I built an AI agent before most people knew the term existed
In late 2023, my team at Atlassian started building what would become Rovo Agent. The brief was straightforward: employees waste hours searching across Jira, Confluence, Slack, and Google Drive for answers. Could we build something that finds the answer for them?
We did not start by building an "agent." We started by hooking an LLM up to a search API. The LLM would read the user's question, decide which tool to call - Jira search, Confluence search, or Slack history - fetch the results, and then write an answer with citations.
That loop - reason, pick a tool, call it, read the result, repeat - is what the industry now calls Agentic AI. At the time we just called it "the search thing." The name is new. The architecture is not.
So what actually makes something "agentic"?
A chatbot takes your question and generates a response from its training data. That is all it does. It cannot look things up. It cannot check if its answer is correct. It cannot break a complex problem into steps and work through them.
An AI agent does all of those things. Here is a concrete example from Rovo:
A product manager asks: "What was the decision on the auth migration timeline? I think it was discussed in a Slack thread last month."
A chatbot would say something vaguely plausible based on general knowledge. Rovo does this instead:
- 1Searches Slack for threads mentioning "auth migration" in the last 60 days
- 2Finds three relevant threads, reads them
- 3Searches Confluence for any decision documents linked in those threads
- 4Cross-references with the Jira epic to check current sprint status
- 5Writes a summary: "The decision was made on March 12 to delay the migration to Q3. Here is the Confluence doc, the Slack thread, and the current Jira status."
Five tool calls, three different data sources, one coherent answer. That is what "agentic" means in practice: the system figures out what to do, does it, checks the results, and keeps going until the job is done.
The architecture under the hood
Every production agent I have worked on - Rovo included - follows roughly the same pattern. It is not complicated. The hard part is making it reliable.
The loop (sometimes called ReAct, for Reasoning + Acting):
The LLM receives the user's question plus a list of tools it can use. It decides: should I answer directly, or should I call a tool first? If it calls a tool, the result comes back, and the LLM decides again. This repeats until it has enough information to answer.
In pseudocode, the core loop looks something like this:
messages = [system_prompt, user_question]
while True:
response = llm.chat(messages, tools=available_tools)
if response.has_tool_call:
result = execute_tool(response.tool_call)
messages.append(tool_result(result))
else:
return response.text # final answerThat is genuinely it. The entire multi-billion-dollar "Agentic AI" industry is variations on this loop. The complexity is in the details: which tools, how to handle errors, how to manage context windows, how to prevent infinite loops, how to make the system safe.
Memory is the piece most people underestimate. A conversation has short-term memory (the current chat). But useful agents also need long-term memory - a vector database that stores documents, past interactions, and domain knowledge the LLM can search when it needs context beyond the current conversation.
Tool design matters more than model choice. I have seen teams spend weeks debating GPT-4 versus Claude when the real problem was that their tool descriptions were ambiguous. If the LLM cannot tell the difference between "search_documents" and "search_knowledge_base," it will pick the wrong one half the time. Clear, specific tool descriptions are the single most impactful thing you can do.
Where I have seen agents actually work in production
I am skeptical of most "agent" demos. A lot of what gets shared on Twitter is a carefully prompted demo that breaks the moment you change the input. Here is what I have seen actually hold up:
Enterprise knowledge retrieval - this is where I have the most direct experience. Rovo handles thousands of queries daily across Atlassian's internal tools. The key to making it work was not the model. It was the retrieval pipeline: how we chunked documents, which embedding model we used, and how we handled the case where no relevant document exists (the agent should say "I don't know" instead of making something up).
Code review and generation agents - GitHub Copilot's coding agent is a good example. It reads an issue, understands the codebase context, writes code across multiple files, runs tests, and opens a pull request. I use this daily and it genuinely works for well-scoped tasks. It falls apart on ambiguous, open-ended requests - which tells you something about where agents are and are not ready.
Customer support pipelines - this is where multi-agent setups shine. One agent classifies the ticket, another retrieves relevant docs, a third drafts the response, and a fourth checks quality. Each agent is simple. The orchestration is where the complexity lives.
What does NOT work well yet: fully autonomous agents that operate for hours without human checkpoints. The error rate compounds. If each step has 90% accuracy, after 10 steps you are at 35% overall accuracy. Production agents need human-in-the-loop for anything high-stakes.
The honest learning path
If you want to build agents, you need to understand the layers underneath. I have seen engineers try to jump straight to LangChain without understanding how embeddings work or why their retrieval is returning garbage. They get stuck fast.
The path that actually works, in my experience:
First, get comfortable with Python beyond the basics. You need to understand async programming, API design, and error handling. Agents make a lot of concurrent calls and things fail constantly.
Second, understand ML fundamentals - not to become a researcher, but to have intuition. When your agent gives bad answers, is it a model problem, a prompt problem, or a retrieval problem? You cannot diagnose this without understanding the basics.
Third, learn LLM APIs directly. Build a simple chat app. Then add function calling. Then add a vector database. Do this from scratch before touching any framework. You need to understand what LangChain is abstracting away, or you will be helpless when it breaks.
Fourth, learn the frameworks. LangChain for basic agents. LangGraph for multi-agent orchestration. Build something real - not a tutorial project, something you actually use.
Fifth, deploy it. Run it for real users. Watch it fail. Fix the failures. This is where the actual learning happens.
I cover this entire path in the bootcamp I teach at Thrive With AI. Twenty weeks, from Python fundamentals through production agent deployment. If self-study is more your style, the path above works - it just takes longer without someone to debug with you.
The part nobody talks about: agent failure modes
I want to end with something practical. These are the failure modes I have personally debugged in production agents, and they are not in any tutorial:
Infinite loops - the agent calls a tool, gets an unhelpful result, and decides to call the same tool again with a slightly different query. Repeat forever. Fix: add a max-iterations limit and a "give up gracefully" fallback.
Tool confusion - the agent has access to both "search_jira" and "search_confluence" and does not know which one to use. It picks the wrong one, gets irrelevant results, and hallucinates an answer. Fix: make tool descriptions extremely specific and non-overlapping.
Context window overflow - the agent retrieves too much data from tool calls and exceeds the context window. The LLM starts ignoring the early parts of the conversation. Fix: summarize intermediate results and be aggressive about what context you keep.
Confident wrong answers - the agent retrieves a slightly relevant document and generates a fluent, confident response that is factually wrong. This is the hardest one. Fix: add citation requirements and build a verification step.
These are the problems that separate demo agents from production agents. They are also the problems you can only learn to solve by building real systems - which is why I am skeptical of courses that only teach the happy path.
If you want the structured version of everything in this post, I wrote a complete Agentic AI learning roadmap that you can follow for free.