# Basic Concepts when building AI Agents Let's cover some basic concepts that we will utilise in our AI agents. We start by calling through to an LLM (OpenAI) via code. Subsequently, we will introduce [LiteLLM](https://www.litellm.ai/) as a wrapper to access LLMs from our code. We will also do some basic evaluation of our results via the [GAIA dataset](https://huggingface.co/datasets/gaia-benchmark/GAIA). Let's get started. * * * ## Project Resources All code examples discussed in this post can be found in the accompanying repo: [GiHub: ai-notebook](https://github.com/rob212/ai-notebook/basics/prerequisites/02-basic-concepts/). If you wish to code along, the details of my setup can be found in my [How I set up a Python AI project](https://ai-notebook.uk/how-i-set-up-a-python-ai-project) post. * * * ## **Concepts** I created `basic-concepts.py` to experiment with some of the basic concepts we will be using as we develop AI agents. * * * ## **🏗️ The Basic Connection** The foundation of any agent is the ability to communicate with a model. We start by creating a simple interface to send a prompt and receive a string. To do this I will use the **gpt-5-mini** model by OpenAI, in order to call it via the OpenAI python sdk you will need to [sign-up and create an API Key](https://platform.openai.com/api-keys). Note you will need to put some credit in your account ($5 is the minimum but should be sufficient for this learning). Ensure you have copied my `.env.example` file to a new `.env` file where you can add your API Keys. ```shell cp .env.example .env ``` > NOTE: ensure that your `.env` file is referenced in your `.gitignore` file if you are planning to push your code to a git repository, to ensure your keys are not exposed. ### A simple call through to OpenAI The foundation of any agent is the ability to communicate with a model. We start by creating a simple interface to send a prompt and receive a response. ```python import os from openai import OpenAI from dotenv import load_dotenv, find_dotenv load_dotenv(find_dotenv()) client = OpenAI(api_key=os.getenv("OPENAI_API_KEY")) response = client.chat.completions.create( model="gpt-5-mini", messages=[{"role": "user", "content": "My name is Rob"}] ) print(response.choices[0].message.content) ``` Note the use of the '**role**' to distinguish the actor that was responsible for the content. This becomes relevant was our context evolves and we need to keep track of the originator of each interaction. The actual response text is found in response.choices\[0\].message.content. The reason choices is a list is that you can request multiple responses using the n parameter, though typically you only use the first one. The OpenAI client supports both the `chat.completions` syntax and the newer `client.responses`. Which was introduced by OpenAI to better reflect the multimodal nature of it's GPT modals. Anthropic, Gemini and other LLMs will all provide a slightly different syntax, increasing the need for boilerplate code in our agent. Thankfully to decouple this tight dependency there are numerous tools that abstract the underlying calls to major LLMs via a wrapper. [**LiteLLM**](https://www.litellm.ai/) is one we will use. * * * ## **LiteLLM - Wrapper to our model call** You can add the `litellm` dependency to your project as follows: ```shell uv add litellm ``` By importing the `completion` function from liteLLM our same call can be made like so: ```python from litellm import completion from dotenv import load_dotenv, find_dotenv load_dotenv(find_dotenv()) ## Basic call to LLM via LiteLLM wrapper response1 = completion( model="gpt-5-mini", messages=[{"role": "user", "content": "My name is Rob"}] ) print(response1.choices[0].message.content) ``` Litellm let's you call over 100 LLMs through this unified completion interface, allowing us to easily swap out LLMs easily. Provided you also have an Anthropic API Key configured in your `.env` file you can test this out by simply updating the model to a Claude model: ```python response1 = completion( model="claude-sonnet-4-5", messages=[{"role": "user", "content": "My name is Rob"}] ) ``` * * * ## **LLM APIs are stateless** When you use ChatGPT or Claude via their web interface, they appear to remember previous conversations. However, LLM APIs are stateless, so each API call is indepenent and has no memory of the previous call. Let's test this out for ourselves: ```python from litellm import completion from dotenv import load_dotenv, find_dotenv load_dotenv(find_dotenv()) response1 = completion( model="gpt-5-mini", messages=[{"role": "user", "content": "Hi, my name is Rob"}] ) print(response1.choices[0].message.content) response2 = completion( model="gpt-5-mini", messages=[{"role": "user", "content": "What is my name?"}] ) print(response2.choices[0].message.content) ``` Even though we introduced ourselves in the first call, the subsequent second call has no memory of it. Therefore to maintain conversation history we must manage it ourselves. ```python messages = [] # First Exchange messages.append({"role": "user", "content": "My name is Rob."}) response = completion(model="gpt-5-mini", messages=messages) assistant_message1 = response.choices[0].message.content messages.append({"role": "assistant", "content": assistant_message1}) print(assistant_message1) # Second Exchange - includes previous conversation history messages.append({"role": "user", "content": "What is my name?"}) response2 = completion(model="gpt-5-mini", messages=messages) assistant_message2 = response2.choices[0].message.content print(assistant_message2) ``` We accumulate all conversation content in the `messages` list and pass the entire history with each call. We ensure that the messages have a corresponding role, the `user` being the human asking the question, and `assistant` being the LLM. * * * ## **Structured Output** We are familiar with the natural languages text that LLMs generate and it is great for humans to read, but incovenient for programs to process. Most modern LLM providers (OpenAI, Anthropic, Gemini etc) support '**Structured Output**', a feature that allows us to instruct LLMs to generate responses in a defiend format, like JSON. A common approach to this is by defining your desired output format using Python's [Pydantic](https://docs.pydantic.dev/latest/) library. Pydantic is a library for data validation that lets you define data structures as classes. By inheriting from `BaseModel` you can create a schema that specifies field names and their types. Let's play with an example `ExtractedInfo` that defines three fields: `name` and `email` are required strings, while `phone` is an optional string that defaults to `None` if not provided. You can add the pydantic dependency to your project as follows: ```shell uv add pydantic ``` ```python from pydantic import BaseModel class ExtractedInfo(BaseModel): name: str email: str phone: str | None = None response4 = completion( model="gpt-5-mini", messages=[ { "role": "user", "content": "My name is John Smith, my email is john@example.com, and my phone number is 07712345678", } ], response_format=ExtractedInfo, ) result = response4.choices[0].message.content print(result) ``` We pass our Pydantic model to `response_format`, a parameter [supported by litellm](https://docs.litellm.ai/docs/completion/json_mode). If you inspect the result from our gpt-5-mini model, you should observe the LLM adhering to our defined ExtractedInfo schema: ```shell {"name":"John Smith","email":"john@example.com","phone":"07712345678"} ``` Try experimenting by altering the initial user question, to remove the phone number and running the prompt again. This time the LLM has no possible way of knowing the phone number so excludes it from it's structured response. How does this differ if you remove one of the none optional fields from the content? Structured output plays a crucial role in agent development. In tool calling, which we'll cover later, the LLM must output which tool to call with which arguments in a structured format. One of the core capabilities of structured output is the ability to convert user intent into appropriate actions. * * * ## **Asynchronous calls** It is possible that you may need to process multiple LLM requests simultaneously when developing your agents. This could be due to comparing responses from multiple models, running a multi-agent system or evaludating dozens of problems in a benchmark test. This is handled as it would be in any Python program via `async/await` and the `asyncio` library. Litellm supports asynchronous calls through the `acompletion` function. ```python import asyncio from litellm import acompletion # Limit to 10 concurrent requests semaphore = asyncio.Semaphore(10) async def call_llm(prompt: str) -> str: """LLM call with rate limiting and automatic retry.""" async with semaphore: response = await acompletion( model="gpt-5-mini", messages=[{"role": "user", "content": prompt}], num_retries=3, # Automatic rety with exponential backoff ) return response.choices[0].message.content # Even if we had 100 prompts, only 10 API calls run at a time prompts = [ "What is 2 + 2?", "What is the capital of Japan?", "Who wrote Romeo and Juliet?", ] # Execute all requests concurrently ## `return_exceptions=True` argument prevents a single failure from cancelling all other tasks, ## Instead, exceptions are returned as values in the results list, allowing us to handle failures gracefully ## while still getting results from successful calls. tasks = [call_llm(p) for p in prompts] results = await asyncio.gather(*tasks, return_exceptions=True) for prompt, result in zip(prompts, results): print(f"Q: {prompt}") print(f"A: {result}\n") ``` This example includes measures to prevent two common issues when sending many requests simultaneously. Rate limits from the API provider and transient failures from network issues or server overload. LiteLLM's `num_retries` parameter handles transient failures with automatice exponential backoff. For rate limiting we can use Python's `asyncio.Semaphore` to limit how many requests run concurrently. By wrapping the `acompletion` call we can ensure that no matter how many tasks are running only a limited number of actual API calls happen at once. * * * ## **System Prompt** System prompts are particularly important for agents. Agents act autonomously across multiple steps. System prompts define the behavioral rules that guide all of these decisions. From a context engineering perspective, the system prompt is information that is always included in the context. Every time the agent calls a tool, analyzes results, or decides on its next action, the system prompt sits at the front of the context, guiding the agent's judgment. This is why the quality of the system prompt determines the quality of the agent's overall behavior. When wirting a system prompt, you typically provide this context to the LLM with the role of 'system'. For example using the `acompletion` call via liteLLM: ```python response = await acompletion( model=model, messages=[ {"role": "system", "content": SYSTEM_PROMPT}, {"role": "user", "content": question}, ], ``` You can see some examples of a production agents system prompt as [Anthropic publishes Claude's system promtps](https://docs.anthropic.com/en/release-notes/system-prompts). Note this Claude system prompt is applied when interacting through the [claude.ai](http://claude.ai) website or mobile chat interface. When calling Claude through the API, developers write their own system prompts. Claude's system prompt serves 4 main roles: defining the products identity, specifying output format and style, setting boundaries on prohibited behaviour and clarifying the limits of it's knowledge.