Context Engineering: Building Smarter AI Agents - Part 1/3

For the past two years, I’ve been building generative AI applications for businesses - everything from straightforward workflows that automate simple tasks to sophisticated AI agents powering complex conversational experiences. Lately, I’ve noticed the term “context engineering” popping up everywhere in the AI community. I dug into what it actually means, only to realise it’s essentially what I’ve already been doing while developing agentic AI systems with frameworks like LangGraph. It turns out, the practices behind context engineering aren’t new - they’ve just finally been given a name.

In this post - the first in a three-part series - I’ll share a deep dive into context engineering, drawing on my experience building agentic AI applications for real-world businesses. I’ll illustrate key concepts with practical examples and walk through LangGraph code snippets to show how it naturally supports context engineering in practice. So, sit back, relax, and enjoy the read! 📖

Beyond Prompt Engineering – The Rise of Context Engineering

The Evolution of LLM Interaction

In the rapidly evolving landscape of Artificial Intelligence, Large Language Models (LLMs) have transitioned from being mere conversational tools to becoming the core intelligence of sophisticated, dynamic agentic systems. Initially, interacting with LLMs primarily involved crafting precise, single-turn prompts – a practice widely known as prompt engineering. This approach, while effective for specific, isolated queries, quickly revealed its limitations when faced with complex, multi-step tasks or ongoing dialogues. The need for LLMs to maintain state, interact with external tools, and engage in multi-turn reasoning led to the development of more advanced interaction paradigms.

What is Context Engineering?

As LLM applications grew in complexity, a new discipline emerged: Context Engineering. At its heart, context engineering is the art and science of meticulously managing the information an LLM uses to think, act, and decide. It’s about constructing dynamic systems that provide the right information and tools, in the optimal format, at precisely the right moment, to enable an LLM to accomplish a given task reliably and effectively.

To distinguish it from prompt engineering: while prompt engineering focuses on the user-facing input to produce a desired response, context engineering is a developer-facing discipline. It delves into the underlying architecture and data flows, ensuring that the LLM receives a comprehensive and well-structured input that goes far beyond a simple prompt. This comprehensive input, or ‘context,’ encompasses a wide array of elements, including:

• System Instructions: High-level directives that define the LLM’s role, persona, and constraints.
• Retrieved Content: Information pulled from external knowledge bases, databases, or documents relevant to the current task.
• Tool Outputs: Results from external tools or APIs that the LLM has invoked (e.g., search results, database queries, code execution outputs).
• Conversation History: The ongoing dialogue, summarised or filtered to maintain coherence and continuity.
• External Data: Any other pertinent data points, such as user preferences, environmental variables, or real-time sensor data.

In essence, context engineering acknowledges that the quality of an LLM’s output is directly proportional to the quality and relevance of the context it receives. It’s about building the intelligent scaffolding around the LLM that allows it to perform at its peak.

Why is Context Engineering Crucial?

The adage “garbage in, garbage out” holds particularly true for LLMs. Without relevant, precise, and well-organised context, even the most advanced LLMs can produce unreliable, irrelevant, or hallucinated outputs. Most failures observed in complex AI agents are not due to the inherent limitations of the LLM itself, but rather to what we term “context failures” – instances where the model was not provided with the necessary information to make an informed decision or generate an accurate response.

This is where LangGraph emerges as a pivotal framework. LangGraph, a powerful library within the LangChain ecosystem, provides developers with granular control over the flow of information and the management of context within LLM applications. By enabling the explicit definition of stateful, cyclic workflows, LangGraph allows for meticulous context engineering, ensuring that the LLM always operates with the most pertinent and up-to-date information. It transforms the abstract concept of context management into a tangible, programmable reality, paving the way for the development of truly intelligent and reliable AI agents.

Pillars of Context in LLM Applications

To effectively engineer context, it’s crucial to understand its multifaceted nature within LLM applications. Context is not a static entity; it’s a dynamic and evolving construct that underpins the LLM’s ability to reason and act.

Dynamic and Evolving Context

Unlike traditional software where inputs are often fixed at the beginning of a process, the context for an LLM in an agentic system is assembled and continuously updated on the fly. As a task progresses, new information becomes available, previous actions yield results, and the LLM’s internal state evolves. This necessitates a system that can dynamically manage and update the context provided to the model.

Consider a customer support chatbot. The initial context might be the user’s first query. As the conversation unfolds, the context expands to include the entire conversation history, the chatbot’s previous responses, and any information retrieved from a knowledge base or CRM system. If the user asks for an order status, the system needs to dynamically add order details to the context before querying the LLM.

Full Contextual Coverage

It’s not enough to provide just the immediate query; for an LLM to perform optimally, it requires full contextual coverage. This means supplying all necessary information that might influence its understanding, reasoning, or generation. This includes not only the explicit user input but also implicit background information, system-level instructions, and relevant external data.

For a code generation agent, full contextual coverage would involve not just the user’s request for a function, but also the project’s coding standards, existing library dependencies, relevant API documentation, and even examples of similar functions within the codebase. Omitting any of these could lead to suboptimal or incorrect code.

Tools and External Information

One of the most significant advancements in LLM applications is their ability to leverage external tools and access real-time information. This capability is a cornerstone of effective context engineering, as it allows LLMs to overcome their inherent knowledge limitations (i.e., being trained on a fixed dataset) and interact with the dynamic real world. Tools can range from simple search engines and calculators to complex APIs for database interaction, code execution, or external service calls.

An LLM agent tasked with planning a trip needs access to real-time flight prices, hotel availability, and weather forecasts. These pieces of information are not part of its pre-trained knowledge; they must be retrieved dynamically through external tools (e.g., flight booking APIs, weather APIs) and then integrated into the context for the LLM to process and generate a coherent travel plan.

Memory Management (Short-term & Long-term)

Effective context engineering relies heavily on robust memory management, which can be broadly categorised into short-term and long-term memory:

• Short-term Memory: This refers to the immediate, transient context of an ongoing interaction. For LLMs, this often involves managing the conversation history within the model’s context window. Since context windows have limits, strategies like summarisation, truncation, or selective retention are crucial to preserve the most relevant parts of the dialogue.

In a multi-turn conversation, a chatbot might summarise the previous 10 turns into a concise paragraph to keep the LLM informed about the conversation’s trajectory without exceeding the context window limit.

• Long-term Memory: This involves persisting and retrieving information that spans across multiple sessions or is relevant to a user’s enduring preferences or past interactions. This often utilises external databases, vector stores, or knowledge graphs.

A personalised shopping assistant might store a user’s past purchases, preferred brands, and sizing information in a long-term memory system. When the user returns, this information can be retrieved and added to the context, allowing the LLM to offer highly relevant product recommendations.

Structured Data and Format

The way information is presented to an LLM significantly impacts its ability to process and utilise that information. Well-organised, structured data (e.g., JSON, XML, or clearly delimited text) is far more effective than unstructured, messy data. Providing context in a consistent and predictable format reduces ambiguity and improves the LLM’s parsing and reasoning capabilities.

When providing search results to an LLM, presenting them as a list of JSON objects with clear keys (e.g., {"title": "...", "url": "...", "snippet": "..."}) is much more effective than simply concatenating raw text. The structured format allows the LLM to easily identify and extract specific pieces of information, leading to more accurate and relevant responses.

Conclusion

In this first part of our three-part series, we’ve unpacked the foundations of context engineering - what it is, why it matters, and how it underpins the next generation of agentic AI systems. By exploring the pillars of context, from dynamic memory management to the integration of external tools and structured data, we’ve set the stage for building truly intelligent, reliable LLM applications.

In Part 2, we’ll dive into how LangGraph provides the architectural framework to put these principles into practice, enabling you to engineer context with precision and flexibility. Stay tuned!