Великі мовні моделі (LLM), такі як GPT-4, Claude, Mistral та інші, здаються розумними у своїх відповідях — але справжня магія полягає в тому, як вони сприймають і інтерпретують контекст. Розуміння того, що входить у контекст LLM і як це впливає на результат, критично важливе для розробників, дослідників і дизайнерів продуктів, які працюють із генеративним ШІ.

У цій публікації я хочу дослідити складові контексту, його структуру, обмеження та взаємодію з найбільш поширеними сценаріями використання, такими як використання інструментів (Tools, MCP) і включення додаткових знань з Retrieval-Augmented Generation (RAG).

I continue to explore one of my favorite topics: how to make AI agents more independent. This blog is my way of organizing ideas and gradually shaping a clear vision of what this might look like in practice.

The Dream That Started It All

When large language models (LLMs) and AI chat tools first started delivering truly impressive results, it felt like we were entering a new era of automation. Back then, I believed it wouldn’t be long before we could hand off any intellectual task to an AI—from a single prompt.

I imagined saying something like:

"Translate this 500-page novel from French to Ukrainian, preserving its original literary style."

And the AI would just do it.

But that dream quickly ran into reality. The context window was a major limitation, and most chat-based AIs had no memory of what they'd done before. Sure, you could translate one page. But across an entire novel? The tone would shift, the style would break, and continuity would be lost.

Large Language Models (LLMs) like GPT-4, Claude, and Mistral appear to produce intelligent responses — but the magic lies in how they consume and interpret context. Understanding what goes into an LLM's context and how it shapes output is critical for developers, researchers, and product designers working with generative AI.

This post explores the components of context, how it's structured, how it's limited, and how advanced use cases like tool usage and retrieval-augmented generation (RAG) interact with it.

Good news! I've extended my lightweight AI orchestrator, CleverChatty, to support Retrieval-Augmented Generation (RAG) by integrating it using the Model Context Protocol (MCP).

Quick Recap

• RAG (Retrieval-Augmented Generation) is an AI technique that enhances language models by retrieving relevant external documents (e.g., from databases or vector stores) based on a user’s query. These documents are then used as additional context during response generation, enabling more accurate, up-to-date, and grounded outputs.

• MCP (Model Context Protocol) is a standard for how external systems—such as tools, memory, or document retrievers—communicate with language models. It enables structured, portable, and extensible context exchange, making it ideal for building complex AI systems like assistants, copilots, or agents.

• CleverChatty is a simple AI orchestrator that connects LLMs with tools over MCP and supports external memory. My goal is to expand it to work with modern AI infrastructure—RAG, memory, tools, agent-to-agent (A2A) interaction, and beyond. It’s provided as a library, and you can explore it via the CLI interface: CleverChatty CLI.

I would like to expose one more benefit of the Model Context Protocol (MCP) — the ability to easily change the transport protocol. There are three different transport protocols available now, and each has its own benefits and drawbacks.

However, if an MCP server is implemented properly using a good SDK, then switching to another transport protocol is easy.

Quick Recap: What is MCP?

• Model Context Protocol (MCP) is a new standard for integrating external tools with AI chat applications. For example, you can add Google Search as an MCP server to Claude Desktop, allowing the LLM to perform live searches to improve its responses. In this case, Claude Desktop is the MCP Host.

There are three common types of MCP server transports:

• STDIO Transport: The MCP server runs locally on the same machine as the MCP Host. Users download a small application (the MCP server), install it, and configure the MCP Host to communicate with it via standard input/output.

• SSE Transport: The MCP server runs as a network service, typically on a remote server (but it can also be on localhost). It's essentially a special kind of website that the MCP Host connects to via Server-Sent Events (SSE).

In recent months, the Model Context Protocol (MCP) has gained a lot of traction as a powerful foundation for building AI assistants. While many developers are familiar with its core request-response flow, there's one feature that I believe remains underappreciated: the ability of MCP servers to send notifications to clients.

Let’s quickly recap the typical flow used by most MCP-based assistants:

• A user sends a prompt to the assistant.
• The assistant attaches a list of available tools and forwards the prompt to the LLM.
• The LLM generates a response, possibly requesting the use of certain tools for additional context.
• The assistant invokes those tools and gathers their responses.
• These tool responses are sent back to the LLM.
• The LLM returns a final answer, which the assistant presents to the user.

This user-initiated flow is incredibly effective—and it’s what powers many AI assistants today.

However, MCP also supports a less obvious but equally powerful capability: tool-initiated communication. That is, tools can trigger actions that cause the MCP server to send real-time notifications to the client, even when the user hasn’t sent a new prompt.