One of the biggest issues with large language models (LLMs) is working with your own data. They may have been trained on terabytes of text from across the internet, but that only provides them with a way to predict and generate text based on your prompts. If we’re to trust their output, we need to use them for what they were intended: providing a natural language interface to our applications and data.

There are many different ways to link LLMs to data, from creating your own embeddings and using retrieval-augmented generation (using graph databases to expose deeper relationships in your data) to using LLMs to summarize and narrate data returned via OpenAPI calls constructed in response to user prompts. Other tools include the plug-in model used by OpenAI’s ChatGPT. And then there is the Model Context Protocol, or MCP.

The Model Context Protocol

At the end of 2024, Anthropic released the specification of the Model Context Protocol, intended to standardize connections between LLMs and your own applications and data. The name is important: It emphasizes that AI applications require context if they’re to deliver coherent outputs. Simply expecting a chat prompt to deliver sensible outputs is at best optimistic, and at worst dangerous. If we’re going to build semi-autonomous agents that build their own workflows around our data, we need a reliable way to deliver that data to them.

MCP is an open source project with SDK implementations for most common languages and a GitHub repository that includes documentation for anyone wanting to either implement or use an MCP server. The development team describes it as a “USB-C port for AI applications,” as it standardizes connections to many different data sources. As these build on a standard, there are already connectors for existing projects, working with many different LLMs and inferencing providers.

The architecture used by MCP is familiar, taking its cues from classic client/server implementations where a broker translates MCP requests into local or remote requests. You could think of MCP as an interface definition language such as CORBA, where it becomes an effective interoperability layer that allows you to quickly change information sources or LLM applications as needed. It uses a JSON RPC connection, so it can be controlled at an individual user level using tools such as Azure API Management.

With MCP it’s possible to think about building generic interfaces to your code for AI, and so it’s not surprising to see multiple MCP implementations appearing across Microsoft’s various AI development platforms, from inside the Semantic Kernel model orchestration tool to MCP servers that work with both Azure OpenAI and Azure AI Foundry. Microsoft is also adding tools to Azure API Management to control access to data using user credentials.

The Azure MCP Server

One early implementation of MCP on the Microsoft’s platform is the open source Azure MCP Server, which recently entered public preview. This provides a common broker for AI access to key Azure services. Like many recent Azure projects, this is open source, with code available on GitHub. It offers access to much of the Azure platform, including databases, storage, and services like the Azure CLI.

Support for the Azure CLI (and the Developer CLI) is interesting as it allows your MCP-powered agents to use Azure directly, treating MCP calls as an operator. This lets you build agents that provide a natural language self-service interface to Azure, for example, taking a description of an infrastructure and then building the Arm templates needed to deploy it. You can even imagine a multimodel agent that analyzes a picture of a whiteboard sketch, creates a description of the resources needed to implement the sketch, and then deploys it so you’re ready to build code. Other system administration services available through the Azure MCP Server include listing current resource groups and using KQL to query Azure Monitoring logs.

Using Azure MCP server with GitHub Copilot Chat

As it’s based on MCP, this new server works from any AI tool that supports MCP, for example, the GitHub Copilot Agent Mode. Simply add the server to your tenant and start asking questions via Copilot, either directly or with the Visual Studio Code integration. In practice, this last option is an effective way to learn how to use MCP and to build the prompts in your own MCP-based AI applications.

Microsoft has yet to release its own MCP tool for its languages, so you’ll need the official SDKs to write your own code. With TypeScript, C#, and Python all supported, you should have most of the tools you need to start writing your own Azure MCP agents. You can experiment from inside Visual Studio Code, using existing Azure credentials.

MCP in Azure AI Foundry

Microsoft is quickly building out a fleet of other MCP tools, exposing existing functionality through MCP or making it consumable in your own agentic applications. It’s a rapid rollout, too; tools for Copilot Studio’s no-code agent development were announced while I was working on this article.

Azure AI Foundry is Microsoft’s main development platform for at-scale AI application development, so it’s not surprising that it too is working on an MCP Server to work alongside Azure’s AI Agent Service. This is intended to link agents running in Azure AI Foundry with clients running as part of other AI applications.

Using MCP servers from Semantic Kernel AI applications

Because MCP is an open standard, the server should work with any client. The GitHub repository includes instructions on how to add a connection using Anthropic’s Claude Desktop, but the real value comes when building your own agent workflows in Semantic Kernel.

Microsoft has delivered some sample code that shows how to build MCP support into a Semantic Kernel orchestration, using it as a kernel plug-in that works with familiar function calls. The same integrations can be wrapped as agents and accessed as needed. Using MCP from Semantic Kernel is still a work in progress, but for now, it seems to fit nicely alongside its existing feature set, requiring very little additional code to expose MCP tools from a server to your AI applications.

Tools like MCP are a crucial element of a modern AI stack, providing a common way to build discoverable interfaces to both local and remote applications. Once defined, MCP tools are easy to call, with a server providing a list of the available tools and MCP giving LLMs a standard way to call those tools and use their outputs. The technique goes a long way to providing a universal grounding tool for AI applications, working with standard APIs, database queries, and AI agents alike.

Explore IT Tech News for the latest advancements in Information Technology & insightful updates from industry experts! 

Source: https://www.infoworld.com/article/3975058/using-the-model-context-protocol-in-azure-and-beyond.html