Agent might be the most overloaded word in AI right now. Everyone uses it, few define it, and the interpretations range wildly.

That confusion didn’t get better when I landed in Microsoft Foundry. I started by building a prompt agent in the portal, assuming it was just “the” way to build a Foundry agent. It wasn’t until I hit its limits — barely any control over orchestration, everything felt locked into a fairly restrictive shape — that I realized I’d only found one of several agent types, and not the one built for production. Hosted agents, self-hosted patterns, custom registered agents: each solves a different problem.

This post is the guide I wish I’d had — some clarity on Microsoft Foundry’s agent types, what each is actually for, and when to reach for which.

What is an agent, really?

Think of a regular AI chat like asking a colleague a question. They think, they answer, done. One turn, one response. An agent is different. It can take multiple steps to get to an answer, and along the way, it can use provided tools: search the web, query a database, run code, call an API. It decides what to do next based on what it got back from the last step. Instead of just responding with text, it can actually do something for you.

An agent is code running somewhere, using an LLM as its brain, deciding what to do next based on its instructions, the conversation history, and the tools available to it. Under the hood, in very simple terms, it might just be a while loop: Call the LLM with the user’s prompt, check if it wants to use a tool, run the tool, feed the result back, repeat. That’s the agentic loop. That loop or that code needs to run somewhere, which is a crucial difference when it comes to the agent types we cover in this blog post.

Foundry agent types

Microsoft Foundry’s Agent Service, as of now, gives you two first-class agent types:

• Prompt agents: Fully managed, defined through configuration (model, system prompt, tools). No code, no container.
• Hosted agents: Your own code, packaged as a container or .zip file, running on Foundry-managed infrastructure.

On top of that, there are two related concepts that often get mixed up:

• Self-hosted agents: Your own code running on your own infrastructure (Azure Container Apps, AKS, your local container environment, etc.), calling the Foundry Responses API for model access. This is not a distinct Foundry agent type, it’s basically your custom application using Foundry’s model APIs.
• Foundry custom (registered) agents: An agent that already runs anywhere — any platform, any infrastructure, any protocol (HTTP or A2A) — that you register into the Foundry Control Plane purely for centralized management and observability. Foundry doesn’t run or host the agent; it fronts it with an API Management gateway, gives you a new endpoint for clients to call, and collects OpenTelemetry traces (via Application Insights) if your agent emits them. It’s basically a governance layer.

These distinctions matter, because choosing the right agent type has a lot to do with the complexity, architecture, availability, connectivity, and scaling requirements of your agent — as well as cost, security, and latency.

Prompt agents

A prompt agent is the fastest and simplest way to get started with agents in Foundry — and a good starting point if you’re new to agents. You can create one directly in the Foundry portal by configuring the model, tools, knowledge, memory, and guardrails, and Foundry runs it for you. After saving, it’s available in the playground or callable via SDK straight away.

You don’t have to worry about containers or compute — that’s fully managed and abstracted. Which also means you have no control over it. The same applies to the orchestration: your only steering mechanism is the system prompt and the tools you configure. You can’t influence how the model uses those tools or how the loop behaves beyond that. Things like traces and evaluations are neatly integrated out of the box though, whereas other agent types and frameworks require additional configuration for that.

What happens in the background when someone calls your prompt agent looks roughly like this:

1. The request arrives at the Foundry API endpoint.
2. Foundry constructs a full prompt from your system prompt, the conversation history, and the tool definitions — plus some internal components you can’t see.
3. That gets sent to the configured model. The model reasons: do I answer directly, or do I need to use a tool first?
4. If the model decides to use a tool — searching an Azure AI Search index, for example — Foundry calls it on the model’s behalf and feeds the result back into the next round of reasoning.
5. This loop continues until the model produces a final answer, which gets returned to the caller.

None of that requires any code from you. The loop, the tool calls, the conversation threading — Foundry manages all of it. The trade-off is that you can’t reach into that loop. What you configure upfront is what runs.

Benefits

• Zero infrastructure overhead: Compute, scaling, and session management are fully managed by Microsoft. There’s no hard limit on the number of prompt agents per Foundry instance, and you pay only for inference and tool usage — not idle compute.
• Fastest path from idea to running agent: You can have something working in the portal in a few minutes. The playground lets you test directly against MCP server connections, file search, and other tools before you publish anything.
• Built-in tool catalog: Built-in tools like Code Interpreter, Web Search, Azure AI Search, and 1,400+ Logic Apps connectors are available without writing any code. Custom MCP servers on Azure Functions are the exception — you’re authoring and hosting that server yourself.
• M365 publishing: Publishing a prompt agent to Teams/M365 Copilot is a guided, no-code flow from the portal — but it still requires a new or existing Azure Bot Service as a routing and auth layer.
• Private networking available: Prompt agents support private networking within your own Azure VNet for data residency and compliance requirements.
• BYOM via APIM: Bring-your-own-model lets a prompt agent call a model behind Azure API Management — centralized quotas and rate limits, and you can swap the backend model without touching the agent definition.

Limitations

• Function calling requires an external executor: When the agent decides to call a custom function, it pauses and hands back the function name and arguments — it doesn’t run anything itself. Your code has to pick that up, execute it, and return the result. That could be an Azure Function, a small API, or any backend process listening for those calls. For simple scenarios that’s fine, but it adds a round-trip you can’t avoid.
• OBO flows don’t reach custom backend APIs: If your agent needs to call a downstream API on behalf of the signed-in user, you’ll hit limitations depending on how that API is connected. For OpenAPI-defined tools and custom function backends, there’s no native way to pass the caller’s token through to an API you control. Workarounds exist, but they add complexity and push the OBO boundary into your own executor code. If user-delegated access to downstream systems is a hard requirement, hosted agents are the better fit.
• One model per agent: A prompt agent is tied to a single model deployment at definition time. You can route through an API Management via BYOM, but the agent still points at one deployment — it can’t dynamically switch models between requests or route different task types to different models. That logic has to live outside the agent.
• No access to the orchestration loop: Foundry owns the reasoning loop. You can shape the behavior through the system prompt and tool configuration, but you can’t intercept reasoning steps, add retry logic inside a step, or wire up custom chaining that the platform doesn’t natively support.
• Tool and region availability: Not all tools work in all regions, it’s worth checking the tool support before committing to a region.

Hosted agents

Hosted agents are newest addition to the Foundry agent service. The main purpose of hosted agents is to provide an execution runtime environment for your custom agent code. Say you’ve built a custom agent application using a framework like Microsoft Agent Framework. The agent runs locally on your machine. The next step would be to make it accessible, considering scaling, availability, security, and so on. You can then pack your code into a container image, push it to a container registry and deploy a hosted agent in foundry. Foundry will then pull the container image and run the agent for you in a managed runtime environment.

What makes hosted agents distinctive is the per-session isolation model. Each user session runs in its own VM-isolated sandbox with a persistent filesystem, so your agent can maintain state across turns without you managing external storage. Sessions that go idle are deprovisioned automatically, but state is preserved and restored when the user returns. Scaling works the same way — Foundry spins up a new sandbox per session on demand, so there’s no replica count or warm pool to configure.

Your container exposes one of two protocols depending on the use case. The Responses protocol is the default for conversational agents and is fully OpenAI-compatible, with Foundry managing conversation history and streaming. The Invocations protocol gives you full control over the HTTP payload, suited for webhooks or custom streaming scenarios like AG-UI. Access to Foundry-managed tools — Code Interpreter, AI Search, Web Search, custom MCP connections — goes through a Toolbox MCP endpoint your code connects to explicitly. Each deployed agent also gets its own Entra ID identity, used at runtime to authenticate against models, tools, and downstream Azure services.

Benefits

• Full control over the agent loop: Your code owns the orchestration and execution — multi-step reasoning, conditional branching, retry strategies, human-in-the-loop checkpoints, sub-agent chaining, and inline function execution without any external executor round-trip or polling for requires_action. You implement exactly the logic you need, unconstrained by what a platform runtime supports.
• OBO flow support: When invoked via Microsoft 365 channels with a user token present, hosted agents can operate in OBO mode — the platform exchanges the token and your agent calls downstream APIs with the actual user’s delegated permissions. For autonomous or background scenarios it falls back to its own managed identity. This is the path you need when user identity genuinely matters to downstream systems.
• Zero-config observability: Application Insights is automatically wired in, and the protocol libraries emit OpenTelemetry traces by default — every model call, tool invocation, and session event is captured without you writing any telemetry code.
• Per-session VM isolation: Every session runs in its own sandbox with an isolated filesystem. No cross-session data leakage, even at high parallel session counts — important for agents that write and execute code or handle sensitive per-user data.
• Framework and model flexibility: Microsoft Agent Framework, LangGraph, OpenAI Agents SDK, custom code — all supported. Because your code makes the model call directly, you can also implement any routing logic you want: failover across regions, task-based model routing, or APIM as a centralized gateway.
• Scale to zero with stateful resume: Sessions deprovision during inactivity with no idle compute cost. When a session resumes, the $HOME filesystem and session state are restored exactly where they left off — useful for long-running or asynchronous workloads.

Limitations

• Limited infrastructure flexibility: Because the runtime environment is fully managed by Microsoft, you have no control over the underlying infrastructure — OS configuration, network stack, installed system dependencies, or runtime tuning. If your agent code has specific infrastructure requirements, you may hit a limit that a self-hosted approach on Azure Container Apps or AKS would not impose.
• ACR must have a public endpoint: The platform cannot pull images from a private-endpoint-only container registry. If your security posture requires fully private registries, this blocks hosted agents today. Note that private networking for downstream services — VNet injection, private endpoints to your own Azure resources — is supported; the public endpoint requirement applies only to the registry itself. Private ACR is now supported for Foundry projects created after June 25, 2026.
• Protocol adapter libraries are Python and C# only: The runtime libraries that handle the communication between Foundry and the agent container are currently only available for Python and C#.

Custom (registered) agents

Custom agents is a way to bring external agents running anywhere into the centralized governance, management, and observability of Microsoft Foundry Control Plane. Rather than migrating your code into a Foundry-managed runtime, you keep your agent wherever it runs and register its endpoint with Foundry. Once registered, Foundry uses Azure API Management to proxy all communication to your agent, adding access control, monitoring, and a consistent entry point across your agent portfolio.

To register a custom agent, it needs to be reachable via a public or private endpoint and communicate using either HTTP or the A2A protocol. Once registered, it appears in the Foundry portal alongside your other agents and gets a Foundry-managed URL that clients use to interact with it. Optionally, your agent can send telemetry to Application Insights for request-level monitoring.

Custom agents are a good choice when you have an existing agent you don’t want to migrate, infrastructure requirements that go beyond what Foundry’s managed runtime offers, or compliance constraints that require you to keep compute in your own environment.

Benefits

• Little to no code changes: You register the agent wherever it already runs — any cloud, on-prem, whatever framework — as long as it exposes a reachable endpoint. Nothing about the agent itself changes.
• One inventory across your whole fleet: Foundry Control Plane discovers and lists custom agents alongside Foundry-native ones (prompt, workflow, hosted), giving you a single view regardless of framework or hosting location.
• Governance without touching the agent: Registration wires the agent through Foundry’s AI Gateway (Azure API Management acting as proxy), which unlocks access control, rate limiting, and security policy enforcement at the network edge — none of it requires modifying your code.
• A stable, managed endpoint: Foundry Control Plane generates a new URL that clients use to reach the agent, decoupling callers from wherever the agent actually lives.
• Cost/usage visibility, if instrumented: Once Application Insights is connected, Foundry Control Plane can surface token usage, cost, and execution traces for the agent — same operational dashboard as your other agents.
• Framework/protocol agnostic: Works with any agent that speaks HTTP or the A2A protocol — not limited to Microsoft frameworks.

Limitations

• No Foundry Guardrails: The Foundry guardrail system (input/output/tool-call intervention points) applies only to agents built in Foundry Agent Service. Custom agents are explicitly excluded. Your only lever is a content-safety policy at the AI Gateway layer — a different, coarser mechanism, not the same feature.
• Observability isn’t automatic: Foundry doesn’t inject tracing into your agent the way it does for hosted agents. You have to emit OpenTelemetry GenAI-semantic-convention data yourself, or you get no traces, tool spans, or LLM spans in the dashboard at all.
• Prerequisites before you can even register: the agent needs an exclusive endpoint, that endpoint must be reachable from the network where your Foundry resource lives, and an AI Gateway plus Application Insights connection must already be configured on the project — none of this is optional setup.
• No managed compute, scaling, or isolation: Unlike hosted agents, Foundry doesn’t run, scale, or sandbox anything here. Compute, uptime, and isolation remain entirely your responsibility on your own infrastructure.
• No session/state management from Foundry: Registration gives you visibility and governance, not runtime features — conversation state, tool execution, and orchestration are still fully DIY.
• It’s the “builder” control plane, not full enterprise governance: Foundry Control Plane gives you observability, guardrail policy assignment for models, and fleet visibility — but broader enterprise controls (Conditional Access, Entra Agent ID, threat detection across your whole org) belong to the separate Agent 365 product. Registering in Foundry Control Plane alone doesn’t get you those.

Self-hosted agents

Self-hosted isn’t a distinct Foundry agent type — it’s a pattern. Your code runs wherever you choose: Azure Functions, Container Apps, AKS, on-prem, anywhere. You call the Foundry Responses API to access models and platform tools. You get Foundry’s model catalog and tools; you manage everything else.

That “everything else” includes things Hosted agents get built in: session persistence, sandboxing, scaling, and identity are all part of the managed runtime, not the Responses API itself. If you want durable state, you’re responsible for building and running it: a session store, a database for conversation history, or your own memory layer if you want the agent to recall things across calls. The Responses API can persist conversation history server-side, but the moment you need something more custom like cross-session memory, isolation between users, resuming long-running work, etc., that’s on your infrastructure to build and maintain.

Benefits

• Maximum infrastructure flexibility: You decide exactly where your compute runs — which region, which platform, which network. For organizations with strict data residency requirements or existing infrastructure investments they want to reuse, this is a meaningful advantage.
• No Foundry-specific operational model: Your application is just an application. You can apply your existing deployment pipelines, monitoring, scaling, and incident response processes without learning Foundry’s agent lifecycle.
• Gradual adoption path: It’s the easiest way to access Foundry’s model catalog from existing codebases without restructuring your application. You call an API; nothing else about your code has to change.
• Full model routing control: Same as hosted agents — your code makes the model call, so you can implement any routing logic, model switching, or APIM integration you want.
• Works with any language and framework: No protocol library constraints. If Foundry has an SDK for your language, use it; if not, the Responses API is a standard HTTP endpoint.

Limitations

• You own all the infrastructure: Scaling, session isolation, state management, identity provisioning, observability wiring — everything that Foundry Agent Service manages for hosted and prompt agents is your problem. For production agent workloads, that’s non-trivial engineering work.
• No managed agent identity: Prompt agents and hosted agents get a dedicated Entra agent identity provisioned automatically. Self-hosted agents authenticate as whatever identity your application uses, which is typically a managed identity or service principal. There’s no agent-native identity concept — scoping permissions per agent, rather than per application, requires explicit design.
• No native Foundry lifecycle integration: No stable Foundry-managed endpoint, no versioning with weighted rollouts, no one-click M365 publishing. If you want any of that, you either build it yourself or move to a hosted agent.
• No native Session state: Hosted agents get persistent $HOME per session automatically. Self-hosted agents need their own state backend if they require multi-turn continuity.

Comparison

Which one, when

If you’re getting started or building something that fits neatly into Foundry’s built-in tool catalog — use a prompt agent. It’s genuinely fast to set up, Foundry handles everything, and the limitations only bite you when you hit them. It’s also the right answer for straightforward M365/Teams integrations where you want minimal operational surface.

If you need custom logic in the agent’s decision loop/orchestration, require OBO with your own backend APIs, need stateful file operations between turns, want per-session process isolation, or need full flexibility over model routing — use a hosted agent. The containerization adds overhead compared to editing a prompt agent’s system prompt, but it’s the right overhead if your agent genuinely needs it.

If you already have code running somewhere and just want access to Foundry’s model catalog without restructuring anything — call the Responses API directly. This is the self-hosted pattern. It’s perfectly legitimate and it’s explicitly the on-ramp to hosted agents if you later decide you want the managed infrastructure.

If your agent already runs and is deployed elsewhere — another cloud, on-prem, a different team’s stack entirely — and you don’t want to touch its code, but you do want it inventoried, governed, and monitored alongside everything else in Foundry, register it as a custom agent. Foundry doesn’t run it; it proxies to it through the AI Gateway (APIM), gives it a stable URL, and puts its telemetry and access policy where your other agents live. The trade-off: you don’t get Foundry Guardrails, hosted-agent isolation, or managed sessions — you get governance and visibility without a migration.

Hope this helps you pick the right agent type faster than I did. Questions, corrections, or a different experience of your own — I’m always happy to hear about it. Cheers!