Microsoft published new architecture guidance on August 19, 2026, for one of the most important Foundry Agent Service decisions: should a workload use a Prompt Agent or a Hosted Agent?

The answer is not primarily about which model the agent uses. It is about where orchestration logic lives and who owns the runtime, state, and operational responsibilities.

Prompt Agents express behavior declaratively through a model, instructions, tools, knowledge, and identity configuration while Foundry operates the runtime. Hosted Agents package application code and a chosen framework for Foundry to run behind managed endpoints, identity, scaling, session, and observability services.

Microsoft's practical recommendation is to start with a Prompt Agent unless the workload has a clear requirement for code-level runtime control.

An Agent Is More Than a Model Call

A basic AI assistant generates an answer. An agent can also decide what happens next, invoke tools, access data, retain context, and complete work across several steps.

Most agents begin with three building blocks:

• A model for language understanding, generation, and reasoning

• Instructions that define the role, task, boundaries, and expected behavior

• Tools that connect the agent to search, APIs, data, code, MCP servers, and business systems

An enterprise implementation needs much more than those three components. Identity, authorization, network controls, content safety, session management, evaluation, tracing, versioning, rollback, reliability, and cost controls determine whether the agent can be operated responsibly.

Foundry Agent Service supplies managed platform capabilities around the agent. The selected agent type determines how much application runtime remains the engineering team's responsibility.

The Real Architecture Questions

Microsoft frames the choice around three questions:

• Who owns orchestration?

• Who owns state?

• Who owns operations?

Those questions are more useful than beginning with a preferred framework.

If Foundry can express and operate the required reasoning and tool-use loop through a declarative definition, a Prompt Agent usually keeps the architecture smaller. If the workload needs deterministic branching, custom dependencies, protocols, state, or middleware, orchestration belongs in application code and a Hosted Agent becomes the stronger fit.

The goal is not to minimize code at all costs. It is to put necessary complexity in a form that can be tested, versioned, and operated clearly.

Prompt Agents Keep the Runtime Declarative

A Prompt Agent is defined through configuration rather than a custom application container.

The team focuses on:

• Model selection and deployment

• Instructions and output expectations

• Approved tools and knowledge sources

• Agent identity and permissions

• Safety configuration

• Evaluation and versioning

Foundry owns the surrounding compute, runtime maintenance, scaling, and patching.

This is often the fastest route from a validated idea to a governed agent. It is especially useful when reviewers need to understand the agent from a relatively small, inspectable definition.

Good Prompt Agent Workloads

Microsoft highlights several patterns that fit a Prompt Agent well.

An enterprise knowledge assistant can retrieve approved policies, standards, procedures, and product documentation while citing the source.

A document review assistant can evaluate a proposal against a fixed rubric and return structured findings, with a human retaining the final decision.

An employee self-service agent can answer common questions and perform a small number of tightly scoped actions such as checking status or creating a support case.

These workloads may use reasoning and tools, but their control flow remains understandable without an application-sized prompt.

Other strong candidates include guided search, grounded summarization, classification followed by one approved action, and bounded data lookup.

Warning Signs That the Prompt Is Becoming Code

A Prompt Agent is probably the wrong boundary when the instruction file begins acting like a programming language.

Warning signs include:

• Large nested branches written in prose

• Retry and timeout behavior expressed as prompt rules

• A deterministic state machine hidden inside instructions

• Complex payload transformation

• Custom framework middleware

• Persistent files or specialized state requirements

• Webhook processing or nonstandard protocols

• Real-time media processing

• Parallel fan-out and fan-in

• Long-running work with human approvals between stages

Prompts are valuable for intent, behavior, tone, policy, and reasoning guidance. They are a fragile place to implement exact transaction boundaries, idempotency, exception handling, or a regulated decision process.

When behavior must be deterministic, testable at branch level, and reviewable as software, write it as software.

Hosted Agents Put Orchestration in Code

A Hosted Agent is a code-based agent application that runs inside Foundry Agent Service.

The team can use Microsoft Agent Framework, LangGraph, Semantic Kernel, OpenAI Agents SDK, another supported framework, or custom code. Current Foundry documentation describes packaging through a container image and also supports source-code deployment paths where the service builds the artifact.

Foundry supplies a dedicated endpoint and agent identity, manages lifecycle and scaling, supports session state, and integrates observability. The application code controls the orchestration logic.

This boundary is appropriate when the agent is truly an application rather than a declarative reasoning configuration.

Workloads That Need Hosted Agents

Microsoft recommends Hosted Agents when teams need capabilities such as:

• A specific agent framework or custom orchestration engine

• Deterministic business rules around model reasoning

• Branching, parallel work, fan-out, fan-in, or approvals

• Custom packages, SDKs, middleware, caching, or algorithms

• Application-owned retries and error handling

• Webhooks or custom request payloads

• Persistent files and custom session state

• Multi-agent coordination

• Real-time voice or other specialized protocols

Examples include financial onboarding, fraud investigation, lending workflows, and security operations.

These scenarios combine model judgment with exact rules, external systems, long-running state, escalation, and audit requirements. A prompt alone should not be responsible for maintaining that contract.

A Framework Does Not Automatically Require Hosting

One common architecture mistake is deciding on a Hosted Agent because the team already uses LangGraph, Semantic Kernel, or another framework.

The framework is an implementation preference, not the requirement.

First identify what the workload needs from the runtime. If the agent follows a simple reasoning-and-tool loop that Foundry already supports declaratively, rebuilding it as a Hosted Agent can add dependency management, deployment, security, cold-start, compute, and operational work without improving the outcome.

Choose the runtime boundary based on behavior, state, protocol, and control requirements. Then choose the framework that implements those requirements cleanly.

Hosted Does Not Mean Fully Outsourced

Foundry reduces platform plumbing for Hosted Agents, but the customer still owns the application.

The engineering team must:

• Secure and patch code and dependencies

• Maintain container or source-build configuration

• Test orchestration and failure handling

• Manage software supply-chain risk

• Select CPU and memory appropriately

• Understand cold starts and scale behavior

• Define state lifecycle and retention

• Control cost and concurrency

• Version and roll back agent releases

The platform can make deployment and operation more consistent. It cannot decide whether custom code is safe or whether a business workflow is correct.

Treat a Hosted Agent like any other production service that happens to use models and tools.

State Ownership Is Often the Deciding Factor

Many agent prototypes store state implicitly in conversation history. Enterprise workflows usually need a more precise model.

Ask:

• What state must survive a turn, session, deployment, or outage?

• Which system is authoritative?

• Can two workers update the same case?

• How are retries made idempotent?

• When does state expire?

• Can users inspect, correct, export, or delete it?

• What evidence belongs in an audit record?

If the workflow requires files, custom state stores, explicit checkpoints, compensating actions, or long-running business status, a code-based orchestration layer may be necessary.

Do not use model context as the only durable system of record for an important process.

Identity and Tool Authority Matter for Both Types

Prompt and Hosted Agents can both call tools and act on downstream systems.

The agent type does not reduce the need for least privilege. Assign a dedicated identity, scope it to the exact resources and actions required, and separate read-only investigation from write-capable remediation.

For Hosted Agents, review both the platform-created agent identity and any credentials used by custom code. For Prompt Agents, review every configured tool, connection, and authentication mode.

Require explicit approval for irreversible or high-impact operations. The approval must show the proposed action, target, supporting evidence, expected impact, and rollback path.

Test prompt injection, malicious tool output, cross-tenant access, credential expiry, and unauthorized action attempts for both architectures.

Observability Must Cover Decisions and Runtime

Prompt Agents make the runtime simpler, but teams still need tracing for model calls, tool use, latency, tokens, errors, and outcome quality.

Hosted Agents add application-level telemetry. Trace orchestration branches, state transitions, queue delays, dependency calls, retries, approvals, and container behavior alongside model and tool spans.

Use a correlation identifier across the entire workflow. An investigation should be able to answer:

• Which agent version handled the request?

• Which prompt and policy version applied?

• What tools were called and with which sanitized inputs?

• Which model deployments participated?

• Where was time spent?

• What state changed?

• Was a human approval required and granted?

• Did the final business outcome succeed?

Measure successful outcomes, not only response generation.

Evaluation Should Test the Chosen Boundary

Agent evaluation is not limited to answer quality.

For a Prompt Agent, evaluate instruction adherence, grounding, tool selection, output schema, safety, and behavior under ambiguous requests.

For a Hosted Agent, add branch coverage, state recovery, concurrency, idempotency, protocol compatibility, dependency failure, version rollback, and resource consumption.

Use the same business-level test cases when comparing both approaches. A Hosted Agent should justify its added operational surface through requirements or measurable outcomes. A Prompt Agent should prove that its simpler definition remains reliable when the workflow becomes messy.

A Practical Decision Sequence

Start by describing the business workflow without naming an agent type or framework.

List inputs, outputs, tools, systems, write actions, approval points, state transitions, latency targets, security boundary, and recovery requirements.

Then ask:

• Can the behavior be expressed as a straightforward reasoning-and-tool loop?

• Are all required integrations available as supported tools?

• Is conversation or platform-managed state sufficient?

• Can failures be handled without a custom state machine?

• Does the client use supported request and response protocols?

• Is custom code necessary for a verified requirement?

If the answers favor the managed path, begin with a Prompt Agent. Move to a Hosted Agent when a real runtime need appears, not when speculative flexibility sounds attractive.

Migration Between Agent Types

Starting declaratively does not prevent a later code-based architecture.

Keep prompts, tool contracts, evaluation cases, security policies, and business schemas outside portal-only experimentation. Version them in source control and define stable interfaces around downstream systems.

If the Prompt Agent reaches a genuine boundary, move orchestration into a Hosted Agent while preserving the evaluated instructions and tool contracts. Compare both versions on the same test set before switching traffic.

Avoid an unnecessary big-bang rebuild. The purpose of migration is to add required runtime control, not to discard evidence accumulated during the simpler implementation.

Who Should Care?

Cloud architects should care because the choice defines ownership of orchestration, state, compute, and operations.

AI platform teams should care because Prompt Agents and Hosted Agents require different delivery, security, observability, and cost controls.

Application developers should care because custom frameworks and packages move more responsibility into their codebase.

Security teams should care because both agent types can reach tools and downstream systems with real authority.

Product owners should care because starting with the smallest suitable runtime usually reduces time to evidence and makes limitations easier to discover.

Practical Cloud Engineer Takeaway

Choose one proposed agent workflow and write its runtime requirements on a single page.

Build the smallest Prompt Agent that can satisfy the core reasoning and tool-use path. Create evaluation cases for normal work, ambiguous input, denied access, tool failure, unsafe requests, and incorrect downstream data.

If the design requires branching, custom packages, protocols, persistent state, long-running execution, or deterministic recovery, implement one Hosted Agent proof of concept against the same cases.

Compare business outcome quality, operational effort, security surface, latency, failure recovery, and cost. Select the simpler option that meets the verified requirements.

Keep the decision reviewable. “We prefer this framework” is not the same as “the workload requires this runtime.”

Bottom Line

Microsoft Foundry's Prompt-versus-Hosted Agent choice is fundamentally a runtime ownership decision.

Prompt Agents are the right default when behavior fits a declarative model, instructions, and supported tools. Hosted Agents are justified when orchestration must live in code because the workload needs custom frameworks, dependencies, protocols, deterministic branching, persistent state, or specialized execution.

Foundry manages more of the infrastructure in both cases, but greater runtime control creates greater application responsibility.

Start with the smallest boundary that can be evaluated and governed. Add a Hosted Agent only when a concrete runtime requirement makes the extra code and operational surface worthwhile.

Sources

Microsoft Azure Architecture guidance, published August 19, 2026: https://techcommunity.microsoft.com/t5/azure-architecture-blog/choosing-the-right-agent-in-microsoft-foundry/ba-p/4547827

Choose how to build with Microsoft Foundry: https://learn.microsoft.com/en-us/azure/foundry/concepts/choose-build-approach

Hosted Agents in Foundry Agent Service: https://learn.microsoft.com/en-us/azure/foundry/agents/concepts/hosted-agents

Prompt Agent quickstart: https://learn.microsoft.com/en-us/azure/foundry/agents/quickstarts/prompt-agent?tabs=python

---

Stay radical, stay curious, and keep pushing the boundaries of what is possible in the cloud.

Chriz Beyond Cloud with Chriz