Stop Parsing Prose for Routing: Inside Microsoft Foundry’s New Decisions API

For the last few years, enterprise AI architecture has harbored an embarrassing secret: we’ve been using hundred-billion-parameter prose engines as glorified if/else statements.

Need to route a support ticket? Send a prompt to a chat endpoint, beg the model to "Respond ONLY with valid JSON and no markdown backticks," cross your fingers, deserialize the string, and pray a temperature spike doesn't inject a polite conversational preamble that crashes your parser. Even with Structured Outputs and constrained decoding, using a generative endpoint for classification or triage means paying generative latency and output-token taxes just to get a boolean or an enum.

With the October 2026 release of Microsoft Foundry, the Decisions API (/openai/v1/decisions) is officially Generally Available. Powered by gpt-6-luna, it decouples evaluating and routing from generating prose—running roughly 10x faster than the standard Responses API with zero output-token charges.

The Core Shift: Evaluation Without Generation

When you call POST /openai/v1/decisions, the model doesn't generate chat completions or stream tokens. Instead, you pass shared evidence (input—which can include text, up to 128 images, raw JSON, or full conversation and tool-call histories) alongside an array of up to 400 bounded questions in a single request.

Look closely at the usage block returned by a Decisions API call:

 {
  "model": "gpt-6-luna",
  "answers": [
    {
      "type": "predicate",
      "name": "refund_requested",
      "probability": 0.95
    }
  ],
  "usage": {
    "input_tokens": 162,
    "output_tokens": 0,
    "total_tokens": 162
  }
}

output_tokens: 0. You are billed strictly on input tokens (around $0.10 per 1M input tokens, with prompt caching discounts when reusing shared context across requests). Instead of sampling a verbose explanation, your application gets calibrated probabilities that deterministic code can threshold and act on immediately.

The Three Primitives: predicate, choice, and score

The Decisions API replaces fragile prompt-engineering templates with three strongly typed question primitives. Every question requires a unique name and explicit instructions, and returns either the typed result or a typed refusal if safety filters trip.

Why score Beats LLM Number Guessing

Ask a standard chat model to rate severity from 1 to 5, and it will overwhelmingly pick 3 or 4 due to token frequency bias in its training data.

The score primitive fixes this by requiring an ordered array of rubric levels (lowest to highest, each with a clear description). Under the hood, gpt-6-luna calculates the probability distribution across every rubric tier and computes the probability-weighted average of the level indices. Instead of an arbitrary integer, your router gets a continuous, nuanced float (e.g., 2.74) backed by the exact probability breakdown across the rubric.

Putting It to Work in C# (.NET)

If you're building on .NET 8/10+, support is baked into the OpenAI NuGet package (v2.15.0+) under the OpenAI.Decisions namespace. Because the typed DecisionsClient is newly minted, you opt in with #pragma warning disable OPENAI001.

Instead of chaining three separate LLM calls or stuffing a massive JSON schema into a prompt, you can evaluate a customer message (and an attached image) across multiple dimensions in a single round-trip:

 using System;
using System.ClientModel;
using OpenAI.Decisions;

#pragma warning disable OPENAI001

string endpoint = "https://YOUR-RESOURCE-NAME.openai.azure.com/openai/v1/";
string apiKey = Environment.GetEnvironmentVariable("AZURE_OPENAI_API_KEY") 
    ?? throw new InvalidOperationException("Missing AZURE_OPENAI_API_KEY.");

var client = new DecisionsClient(
    new ApiKeyCredential(apiKey),
    new DecisionsClientOptions { Endpoint = new Uri(endpoint) });

// 1. Shared Evidence: Evaluated once across all questions
var input = new DecisionsUserMessageItem(
[
    new DecisionsInputTextContentPart(
        "I was charged twice for our enterprise license renewal, and now our production tenant is locked out!")
]);

// 2. Define Typed Questions (Predicate + Choice)
var refundCheck = new DecisionsPredicateQuestion(
[
    new DecisionsInputTextInstructionsPart("Does the customer explicitly ask for a refund or billing reversal?")
]) { Name = "needs_billing_reversal" };

var departmentRoute = new DecisionsChoiceQuestion(
    instructions: [new DecisionsInputTextInstructionsPart("Which team should own primary triage?")],
    choices: 
    [
        new DecisionsChoiceOption("billing", "Payment disputes, double charges, and invoices."),
        new DecisionsChoiceOption("tenant_ops", "Production outages, locked tenants, and access emergencies."),
        new DecisionsChoiceOption("general_support", "Standard how-to questions and feature requests.")
    ]) { Name = "triage_owner" };

// 3. Execute in a single zero-generation pass
DecisionResource decision = await client.CreateDecisionAsync(
    new DecisionCreationOptions([input], "gpt-6-luna", [refundCheck, departmentRoute]));

// 4. Handle deterministic, strongly-typed answers
foreach (var item in decision.Answers)
{
    switch (item)
    {
        case DecisionsPredicateAnswer predicate when predicate.Name == "needs_billing_reversal":
            Console.WriteLine($"[Predicate] Billing Reversal Probability: {predicate.Probability:P1}");
            if (predicate.Probability >= 0.85)
            {
                // Trigger automated ledger lookup before a human even sees the ticket
            }
            break;

        case DecisionsChoiceAnswer choice when choice.Name == "triage_owner":
            Console.WriteLine($"[Choice] Route to: {choice.Choice.StringValue} (Confidence: {choice.Confidence:P1})");
            break;

        case DecisionsRefusalAnswer refusal:
            Console.WriteLine($"[Refusal] Model declined question: {refusal.Name}");
            break;
    }
}

Architectural Patterns: Where Decisions Belongs

The Decisions API isn't a replacement for the Responses API or Function Calling—it’s the control plane that sits in front of them.

1. Pre-Execution Guardrails for Autonomous Agents: Before allowing a hosted agent or a Playwright Browser Automation session to execute a state-changing tool call, pass the conversation history and proposed tool call to /decisions. Use a predicate question: "Is this action reversible, low-impact, and strictly within the user's stated request?" If probability is less than 0.80, pause the agent loop and trigger a human-in-the-loop approval.
2. High-Speed Multimodal Triage: Because a single request accepts up to 128 images (up to 32 MB total) and allows per-question reference images in the instructions, you can evaluate visual claims (e.g., "Does product.png show physical chassis damage, ignoring packaging?") alongside JSON metadata in ~170ms.
3. One-Pass Rubric Grading for RAG & Evals: Instead of running slow "LLM-as-a-judge" chat prompts that write three paragraphs of critique before spitting out a score, batch 50 retrieved chunks or agent traces into a single Decisions request and score them against a strict 4-level relevance rubric in milliseconds.

Three Gotchas Before You Deploy

Before you rip out your existing routers this afternoon, keep three sharp edges in mind:

• The East US 2 Blind Spot: While the Decisions API is GA across Azure regions where gpt-6-luna is hosted, East US 2 is excluded from the initial rollout. Check your regional quota before hardcoding endpoints.
• Model Lock-In (For Now): gpt-6-luna is currently the only supported model on /openai/v1/decisions. Remember that Azure requires your deployment name in the model parameter, not the raw model string, unless they match.
• No Retries on Blind Timeouts: Because /decisions is a non-streaming endpoint billed on input evaluation, treat network timeouts carefully—and never wire an unthresholded choice output directly to a destructive execution path without checking both confidence and DecisionsRefusalAnswer.

Generative models are fantastic at writing, summarizing, and synthesizing. Stop forcing them to talk when all you need them to do is decide.

Sources:

• What’s new in Microsoft Foundry – October 2026
• Microsoft Learn: Azure OpenAI Decisions API how-to
• NuGet: OpenAI 2.15.0
• Unite.AI: OpenAI Makes Decisions API Generally Available
• OrcaRouter: OpenAI’s Decisions API