AI Agent Quality Control Loop with AI Judge | .NET 10, Microsoft Agent Framework & Ollama

Authorised Territory demonstrates how to build a .NET agent that generates an answer, has a second model judge it against criteria, and retries automatically until it passes or hits an iteration limit, using Microsoft Agent Framework with local models running in Ollama.

Overview

The video walks through implementing a quality-control loop for an AI agent using two different LLM roles:

• Worker model: generates the initial answer to the user’s request.
• Judge model: evaluates the worker’s answer against the original request and a set of criteria.

Architecture: worker, judge, and orchestrator

Worker model (answer generation)

• The worker is responsible for producing an answer to the user’s question.
• In the example setup, the worker model is Granite 3.3.

Judge model (answer evaluation)

• The judge compares:
  • the original request, and
  • the worker’s answer
• It decides whether the answer meets the required criteria.
• In the example setup, the judge model is Granite 4.1.

LoopAgent (control loop)

• A LoopAgent sits above both models and orchestrates the process.
• Control flow described in the video:
  • If the judge says the answer is good, return the answer.
  • If the judge says the answer is not good enough, retry generation.
  • Retry up to a maximum of three iterations.

Local model runtime with Ollama

• The tutorial uses an LLM running locally via Ollama.
• The approach relies on two models being available (one for the worker role and one for the judge role).