Microsoft Developer (featuring Bethany Jepchumba) walks through what to do after fine-tuning in Microsoft Foundry: evaluate a customized model with a custom grader, manage long-term inference costs, and deploy the model for production use.

Model Optimization in Microsoft Foundry: Deployment and Evaluations

This video covers the post-training phase for customized models in Microsoft Foundry / Azure AI Foundry, focusing on how to evaluate and deploy fine-tuned models while keeping inference costs under control.

What this installment covers

After training is complete, the video focuses on three practical areas:

• Post-training evaluation
  • How to evaluate a fine-tuned model after training
  • A demo of custom grader logic to validate model quality
• Cost management
  • How to think about and manage long-term inference costs
• Deployment
  • How to deploy fine-tuned models effectively
  • Emphasis on production-grade reliability

Timeline

• 00:03 — Welcome and scenario
• 00:52 — Post-training evaluation
• 01:20 — Demo: Using a custom grader for evaluations
• 03:35 — Cost management
• 05:50 — Model deployment

Links from the description

• Microsoft Foundry: https://aka.ms/foundry-ft
• Foundry fine-tuning demos on GitHub: https://aka.ms/ft-demos
• Azure OpenAI fine-tuning costs: https://aka.ms/aoai-ft-cost

People

• Bethany Jepchumba (Twitter/X): https://twitter.com/bethanyjep
• Bethany Jepchumba (LinkedIn): https://www.linkedin.com/in/bethany-jep/
• Bethany Jepchumba (GitHub): https://github.com/bethanyjep