Browse All Machine Learning Content (230)

Matt Basile introduces OneLake storage tiers and lifecycle management in Microsoft Fabric (preview), explaining how admins can reduce long-term storage costs by automatically moving files between hot, cool, and cold tiers while accounting for the higher transaction and retrieval costs of colder tiers.

Ambika Jagadish announces a preview feature for Microsoft Fabric Warehouse that lets teams configure how many days of historical data versions are retained, enabling time travel queries, point-in-time clones, restore points, and snapshot history with a single T-SQL command.

FaizaanMerchant explains a Zero Trust network design for Azure Databricks that avoids public workspace exposure by fronting external access with Azure Application Gateway WAF and routing traffic to the workspace through Private Endpoints, while keeping internal access on private connectivity (VPN/ExpressRoute).

Mariya Ali and Twinkle Cyril introduce Data Warehouse Monitor (Preview) in Microsoft Fabric Data Warehouse, a unified UI for viewing running and completed queries together, analyzing performance across executions, spotting regressions and recurring slow queries, and canceling problematic queries directly from the monitoring experience.

stclarke describes a Microsoft Research open dataset and pipeline that builds geographically grounded, electrically coherent U.S. transmission-grid models from public data, enabling AC optimal power flow studies at scales from small state networks to the full Eastern Interconnection.

Shireesh Thota summarizes the main architecture trends from Cosmos DB Conf 2026, focusing on how teams are building AI-native apps on Azure Cosmos DB with flexible data models, serverless scale, and first-class semantic/vector search, plus practical patterns for agent memory, cost visibility, and multi-user security.

robece announces General Availability of Stripe as a partner event source for Azure Event Grid, and outlines how to route Stripe events into Azure services (Functions, Logic Apps, Event Hubs, Service Bus) and Microsoft Fabric Eventstream for real-time processing and analytics.

John Savill explains why enterprises need a data virtualization layer and how to build one using Microsoft Fabric OneLake, including a single namespace approach, shortcuts, mirroring, governance, and semantic models to make data easier to use for analytics and AI.

John Savill's Technical Training gives a quick overview of why enterprises often need a data virtualization layer, and how it helps provide a unified way to access data across different systems.

This roundup tracks a clear shift from agent capability to agent governance: more context, more observability, and more policy controls across Copilot, VS Code, and the CLI. On the platform side, Microsoft tightened the path from prototype to production with .NET agent building blocks, Azure AI Foundry deployment patterns, and data governance improvements that make RAG and operations easier to standardize. We also cover the less flashy work that keeps systems dependable at scale, including Fabric and Databricks operational updates, GitHub migration and ruleset changes, and security research that keeps token theft, privilege escalation, and supply chain risk in focus.

mscagliola shows how to use GitHub Copilot skills for spec-driven development, turning a Medallion Architecture blog post into a repeatable repo that generates Terraform for Azure platform setup and Databricks bundle files for workloads, while enforcing strict placeholder/TODO rules to avoid invented environment values.

Microsoft Developer recaps key themes from Azure Cosmos DB Conf 2026, focusing on what engineers are doing in production to scale reliably, keep costs under control, and support AI-driven workloads—especially vector search and modern search patterns built into Cosmos DB.

John Savill breaks down practical ways to change an AI model’s behavior, from prompt and context techniques through to retrieval-augmented generation (RAG) and fine-tuning approaches like LoRA.

Juan Montes reports on how Porsche Cup Brasil built an AI-assisted crash analysis and telemetry workflow on Microsoft platforms, cutting damage assessment time and improving race operations with human-in-the-loop validation.

David Ortinau and Gerald Versluis are joined by Nick Kovalsky to demo .NET MAUI work that combines Rust, SkiaSharp, and a drawn-UI approach, plus AI/ML live processing techniques he’s been building.

Aaron Merrill announces the general availability of OneLake security in Microsoft Fabric, outlining the rollout plan, UI improvements for role management, safer authoring with inline RLS validation, and new APIs for managing roles. The post also recaps key fixes and capabilities like mirroring security and ReadWrite permissions.

Gaurav Mittal explains how to validate GitHub Copilot Coding Agent runs in CI when agent behavior is non-deterministic, by building an independent “trust layer” that checks essential outcomes instead of brittle step-by-step scripts.

Avinanda Chattapadday introduces High Concurrency (HC) sessions for the Microsoft Fabric Livy API, enabling multiple Spark workloads to run in parallel while reusing sessions, isolating execution, and improving monitoring and cost efficiency for automated pipelines.

Michael Bruhjell introduces a preview feature in Microsoft Fabric’s monitoring hub that centralizes failure notification management for scheduled items, making it easier to configure, edit, and remove email recipients across many Pipelines, Notebooks, Dataflows Gen2, and other workspace items.

Miquella de Boer announces general availability of native OneLake catalog browsing inside Azure AI Foundry, so teams can discover governed Fabric data and turn it into knowledge sources without switching tools. The post includes prerequisites and a walkthrough for creating a Foundry knowledge base backed by OneLake and Azure AI Search.

pauledwards explains how to cut “model weight pre-flight” time on multi-node Azure GPU clusters by sharding downloads from Azure storage and broadcasting the remaining data over InfiniBand using MPI, with practical launch patterns for both Slurm and AKS.

KonstantinaF outlines a practical, phased disaster recovery strategy for Azure Databricks, focused on cross-region resilience for lakehouse workloads. The post explains RTO/RPO trade-offs, compares active-active vs warm standby patterns, and details how to replicate Unity Catalog metadata and Delta data using IaC, CI/CD, and repeatable DR pipelines.

Amit Damle and RK Iyer describe a “Discovery” utility for Azure Databricks that inventories workspace assets into Unity Catalog-backed Delta tables and a Lakeview dashboard, helping platform teams quickly understand clusters, jobs, warehouses, pipelines, security settings, and DBU usage.

Pankaj Arora announces general availability updates to Microsoft Fabric’s capacity metrics and chargeback apps, adding a new health page plus timepoint summary/detail views to pinpoint compute usage and throttling. The post also outlines how chargeback reporting helps allocate Fabric costs across workspaces, SKUs, and workload types.

Nadav Schachter introduces the OneLake Catalog Search REST API (preview) and related tools that let you discover Microsoft Fabric items across all accessible workspaces from code, AI agent workflows (via the Fabric Core MCP Server), and the terminal using the Fabric CLI.

Twinkle Cyril announces a preview feature in Microsoft Fabric Data Warehouse that adds T-SQL ALTER TABLE ... ALTER COLUMN support for metadata-only schema changes, reducing the need for table rebuilds and helping teams evolve warehouse schemas with less disruption to pipelines and downstream dependencies.

Miguel Escobar introduces the preview “My queries” feature in Microsoft Fabric Dataflow Gen2, which lets you save Power Query (M) transformations into a personal library and reuse them across dataflows to reduce duplication and keep data prep logic consistent.

Justin Barry explains how Direct Lake on SQL works with Microsoft Fabric Data Warehouse, including how semantic models page Delta Parquet data into memory, what drives performance (especially cardinality and model design), and how to avoid or diagnose fallback to DirectQuery.

This week’s roundup is about turning agentic tooling into something teams can run, budget, and govern. GitHub Copilot’s shift to token-based billing and AI Credits makes cost a first-class part of rollout checklists, especially as agent-style IDE and PR workflows expand and code review begins consuming both AI Credits and GitHub Actions minutes. On the platform side, GPT-5.5 in Microsoft Foundry, Microsoft Agent Framework 1.0, and A2A/MCP interoperability point toward more standardized agent runtimes, while Azure and Fabric updates reinforce the same operational theme: tighter identity, clearer observability, and more precise controls in both connected and constrained environments.

sameeraman explains how Microsoft Discovery can automate a scientific simulation workflow using a coordinated set of AI agents, reducing manual scripting and job monitoring while keeping scientific decision-making with researchers.

Marisa Mathews rounds up April 2026 community picks from Microsoft MVPs and Fabric Super Users, linking to deep dives on Fabric Warehousing, Power BI/DAX performance, SQL Server 2025 mirroring, ingestion/orchestration choices, metadata-driven integration frameworks, governance features, and real-time dashboards built with Eventstream, Eventhouse, and KQL.

David Levy joins Data Exposed to recap a burst of recent Microsoft SQL driver and SDK releases, including mssql-python updates with Bulk Copy and Apache Arrow support, plus new versions of SqlClient and JDBC.

Microsoft Fabric Blog announces a preview of the OneLake Catalog Search REST API plus MCP and Fabric CLI support, letting developers search Fabric items across workspaces by metadata and use the returned identifiers in scripts, internal tools, and agent-driven workflows.

Microsoft Fabric Blog explains how Fabric SQL developers can move from Azure Data Studio to VS Code, keeping SQL Database Projects and adopting a Git-based workflow with pull requests, schema compare, publish script previews, and optional GitHub Copilot assistance in the MSSQL extension.

Connie Xu explains how Microsoft Fabric pipelines are shifting from classic ETL into end-to-end workflow orchestration, including long-running processes and human approval steps. The post introduces the new Approval activity (preview) in Fabric Data Factory and outlines why monitoring, governance, and usability matter as pipelines expand beyond data movement.

Katie Murray summarizes the April 2026 Microsoft Fabric updates across platform UX, data engineering, data science/ML, warehousing, and real-time intelligence, including VS Code workspace management, notebook retry policies, MLflow cross-workspace logging, Eventstreams observability, and new security options like Key Vault-backed custom CA and mTLS.

Anasheh Boisvert introduces Microsoft Fabric workspace monitoring (preview) for Eventstreams, which creates a managed monitoring Eventhouse and tables so teams can query streaming health, throughput, and errors with KQL and build dashboards and alerts with minimal setup.

PrabalDeb lays out a practical reference architecture for running diffusion model workloads on Azure Kubernetes Service (AKS), focusing on GPU/CPU lane separation, dispatch and autoscaling options (Kubernetes-native vs Service Bus + KEDA), secure ingress and identity, durable storage for outputs and model caches, and end-to-end observability for both apps and GPU hardware.

Vaibhav Shirvastava announces the general availability of the SQL operator in Microsoft Fabric Eventstreams, focusing on code-first stream transformations, routing to multiple destinations, built-in testing, and event-time processing features for handling late and out-of-order data in real-time pipelines.

Matthew Hicks announces a preview feature in Microsoft Fabric that mirrors Dremio-managed Apache Iceberg tables into OneLake using a zero-copy approach, so teams can query and use the data across Fabric workloads without duplicating data or building ETL pipelines.