Browse All Machine Learning Content (188)

Michaela Isaacs introduces pq-adbc-advisor, a self-serve notebook-based scanner that inventories Microsoft Fabric workspace items still using legacy ODBC drivers and helps teams plan the ODBC-to-ADBC cutover. It produces a per-item impact report, classifies cutover risk, and recommends concrete fixes to validate ahead of connector-specific deadlines.

Mariyaali announces the general availability of the SQL DW operations skill for Microsoft Fabric Data Warehouse, which lets teams use natural-language prompts (for example via GitHub Copilot CLI) to run bounded, read-only diagnostics and get structured findings, evidence, and recommended next steps for common warehouse incidents.

This week's ML roundup focuses on making AI systems more reliable by tightening the connection between governed data and agent grounding, from Fabric's lineage-first item relations API to table-level discovery in OneLake Catalog search. We also saw practical Spark work that pushes latency down and correctness checks earlier, including Structured Streaming real-time mode for fast quarantine and native JSON parsing in Fabric's Native Execution Engine. Finally, three production case studies (supply chain planning, closed-loop lab discovery, and feedback-to-work-item automation) show what agentic systems look like when they ship with traceability, safety controls, and human review.

Wiliam_Rosa breaks down how Spark Structured Streaming’s Real-time mode (as implemented in Azure Databricks) can cut anomaly-detection latency from seconds to sub-millisecond, using an Ethereum transaction pipeline that classifies events as ALLOW or QUARANTINE based on protocol invariants and sensitive-payload checks.

KimVaddi explains why many oil & gas AI initiatives stall on inconsistent business meaning rather than data volume, and argues that the “meaning layer” is the real readiness bottleneck. The post connects OSDU, Azure Data Manager for Energy, and Microsoft Fabric IQ’s preview ontology capability as a practical path to governed, cross-domain reasoning.

yaronc announces a preview Fabric REST API that exposes item lineage as a dependency graph you can query from code, enabling automated impact analysis, documentation/catalog scenarios, CI/CD checks, and richer context for AI agents that need to reason about upstream sources and downstream consumers.

Santhosh_Ravin1 announces a preview update to Microsoft Fabric’s Spark Native Execution Engine that adds native JSON reading and parsing. The post explains why JSON parsing often becomes a pipeline bottleneck in lakehouse workloads, and how keeping JSON in a columnar native path can improve end-to-end Spark job performance.

Microsoft Developer covers automatic index compaction (public preview) and how it can reduce or eliminate traditional index maintenance work for Azure SQL Database, Azure SQL Managed Instance, and SQL database in Microsoft Fabric.

Juan Montes profiles Kinaxis’ Maestro platform and how it uses Azure (including AKS, Databricks, Azure OpenAI, Cosmos DB, and Azure AI Content Safety) to run large-scale supply chain scenario modeling, forecasting, and agentic AI experiences. The piece also touches on Kinaxis’ internal use of GitHub Copilot for an agentic, PR-driven development workflow.

Marco Casalaina breaks down Foundry IQ, Fabric IQ, Work IQ, and Web IQ, explaining them as grounding capabilities that give AI agents the right context. He maps each “IQ” to the underlying data source (knowledge bases, OneLake, Microsoft 365 apps, and the public web) and what it enables.

nschachter announces a Microsoft Fabric preview that adds table-level discovery to OneLake Catalog search, including Global Search and the OneLake Catalog Search API. The post explains how permission trimming works, what object-level security excludes, and which tenant setting admins should review before the late-September rollout.

bharadwajr explains how the Azure team built GEM (Great Experiences Matter), an AI-enabled “feedback-to-fix” loop that aggregates customer feedback at scale, turns it into prioritized insights, and creates actionable Azure DevOps work items with an auditable evidence trail and human review.

salilkanade1 announces updates to the Microsoft Fabric SQL Query Editor, focused on scaling web-based SQL development for large warehouses with a faster results grid, improved object exploration and IntelliSense, better query management, and tighter connections to OneLake analytics, semantic models, and operational workflows.

newmancheng explains how Microsoft Discovery combined agentic AI, computational predictors, and wet-lab feedback to design and validate a novel organic negolyte for redox flow batteries, with a focus on preserving negative results as durable knowledge that improves subsequent design rounds.

This week in Machine Learning, the focus shifts from prototypes to production guardrails, with Fabric data agents now generally available in Copilot Studio and new preview work on advanced DAX generation for semantic models. Fabric governance tightened with workspace outbound access controls, tenant-wide networking policy auditing, and connection recency signals that help reduce stale and single-owner risk. On data movement and interoperability, teams get clearer paths for private connectivity (Snowflake, Eventstream) and a GA transition from ODBC to ADBC that is worth validating early, while platform operations mature through capacity telemetry in Real-Time Hub and new CI/CD guidance to standardize deployments.

Topi Nuotos explains why his team replaced a shared Excel tracker with a Microsoft Fabric App, and how Fabric Apps adds an application layer on top of Fabric for building data-driven web apps with preconfigured infrastructure.

Authorised Territory demonstrates how to build an AI agent with the .NET Microsoft Agent Framework using a local Ollama model, then evaluate the agent locally with built-in checks to confirm response quality and that required tools were actually invoked.

Scirqueira shares practical operating rules for keeping a Microsoft Fabric Data Warehouse medallion architecture healthy in production, focusing on batch ingestion, rerunnable Silver transformations, consumption-ready Gold modeling, using Fabric DW defaults effectively, and monitoring performance and failures by layer.

Andre Terceros and Bodhisatva Gautam announce a preview of Workspace Outbound Access Protection (OAP) in Microsoft Fabric for Operations Agent and Fabric Maps, explaining how workspace admins can enforce a default-deny model for outbound connections, allow only approved endpoints, and get clearer visibility into blocked actions.

ryanjadams explains how Fabric’s “connection recency” signals help admins govern connections at enterprise scale, and how to use Fabric REST APIs plus a companion Python notebook to find stale, duplicate, and single-owner connections before taking cleanup actions.

Jeremy Winter argues that getting enterprise AI into production is less about picking a single model and more about running an end-to-end system on Azure—spanning infrastructure, data, governance, security, and operations. The post highlights Microsoft AI Foundry, Fabric, Purview, and Azure databases, plus modernization workflows that include GitHub Copilot.

pranabpaul walks through the first part of building a retail assistant demo using Microsoft Fabric and Azure AI Foundry, focusing on setting up the Azure environment, provisioning Fabric capacity, loading sample data into a Lakehouse, creating an ontology, and wiring a Fabric Data Agent to query that data.

ryanjadams introduces Microsoft Fabric Connection Recency (preview) and the related REST APIs for auditing and governing connections at enterprise scale, including how to identify stale and duplicate connections and reduce ownership risk using role assignments and Entra group ownership.

pankajarora announces a preview of Capacity Operation Events in Microsoft Fabric Real-Time Hub, adding near real-time, operation-level telemetry to help capacity admins pinpoint which operations, workspaces, and artifacts drive consumption, throttling, and performance issues, and to build custom monitoring and alerting workflows.

ryanjadams explains how to keep Fabric Data Factory pipelines and Copy Jobs fully private when moving data to or from Snowflake, using Workspace-level Private Link, a private Azure Storage staging account, and Snowflake’s Azure Private Link storage integration.

AmirJafari announces general availability of the Microsoft Fabric data agent integration in Copilot Studio, enabling agents to call governed Fabric data as a tool (Fabric IQ Data MCP) and return permission-respecting, data-grounded answers. The post highlights the new tool-based authoring experience, improved orchestration, and expanded publishing targets.

wolfgangdesalvador breaks down Azure’s MLPerf Storage v3.0 results for Azure Managed Lustre, focusing on how sustained throughput and fast checkpoint I/O help keep large AI training clusters from stalling.

Michaela Isaacs explains the GA transition in Power BI and Microsoft Fabric from embedded ODBC drivers to Apache Arrow Database Connectivity (ADBC) drivers, why it improves large-result performance and security posture, and how to validate the change ahead of the tenant-wide cutover.

bgautam announces general availability of the Microsoft Fabric Admin API for listing workspace networking communication policies tenant-wide, aimed at improving security auditing, compliance monitoring, and governance reporting across Fabric workspaces.

Nimrod Shalit introduces a new set of Microsoft Learn resources for implementing CI/CD in Microsoft Fabric, covering platform concepts, best practices, and hands-on tutorials for deploying Fabric items across dev/test/prod using Git integration, Azure DevOps pipelines, Terraform, the Fabric CLI, and the fabric-cicd library.

Virginia_R announces a Preview feature in Fabric data agents that upgrades natural-language-to-DAX translation for Power BI semantic models using an agentic, iterative approach. The post explains how planning, tool use, intermediate-result inspection, and instance value indexing improve filter accuracy, consistency across Microsoft experiences, and overall query reliability.

Rafia Aqil explains how to diagnose and resolve Azure Databricks “capacity” incidents by separating true regional VM SKU shortages from subscription quota limits and Databricks platform ceilings, then applying practical mitigations like SKU swaps, off-peak retries, instance pools, and capacity reservation groups.

meghasony shares guidance on structuring dbt jobs in Microsoft Fabric, focusing on how to place Bronze/Silver/Gold layers across Lakehouse and Warehouse, and how to separate architecture decisions from orchestration choices using Fabric pipelines or native scheduling.

bytecascade explains how Microsoft Fabric Eventstream can securely ingest private operational data (like factory MQTT telemetry) for real-time analytics by using an Azure virtual network bridge and a Fabric streaming virtual network data gateway, without exposing public endpoints.

FabSubhojit announces the general availability of Fabric Migration Assistant for SQL database, a guided workflow for assessing schema compatibility, provisioning SQL database in Fabric, deploying schema with fix suggestions, and copying data. The update adds a Validate step that analyzes a DACPAC before any Fabric resources are created.

Naorbt announces a preview feature that lets you embed a Fabric Map as a reusable tile inside Microsoft Fabric Real-Time Dashboards, so teams can add geospatial context to operational monitoring without duplicating map queries, layers, filters, or styling across dashboards.

robece-msft continues the Microsoft Fabric Business Events series by outlining five real-world business event scenarios and the shared event-driven pattern used to turn business moments into trusted signals and downstream actions across Fabric.

This week in ML and data engineering, Microsoft Fabric pushed further on standardization and operability with Runtime 2.0 (Spark 4.1 and Delta Lake 4.2), a GPU query acceleration preview, and more policy-driven OneLake governance, including a preview mirror of Google Lakehouse Runtime Catalog metadata for Iceberg tables. On the workflow side, Fabric Warehouse CI/CD moved closer to familiar database project patterns with DacFx and a VS Code Schema Compare path, while agent updates (including GPT-5.1 references and MCP-based connections) continued the shift from assisted authoring to tool-connected automation that still needs validation and logging. Rounding out the week, new guidance focused on practical cost loops in Azure Databricks and on building AI-ready apps with vector search and RAG in Azure SQL Database, plus a broader view of how Work IQ, Foundry IQ, and Fabric IQ frame RAG as the grounding layer for enterprise agents.

goupadhy announces the August 2026 on-premises data gateway release (version 3000.330), highlighting Power BI Desktop compatibility plus updates across security, authentication, diagnostics, and reliability, along with guidance to upgrade to stay current with dependency and CVE-related fixes.

robece-msft continues the Microsoft Fabric Business Events series by showing how to react to business events using Fabric Activator and Eventhouse, focusing on practical event-driven patterns for turning event streams into signals and actions across Fabric.