Content by Rafia Aqil (4)

Rafia Aqil outlines how to enable Azure Databricks’ Compliance Security Profile (CSP) for HIPAA workloads, including the September 1, 2026 deadline, required prerequisites like Azure VNet encryption and supported VM series, and a rollout approach to validate cluster startup and end-to-end connectivity before production.

Rafia Aqil explains Microsoft’s IQ Platform (Work IQ, Fabric IQ, and Foundry IQ) and how it adds business and organizational context to AI systems. The post breaks down Fabric’s OneLake-based data layer, ontology-driven meaning, and Foundry IQ’s managed services for RAG, memory, ranking, and citations.

Rafia Aqil explains how to diagnose and respond when Azure Databricks clusters can’t start or scale due to Azure regional VM capacity constraints, including what to send to Microsoft support, which VM families to switch to, and longer-term design choices like instance pools, serverless compute, and multi-region deployments.

Rafia_Aqil outlines a reference architecture for ingesting both streaming and batch data through Microsoft Fabric into Azure Databricks, using OneLake/ADLS and a medallion (Bronze/Silver/Gold) layout. The post breaks down five Fabric-to-Databricks integration paths and calls out security, governance, and monitoring considerations.