The Microsoft Fabric Blog team introduces new Copy Job features—resettable incremental copy, auto table creation, and JSON support—to improve data ingestion workflows for technical users.

Enhancements to Microsoft Fabric Copy Job: Reset Incremental Copy, Auto Table Creation, and JSON Support

Introduction

The Microsoft Fabric Blog has announced a set of powerful new features for Copy Job in Fabric Data Factory. These enhancements are designed to streamline data movement and ingestion across a variety of platforms—cloud, on-premises, and hybrid environments.

What’s New in Copy Job

1. Reset Incremental Copy

  • Greater Flexibility: Incremental copy allows for efficient data synchronization by transferring only new or updated records after the initial full load.
  • On-Demand Full Refresh: Users can now easily reset the incremental state for any table, triggering a full data copy on the next run. This is particularly useful for resolving data discrepancies between source and destination without affecting other tables.
  • Per-Table Control: Reset operations can occur on individual tables, enabling targeted troubleshooting and minimizing disruption.

2. Auto Table Creation on Destination

  • Frictionless Data Movement: If a destination table does not exist, Copy Job will automatically generate the table schema and create it for you—no manual pre-configuration required.
  • Supported Destinations:
    • SQL Server
    • Azure SQL Database
    • Fabric Lakehouse Table
    • Snowflake
    • Azure SQL Managed Instance
    • Fabric SQL Database

3. JSON Format Support

  • Expanded File Format Compatibility: In addition to existing support for CSV and Parquet, Copy Job now enables high-throughput movement of JSON files to and from table destinations, giving data engineers more flexibility with modern data formats.

4. Quick Access to Connection Details

  • Improved User Experience: Hover over any data connection in the Copy Job panel to quickly view service and workspace details. This streamlines configuration and helps troubleshoot connection issues.

How These Changes Help

These improvements are aimed at developers and data professionals seeking to minimize manual intervention, automate common tasks, and handle diverse data sources and targets efficiently. They address challenges in resetting pipelines, automating schema creation, and working with structured or semi-structured data formats.

Additional Resources

Conclusion

The latest update to Microsoft Fabric’s Copy Job can lead to greater productivity and ease of use for data engineers and technical teams. Features like incremental copy reset, automatic table creation, expanded JSON support, and quicker connection diagnostics make managing large-scale data pipelines more seamless.

This post appeared first on “Microsoft Fabric Blog”. Read the entire article here