Tag Archives: Azure Data Factory

Operational Edge Projection + CDC Write-Back: A Minimal Pattern for Low-Latency Authority Without Breaking Temporal Truth

This article introduces a practical operational extension to the UK FS SCD2 Bronze/Silver architecture. Rather than forcing operational applications to interact directly with the lakehouse, governed Silver current-state views are projected into a low-latency SQL database for operational consumption. Business decisions are made and stored locally, captured through SQL Change Data Capture (CDC), landed append-only into Bronze, and subsequently reconciled into governed Silver. The pattern preserves temporal correctness, auditability, replayability, and regulatory defensibility while providing the transactional performance and simplicity required by operational systems. It separates operational authority from institutional memory, allowing each layer to optimise for its intended purpose without compromising governance.

Continue reading

Azure Data Factory: Why Can’t You Just Do the Simple Thing?

Azure Data Factory can route traffic through a corporate firewall with a fixed outbound IP… but only after you abandon the idea of “simple”. This article explores why a basic enterprise requirement turns into architectural theatre, and what that says about modern cloud platforms.

Continue reading