Tag Archives: Data Platform

Operational Edge Projection + CDC Write-Back: A Minimal Pattern for Low-Latency Authority Without Breaking Temporal Truth

This article introduces a practical operational extension to the UK FS SCD2 Bronze/Silver architecture. Rather than forcing operational applications to interact directly with the lakehouse, governed Silver current-state views are projected into a low-latency SQL database for operational consumption. Business decisions are made and stored locally, captured through SQL Change Data Capture (CDC), landed append-only into Bronze, and subsequently reconciled into governed Silver. The pattern preserves temporal correctness, auditability, replayability, and regulatory defensibility while providing the transactional performance and simplicity required by operational systems. It separates operational authority from institutional memory, allowing each layer to optimise for its intended purpose without compromising governance.

Continue reading

Complex Precedence & Out-of-Sequence Safety in Bronze-Layer SCD2 (Regulated FS)

This article defines how to implement SCD2 in the Bronze layer to safely handle multi-source precedence, out-of-sequence data, partial and full loads, deletions, and transaction patterns in regulated Financial Services. It introduces a metadata-driven approach that preserves temporal truth, prevents ingestion-order corruption, and enables deterministic is_current. The result is a defensible, replayable foundation that simplifies downstream Silver layers and supports point-in-time reconstruction under audit.

Continue reading

Measuring Value in a Modern FS Data Platform: Framework for Understanding, Quantifying, and Communicating Data Value in FS

Measuring Value in a Modern FS Data Platform reframes how Financial Services organisations should evaluate data platforms. Rather than measuring pipelines, volumes, or dashboards, true value emerges from consumption, velocity, optionality, semantic alignment, and control. By landing raw data, accelerating delivery through reuse, organising around business domains, and unifying meaning in a layered Bronze–Silver–Gold–Platinum architecture, modern platforms enable faster decisions, richer analytics, regulatory confidence, and long-term adaptability. This article provides a practical, consumption-driven framework for CDOs and CIOs to quantify and communicate real data value.

Continue reading