Tag Archives: open source

Databricks vs Snowflake: A Critical Comparison of Modern Data Platforms

This article provides a critical, side-by-side comparison of Databricks and Snowflake, drawing on real-world experience leading enterprise data platform teams. It covers their origins, architecture, programming language support, workload fit, operational complexity, governance, AI capabilities, and ecosystem maturity. The guide helps architects and data leaders understand the philosophical and technical trade-offs, whether prioritising AI-native flexibility and open-source alignment with Databricks or streamlined governance and SQL-first simplicity with Snowflake. Practical recommendations, strategic considerations, and guidance by team persona equip readers to choose or combine these platforms to align with their data strategy and talent strengths.

Continue reading

Revisiting the Home Office’s Big Data Initiative: A Success Story in Modernising Border Security

This article revisits the Home Office’s 2013 initiative to modernise border security using Big Data and Machine Learning. It highlights significant cost reductions, enhanced functionality through advanced data integration, and the strategic shift to open-source technologies, which improved security and flexibility. The piece also reflects on the challenges of avoiding vendor lock-in while strengthening UK border operations.

Continue reading

Transforming Border Security: The Home Office’s Big Data and Machine Learning Strategy

In September 2015, I had the opportunity to represent the Home Office and Border Force in a panel discussion at an event covered by Diginomica, where we delved into the transformative potential of Big Data and Machine Learning technologies. Alongside industry leaders from Jaguar Land Rover and Zurich Insurance, I discussed how the Home Office is actively leveraging Hadoop and other open-source tools to modernise its data management and decision-making processes, particularly within border security operations.

Continue reading

Exploring Real-World Applications of Hadoop and Big Data, with HortonWorks, Jaguar Land Rover, Zurich Insurance, and The Home Office

In a recent event, celebrating the opening of HortonWorks’ new London offices, three prominent organizations, Jaguar Land Rover, Zurich Insurance, and the UK’s Home Office, were invited to highlight their pragmatic use of Hadoop and big data technologies to extract value from unstructured data. I was pleased to be asked to represent the Home Office.

Continue reading