Apache Kafka outbox pattern
In the world of Apache Kafka, a seemingly innocent pattern is quietly corrupting data integrity across countless microservice architectures. It’s called “dual writes,” and it…
Read MoreApache Kafka outbox pattern
Why Google’s Agent2Agent Protocol Needs Apache Kafka®
For agents to operate at enterprise scale, they need a communication backbone that’s resilient, decoupled, and built for many-to-many collaboration. That’s where Apache Kafka comes…
Guide to ZooKeeper to KRaft migration
With Apache Kafka 4.0’s release, the long-awaited future without ZooKeeper has arrived. This version marks a historic milestone – the first release to completely remove…
Building Streaming Data Pipelines, Part 1: Data Exploration With Tableflow
This blog post demonstrates how to use Tableflow to easily transform Kafka topics into queryable Iceberg tables using UK Environment Agency sensor data as a…
Confluent Champion: Devising Real-World Solutions for Customers
Learn how Confluent champion Tejal Bhatt helps customers on their data streaming journey as part of the Dedicated Solutions Engineering team. This post first appeared…
From Reactive to Orchestrated: Building Real-Time Multi-Agent AI With Confluent
Learn how Agent Taskflow uses the Confluent data streaming platform as the foundation of its event-driven multi-agent system. This post first appeared on Read More
3 Strategies for Achieving Data Efficiency in Modern Organizations
The efficient management of exponentially growing data is achieved with a multipronged approach based around left-shifted (early-in-the-pipeline) governance and stream processing. This post first appeared…
Guide to Consumer Offsets: Manual Control, Challenges, and the Innovations of KIP-1094
Learn how to achieve data consistency and reliability with a complete Apache Kafka consumer offsets guide covering key principles, offset management, and KIP-1094 innovations. This…
Agencies Win With Data Streaming: Evolving Data Integration to Enable AI
Shift-left, streams-first integrations unlock data modernization in government agencies. Learn how data streaming enables public sector innovation with Public Sector Summit recaps. This post first…
Why Is My Apache Flink® Job Not Producing Results?
Learn how watermarks affect Flink jobs and common pitfalls, including idle streams/partitions, watermark strategy issues, etc., which can cause a job to not produce any…
Beyond Boundaries: Leveraging Confluent for Secure Inter-Organizational Data Sharing
Discover how Confluent enables secure inter-organizational data sharing to maximize data value, strengthen partnerships, and meet regulatory requirements in real time. This post first appeared…
Unlocking Data Insights with Confluent Tableflow: Querying Apache Iceberg™️ Tables with Jupyter Notebooks
Learn how to integrate Confluent Tableflow with Trino to query Apache Iceberg tables, and unlock real-time and historical analytics with Jupyter Notebooks This post first…
Chopped: AI Edition – Building a Meal Planner
Learn how to build an event-driven architecture that coordinates a team of four family-meal-planning agents, and leverages LangChain, Anthropic’s Claude, Kafka, and Flink. This post…
The AI Silo Problem: How Data Streaming Can Unify Enterprise AI Agents
To solve the AI silo problem, enterprises need a shared communication layer for AI agents—a real-time, event-driven approach that lets agents share intelligence and take…
Shifting Left: How Data Contracts Underpin People, Processes, and Technology
Application engineers situated at the beginning of a data pipeline (“left side”) should apply data contracts and products as rigorously as the data engineers further…