Streaming Audio: Apache Kafka® & Real-Time Data
Streaming Audio features all things Apache Kafka®, Confluent, real-time data, and the cloud. We cover frequently asked questions, best practices, and use cases from the Kafka community—from Kafka connectors and distributed systems, to data mesh, data integration, modern data architectures, and data mesh built with Confluent and cloud Kafka as a service. Join our hosts as they stream through a series of interviews, stories, and use cases with guests from the data streaming industry. Apache®️, Apache Kafka, Kafka, and the Kafka logo are either registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries. No endorsement by The Apache Software Foundation is implied by the use of these marks.
Episodes
264 episodes
Apache Kafka 3.5 - Kafka Core, Connect, Streams, & Client Updates
Apache Kafka® 3.5 is here with the capability of previewing migrations between ZooKeeper clusters to KRaft mode. Follow along as Danica Fine highlights key release updates.Kafka Core:KIP-833 provides an updated timeline for ...
•
Season 1
•
Episode 265
•
11:25
A Special Announcement from Streaming Audio
After recording 64 episodes and featuring 58 amazing guests, the Streaming Audio podcast series has amassed over 130,000 plays on YouTube in the last year. We're extremely proud of these achievements and feel that it's time to take a well-deser...
•
Season 1
•
Episode 264
•
1:18
How to use Data Contracts for Long-Term Schema Management
Have you ever struggled with managing data long term, especially as the schema changes over time? In order to manage and leverage data across an organization, it’s essential to have well-defined guidelines and standards in place around data qua...
•
Season 1
•
Episode 263
•
57:28
How to use Python with Apache Kafka
Can you use Apache Kafka® and Python together? What’s the current state of Python support? And what are the best options to get started? In this episode, Dave Klein joins Kris to talk about all things Kafka and Python: the libraries, the tools,...
•
Season 1
•
Episode 262
•
31:57
Next-Gen Data Modeling, Integrity, and Governance with YODA
In this episode, Kris interviews Doron Porat, Director of Infrastructure at Yotpo, and Liran Yogev, Director of Engineering at ZipRecruiter (formerly at Yotpo), about their experiences and strategies in dealing with data modeling at scale.<...
•
Season 1
•
Episode 261
•
55:55
Migrate Your Kafka Cluster with Minimal Downtime
Migrating Apache Kafka® clusters can be challenging, especially when moving large amounts of data while minimizing downtime. Michael Dunn (Solutions Architect, Confluent) has worked in the data space for many years, designing and managing syste...
•
Season 1
•
Episode 260
•
1:01:30
Real-Time Data Transformation and Analytics with dbt Labs
dbt is known as being part of the Modern Data Stack for ELT processes. Being in the MDS, dbt Labs believes in having the best of breed for every part of the stack. Oftentimes folks are using an EL tool like Fivetran to pull data from the databa...
•
Season 1
•
Episode 259
•
43:41
What is the Future of Streaming Data?
What’s the next big thing in the future of streaming data? In this episode, Greg DeMichillie (VP of Product and Solutions Marketing, Confluent) talks to Kris about the future of stream processing in environments where the value of data lies in ...
•
Season 1
•
Episode 258
•
41:29
What can Apache Kafka Developers learn from Online Gaming?
What can online gaming teach us about making large-scale event management more collaborative in real-time? Ben Gamble (Developer Relations Manager, Aiven) has come to the world of real-time event streaming from an usual source: the video ...
•
Season 1
•
Episode 257
•
55:32
Apache Kafka 3.4 - New Features & Improvements
Apache Kafka® 3.4 is released! In this special episode, Danica Fine (Senior Developer Advocate, Confluent), shares highlights of the Apache Kafka 3.4 release. This release introduces new KIPs in Kafka Core, Kafka Streams, and Kafka Connect....
•
Season 1
•
Episode 256
•
5:13
How to use OpenTelemetry to Trace and Monitor Apache Kafka Systems
How can you use OpenTelemetry to gain insight into your Apache Kafka® event systems? Roman Kolesnev, Staff Customer Innovation Engineer at Confluent, is a member of the Customer Solutions & Innovation Division Labs team working to build bus...
•
Season 1
•
Episode 255
•
50:01
What is Data Democratization and Why is it Important?
Data democratization allows everyone in an organization to have access to the data they need, and the necessary tools needed to use this data effectively. In short, data democratization enables better business decisions. In this ep...
•
Season 1
•
Episode 254
•
47:27
Git for Data: Managing Data like Code with lakeFS
Is it possible to manage and test data like code? lakeFS is an open-source data version control tool that transforms object storage into Git-like repositories, offering teams a way to use the same workflows for code and data. In this episode, K...
•
Season 1
•
Episode 253
•
30:42
Using Kafka-Leader-Election to Improve Scalability and Performance
How does leader election work in Apache Kafka®? For the past 2 ½ years, Adithya Chandra, Staff Software Engineer at Confluent, has been working on Kafka scalability and performance, specifically partition leader election. In this episode, he gi...
•
Season 1
•
Episode 252
•
51:06
Real-Time Machine Learning and Smarter AI with Data Streaming
Are bad customer experiences really just data integration problems? Can real-time data streaming and machine learning be democratized in order to deliver a better customer experience? Airy, an open-source data-streaming platform, uses Apache Ka...
•
Season 1
•
Episode 251
•
38:56
The Present and Future of Stream Processing
The past year saw new trends emerge in the world of data streaming technologies, as well as some unexpected and novel use cases for Apache Kafka®. New reflections on the future of stream processing and when companies should adopt microservice a...
•
Season 1
•
Episode 250
•
31:19
Top 6 Worst Apache Kafka JIRA Bugs
Entomophiliac, Anna McDonald (Principal Customer Success Technical Architect, Confluent) has seen her fair share of Apache Kafka® bugs. For her annual holiday roundup of the most noteworthy Kafka bugs, Anna tells Kris Jenkins about some of the ...
•
Season 1
•
Episode 249
•
1:10:58
Learn How Stream-Processing Works The Simplest Way Possible
Could you explain Apache Kafka® in ways that a small child could understand? When Mitch Seymour, author of Mastering Kafka Streams and ksqlDB, wanted a way to communicate the basics of Kafka and event-based stream processing, he decide...
•
Season 1
•
Episode 248
•
31:29
Building and Designing Events and Event Streams with Apache Kafka
What are the key factors to consider when developing event-driven architecture? When properly designed, events can connect existing systems with a common language and allow data exchange in near real time. They also help reduce complexity by pr...
•
Season 1
•
Episode 247
•
53:06
Rethinking Apache Kafka Security and Account Management
Is there a better way to manage access to resources without compromising security? New employees need access to a variety of resources within a company's tech stack. But manually granting access can be error-prone. And when employees leave, the...
•
Season 1
•
Episode 246
•
41:23
Real-time Threat Detection Using Machine Learning and Apache Kafka
Can we use machine learning to detect security threats in real-time? As organizations increasingly rely on distributed systems, it is becoming more important to analyze the traffic that passes through those systems quickly. Confluent Hackathon ...
•
Season 1
•
Episode 245
•
29:18
Improving Apache Kafka Scalability and Elasticity with Tiered Storage
What happens when you need to store more than a few petabytes of data? Rittika Adhikari (Software Engineer, Confluent) discusses how her team implemented tiered storage, a method for improving the scalability and elasticity of data storage in A...
•
Season 1
•
Episode 244
•
29:32
Decoupling with Event-Driven Architecture
In principle, data mesh architecture should liberate teams to build their systems and gather data in a distributed way, without having to explicitly coordinate. Data is the thing that can and should decouple teams, but proper implementation has...
•
Season 1
•
Episode 243
•
38:38
If Streaming Is the Answer, Why Are We Still Doing Batch?
Is real-time data streaming the future, or will batch processing always be with us? Interest in streaming data architecture is booming, but just as many teams are still happily batching away. Batch processing is still simpler to implement than ...
•
Season 1
•
Episode 242
•
43:58
Security for Real-Time Data Stream Processing with Confluent Cloud
Streaming real-time data at scale and processing it efficiently is critical to cybersecurity organizations like SecurityScorecard. Jared Smith, Senior Director of Threat Intelligence, and Brandon Brown, Senior Staff Software Engineer, Data Plat...
•
Season 1
•
Episode 241
•
48:33