The 1 Billion Row Challenge with Gunnar Morling | Ep. 23

Tim Berglund talks to Gunnar Morling (Confluent) about his career in open source Java and data streaming. Gunnar’s first job: a student PHP developer in AMD’s e-learning group. His challenge: working at Decodable on the 1 Billion Row Challenge.

SEASON 2
Hosted by Tim Berglund, Adi Polak and Viktor Gamov
Produced and Edited by Noelle Gallagher, Peter Furia and Nurie Mohamed
Music by Coastal Kites 
Artwork by Phil Vo 

•  🎧 Subscribe to Confluent Developer wherever you listen to podcasts. 
• ▶️ Subscribe on YouTube, and hit the 🔔 to catch new episodes.
• 👍 If you enjoyed this, please leave us a rating. 
• 🎧 Confluent also has a podcast for tech leaders: "Life Is But A Stream" hosted by our friend, Joseph Morais.

Transcript

SPEAKER_00 0:00

And this was just amazing to see like all those machines, oh this cost like 50 million, this cost 60 million, and like okay, okay, that's that's kinda impressive. For a while already I had this idea. I want to do something with a billion ropes. I still believe, you know, if you want to set out this goal, hey, I want to organize a viral coding challenge, I feel odds for this to work out are pretty low. And here it was the opposite.

SPEAKER_01 0:24

In this episode, I'm talking to Gunnar Morling, a principal technologist for Confluent. Don't worry, he explains what a principal technologist does in the interview. Gunnar's been a friend of mine for years, and he's also been a driving force behind several influential projects like Hibernate and Debezium. Today, we're talking about his infamous one billion row challenge from January of 2024, a viral project where he inspired dozens, maybe hundreds of developers to tackle a deceptively simple problem. Let's listen. It is great to have you here. Um before we get started, what does a principal technologist do?

SPEAKER_00 1:17

You you start with the tough questions right out of the gate, I guess.

SPEAKER_01 1:21

What would you say you do here?

SPEAKER_00 1:22

Yeah, I guess I'm still trying to figure out, you know. No, I mean, more seriously, the way I like to think about it is um there's roughly speaking two facets to it. There's an outwards uh-facing facet, uh, which I guess folks are more familiar with. So this you know relates to writing. So I spend a lot of my time on writing for my blog about all kinds of interesting stuff: Kafka, Flink, Debesium, databases, um, you know, the usual Gunnar suspects. Yeah. The usual uh Gunnar stuff, exactly. So I spend lots of time on that. I spend lots of time, or let's say a reasonable part of my time uh with doing talks. So I go to conferences like uh current or other places and yeah, speak about the stuff. And so, you know, all that is like outward outwards uh facing, I would say. And then there's another non-public side to that role, which is about yes, internal uh consultancies. So sometimes, you know, our leadership they have questions around technology. So maybe somebody wants to know, hey, how do watermarks and flink work? And I would put together like one pager.

SPEAKER_01 2:24

And you say, Well, I don't know, nobody knows.

SPEAKER_00 2:26

I nobody knows, nobody, but here's how I understand it. Um, you know, and or maybe we want to think about so does it make sense for us to invest into certain technologies? Is this something we should pay attention to, or you know, this is a f uh doesn't matter, this is going to go away in six months, so we don't have to care about it. So I would provide my input on that. I would provide input on the product. So, you know, it's a very uh multi-faceted uh role, and I guess you would call it like um make your own adventure or something. So it's it's a bit like that.

SPEAKER_01 2:53

Yeah, yeah, yeah. A little bit of choose your own adventure. And I I have the privilege of being in some of those conversations with you. Sometimes I get invited for when I want somebody to tell jokes. Um and that's always good to uh to be a part of those.

SPEAKER_00 3:05

Yeah, yeah, yeah. No, that's that's that's pretty much it.

SPEAKER_01 3:07

From there, tell us about your first job.

SPEAKER_00 3:10

Wow, uh, I feel like you want to make me feel old. I mean, this is I you know gotta go. Yeah, gotta go a long time back. It's not me. Yeah. Um so actually, let me start. I want to mention two things. Um so one, I guess it wasn't technically speaking my first real job, but while I was at a student at a university, actually I used to work at AMD, the uh semiconductor manufacturer, and they uh used to have a big uh site in in Dresden where I used to study. Um, you know, now this is actually this old called Global Foundries now. But I was a um yeah, a student employee there. So I worked like, I don't know, 12 to 20 hours or something uh per week.

SPEAKER_01 3:50

Is it still a fab now?

SPEAKER_00 3:51

It was a fab then and it's it's still a fab there uh now, but uh you know they kind of uh are not on like the latest uh process uh generations. So they're sure it's one of those.

SPEAKER_01 4:01

Yeah, those are still important processes.

SPEAKER_00 4:04

It was like uh they did like the ethlon thing, and it was like the first time when they actually made it like really hard for Intel again. Uh so this was right when I joined, actually. This was very cool to see. You know, I I joined and then it all all took off. Um but yeah, so I was working there as a student and uh not in the manufacturing part but in the uh e-commerce and sorry, um e-learning part. So you know, we help people to learn about all that stuff. Um, for instance, if you wanted to go into the clean room, you had to do some training so you would know how to put on like your bunny suit and all this kind of stuff. And I was yeah, working as a I hardly dare to say PHP programmer, and you know, we worked on those web-based uh trainings, and it was actually big uh big fun because it was like a super interesting, you know, area and very just super exciting to be a part of this uh still kind of out-of-world technology in a sense, you know.

SPEAKER_01 4:53

Yeah, yeah. Yeah, so you know, so that's it. For me it was Burger King, but uh you know, an AMD fab is is is also a pretty good job. Yeah, but hey, PHP, right?

SPEAKER_00 5:04

It was PHP and then it was like ASP.net, uh whatever version back then. Oh okay. So yeah, all the good stuff, and you know, of course I did like lots of mistakes, like putting everything into session state and wondering why nothing would work, but that was a good learning experience, and people were friendly. And I know actually at one point I even had the opportunity to go into the clean room because we wanted to take some pictures for new training, and this was just amazing to see like all those machines, you know, and they oh this costs like 50 million, this costs 60 million, and like okay, okay, that's that's kind of impressive.

SPEAKER_01 5:34

Yeah, it'd be massively expensive, and that uh these days, you know, fab equipment is isn't it?

SPEAKER_00 5:38

Yeah, you can add like a zero desk, like 200 million now.

SPEAKER_01 5:42

But um what tell us about it's uh a very interesting, maybe one of the most interesting problems you've you've tackled. Uh you've you've done a number of of pretty cool things.

unknown 5:52

Right.

SPEAKER_01 5:53

So what comes to the top of the list?

SPEAKER_00 5:54

Yeah, I mean there's so many things I guess we could talk about, but um as I was preparing for that, I thought it would be interesting to speak about what's called the uh one billion row challenge, and this was a viral coding challenge, um, which I did uh last year, and we can you know drill into all the details of that. Um I guess I just should say, you know, technically speaking, I was organizing that challenge for the Java community, so I didn't really do the challenge myself, but I made the challenge for other people, but I still think it's an interesting topic to talk about.

SPEAKER_01 6:24

It it a thousand percent belongs on this podcast. Uh it was uh it was just a I was standing back and and watching it, uh it was an incredible achievement achievement in a number of ways. And it's yeah, I would love to hear like how did you get the idea? Had you ever tried anything like that before? Right. Just tell us the whole story. What how did it start and how did it unfold?

SPEAKER_00 6:46

Absolutely. I mean, as with so many of the things, it was kind of coincidential. And I still believe, you know, if you want to set out this goal, hey, I want to organize a viral coding challenge. I feel odds for this to work out are pretty low. And here it was too.

SPEAKER_01 7:04

Let me go give that give that to my team. Hey, folks, go make a viral challenge.

SPEAKER_00 7:07

Yeah, make it go viral. So make it viral exactly. So this is just how it works, right? And it this was not how it was the case here. This was like really coincidential. So it was actually during the uh holiday season in the year before last year, uh so you know, between Christmas and and New Year's Eve, I had uh you know, some some uh slow time, and I was uh you know thinking, hey, what uh what should I do? Um and as you know, as we nuts do, I went off and tinkered around with stuff. And then I had for some reason I had for a while already, I had this idea. I want to do something with a billion rows. Um, you know, it's just like it's a round number, everybody can kind of relate to it. And I didn't really have a specific idea, but this this somehow this number was on my mind. And then yeah, as I had this time at my hands, I felt, okay, yeah, maybe I can evolve this into some kind of challenge. And then the idea was, well, you know, let's see what it takes to write a program, a Java program, which processes a file with uh one million rows, and it you know creates some basic calculation on that. And so the example which I said there was like this a one billion row file, it has um temperature measurements uh keyed by a weather station. So, you know, that would be measurements for uh Denver or Austin and Hamburg where I live, and uh, you know, and you had to calculate the min, the max, and the average value per weather station across those one billion rows. So this was the idea.

SPEAKER_01 8:34

What was the cardinality of the weather stations? There's probably not a billion of those.

SPEAKER_00 8:37

Oh, yes, uh, that's actually a very good question because this is already where it uh started to you know uh deviate from the assumptions I probably had implicitly. Um but so let me just uh uh briefly wrap this up. So, you know, um um you know so I set this out and I thought, okay, maybe like a handful of people will participate. And actually, my wife uh she was uh I mean, usually obviously she's smarter than me, and and she said, okay, you know, if you put this out, you have to put some limit in there so you don't create like an unlimited amount of work for yourself. Um if everybody comes and participates. And I was like, ah, whatever, you know, I'm gonna do it like this. She's she believes in you, she thinks it's she believes in me more than I do, uh apparently, because I felt like you know, uh this is like five people are gonna do it, so who is gonna care? And well, obviously she was more right than I was because then actually a lot of people participated, and uh I created yes, uh literally a huge amount of work uh for myself, for myself. Um but so yeah, then I put it out on January 1st and the challenge ran for the entire month of January, and we can uh we can talk about all the details. But as you ask about the cardinality, so this is a good uh it's a great question actually because this was one of the learnings already. So, you know, the thing is this one billion row file, this has it has a size of um 13 to 14 gigabytes, so it's a quite a sizable file, right? So I couldn't put it to GitHub for people to download it and everybody could work the same file. So instead, what I did is I created a test uh data generator, so people would just have to get this generator, run it, and then they could create their own 1 billion row file. And now for that file for that generator, I used um you know some stage names which I got of uh Wikipedia, and it was like 400 something or so. And now what I didn't expect is people started immediately to optimize heavily for this particular key set. So they provided implementations which would perform very well for those 413 station names in that example dataset generator, and maybe they would perform worse if you had like more station names or just other station names. And now, of course, for me this wasn't what I had in mind, right? For me, this was oh, this is just an example data generator, but you should create an implementation which generally works well. Um, but then you know what do you do? Because people already spent time, so they optimized and they started and you know they put lots of effort to make it work well for those particular names. And then there was another fraction of people, and they said, No, we want to solve it for like 10k stage names, we want to solve the general problem. And so what we ended up doing is we created like two kinds of evaluations, and actually, different people were at the top of those two different evaluations because they just uh you know implemented the problem in different ways.

SPEAKER_01 11:14

Now, a quick word from our sponsor. Confluent developer the podcast is brought to you by Confluent Developer the website, which has everything you need as a developer of data streaming systems, and it's completely free. We've got curriculum, hands-on exercises, executable tutorials, the online data streaming engineer certification, also free, a way to find a meetup near you, those are free. Everything is there. I really want you to be successful in your journey as a data streaming engineer, and this is the site that has what you need. Check it out at developer.confluent.io. That's developer.confluent.io. Now back to the show. And one of those was optimized, just lowest time for my set, and the other one was lowest time generally.

SPEAKER_00 12:02

Exactly, right. Because what happened is um so you know, if you want to aggregate those measurements, you need to have some sort of a map, right? So you want to uh put those measurements into a map, and then you get like the next row for Denver, you want to put it into the accumulator for Denver, which you have, right? So you need to have some sort of map structure. And now, of course, Java provides a general purpose map as part of its standard library. But if you know what the keys are which you want to put into this map, you can create a more optimal custom map just for those keys. And this is what people did. So they created their own map-like implementations which would give the best performance for those known stage names. And they did this by choosing, for instance, you know, particular hashing functions which would never collide for those uh known names. So that's how you do it, right? And again, of course, this is not what I had on my mind, but I also felt okay, you know, it's fair. People interpreted the problem that way, and it's uh I shouldn't stop them.

SPEAKER_01 13:01

There's only a f a only a few hundred station names, you could have a very sparse map, you know, the the the the no-colliding sparse map. You could burn a bunch of memory on something that is going to be faster.

SPEAKER_00 13:15

So what people did is essentially they you know they uh implemented what's called an open addressing scheme. So essentially you take the uh key of the name like the name and you convert it into an integer and you use this as an index, which you then uh use to as an uh as an index within an array. And this is where you store the the value. And now if you you of course you want to find a hash function which gives you that integer derived from the names, which doesn't collide uh for those uh for those known names, because then it's you know just the fastest.

SPEAKER_01 13:50

What was the experience like as it's unfolding? Because I was seeing it explode on on social media, and I I confess I never actually did the billion row challenge. You know, I was pretty busy doing stuff, but I was watching it happen. Uh what was it like uh to see it start to explode?

SPEAKER_00 14:08

Right. I mean, to be honest, it was a huge emotional journey, I would say. Um because I mean there's so many things to say around this. So, first of all, there is this uh viral nature, and I felt it actually, in a sense, it is kind of scary and intimidating because you know you put out this thing and you kind of lose then control over what is going to happen with it because people take it and they make out of it uh what they want to do with it, and you may or may not like this. So, for instance, um I made this uh specifically for the uh Java community uh for two reasons essentially. I first, you know, that's the language I'm familiar with. I know the tool set, I know how to run the stuff. I didn't want to be in the business of installing Rust tool chains and C compilers and all this kind of stuff.

SPEAKER_01 14:54

I was just about to say somebody's gonna want to do this in Rust.

SPEAKER_00 14:57

Yeah, and and so I, you know, and this is exactly what happened. So I did it for Java, also because I wanted to promote it a bit. But then people took it, and of course they did it in like all kinds of languages, even COBOL and you know, uh Rust, of course, and C, and I don't know, like literally every language you could imagine, and also tools like databases like Postgres and Snowflake, and really like uh it was intense. But, anyways, now some of those people made it like super competitive. So, hey, we are better than Java, and you know, this is uh how amazing we are. And for me, this was not really the goal. Um, even like getting to who is the fastest, this wasn't even my goal. My goal was more okay, let's make this a learning experience. So, this you know, I felt okay, let's collaborate and you know, see how we can all learn together. So, for instance, also it was um unlike in other challenges, you actually could look at the code. So, I you know, if you had participated, I could take a look at your code and say, Oh, Tim is doing something amazing there. This is a cool trick, and I can uh yeah, I can take it and I can improve my own implementation with it. I mean, obviously, I just don't take your code and say this is mine, this would be silly, right? But people could and uh take inspiration, and this was happening. So for me, it was more about this learning, and then some there was this super competitive notion at some point, and I really didn't like it. This was not what I wanted it to be, but yeah, you you lose control, right? Because I mean there's hundreds of people, and I mean you don't get to tell them what to do, right? So this was kind of scary in the sense, and I mean, here this had overall very positive energy, I would say, right? And everybody kind of liked it, and it was like a cool thing, but then I also felt like if you are at the center of that, and it's actually like a negative uh thing, you know. Let's say you say something stupid online, and like everybody comes and like you know, um uh hits you on the head, then this can be like really devastating, right? So I felt this is kind of um yeah, it's really like uh scary in the sense to have this this massive energy, and you kind of or just like you're not in a triver seat necessarily.

SPEAKER_01 16:51

Yeah, yeah, you're really not. Um speaking of alternate solutions, at the time, I was at Star Tree with uh on the the Apache Pinot community.

SPEAKER_00 17:01

Right.

SPEAKER_01 17:02

And we solved it with Pinot, which was an example of a of a kind of an off-label. Right. Uh this isn't really the the challenge, we're not entering this as a result. But wouldn't it be interesting? Absolutely. And um there, I mean Pinot has an index that's super well optimized for that kind of thing. As long as you don't count the ingest time, it it was amazing. And that was that was the thing I think with the database-based solutions was do you count ingest? Do you just count the queries? Right, yeah, exactly. All these kinds of things.

SPEAKER_00 17:31

And that's also, you know, I mean, I just didn't have time to install something like Pino or whatever. So I really I figured just on those, you know, Java implementations which I uh could actually evaluate. And then also, as you ask how it was to experience, well, actually what also happened, so yes, people came and wanted to participate, but also people came and they wanted to help run it and help organize it. And this was just amazing to see how people, you know, build infrastructure to uh uh uh tooling like uh shell scripts to you know automate that process of maintaining the leaderboard and uh having like a test suite. Um, you know, they they helped uh to make it work, which I found really cool because you know what is in it for people to do that? They just do it because they like being part of the community, right?

SPEAKER_01 18:16

In terms of evaluating the results, you had a standard instance that you ran those on in a kind of an objective way to say.

SPEAKER_00 18:22

Right, yeah, exactly. So that was the idea. Um I you know I rented a machine and um like in a um um how do you call it like not one of the big cloud providers, but like a colo space or like where you could have a dedicated server. Uh so that was the idea.

SPEAKER_01 18:36

And now initially seriously, oh okay. I thought it was an EC2 instance.

SPEAKER_00 18:39

No, it was not EC2 because then you you know you might have noisy neighbors and they might give you like uh burst uh um behaviors, right? So it's uh much um much harder to compare. So what I thought, so I went you know to this uh uh yeah cloud provider, which give me gave me like my own machine. But initially I thought, okay, you know, I'm gonna go there, and I use like what they would call exclusive virtual course. So I thought, okay, you know, it's in a virtual machine, but they tell me I have my own course. So I'm I'm free from like noisy neighbors, and this should be objective. And this worked for like the first week or so, and then you know, uh every night, every day, people would create their PRs with the new implementations. I would wake up and I would take the 20, 30 or whatever and run them in the morning. And I one day I came back and like all the results were suddenly off. So this it seemed like wait, wait a minute, suddenly everything is like much faster than before. So I ran some of the implementations which I already had run, and they were like twice as fast. And so I was okay, what's happening? And then what turned out, well, it was a virtual machine, and yes, it had like exclusive cores, but then they just decided to move that workload to another instance which was more capable.

SPEAKER_01 19:47

And yeah, faster ends are giving you more value.

SPEAKER_00 19:49

Yeah, exactly. They were giving more value. So maybe you know they saw this. There you go. And like well, it really messed messed up uh the things for me, of course. So what I then ended what I ended up doing is uh I actually got my own exclusive machine, and again somebody from the community came to help me because I mean I'm not a big uh Linux admin person, and you know, I knew okay, I want to do things like disabling turbo boost and disabling hyper threading and sort of stuff to to you know make it like uh more objective, but I didn't really know how to do it.

SPEAKER_01 20:20

I would also need help with those things. I'd let somebody look over my shoulder and make sure that it's a good idea. Exactly.

SPEAKER_00 20:23

And and so I went online and asked, hey, is there anybody who can help me with that? And then some guy, which I didn't know before, he said, Yes, I can help you with it. I have experience. So we hopped on a call and I, you know, I got a good feel. And then the next minute he sent me his SSH key, I uploaded his key to the machine, and then he could go and configure the thing, and it worked. And this was just amazing to see people do all those uh things.

SPEAKER_01 20:47

Two things about that. Number one, uh as you know, we're p nerds, and when there's the thing that we're good at, we really like to do. And if somebody needs help with the thing that we're narrowly specialized in, um, you know, our our specialties, if you just look at the whole world of things humans are good at, we have these incredibly narrow specialties, right? So when somebody needs that, that's the old XKCD, you know, uh stand back, I know regular expressions saving the day. Um and so that that's an impulse that we have anyway. And secondarily, I think when you have something that's so virally successful as OneBRC was, um, that's a train people want to get on board. Right. It's easier to get help. It's it's not some little backwater project struggling, and you're like, oh, does anybody care about me? Everybody was talking about it. And so it was kind of an honor to be involved.

SPEAKER_00 21:44

Yeah, yeah.

SPEAKER_01 21:44

I I guess. And that's that's a great, that's a great dynamic about something that really takes off like that.

SPEAKER_00 21:50

Yeah. So this was really, I don't know, uh, you know, just a positive experience to see all that. I had to like there was one single interaction uh with a person who was like really hostile. And I ended up like uh muting them on GitHub or whatever. But this was like the only thing other than that, like everything was like super supportive and very helpful. And yeah, I felt, man, this is just so great to see.

SPEAKER_01 22:11

And that's that's typical if you know I look back on the communities I've been a part of. I have been uh I think I would say blessed to be in healthy communities where that kind of behavior is unusual. Right, right, right. It happens and you drop a hammer, yeah, and uh, you know, uh a healthy ecosystem. Absolutely toxins.

SPEAKER_00 22:34

And uh, yeah, you gotta deal with it quickly. Uh that's also what I always felt in the open source communities. I participated in you have to stop those things early on, and then actually you can have a very healthy dynamic.

SPEAKER_01 22:48

Yeah. And important to note, so if you don't know who Gunnar is, and you know, at the beginning I gave a little bit of an intro, uh this is not his first rodeo. This you've been uh, you know, you you have been uh a maintainer of and participant in some fairly prominent open source talks. So so you've got you've you've been able to build those muscles.

SPEAKER_00 23:09

Yeah, yeah, I I would say so, yeah.

SPEAKER_01 23:12

Let me ask you this. What is can you think of anything that you've done like this that didn't work? Because I know at the time my my you know, people, leaders near me in the company I was at were saying things like, well, we should do something like this. Right. And and like we were joking, you know, the the the impulse is, oh, okay, team, you know, go do a go do a viral thing. Um you you don't just do a viral thing. So have you tried projects like this, uh, or you know, even games like this, and they haven't gone on to this kind of success? What's what's that track record like?

SPEAKER_00 23:48

Right. I mean, no, I don't I haven't done any things uh you know similar to a challenge like this. I mean, I've worked on different uh different open source projects, um, led some of them, some were successful, some didn't go um, you know, very far, but I guess that's just the nature of the game, right? So you want to try out uh stuff and see what's happening.

SPEAKER_01 24:09

It's funny, I I I don't know the ones that didn't go far, but I guess that's because they didn't go far. I know I know I know DB EZM, I know Hibernate, and those are mild, moderate success. So there's a little bit of a little bit of uh results.

SPEAKER_00 24:20

But for instance, Hibernate is a good example because for some while we tried what's called Hibernate OGM, or what was called Hibernate OGM object uh grid mapper. So this was the idea to you know take the Hibernate or M APIs and apply them to NoSQL store. So like back then, like MongoDB was I mean it's still a thing, but you know, back then it was like really all the rage and Kazangra and all the everybody's talking about it, yeah. Exactly. And so we tried to make Hibernate work with that, and for various reasons it didn't uh really uh go anywhere. I guess that was just too much of a mismatch, I would say, between those APIs as they were back then and what those um databases could you know work with?

SPEAKER_01 24:55

Yeah. Okay, interesting. So uh again, the things that don't work, I think you just don't know about until you gotta keep trying. Telling the the historical stories, yeah. Um what did you take away from it? Like uh, you know, were there results for the company you worked for, results for you? Right. What do you do? How are you different now?

SPEAKER_00 25:16

Right. Yeah, that's a great question. I mean, um for the company. I mean, back then I was you know at a start-up, a startup called Decotable, also in the Flink Space uh back then. And for them or for us back then, I would say it was maybe uh yeah, it was a marketing success, I would say, but very importantly, it was not planned like that. But this was really, you know, my own idea. I didn't even talk to anybody before because I just didn't expect it to be a thing, really. And then January uh happened and the you know it kind of blew off, and you know, first week of January is still kind of slow, and I thought, okay, I'm I'm I'm fine doing it, you know. But then it like kept growing, and I spent like literally 12 hours per day doing the thing, and I felt okay, gotta raise a flag and say, hey guys, I you know I have this challenge and I feel like I want to keep it going, so what do we do? And they were like super supportive. So really like this entire month, I I worked on that thing, and then you know, they helped me to pay for that environment and they helped me to um uh organize, for instance, some prices. So I actually I sent t-shirts and coffee mugs to like the top participants in the end. So we put a light touch of marketing, but it never was around uh this was never the idea, it was more like okay, they were supportive of it, so of course I would add it to the README and the GitHub repo and stuff. Um, so this was uh what you know what happened for them. I'm still super grateful that they were uh that supportive. And for me, I mean I learned so many things, right? I mean, I also learned how to send t-shirts and coffee mugs to people around the world, which was like a massive challenge by itself because if you want to do this stuff, you want to send to people in South Korea and uh you know uh Brazil and uh I don't know, all over the place. It actually just took me like I don't know, a couple of days just to make uh that part happen, actually, in the end.

SPEAKER_01 27:01

That is it, that's funny. I mean, this is why I have the great Ale Gutierrez on my team because that's it.

SPEAKER_00 27:06

I I would have loved to have somebody for me to do that.

SPEAKER_01 27:09

That's just a button they press and it happens. But yeah, if somebody walks up to you and you're a person like you or me and they say, We we need to send a t-shirt to uh you know somebody in Jakarta.

SPEAKER_00 27:20

Right, yeah, exactly. And this was happening. I had I think it had like Indonesia had Indonesia, but definitely countries there, um uh you know, Japan. So it was like really super expensive just in terms of the shipment, actually.

SPEAKER_01 27:32

Yeah, yeah. And you know, and there are services. There are there are people who have created abstraction layers that actually make it push button easy, but you don't you don't have access, you don't know what those are. Exactly. And people do, so that's a that's a little thing.

SPEAKER_00 27:44

And then I also wanted to give everybody like their own individual um letter of uh certification. So actually I learned how to use the mail order uh what's it the mail merge uh facility in macOS so I can you know uh print like 165 PDF files with like the individual name and the time, all this good stuff. So it was was just very interesting to organize it. And then of course you know like steering uh more or less that community and try to keep everything on track. So it's definitely was a big uh personal learning experience.

SPEAKER_01 28:15

Next up, the billion address mail merged.

SPEAKER_00 28:20

Yeah, yeah, yeah. That that would be that would be a challenge for sure.

SPEAKER_01 28:25

Um you think you'll do something like this again sometime?

SPEAKER_00 28:28

Yeah, so people keep asking about it, and I mean I'm still a bit unsure. So the thing is overall, yes, I would like to do it again. But I mean, obviously, it needs to be something which is like also kind of clural and which is different, right? So you don't want to do like a two-billion row challenge, it would be kind of useless. So you wanna you wanna do another kind of challenge, and I felt it was so successful because it hit a very specific uh combination of being very easy to explain. So, like you know, at the beginning of our conversation, I could explain it to you like to you in 30 seconds, and you pretty much to every person you can explain this in 30 seconds, and at the same time, this problem has lots of potential to keep those people busy for an entire month. And this is like you know, really the smartest developers in the Java community, right? Also, by the way, Cliff Click and like the OGs working on the Java uh Hotspot VM, people like those Alex Chipilev, they participated. So like really high-profile people, right? So and so you want to keep those engaged for months, it has to have some sort of depth to it, right? And so far, I just really haven't found another problem which would combine those two traits, being easy to explain and still having this kind of unlimited uh potential. So I'm still looking for ideas. If somebody has an idea which kind of checks those two boxes, I would be uh all ears.

SPEAKER_01 29:50

My guest today has been Gunnar Morling. Gunnar, thanks for being a part of the Confluent Developer Podcast.

SPEAKER_00 29:54

Thank you so much, Tim, for having me. This was a great pleasure. Thank you.