Agent Mesh: A Microservice With a Brain ft John Miller & Eric Broda | Ep. 33

Tim Berglund talks to John Miller (Enid Technologies) and Eric Broda (Agentic Mesh Company) about their work on agent mesh and enterprise agents. Their challenge: rethinking agents as enterprise-grade participants in business processes.

SEASON 2
Hosted by Tim Berglund, Adi Polak and Viktor Gamov
Produced and Edited by Noelle Gallagher, Peter Furia and Nurie Mohamed
Music by Coastal Kites 
Artwork by Phil Vo 

•  🎧 Subscribe to Confluent Developer wherever you listen to podcasts. 
• ▶️ Subscribe on YouTube, and hit the 🔔 to catch new episodes.
• 👍 If you enjoyed this, please leave us a rating. 
• 🎧 Confluent also has a podcast for tech leaders: "Life Is But A Stream" hosted by our friend, Joseph Morais.

Transcript

SPEAKER_00 0:00

What is a microservice with the brain? So it doesn't just talk using APIs, although it may. It talks natural language, for example.

SPEAKER_02 0:08

All we were talking about was coding agents. And Eric and I started the podcast talking about enterprise agents. And I said, listen, Eric, you need to have a conversation about coding agents so that we can participate in the discourse.

SPEAKER_01 0:22

Hey, welcome to another episode of the Confluent Developer Podcast. I am your host, Tim Bergland, and today I got to talk to Eric Broda and John Miller about the idea of an agent mesh, which is about what it sounds like. Now, who is John and who is Eric? John is uh the co-founder of a company called Enid Technologies. He's a AI systems builder and a data architect, uh, goes way back in that space. Eric is a guy who comes from technology exec positions in finance, and he really likes meshes. He wrote a book on service mesh, data mesh, and now agent mesh. Meshes rule everything around Eric. And, you know, I actually really love this conversation today. It was just super interesting. Eric and John talk about their notion of uh this web of enterprise agents collaborating and all of the typical enterprise concerns that that need to be addressed in that and kind of a framework for doing that. And they have a really provocative view, I thought provocative view of how to build an agent. You know, you're wondering which framework is it? Is it uh, you know, the spring? Is it what what which are they doing it uh well? No, uh, I'm not gonna tell you in the intro. You're gonna have to listen in the podcast. We spend, frankly, most of the episode talking about that and only a little bit talking about the mesh. I hope I can have the back on the show again soon to get more. So great conversation. Let's dive in. Eric and John, welcome to the show. Glad to have both of you here.

SPEAKER_02 2:01

Great to be here. Yeah, it's great. Great to see you uh from London. Uh I'm joining you from the hot climates of northern Europe where it's become uh very hot in the springtime. So it's uh it's great to see you.

SPEAKER_01 2:16

For the record, we're recording this May 27th, 2026. Check the weather. It's not it's not a good afternoon to be in London. No, it's not.

SPEAKER_02 2:26

I know. I'm seeing people in the city in shorts. It's very rare uh in the financial district. Yeah.

SPEAKER_01 2:32

Not how it happens. Uh so you guys together uh talk about this concept of the agentic mesh. Uh you got a YouTube podcast, YouTube podcast, you got a podcast, you got a YouTube channel, you got a uh Substack. Uh Eric, you're co-author of a book um by that name. Uh what is that and uh why is it important in uh here we are in the middle of 2006? Yeah, sure.

SPEAKER_00 3:01

Uh so I think everybody's heard of agents, but most people think of coding agents, Cloud Code, Codecs. These are these things are fantastic productivity tools. Um but what we believe is that you don't work with one agent in an enterprise or 10 of them. In fact, you're gonna have thousands, maybe millions of them. If you believe Jensen Huang and the others, we're gonna have a vast ecosystem of agents. And they're gonna operate very differently. They're not gonna be tied to an individual user or user interface. They will be participants in a business process. And that means that they're gonna have to have a whole set of capabilities and um constraints, if you will, that are different than what folks are used to dealing with today. So agentic mesh, the book and the concept, is really about uh organizing the ecosystem of agents. And what do you need to do to allow agents to safely uh, well, first off, find each other and then safely collaborate, interact, and even transact. Um so this means this is all about enterprise grade agents and the enterprise grade agent ecosystem that they run in.

SPEAKER_01 4:15

Let me uh ask a potentially spicy question right out of the gate there. What it sounds like you just described microservices. Um and I kind of I like that account of agents, but why are those units of software you just described, why are they not the things we would have been talking about five, six years ago?

SPEAKER_00 4:35

Yeah, sure. That's a that's a great question. So so microservices are uh they're they're obviously headless, they're ambient, um, but I'll use some simplistic terms. They're not smart. Uh so in our terminology, an agent is actually, if you deploy it as part of a business process, headless if you will, it is a microservice with a brain. Um so it doesn't just talk using APIs, although it may, uh it talks natural language, for example. And it responds using the intelligence in the large language models behind it. So we actually think they're they're closely related, but again, to turn a really simple, pithy phrase, uh they're not just microservices, they're smart microservices. And that intelligence allows them to far exceed the capability of a microservice and allows them to actually, like I said, find each other and safely interact and collaborate in new and innovative ways. So so they're they're they're related but a little bit different.

SPEAKER_01 5:37

Would it be fair to say that the SMART is uh an LLM? It's a they there's there's a large language model that they're calling in there. Is that it?

SPEAKER_00 5:46

Absolutely. Uh I would extend that a little bit. So what we've done is we put LLMs, small language models, large language models, and everything in between in the microservices, but we also put cloud code in a microservice, in a container. So so we we do that with codecs also. So what ends up happening now is where an LLM, we probably know the difference, but for the audience, I'll elaborate a bit. An LLM has, for lack of a better word, intelligence. The Cloud Code or Codecs have the harness that allows them to do very compelling, long-running work. So when we package these LLMs first in a microservice, you get some capability. But we actually found a way to very securely package Cloud Code or Codecs and some of the other uh um innovative coding agents. And by the way, I call them coding agents, and that's not meant to be pejorative. They can do much more than coding agents. But the key here is we actually package agents that can do not just simple tasks or respond to simple queries in the natural language, mind you, but they can actually do very sophisticated, long-running, complex tasks. So what we have now is not just a simple microservice or a smart microservice. We actually have a microservice that can do extended amounts of work, which is not something that you can get easily unless you actually combine the technologies, as I mentioned.

SPEAKER_01 7:13

Okay, that is fascinating. So I want to I want to get into the mesh stuff, okay? Because I that's sure what your property is is all about and what your thinking together is is all about, but you're you're saying some things that are just kind of getting my wheels turning here.

SPEAKER_02 7:30

So yeah, well, yeah, and Tim, as you uh asked that question about isn't this just microservices? I think I would love to unpack some of that because at the foreword of the book and the way uh uh the agentic mesh idea uh came to light was very much uh the next evolution of distributed computing, right? So it wasn't novel that microservices would be the foundation layer for distributed computing, but how do you incorporate LLMs and agents inside that? And so when you asked that question, I thought, huh, I wonder if you were gonna go a different direction uh in how you were thinking about distributed services and microservices. So I'm sort of throwing it back to you a little bit.

SPEAKER_01 8:26

There, there, uh there's I've got in terms of the mesh and the orchestration and everything, definitely places I want to go there. Okay. I I want to I want to get there though. Uh okay, good. And by the way, just uh it I my opinion on this is is not really what this podcast is about, but I I happen to love the definition of it's a microservice that calls an LLM. Um and does something. I I I I would like to add, you know, it causes some side effect in the world. It's not it's not a chat bot.

SPEAKER_02 8:59

Well, and and and I think that's important because what I have found is Eric and I talked uh to each other and to other folks, the level of understanding around what is an LLM, the difference between a chatbot and an agent is sort of one level of understanding, the understanding of how does an agent use memory and how we think about trust, every dimension of a typical system design must be thought of when you're building a whole fleet of agents. And so let's go back, let's jump around a little bit, but it's fine to have fun with this discussion because I think as the more we talk about the concepts on our podcast, the more we realize that a lot of these common themes, like how do you uh identify an agent? Does it have its own ID? A lot of these basic concepts of distributed computing keep coming back over and over again. We've solved them, but now we're revisiting them with AI. So, you know, we could talk all day about this, which which might be fun, but uh but I uh would love uh if there's a train of thought that you wanted to uh take us on.

SPEAKER_01 10:26

That's that's great too. Now a quick word from our sponsor. Confluent developer the podcast is brought to you by Confluent Developer the website, which has everything you need as a developer of data streaming systems. And it's completely free. We've got curriculum, hands-on exercises, executable tutorials, the online data streaming engineer certification, also free, a way to find a meetup near you, those are free. Everything is there. I really want you to be successful in your journey as a data streaming engineer, and this is the site that has what you need. Check it out at developer.confluent.io. That's developer.confluent.io. Now back to the show. I I think we need to get there. Uh okay. But uh, and by the way, everybody, don't miss that John said the podcast, the agentic mesh podcast, agentic mesh podcast, linked in the show notes. Um, I think you're at this point morally obligated to check that out. Um yeah, no, definitely do that. This is this is because this is really important stuff. Um we're we're I think up-leveling the the conversation, the sort of a uh this kind of 60th percentile Twitter conversation about agents that Eric, you hinted at this, is is just about Claude code or you know, yeah, no, no, Codecs. And you know, okay, you know, that's we can have that. And they're great. I mean, I was working on a project this morning before work with Claude. Um, love that life, but uh, you know, the agents that we write to power the business, I think is the much more important thing. And Eric, you said something that's got me my head spinning a little bit. Um, using Claude Code itself as the harness inside one of those, I'll just say for for our purposes now, an enterprise agent, an agent in a business doing stuff. I have thought, and most of the conversation I've seen, um and folks I've talked to, and you know, looking at the frameworks and everything. It's all early days, so it's not like all the stories of all of the successful agent deployments are you know, you're scraping them out of your shoe and you're walking to the car in the morning. They're not they're not exactly all over the place. We're talking about it more than we're doing it. But I have thought that that that harness, that agent loop is a thing that really we write in the enterprise agent and not just bring in clawed code. So before we get to the mesh stuff, could you expand on that? Yeah, yeah, yeah, sure. So either one of you, these are all jump balls uh for my two guests. You guys, Kim.

SPEAKER_00 13:02

Yeah. Um so so here's here's the the the analogy that I think I would use. And it it may stretch a little bit too far, so my apologies, but uh it is quite appropriate, at least for this part of the discussion. Every organization has employees, some big organizations have thousands of employees. These employees do work. These were these employees are able to do the work because they have tools and they have the intelligence and smarts to fulfill a job description and complete tasks. Our proposition is if you have a sufficiently capable agent like Claude Code, the analogy is very appropriate. Claude Code or Codex, I'll use them in general, the these coding agents have the capability to do remarkable amounts of work, not just coding, although that is the primary purpose that we see now. We can see anthropic moving into financial, you know, providing skills and capabilities around financial services and regulatory and compliance. They know the fact that Claude Code is not just a coding agent, and that's why they renamed, well, they have Claude Co-work also. Here's the fundamental thing is these agents are supremely capable and they can function with a little bit of help, I suppose, and they're not necessarily 100% at all replacements for people, maybe augmenting the capability, but they can participate in business processes, they have the intelligence and the smarts to participate in a business process. A simple LLM does not. You need to write the full harness, the looping, and all the rest of the stuff. But things like Claude Code, Anthropic Opening AI with Codex have figured this out. We take advantage of that. Our mission is to actually enable these agents to, again, safely find each other, collaborate, interact, and transact. That's when the real value occurs. That means that we need to have agents that are smart enough, Claude Code, Codex, that can actually participate in a real, practical way as participants in the business process. How to do that? You need to wrap them in a container, deploy them as a microservice, for example, and you need to provide a very robust communications backbone. And like I said, we're we're pretty biased in this. We think event-based communications is the way to go. Things like Confluence Kafka, um, and there's a few others that are out there. This is the communication fabric for agents that enable them to collaborate at scale, allows us to take advantage of the intelligence that is only growing of things like Cloud Code and Codex. That's what we we think our perspective is a little bit different than that that 60% traffic that you see on Twitter, perhaps. We think this is where the future actually is, Tim.

SPEAKER_01 15:59

Okay. I it's sometimes as a podcast host, when uh you have your mind a little bit blown in the middle of an interview, you have to kind of like stop and say, I am I am rethinking some things now. But I I like where you're going. I guess that means um I mean you you just said you take Claude Code and wrap it with communications harness, you know, I would put Kafka endpoints, uh, you know, a producer-consumer sorts of things on there. And um I want to ask you next, like, oh okay, tell me about this mesh now. What is inside that? But then then um quote unquote writing an agent would be reduced to skills, uh uh MCP connections if necessary. It it's really for that to be in any sense specialized to a business process would mean it has access to skills, but this agent of yours is a fairly generalized thing.

SPEAKER_00 16:59

Uh I'm gonna say yes and no. Um so so there's a bunch of things you have to actually wrap around this microservice containerized version of cloud code in order to make it work in the enterprise. It needs to have an identity, it needs to have access controls, it needs to be discoverable, observable, operable, it needs to be explainable. We need to ever every action and information that it processes needs to be audited. So there's a bunch of capability that actually needs to be wrapped around these that actually we believe are not necessarily just part of the agent, but they're part of the mesh. They're part of the fabric, not just the communication fabric, the identity fabric, the authorization fabric, the trust fabric that you need to actually implement. That's part of the mesh capability. And then the last thing I would say, so one last thing, John. The last thing I would say is that the mesh sounds like you got thousands of these things running around. The form factor that we see for the mesh is actually a business process. And that's why I come back to agents are participants in the business process. That's how they are arranged. Now, the task definition as part of a process, that's where you actually embody the skills and the tools. Because we strongly believe that if you have these unbounded agents, they can access bash, they can access the internet, which is what you can do today. That is not the model that any chief security officer is going to let you get away with. It'd be silly to actually put those things allowing that type of access. So, what we believe is you need to have a very sophisticated framework around the task that the agent wants to do and the specific skills and the specific tools that need to be accessed at that point in time for that point in time, just for the duration of the task. So there's actually a lot of capability. Once you believe you need these things participate in an enterprise business process, there's a a whole ecosystem of capability you have to actually add. John, I know you've been itching to say stuff.

SPEAKER_02 19:03

Well, I know, and this is what I love about our podcast is we uh we send cues, you know. Hey, uh Eric, I want to interrupt. Uh how to how to negotiate the mic, exactly. Very important. But this is what I love about this thing is that we are also learning, you know, six months ago it was context engineering, you know, then it became decision traces. And now we're talking about memory. And now we're talking about the combination of structured data management principles. You know, I'm a board member of the Data Management Association in LA and in the Bay Area. And we talk a lot about data management, but we also know that there's a whole practice around knowledge management that comes from a library sciences background. And when you start to think about a business process or anything you're building in Claude Code, without memory, you're kind of just making it all up. You're kind of just saying, Well, I think I need all these technical things to work, but you also need to know what's the problem you're trying to solve, and often in businesses. That's one of the hardest things to do. And if I were to step back, here's what I would say. I think we can all agree is that getting business requirements for complex domains is a non-trivial task. And so what we have found is that part of building systems of agents is figuring out how you're going to provide them with the knowledge required so that they can participate inside a business process or they can build new modules of code. So I think a lot of people are finding that when they're doing uh long-running clawed sessions, uh clawed code sessions, they need to prep the agent with a bunch of memory. Well, that same memory should define the business key business memory that is important to the business process. So I'm a data guy from way back, so I always like to make sure that we have the knowledge and data background to complement what was a very distributed computing world. You know, when we talk about microservices, everybody forgets that they don't have state. So there's no data. So it's like microservices, yes, and there's a whole body of knowledge around how systems use information and data to uh to run processes and to make clawed code much better. Right. So I think a lot of people are finding that uh you know, creating a second brain or creating some form of memory that they can feed to their agents is not optional. It's actually part and parcel of building enterprise grade. We did a a lot of writing and talking about this. We we called it uh virtual memory management or what is the minimum viable context for an agent to be able to answer a question? Well, uh the knowledge part of how agents reason becomes a first class concern because you know, forget about microservices. An LLM has a limited amount of memory or context it can hold at one time. It's like a human short-term memory. If you don't know about it and you can't recall, you can't make a decision. I I'm on a uh a soapbox. Let me pause for a second.

SPEAKER_01 22:49

And I like you said, context engineering as a leading edge topic is six, twelve months old. It's it's um not the thing people are talking about. It's not not a thing anymore because context is still a scarce resource in our LM calls, so it's it's um it's a thing we do, and I view memory as a part of that. So how uh does this work? So maybe from an infrastructure standpoint, and I'm I'm always the one who wants to talk about implementation, yeah, which the uh dot dot dot mesh people tend not to like, but uh uh you know indulge me. Uh what are the forms that you see memory taking in the mesh? And how do you wire that into cloud code? Like how do you is that a skill? How do you motivate it right?

SPEAKER_02 23:38

Yeah, so uh Eric, uh you might want to go first. When we we just recorded episode 19 uh a few uh months ago, and it was all about memory. So so we've got a well-rehearsed speech how to think about memory.

SPEAKER_00 23:56

Yes. Yeah, so so we got we were pretty opinionated on this, but let me let me kind of kind of set the stage. There's there's probably, and I think we did talk about this at at length in the discussion in the podcast, plus uh in the article. There's there's five, I think I'm gonna say five different types of memory, and you need to think about all these. So I want to separate the types from how you actually store and and make it available, but there's obviously a static memory. This is what the LLM comes with or the agent. So I'm gonna use LLM and agent interchangeably for this point of the discussion. This is the pre-trained stuff. Um, the second piece though is long-term enterprise memory. So so if I have an enterprise agent and it wants to um uh execute a compliance process, it behooves it to make that information about how the company or the enterprise wants to do that, how specific it may be. Uh that that is what we call enterprise memory. So so that's the stuff that gets boiled down and put into the context. There's semantic memory.

SPEAKER_01 24:55

Is that is that the implementation? I hesitate to use such a gauche term in 2026. Would that be rag, Eric?

SPEAKER_00 25:04

Uh rag would be a very um simplistic uh view of that. We think there is a bunch more uh without going into too much detail, um, knowledge graphs, ontologies, and uh these all are opportunities available, and we use some of them. Um but that's part of the semantic memory uh bucket. So concepts, definitions, um, and ontologies, policies on top of that need to be in there. Because all too often we store the concepts, what is an invoice? But there's there's processing instructions, tacit knowledge that needs to be captured around the policies and interpretation of what happens, for example, when the invoice amount is too high or there's an error in the invoice. So so there's there's policies that also need to be captured. Um there's shared memory amongst agents. So if you have agents doing work together and they're solving a bigger problem, they need to actually see each other's conversations. So that's part of that. And then we all need to boil this down into this token aware thing we call the minimum viable context. That assembles all of these things based off of an appropriate set of criteria, and it puts it into, and I think this is actually very, very crucial if you read some of the news on Twitter and such, in a token aware fashion. So that's why we emphasize it's not just the the um viable context, it's the minimum viable context that serves the right content at the right time in a token aware, token budget aware fashion. So so that's the different types of memory. John is an is is I would call an expert in a bunch of the implementation techniques. So John, I'll I'll let you finish off this part of the discussion.

SPEAKER_02 26:47

Well, Tim, uh, did you have any questions about that? Because uh, you know, a lot of people ask me, well, what is it, how do you implement that?

SPEAKER_00 26:56

And so and uh I I mean I'm just I'm thinking I guess there are you know skills that get triggered that call CLIs that call your semantic uh augmentation layer or or Yeah, we we we end up calling it we like anybody we have a really cool acronym for we call it the Agentic Knowledge Fabric or AKF, but it's the thing that bundles all these capabilities together.

SPEAKER_01 27:23

Um nice. Okay, all right. So the agentic knowledge fabric. Uh yes, another three-letter. I think I can imagine that. Yeah, it just with the the tools I said. Because again, and guys, you know, your whole thing is the mesh, and I'm super obsessed with this. Wait, you're doing this with cloud code? You know, and so a lot of my questions are are kind of around that and how how you connect it to the mesh. Um, I will not also I won't apologize for that. I think it's an interesting issue.

SPEAKER_02 27:54

No, it did I think it's fundamental, Tim, to what we're hearing. You know, I was uh being based in San Francisco, all we were talking about was coding agents. And I was and Eric and I started the podcast talking about enterprise agents. And I said, listen, Eric, we need to have a conversation about coding agents so that we can participate in the discourse of our community, right? It's all about coding agents. But there's actually more to it than just that. So your perspective on thinking about coding agents as a first class agent development platform, it's true. And it's a yes and you need to think about it.

SPEAKER_01 28:39

So use them as their harness. Yeah, yeah.

SPEAKER_00 28:42

So it's kind of interesting because um, I'll be honest with you, when we started our podcast in the Substack and even the book, um, it was almost like we're shouting from the mountaintop and nobody's listening because we had such a radically different uh way of looking at this. The reason I think is because John and I both come from you know huge enterprise backgrounds. We solve big problems using things like microservices, event-based systems. And that so this is kind of natural for us to look at this as a form factor implementation approach. And it was a natural byproduct to figure out how we can make these microservices um intelligent and how we could actually connect them using an event fabric, event-based fabric. So to us, this was actually very natural. But you know, only now are we starting to hear folks actually say, ah, but we do still have to explain it. So we can say coding agents love them, fantastic, remarkable things. We use them every single day. But here's a problem. First off, there are personal resources, individual resources, running under your own resource uh permissions, your own user ID, and it has access to that environment, whatever secrets you have in your environment, variable, whatever. You know, what we do is we use the capability, but we wrap them uh and and allow them to communicate. Only now are we starting to get people to actually go, oh, you're right. There's a whole new opportunity here. So um we're starting to get some traction and people are starting to listen, which is fantastic.

SPEAKER_01 30:14

Now, um sensitive to time. I there's a couple more things I want to I want to dig into here. Um Eric, you you rattled off a list of all the things that you need an agent to be able to do to be a participant in an enterprise system. It needs to be observable, um, authenticated, authorized. Um you listed some other things. And and solving those, I imagine, is a significant part of what the mesh really needs to do. Am I right there?

SPEAKER_00 30:46

Absolutely. So uh we have been trying to put together all of these concepts into an open source capability. We'll be hopefully releasing that in the next give or take month or two. And the the the notion here is is this is how do you get enterprise grade agents and enterprise grade agent ecosystem using a form factor of process automation? We are implementing all of those capabilities. So we've wrapped in identity. And once you have an identity, you can attach roles and permissions. Once you have those, you can actually attach skills and tools. So all of a sudden, when you start to put all these pieces together, you can actually assemble ecosystems, not individual agents, but processes in gigantic ecosystems. So we're putting that together. Um, and that should be already in probably about two months' time, maybe a little bit less if we're fortunate. But it does all the things we mentioned, but it gives you explainability, it gives you observability, it gives you traceability, gives you audit logs, it gives you trust accounting or sorry, token accounting. You can actually see in real time the tokens that are being consumed by a task or a business process. Um, so so this is a capability that we think there's a there's a huge opportunity for, and we think you know, being able to give back a little bit to the community is probably the right way to bring it to market.

SPEAKER_01 32:14

Of course. My guests today have been John Miller and Eric Broda. John and Eric, thank you so much for being a part of the conflict developer podcast.