Speaker A
You can tell we’re API nerds because we're saying agents should talk to each other through API.
Explore the evolution of agentic infrastructure with Claude, focusing on managed agents, API communication, harness innovation, and complex problem-solving.
Key Takeaways
- Agent communication via APIs is key to scalable, modular agentic infrastructure.
- Reduced model nondeterminism enables more flexible and thinner harnesses for agent workflows.
- Agent identity management and permissioning are crucial for trust and auditability.
- Innovative multi-agent strategies improve problem-solving and workflow efficiency.
- Agentic systems can replicate expert knowledge, accelerating complex industrial processes.
Summary
- Claude Platform has evolved from a simple API for model inference to a rich infrastructure supporting managed agents and complex workflows.
- Agents communicate through APIs, enabling modular and scalable interactions between autonomous systems.
- Agent identity and trust are emerging concepts, with agents requesting access permissions and creating service accounts for auditing.
- Model improvements have reduced nondeterminism, allowing agents to operate with less restrictive harnesses and more flexible workflows.
- Harnesses are evolving to become thinner and more strategic, sometimes involving multiple agents competing or collaborating to solve problems.
- Innovative strategies like advisor models enable agents to call on smarter agents when stuck, enhancing problem-solving capabilities.
- A highlighted use case involves manufacturing facilities using agents to replicate expert knowledge for machinery monitoring and maintenance.
- Security, compliance, and guardrails remain critical for agent deployment in enterprise environments.
- The future of agentic development points toward hybrid strategies, expansive architectures, and possibly operating system-like frameworks for agents.
- Engineering teams are redefined as orchestrators of multiple agents, leveraging cost-effective and scalable AI-driven workflows.
Chapters
- 00:00Introduction to Agent Communication via API
- 00:56Customer Feedback on Managed Agents
- 01:34Agent Identity and Trust Considerations
- 02:14How Agents Interact Through APIs
- 03:12Model Improvements and Reduced Nondeterminism
- 03:52Harness Layer Evolution and Thinning
- 05:21Innovative Multi-Agent Strategies
- 06:03Complex Use Case: Manufacturing Expert Agents
Full Transcript — Download SRT & Markdown
Speaker A
It's been a crazy six months. Because I think if we look back six months, most of what the Claude Platform was, was an API that got you access to inference and tokens.
Speaker A
And out of the model. And sure, we had started to build some interesting tools around the model that could get you more intelligence, or could help you lower your costs or get more speed.
Speaker A
But I think of late we've started to launch some really rich features within the platform that help you get a ton more out of the model, that help take infrastructure problems off your hands, and help take harness engineering problems off your hands,
Speaker A
so that you can really get that intelligence at a much lower cost for your team.
Speaker A
Angela, what has been some of the most interesting feedback patterns you've seen from customers?
Speaker A
I think the most exciting ones have been around managed agents, which has been really awesome, and I know you obviously product manage to manage agents.
Speaker A
But it's been great. I think the feedback has been really awesome, especially around concepts like memory and concepts like outcomes and dreaming.
Speaker A
But I think my favorite one actually is a bit old school, but I did hear a developer who was using it say they just really love the abstraction levels.
Speaker A
And that always warms my heart a little bit. Even as these workflows are evolving, there's still this concept of agent identity.
Speaker A
And so how do we think about the idea of workflows combined with agent identity?
Speaker A
Yeah, I think agent identity will actually probably need to be almost somewhat separate. Like almost the agent needs to have its own identity.
Speaker A
I think today it's still a little bit early. People are still discovering interesting use cases, and there's still a lot of trust that you need to give an agent.
Speaker A
But I think increasingly that agent identity is closer to something where the agent listens to an outcome that you want, and then it probably comes back to you and asks, like, "Hey, in order to accomplish the outcome that you gave me,
Speaker A
I need access to A, B, C, and D," and you may say, "You know, A, B, and C are okay, but I don't want you to touch D," and then the agent is able to go and see if you can accomplish that.
Speaker A
And then when it does do that, it's able to kind of almost create a service account for itself.
Speaker A
And in that world then you can audit it. You can make sure that it's successfully doing the things that you want.
Speaker A
And I think that is probably closer to the operating model we’ll see with the identity layer kind of evolving.
Speaker A
So a lot of what's interesting about how agents can talk to each other is like, you can build an agent and you can expose an API, or you can expose just some mechanism by which another agent can talk to that agent
Speaker A
the same way that a person might interact with that agent. And we've seen people doing some really interesting stuff from this perspective, like some folks have built on Claude managed agents within the platform, and then have built like a nice thin MCP server
Speaker A
that they can go and expose, and then you can have another agent just know exactly how to call in to you talking to that agent.
Speaker A
And so, I think that is extremely helpful. And, you know, just people have been able to do really creative stuff like that.
Speaker A
So, why are these kinds of workflows legitimate now? Like what had to change at the model or at the infrastructure layer for this to be possible?
Speaker A
Models have obviously just gotten better. I think that's a huge part of it, because I think previously you would have had to maybe build a whole ton of scaffolding around the model and standard operating procedure type stuff that make sure that you have,
Speaker A
this stuff happens and then that stuff happens, right? Like the nondeterminism of the model used to be a lot more problematic than it is today.
Speaker A
I think today the models are really great and can actually figure out within some reasonable guardrails, like what are the steps I need to take?
Speaker A
And what are the things that I need to do? I think they've also been able to run for longer.
Speaker A
Like we have the right infrastructure to let an agent be ambient within a workspace, and it can get triggered by something, and then it can go off and run at some workflow for some period of time,
Speaker A
and, you know, come back when it's actually ready. So I think some of this is evolution of the model.
Speaker A
I think some of this is evolution of the infrastructure in the way that people are able to actually stand up and build agents around it.
Speaker A
Angela, I would love to hear your thoughts on how this nondeterminism that Katelyn is talking about has really evolved the harness layer.
Speaker A
Yeah, I think in the past, it just felt like a couple months ago, really, where people would actually create these very complex boxes of business processes that they'd put together.
Speaker A
So first it needs to go through step A, and then step A can only go to B if and only if all this kind of stuff.
Speaker A
And it's a very complex web of things. And it ended up being like very fragile, I think,
Speaker A
in terms of an agent actually being able to exhibit the intelligence that you were hoping it to have, and you were obviously trying to automate something or make it a, you know, a little bit more useful.
Speaker A
And we've kind of tried to box in a lot of the model. As Katelyn mentioned, with the model getting smarter and I think more and more capable, more agentic in its tool calling and also deeper in its own reasoning,
Speaker A
it's actually able to get to a place where you can kind of start to delete some of those, you know, restrictive parts of the harness.
Speaker A
And so I see harnesses actually getting thinner and thinner over time. The other thing I see is that with harnesses getting thinner, you almost have like meta harnesses.
Speaker A
I don't know what to call those things. Maybe they're like saddles or something in the future.
Speaker A
But, I do think they tend to combine strategies. So, for example, we've seen people do really innovative stuff where it's multiple agents, and this is built into the harness in itself, but multiple agents actually compete to go try and solve a problem together.
Speaker A
And there's another instance in which you spin up two of them, and one of them generates an idea, and then the other one's adversarial to it.
Speaker A
And there's so many other strategies you can build on top of it. We recently did advisor strategy as an example, where if the model can't figure out what to do, it actually just goes, reaches out, and calls a friend, and that friend
Speaker A
is hopefully smarter and helps it figure out. So I think things like that are increasingly where harness innovation will go.
Speaker A
And we'll see a lot of incredible ways where agents, through that expansive architecture, are able to solve more and more complex problems.
Speaker A
Yeah. I think this is really getting to the heart of how we think about harness evolution, because ultimately you have all these different composite strategies, and over time, we'll be evolving to having hybrids of these strategies like you could start with the best of end,
Speaker A
sort of like expansive approach. And then once you decide on the right framework, then you can iterate on that single framework.
Speaker A
So you mentioned complex problems. Tell me about a particularly complex problem that has inspired you recently.
Speaker A
There was this one really great hackathon winner that was super inspirational. I believe it's called Urrea.
Speaker A
I might be mispronouncing that, but they, you know, had this problem where inside a bunch of different manufacturing facilities, there is this need for an expert who really understands the machinery.
Speaker A
And then you have to kind of listen to and monitor whether or not certain machines are going up or down.
Speaker A
And then you also have to then read the manuals for like that specific component inside the machine.
Speaker A
And traditionally there's one person in that factory who has figured that out over the course of maybe like 10, 15, 20 years working in that plant, and then they...
Speaker A
And then you just, you don't have that expertise anymore. This person was actually able to take all of these types of pieces and say, okay, if I upload the standard operating procedure, if I build in monitoring, I just attract signals
Speaker A
from all the different types of parts of the plant. Then I can actually have agents try to mimic that human judgment.
Speaker A
And it was really cool to see them actually be able to take a significant proportion of that and then build in what is basically redundancy in, you know, that person retiring and actually being able to return to that agentic system and say, okay,
Speaker A
I can rely on this as a place to accumulate the really important, you know, factory knowledge that's necessary.
Speaker A
What about you, Katelyn? What's an interesting problem you've seen solved recently? Yeah. I think one of the cool things that's happening within engineering team specifically, or not even actually engineering teams just like development teams that are trying to get work done,
Speaker A
is people are coming up with agents that are just more powerful than just, I can get some code written, like we have excellent like, Claude Code is an excellent product, a whole bunch of excellent products around, how can you actually write code?
Speaker A
And I'm seeing people now go further than that and say, okay, if I'm starting from the very beginning of a project, what are all the things that are going to have to happen in order for that work to get done?
Speaker A
How do I need to think about running my development environment to actually test the code that I'm running?
Speaker A
Right? Like, How do I, you know, actually write the PR up front, like the requirements documents and then like later on verify all the QA testing sort of things that need to happen.
Speaker A
And so there's a few examples out there right now where people, in larger companies have actually put together full agenda systems and platforms.
Speaker A
That help them do pretty custom end to end development. And I think a really good example is Shopify recently talked about doing this with, I think they called it River.
Speaker A
And, you know, there's been a few other examples like this. And I think this is one of the evolutions that has been made possible by just the evolution of what agents are capable of.
Speaker A
So it's clear that some organizations are seeing a massive amount of ceiling raising on the problems that they're able to solve.
Speaker A
But not all organizations are feeling that. So what do you think is the biggest barrier to that?
Speaker A
Actual boundaries, like the things that can get in the way for people. Security and compliance guardrails that they need to have in place in order to feel comfortable, like having agents do the work that they're able to do.
Speaker A
Evals, I think is a big one, too. In order to actually get the most out of the technology.
Speaker A
Security and compliance is definitely a big one here, because I think that a lot of teams are operating on, security assumptions that prevailed, you know, 20 years ago.
Speaker A
And now agents are fundamentally changing everything. And along with that comes security. So I think that we have a pretty fundamental idea of what makes a safe agent with strong guardrails.
Speaker A
I think pushing that concept uphill and revising the checklist, has been a bit of a, a bit of a journey.
Speaker A
Yes. Yeah. And Curious for how you think about ROI of agents and how enterprises can manage that?
Speaker A
Yeah, I think it's a very, top of mind question for a ton of different companies right now as they look at how everything is accelerating and how they can accelerate themselves.
Speaker A
I think, you know, there's a couple different ways to think about this, but I tried to kind of encourage a slightly more like simplified mental model.
Speaker A
I think oftentimes people want to jump to like, how do I agentify gigantic old school processes that I have, and maybe they have 120 of them and they want to go down the laundry list or they're like, great, this is my moment.
Speaker A
If consumer preferences are changing to fundamentally transform our entire product service. And I think all those things are great, but it's obviously very hard to go through all of those.
Speaker A
And there's a very, very wide array of things there. I do think that from an ROI point of view, it's easier to almost start from the individual, actually, and to be focused on how much faster that individual is getting.
Speaker A
And it sounds really almost like simple and kind of, you know, like not sophisticated enough.
Speaker A
But I actually think if you start there and you are able to accelerate that one individual, you can move from that to then a team.
Speaker A
And as you get to the team level, you start to think about the speed that you're generating there.
Speaker A
And then, and probably only then I would actually start then to encourage say, okay, if I can make this team faster, now what is my ability to take this team and think through a process across my company?
Speaker A
And realistically, a process typically requires many different teams and many different individuals who don't share standard operating procedures, who don't share maybe the exact same expertise.
Speaker A
And that's why we have all them together probably, you know, duct taping a bunch of things together.
Speaker A
But if you go through that kind of phase where you think about speed at the individual layer, team and their productivity, and then lastly followed by how you're able to string these workflows together.
Speaker A
I think you're able as a company, to basically almost light up every single bar that you eventually care about, so you can eventually make your way to that kind of list of 120 workflows that you wish you could be agentifying.
Speaker A
And I think if you think about it from that point of view, most of your ROI calculations should probably be on speed first and foremost, and productivity.
Speaker A
It tends to be much more leading and successful as the primary mechanism to take a look at as that starts to, I think, flower a little bit inside of a company, then it makes more sense to kind of transition a bit to be like,
Speaker A
okay well you know, maybe there are financial metrics that I wanted to push. Maybe there are, user metrics that will help drive the overall like outcome productivity that I care about.
Speaker A
And I think if leaders and companies are able to follow that kind of process.
Speaker A
They're actually able to drive more of that ROI. And see, it's stage by stage.
Speaker A
Katelyn, you keep saying engineering teams and I want to double click on that and say what is an engineering team anymore?
Speaker A
So engineering team for us is a set of humans that actually doesn't look that different than an engineering team looked 6 months ago, 12 months ago.
Speaker A
You still kind of need that set of humans to understand the system that they are building, how to operate it, how to be on call when something's going wrong, all of these sorts of things.
Speaker A
But each of those humans is just insanely turbo charged by agents that can help them get their work done.
Speaker A
And so what we're kind of seeing is a bit of a shift from an engineering team that was maybe, like one technical lead who's got, you know, opinions on how we should design the system, right?
Speaker A
And then a whole bunch of engineers who are, you know, picking up tickets and getting work done to almost the whole team are people who have strong opinions on here's how we end to end build a product or build a system.
Speaker A
Here's what the technical design needs to look like. And then they're kind of orchestrating their Claudes, for lack of a better term, to get the work done.
Speaker A
And so, for us, our engineering teams actually look pretty similar. But they're able to just get so much more work done, than they were in the past.
Speaker A
I'm curious how you think about potential failure modes of reliance on agents in an organization?
Speaker A
I do think it does create a sense of like, hyper independence in a way that is maybe slightly false.
Speaker A
Like, you think that. Of course, now everyone's a builder, so I can go and build and yes, I can like spin up like ten prototypes.
Speaker A
And for you, normally you'd be like, well, which of these options is the best option?
Speaker A
You do a little bit thinking you now have the ability to be like, well, this is cheap.
Speaker A
Like, why don't I just like launch all ten and then whichever one's the winner is the one that they will pick.
Speaker A
And so I do think it creates this like hyper independence on every single person.
Speaker A
But oftentimes, you know, I think the quality that comes together from something that's like more systematic, more holistic tends to then be a little bit harder to coordinate.
Speaker A
So I do think that's a little bit of the failure mode is like if by giving everyone hyper independence, but not necessarily organizing them together to a concrete direction, you might see like sprawl that looks a little bit like this.
Speaker A
And that can have pros and cons, but I think there's definitely failure modes that result from things like that.
Speaker A
So where do you think the puck is going? What is the future of agentic development?
Speaker A
I think it's like deeply embedded inside the organization to a point where you probably don't really use tools in the way that's like so obviously instantiated.
Speaker A
And what I mean by that is you know, today we all are reaching for this agentic tool or that agentic tool.
Speaker A
And this one's good at this and that's good at that. And I think in the future it'll be closer to like maybe there's some common substrate where we all kind of engage.
Speaker A
It's like familiar interfaces and everything like that. But you kind of are just able to kind of tag an agent, spin it up and down as you see fit, and it just does a lot of like work by itself invisibly.
Speaker A
And so and it might even actually be proactive. Maybe it's the one coming to you and saying, hey, you know, we noticed this thing went down.
Speaker A
And so I dug into the details and I figured it out and I fixed it.
Speaker A
And here's the PR maybe you want to review it. And maybe you even told it, like next time in the future for things so small, don't bother me.
Speaker A
Just go ahead and, like, ship it. And I think it'll look a little bit like that where it almost feels like, almost like an invisible like substrate that you engage with.
Speaker A
And in that world, I think that every person ends up being at a place where they almost kind of like, are able to build kind of team oriented agents, but maybe not in the sense of like it's like another team mate,
Speaker A
but more that like the team orientation of some workflow that say, you and I have there's an agent for that, and it's able to actually map to the kind of preferences the two of us have been in.
Speaker A
The three of us have a team based agent and it like, understands a slightly different preference, and it starts to kind of fill in a lot of the gaps that are necessary.
Speaker A
And again, I think still have the common interfaces and things we have, but we might see it a bit more of as like an operating system or, or something like that instead of, specific tools that we actively reach out for.
Speaker A
And how do you think Claude Platform will help us get to that vision? Yeah, I think one of the bigger things that we've been kind of trying to push the boundaries on recently are some concepts, like outcomes is a big one.
Speaker A
Like we shipped in Claude managed angents this idea of an outcome where you tell Claude, like, here's what good looks like.
Speaker A
Give it a rubric. How many times can it iterate to go and try and get that outcome successfully before it stops, right?
Speaker A
And I think as we evolve that concept, we'll probably get closer and closer to a world where you're talking to Claude and you're saying, I want this outcome and here's a budget.
Speaker A
Go. You know, and you don't really have to think beyond that. And I think the idea of what we're trying to get out with the platform is make that so easy.
Speaker A
And being in a world where you're spinning up an agent every day because you're like, today, I have to summarize some interview notes and put together a packet of feedback, right?
Speaker A
Or something like that. You can say, okay, great agents, I want notes look good, looks like this.
Speaker A
And you can spend this amount. Go. Right? And then you'll get back what you want and making it so that you don't have to work harder, think hard to actually create that, is the gap that we're trying to fill.
Speaker A
Making that super easy. This has been so much fun talking about the future of agents.
Speaker A
Thanks so much, guys.
Topics:Claude Platformagentic infrastructuremanaged agentsAPI communicationagent identitymodel nondeterminismharness innovationmulti-agent systemsAI workflowsenterprise AI
