Multi-Agent Interaction with Sierra AI

Discussion on multi-agent interaction, agent capabilities, and building real-world AI agents by Karthik from Sierra AI at Greylock's Change Agents series.

Ask about this video. Answers come from its transcript only — with the timestamp, so you can check them.

Generated from the transcript and can be wrong — check the timestamp.

Key Takeaways

Multi-agent interaction is fundamental for advanced AI and AGI development.
Agents combine language understanding with decision-making to operate autonomously in complex environments.
Real-world applications like customer support benefit greatly from AI agents that interact and act in partially observable settings.
A structured framework helps evaluate and improve agent capabilities progressively.
Building reliable, scalable agents requires balancing research innovation and practical deployment challenges.

What the video covers

Multi-agent interaction is essential for achieving AGI, requiring agents to handle complex communication and collaboration.
Karthik, head of research at Sierra and Princeton professor, shares his decade-long experience in AI focusing on NLP, reinforcement learning, and agents.
The concept of agents dates back to early AI, defined as entities that can affect change in their environment.
Sierra AI focuses on building real-world agents for enterprise use cases, especially customer support automation.
Customer support is a prime example where agents must understand language, communicate effectively, and take actions that affect real-world states.
A three-part framework for agent capabilities includes reasoning and acting, learning new skills via language, and controlled interaction with guardrails.
The synergy between language understanding and decision-making is key to advancing autonomous agents.
Research at Sierra balances developing reliable, high-performing agents and making agent development easier.
The conversation covers current agent capabilities, real-world agent building challenges, and future directions in agent technology.
The session is part of Greylock's Change Agents series, which explores building agentic software through expert discussions.

Chapters

What is the significance of multi-agent interaction in AI?

Multi-agent interaction is crucial for achieving AGI because agents need to communicate, collaborate, and operate effectively in complex environments, reflecting real-world intelligence.

What kind of problems is Sierra AI focusing on?

Sierra AI is building real-world autonomous agents primarily for enterprise applications, starting with customer support, where agents must understand language and take actions that impact real-world states.

What are the key capabilities of modern AI agents according to the framework discussed?

The framework includes agents that can reason and act, learn new skills through language, and be controlled with natural language while maintaining safety and reliability.

Full Transcript — Download SRT & Markdown

Speaker A

I mean, if you take a really, really high-level view, multi-agent interaction is basically essential for AGI, right? We're all doing multi-agent interactions now, right? Um, we're all agents, right? So, in one sense, if you want agents to be close to this level of intelligence, they need to be able to handle multi-agent interactions, right? Well, thank you everyone for coming. Uh, as many of you know, this is actually the second session of Greylock's Change Agents.

Speaker A

Change Agents is a new series that we're hosting that explores what it means to be building agentic software, and we're doing that through conversations with researchers and CEOs who really are living these problems every single day. Uh, I'm Karin Riley. I'm one of the partners here at Greylock, and I focus on early-stage B2B, uh, enterprise infrastructure and applications. And, uh, tonight I'm joined by Karthik. Uh, Karthik is the head of research at Sierra and associate professor of computer science at Princeton. Uh, I'll let him give a more wholesome introduction of himself, but yeah, welcome to Change Agents.

Speaker A

Oh, uh, thanks Karin, and thanks for having me here. Um, hi everyone. I'm Karthik. Uh, like Karin said, uh, I lead the research team at Sierra. I'm also an associate professor at Princeton. Um, short intro is I've been working, um, in this field that we call agents for quite a long while. Um, I'm really excited by the potential of agents to really impact, you know, real world, um, and help us solve real world problems. Um, a bit

Speaker A

of background, uh, I think before we call it agent—I mean, we're calling it agents now, but like most of my work in a technical sense has been around the fields of NLP, natural language processing, reinforcement learning. Uh, so

Speaker A

I've been doing that, uh, both at, uh, my time as a faculty member at Princeton, uh, and before that as well, um, where I had a short stint at OpenAI for a year where I was lucky enough to be involved

Speaker A

with, uh, the very first GBD model. Perfect. Perfect. And, uh, and actually that perfectly goes into the first question, which is, uh, you know, like you said, you've been working in this area for about 10 years, and now we all use

Speaker A

the word agent every single day, and if you open LinkedIn, every single post is about agents. Uh, but, you know, if you read back to your research back to 2015 and everything that you've done until now, a lot of it is around this concept.

Speaker A

So maybe actually walk us through a little bit more detail on the work that you've done and then actually why don't you give everyone—we have some people from Sierra in the crowd—but actually give us a description of Sierra and

Speaker A

what you do there. Uh, for sure. Yeah. Um, if you want to start with agents, uh, I think, uh, I mean many people here probably know it's sort of like you can go back all the way to the beginning of AI in some sense as

Speaker A

as, uh, where, you know, the term agent was coined, right? So an agent is not just an AI agent, like all of us are agents, right? So an agent is basically something that can affect change, um, in its environment,

Speaker A

right? Um, and so the way I tend to look at it is like, you know, when the field started, uh, this was a dream, right? Build autonomous agents that can accomplish things in the world where there's physical, virtual, whatever. And, um, we

Speaker A

sort of realized very quickly, um, you know, through the 60s and 70s and 80s that this is a very hard problem, right?

Speaker A

Um, and so we split up into these different fields, you know, computer vision, natural language processing, RL, and all of that. And for me it was always like right from, um, the time I was introduced to AI and NLP and RL, it

Speaker A

always seemed like these are subfields working towards the same thing but just trying to solve parts of this holistic problem, right? And, uh, it's super exciting now because, you know, we have language models that do really well solve

Speaker A

problems that have been longstanding in NLP. Um, we have, uh, agents that, you know, that from the reinforcement learning sense are also like working really well. So, you know, we can work towards like combining these two and, and

Speaker A

and, uh, uh, have agents that both understand language, can communicate with humans and all of that, um, but also actually do things in the real world.

Speaker A

And so, a part of this is what, um, uh, we're trying to do at Sierra, right? So, for those of you who didn't, don't know, um, at Sierra, we're trying to build, uh, real world agents for the enterprise. Uh,

Speaker A

starting with customer support as, as like one vertical. Um, but you can see it's a tailor-made problem for AI agents to really help us automate parts of this, right? So, um, in customer service you're basically having these interactions

Speaker A

where, you know, you need to communicate live with the human who is sharing information with you often in a partially observable kind of setting, right? You, you don't get all the information in one shot, so you have to

Speaker A

you know, have multiple rounds of communication, um, and at the same time you have to figure out what actions need to be done on the back end, whether it's like changing your flight reservation or, you know, walking through a radio refresh or

Speaker A

whatever, uh, it is, right? So, um, it has this nice mix of these two things, right? You need lang—you need to understand language, you need to process language, you need to generate language, uh, but at the same time you need to take

Speaker A

actions that affect, um, the state of the world in some sense. Perfect. Yeah, and so today we're going to divide our conversation into three broad topics. Uh, the first is just agent capabilities: what can agents do well today, and you have a really great

Speaker A

framework that you've spoken about in the past about this. The second is, you know, building agents in the real world and what that looks like. And lastly is, you know, looking at the frontier of agents, what you see coming in the

Speaker A

future. And so, uh, starting with agent capabilities, you have this framework that you've spoken about in the past of, you know, we're shifting from language models to language-enabled agents. And, you know, you talked about a sequencing of key capabilities into three parts.

Speaker A

Part one being, you know, we have agents, language agents that can reason and act, and that ties closely to your REACT paper. Uh, part two is agents that can learn new skills through language in addition to just doing and

Speaker A

experimenting. And then part three is, you know, the point where we can actually control agents with natural language and have really great, you know, guardrails around them. And so, you know, walk us through this three-part framework that

Speaker A

you came up with and, you know, why are these the dimensions that you think matter?

Speaker A

Um, I would just add like it's not something that I came up with necessarily. I mean, a lot of other people have also, like, thought of this.

Speaker A

Um, I think this ties back to what I said, um, just now, right? Um, the goal of these different subfields of AI have always been to, like, move towards autonomous agents. And for me, it's always been, um, uh, useful to think of it

Speaker A

as like a synergy between, you know, language capabilities and decision-making capabilities. So, um, that sort of plays into this, right? So, you know, when much of the field is thinking about, you know, NLP has a lot of different problems you

Speaker A

can think of like question answering, information extraction, all of that, but the end goal for all of that is to be part of a larger system that actually does things, right, in the world in some sense. Um, and so the three-part

Speaker A

framework you can think of as like at the very core, um, you want agents to, you know, uh, you want basically these systems—let's not, let's not use agents for a bit, right? Autonomous systems to both use language and be able to take actions, and

Speaker A

so that means leveraging the synergy here, right? So, in reinforcement learning we've often had, you know, this concept of taking actions, getting feedback, getting rewards, and then like taking better actions, learning from that whole thing, but language provides us a a

Speaker A

mechanism for reasoning or thought, right? I mean, that'

Speaker A

can think of like question answering information extraction all of that but the the end goal for all of that is to be part of a larger system that actually does things right in the in the world in some sense. Um and so the three-part

Speaker A

framework you can think of as like at the very core um you want agents to you know uh you want basically these systems let's not let's not use agents for a bit right autonomous systems to both use language and be able to take actions and

Speaker A

so that means leveraging the synergy here right so in in reinforcement learning we've often had you know this concept of taking actions getting feedback getting rewards and then like taking better actions learning from that whole thing but language provides us a a

Speaker A

mechanism for reasoning or thought, right? I mean, that's just one mechanism. You can have argue other mechanisms exist, right? Um and so reasoning and acting seem like a you know, great fit up uh straight up there, right? So, um uh using language as a

Speaker A

mechanism for just like writing down your thoughts, reasoning through it, all that. This is something intuitively we do as humans, right? Um and that's that was sort of like the basis of for for for the React um kind of uh framework.

Speaker A

Uh the other two parts are actually uh something that's uh quite close to my heart. Um uh the part about learning from reading is something that um we do a lot, right? So think about like you know we were born um you know as as

Speaker A

children there's so much that you need to learn. A part of what we learn is by just trial and error, right? And this is the basis of reinforcement learning.

Speaker A

Like you try things like I mean you you try to hit the tennis ball with your racket like keep trying, keep trying.

Speaker A

first 10,000 times you're bad and then you know you start to get better and so on. And so that's the basic premise of reinforcement learning. But if you really think about learning um uh efficiency of like how we learn as

Speaker A

humans, if we only did trial and error um we would never be where we are right now, right? Um so there's also this component of learning through just reading about things, right? So and this is how we sort of share experiences

Speaker A

amongst each other um through generations and all of that. And ideally for a very intelligent agent, it should have the same capability, right? So, uh, in this tennis example, this would be something like, you know, you have a

Speaker A

coach that looks at how you're playing and tells you, hey, maybe hold the racket up, you know, a little bit further up or something like that, right? Um, and so that sort of takes away lots of cycles with this trial and

Speaker A

error process. Um, and so that's that's something that I think like we need to have agents agents, AI agents do the same thing. And um that's more within our grasp right now because language models are just like so good at

Speaker A

processing text right now and a majority of our world's knowledge is is on the internet in the form of text right and videos and so on but still a lot of it is in text um and then the third part um

Speaker A

which is again like I mean this is something that's not new right so we all want to be able to as humans uh as a human race we want to be able to like develop these agents but we also want

Speaker A

them to do things for us and for that like our natural mechanism of communication is through you know languages. Um, and so we'd want to be able to um, express whatever we want these agents to do in in free form

Speaker A

natural language. Um, not have to code it up in very very specific forms, right? This just leads to more natural interactions.

Speaker A

And if you think about one and two in particular, right? Uh, this is a purposely difficult question, right? if you were to give us a score of where we're at today uh on our capabilities on one right to do that synergy between

Speaker A

between reasoning and action and then also on part two which is to understand based on language like where where would our c ability in agents score today um it is a hard question um I I and I think it depends on the task

Speaker A

okay uh if I were to just give like one number just for fun right um I would say on number one the ability to reason and act reliably um maybe like 50 out of 100. Um but the one about like learning um from reading

Speaker A

and and and being able to do that very fast, maybe very lower like 30 or 20. Uh there are probably some tasks where like you can quickly get it and you know and be able to um uh assimilate that but

Speaker A

it's it's hard to say. And and what's going to bridge that gap? Um that's a good point. Um I I think for the reasoning part um we're we're definitely on a trajectory towards making these models um smarter and and be able to like um navigate

Speaker A

enough of the uncertainty in the way that language models decode and all of that. Um I would suspect like I mean part of this answer goes towards like you know better data and and and uh how we train the models and all of that. But

Speaker A

actually there's a there's an interesting part that I think could be even more important is just how we structure the environment um to be able to uh facilitate this kind of learning right and so uh just as a highle summary

Speaker A

of that thought is basically you know we've had notions of environment and agents and all of that in the in the field of reinforcement learning for a very very long time and I think a lot of those notions can

Speaker A

be challenged or revisited simply because we're in a totally different paradigm to where we were in the 80s or 90s.

Speaker A

Okay. And that's why you're working on working on that at Sierra totally. So, so let's go into the second core topic which is you know building agents in the real world and uh you know you often talk about you know the eval of

Speaker A

agent being one of one of the core bottlenecks right to getting something in production and one of the most immediate challenges and uh you know actually if we think about the two most you know cited or even used use cases

Speaker A

for agents today people always think about customer support and then they think about coding right and actually you've worked on the two probably most important benchmarks in those areas Right. So, Towbench for Sierra uh in the customer support space and then uh

Speaker A

Sweetbench as well and coding agents. So, maybe you let's start with TWBench actually. Uh walk us through that right and how how does it work for Sierra and obviously it's been used even past Sierra and I think it's been cited in

Speaker A

anthropics work as well. Yeah, walk us through your work there. Um yeah, I I would say uh this comes back to the point I made about environments, right? So uh like like like I said before you know at Sierra

Speaker A

we're building agents for u the specific use case at least initially of of customer support and so the way that Towbench came about was like as you know like um in order to build good agents you always wanted ways to measure their

Speaker A

capabilities really well and there's been a ton of work in the space of you know dialog systems as well as like um agents for you know web based tasks you know web shopping or whatnot And so what we were trying to do was was

Speaker A

really come up with good systems to benchmark the agents that we're building both internally but also like broadly for the space um in an academic sense.

Speaker A

And um this was an interesting story because like uh what we realized was like while there exists both these spheres you know dialog systems and web agents and all of that there wasn't something that really was in this

Speaker A

missing part or gap in between um where you sort of want multi-turn um dialogue but you also want to be able to like take specific actions um and uh the information is revealed to you in sort of like this like partially observable

Speaker A

adversarial setting. because sometimes you know as you know customers don't want to tell you everything right they just want the issue solved in like two seconds um so that kind of led to uh us creating this benchmark and it's I think

Speaker A

it's also the right timing because you know language models just are so good right now that you can do things like simulating the user in a much more realistic fashion um as opposed to how it was like even like five years ago.

Speaker A

Yeah. Yeah. and and like do you envision let's say I'm starting a company I'm in the audience I'm starting a company today not in the customer support space uh you know obviously it's case by case dependent but like do you envision a

Speaker A

world where like companies like Sierra need to build their own benchmarks continuously or like what do you think evolves looks like in the gold standard you know steadystate world um I think for any uh systematic methodology that you want for improving

Speaker A

your agents you do need to have some kind of a benchmark or some kind of a metric to measure against right so in that sense yes you would definitely have to build evals but I think eval is maybe

Speaker A

the is is a sort of uh vague term for this I would actually say like focusing more on the um the environment in which you want your agent to act um because uh uh that is a little bit more than just like eval

Speaker A

right because you want to also think about you know the the the actions that your agent can take um which are called affordances right? Um or you know the the signals you want to feed it the you know which are the observations in some

Speaker A

sense and then how you also determine the reward function in some sense. So you want a combination of all those and then you also obviously as I mentioned worked on SWEBench and uh you know I think any of us following the the

Speaker A

coding world a lot of times SweetBench gets gets cited as right like a criteria that you know you're really marking yourself against. Yeah. What do you think OB and a lot has changed also since SweetBench was first released. Uh

Speaker A

yeah, what do you think of what looks like the future of eval particularly in the coding space and uh you know if you were to do SweetBench 2.0 uh today how would it be different?

Speaker A

Um yeah, I think coding is a very interesting use case, right? Um we've had um just to recount the story of SUBEN, uh we we've had like lots of benchmarks for for coding, right? Um and many of them focused on the task of

Speaker A

actually just producing code, right? So you have a you sort of have a prompt how well can you write this function that reverses a palendrome or something whatever, right? You you have this particular setup. But if you take the

Speaker A

task of software engineering, software engineering is much more than coding, right? In fact, you spend probably like 20% of your time writing the code, 80% debugging it and doing all the other things. Um and so that's how the idea

Speaker A

for Sweet Bench came about like you know being able to say um can you can we test these models um and agents later on um at their ability to take a piece of codebase and then take an issue and make

Speaker A

the surgical change to this codebase based on whatever the issue the bug or something whatever you have uh is telling you to do right um I think that was a good start now almost two years ago right um and I think we've made a

Speaker A

lot progress which is great to see right so initially those models were less than 10% I think on this but now we have close to 70%. Um, and I think that's great, right? In in terms of like making

Speaker A

these models more applicable for the real world and like uh we're already seeing, you know, companies like Kodium and cursor and all these like benefiting from this. Uh, as for a we think we should just pull on this thread

Speaker A

a little bit more, right? So thinking about what are the different aspects of the life cycle of a you know type um uh role would look like right um you could imagine like going up levels of abstraction right so fixing a bug still

Speaker A

seems like a one surgical thing what if you could take 100 bugs and figure out what are the three factors necessary to do something right um so there's lots of ideas like I think you can go levels of

Speaker A

abstraction you can think of different modalities there was a follow-up um that was more on like multimodal sweet bench and all of that Um uh so yeah there's a lot to be done but I think like it's just you know once we have this kernel

Speaker A

we can go in different ways. Yeah. Yeah. And uh so let's talk about the second part of of uh you know building in real life here is is the orchestration side of things. So there's a number of of frameworks available open

Speaker A

source that you were for purchase. You know how do you think about how you connect all this together and and I'll leave that open-ended and then we'll go into more details. I think the frameworks are definitely useful to

Speaker A

start thinking about how um you can connect the different components you want in your agent. Uh but I think the bigger question just you know comes back to um how you how you sort of structure your environment and how you want the agent

Speaker A

to operate in that environment. So what I mean here is uh you know like I said like the things like the small things like you know determining the affordances or uh the observations that the agent take are probably more

Speaker A

important in my in my mind than the actual um framework you use to like orchestrate them in the sense that um you are I mean most of the open source tools available right now are basically doing similar things right um it's it's it's

Speaker A

sort of like just like an abstraction layer that provides you a little bit more um ease of you know writing the code to define the agent but the real real thing that you really want is you know um something deeper

Speaker A

beyond that right so uh how do you how do you make sure that you you have that right balance of control uh but also you don't want to work too hard to define right so you want that and and actually this is the exactly the

Speaker A

next question which I was calling it like budgeting agents right and so you know when we think about um orchestration specifically You know, our understanding is, you know, an agent has the capability to keep on iterating on a task until it

Speaker A

gets a completion and finds a result. Now, we've seen in products, you know, this capability to kind of like budget an allowance for how much you should think, right? And really set a cap on what the agent should be doing or how

Speaker A

much it should be iterating. And I think even Enthropic released like the think tool so you can exactly tell it when to stop and think about what it's doing.

Speaker A

Now, how do you think about whether that works? And then also as it relates to a customer, right, like you said, often you're dealing with a frustrated customer. How does that impact like the the UX from a from a customer using your

Speaker A

agent perspective? Yeah, from a I think from a technical perspective, uh you can always look at the results around like you know test time scaling and um uh the ability to use like trade off I guess more compute

Speaker A

at test time for basically more um accurate or reliable answers. It it depends on really like where we put the test time compute on, right?

Speaker A

It's um uh the in in the paradigm. I think there's still like a lot of headroom for us like we've still not like fully exhausted it. But you can also imagine right at some point um just thinking more doesn't solve your

Speaker A

problem. Um either it's it's an issue with like the information you have or you know your the actions you can take or something like that. Um, so, uh, I I think this is a trade-off to be had and

Speaker A

in different domains like you'll have to think about this, right? There are costs associated just like you said, you know, like it might increase your latency or um the cost of what you want to do. Uh, but one thing for sure, I think like um

Speaker A

there there there's a lot of research to be done in terms of like how do you actually like distill those behaviors back into ways that don't affect those those different parallels. Do you find that customers want that level of granularity or are we

Speaker A

still far away from them even wanting to kind of determine how much the agent thinks or how much it iterates?

Speaker A

Oh, um that's yeah I I I don't know if that should be a knob that the customer decides or not. That's a different Yeah, that's a different question. Um but uh it could be. Yeah, could be. Um so actually let's connect

Speaker A

this back to to Sierra, right? And so if I think about uh simplistically a customer support workflow, right? I think of you know the old way of doing it with standard standardized SOPs with uh you know tree decision- making. And

Speaker A

so if I think about that, I can easily imagine a world where you can uh you know chain some prompts together and have an LLM do that. So why is it that agentic reasoning is useful to Sierra or

Speaker A

to customer support in in in general? Um I I think there's uh there's maybe two reasons for that. Um so one is around uh you know you could you could hook up you know some some kind of a a loop and and and have build a quick

Speaker A

demo right so but but one of the key challenges I would say is like the reliability of an agent is um in the real world is very different from building a demo quickly right um and I mean TaBbench was like the the

Speaker A

whole point of Towbench is actually to showcase that you know in one of the dimensions and I think part of this comes down to LLMs being not fully deterministic systems, right? And they're not um they're they're still not

Speaker A

at that level where you know all the different answers that you generate from this proistic inference are actually correct, right? So there's there's still like lots of paths where you know even a single token can lead them astray and

Speaker A

all that. So I think there's this like one issue of how do you ensure um reliability and repeated accuracy, right? um that's not easy if you just like build it with like a loop and you know add some tools which is exactly

Speaker A

what we show in TBEN and the other part I would say is like at the same time you don't want to just like you said you know you could try to manually write down the whole SA SOP in some sort of

Speaker A

decision tree but that just goes back all the way to like what we had in like shrewloo and all these old systems right so uh this would just be like too much work to maintain and um and even just

Speaker A

like build an agent that that can handle all these nuanced you know tail cases.

Speaker A

So you want you sort of want this right balance between like ease of use and you know specification. How do you how do you basically build the agent very quickly and um at the same time like have it reliable enough and I think

Speaker A

that's what we're doing at Sierra like having this right combination um such that it's it's it's a delightful experience with a customer but it's also like a very very scalable reliable way of building agents and and and actually yeah that uh goes

Speaker A

perfectly into how do you make it customizable for agents and and and maybe even just abstracting that even further if you think about the way a gentic products can and should work right um should they be customized to

Speaker A

every single customer or do you think that there's like a generalizability that we can achieve such that the next you know 100 Sierra customers can really be adopting this in a very similar way?

Speaker A

Yeah, that's a good point. Uh we do have uh something that we call the agent SDK or agent OS um which has these modular components that can be easily comp composed in in in different ways for for building your agents. That's exactly

Speaker A

part of the part of the thing that we uh make uh to to ensure that's like it's not like a tedious process where like you're building an agent but at the same time you get you get that guaranteed

Speaker A

level of reliability that that you want in the in the actual agent. Great. So let's go to the last topic which is uh kind of frontier regions and uh you know in this topic in particular one of the things I want to talk about

Speaker A

is is multi-agent interaction and this is something that you've spoken about in your lectures prior and people bring up all the time I actually was texting a friend of mine who um you know works at a very large tech company that we would

Speaker A

all know and they say you know our customers ask us all the time if we have multi-agent interaction available and then you know you press them on why they exactly want it and and you know there's no real reason, right? But it's

Speaker A

something that people have talked about enough that, you know, it's it's in the common lingo of customers. Now, if we asked probably everyone in this room of how they would define multi-agent interaction, I bet we would have, you

Speaker A

know, 40 different responses. And so, maybe turning it over to you, like what do you think multi-agent interaction is?

Speaker A

Let's start with that and then we can go deeper. Um I mean if you take a really really high level view multi- aent interaction is basically like essential for AGI right um we're all doing multi- aent interactions now right um we're all

Speaker A

agents right so you so in in in one sense like if you want agents to be close to this level of intelligence they need to be able to handle multi-agent interactions right um but to be a go down bit more you know concrete right I

Speaker A

think of it as like the you always have want to have some kind of asymmetry um between different agents for it to be truly multi-agent right so if it's just like you take um um I mean you could

Speaker A

even consider the prompt of two different agents to be you know different but that's like it's a little bit shallow right so you basically have the same model underlying and you just like prompted them in different ways um

Speaker A

technically you can still call that multi- aent but it just seems a little bit a lot of people are I think yeah um I think cases where uh it could make more sense are you know there's like some kind of knowledge as symmetry.

Speaker A

Let's say you know you have two robots and you know one is in one corner of the world gathering some input sensors and one is in the other corner and they do something or in the more like less

Speaker A

physical sense you could even imagine you know um places where information is just like sort of guarded due to privacy concerns or let's say like different companies have their own siloed information but you want agents between them to be interacting with each other.

Speaker A

That's when like you truly need, you know, the protocols for agents to communicate and like figure out how they want to work together and all of that.

Speaker A

Um I think we'll start seeing some of these soon these use cases. But um why do you say that? Yeah. What or what use cases are on the horizon? Like what makes you believe that?

Speaker A

Oh, I I I just say this from a um uh from a more um high level overview point. The fact that we're all like, you know, we're living in a multi- aent simulation ourselves. Um, and so if you

Speaker A

want agents to to act in the world and be together, work together with humans, um, they would also have to work together with each other. And so you talked about, you know, um, um, agents taking on suite tasks or coding tasks,

Speaker A

right? Or agents performing customer support. At some point, you'll have these different functions needing to communicate with each other, whether within a company or across companies and all of that. And so I don't think we can avoid it. Um and

Speaker A

and it has to be like on the path to AGI and and you know what needs to happen to get there or even maybe even more granularly where is it going to come out of like is that something that you know

Speaker A

your team at Sierra can work on and get done very well in your context or is that something that needs to come out truly from the model layer?

Speaker A

um which part the the need for the ability the ability for these agents to be communicating and acting uh in in tandem.

Speaker A

Yeah, I think it shouldn't be a forced um sort of thing like you mean you should have a clear reason for having a multi- aent setup, right? And uh just like I said, I mean there there's likely some sort of asymmetry whether it's like

Speaker A

skills or information or something that enables it. Um if it seems forced then you're probably better off like having a single agent.

Speaker A

You should probably not have it in the first place. That's just my Yeah. Uh are there are there specific use cases at uh Sierra or in customer support more broadly that you think are valuable for multi- aent?

Speaker A

Um, I think the the probably the not specifically for Sierra, but like probably the the use case I can think of that that would likely be soon true enough is um true soon enough is that um you know you just have information

Speaker A

that's like guarded between behind walls and you you have to have some way I would say in some sense like you can you can argue uh protocols like MCP are moving towards that you know you want to hide some information away. Um at Sierra

Speaker A

specifically like without going too much into detail we do have some systems that that sort of do this guarding like you know in terms of guard railing the whole uh process and making sure the agent doesn't do things you don't want it to

Speaker A

do right so you could you could argue that that's part of that framework and then uh I'm going to ask one parting question and then uh I'm going to open it to the group if there's any questions from the group uh please speak up. Um my

Speaker A

parting question is is uh you know you know you've had an incredible career both in academia and and in industry now uh you know if you were advising you know a younger student on what they should go research like what they should

Speaker A

focus you know their their academic life on what would that be and then I'm going to ask you the same question of what you're excited about in industry.

Speaker A

Um okay so first one is on like working in academia. I think there's there's a t like there's a ton of things. I think the high level point is to embrace the current state of technology and to think

Speaker A

forward a little bit, right? Um so to give you an example, I think one of the things where u I think there's still a lot to be done is uh in thinking about what is the next um thinking about continuously evolving or

Speaker A

learning agents in general. Right? So I mean right now what we're building um uh are basically you know you you have some kind of a underlying model and you're orchestrating it to do you know automate some set of tasks but how do we think

Speaker A

about these as continuously evolving and autonomously learning on their own over time right and so this is where like like I said before right so notions of things like reinforcement learning that we've built over decades might need to

Speaker A

be reinvented or even rethought of and in fact like um uh I think David Silver and Rich Sutton had a paper recently that talks about a little bit on this like era of experience, right? I think that's that's along the same path like

Speaker A

you know um I think this is a this is a very fascinating topic and honestly like there's so many different ways you can attack it and I think that's ripe for uh something you can look at. Um from the

Speaker A

industry standpoint I think uh there's just a lot to be done in still making this transition from agents that demo really well to agents that actually work in the real world. And a lot of this is may not be

Speaker A

you know um uh something that requires um a completely new paradigm shift. Some of it will be but a lot of this is is is relies along you know figuring out the right problems you want to solve and

Speaker A

measuring them and making these systematic um sort of changes or improvements. I think that's where you know uh focusing on the things that really matter right like so customer support it's like it's not like you're having two conversations with the user

Speaker A

right you're having millions of conversations per day and even for a very specific conversation you're having the same kind of radio refresh happening like you know 10,000 times you don't want even like a 0.1% failure rate there is actually 100 conversations you don't

Speaker A

want that right um and so uh you um like focusing on these aspects of like what makes agents actually take in the real world is is super important um to increase industry adoption and make sure make sure we actually you know leverage

Speaker A

this technology. Great. Are there any questions from the audience? Yes. Right. What does your dayto-day look like as head of research? And then also repeat the question. Um what does your day-to-day look like as head of research? Um also like if you're adjunct

Speaker A

at Princeton like how are you managing the split between that? Um, and then also just like can you walk us through like what the different parts of research are that you guys are working on at Sierra as in for example you're

Speaker A

talking about guardrails right so you probably have some safety people that are doing research there you probably have some people that are working on like industry adoption working closely with the engineers um uh you know like what are the all the different teams

Speaker A

what are the sizes of the teams uh maybe I'll answer the last part first because like that to the the day-to-day we have a ton of problems that we're trying to solve right it's again like coming back to this like the overall

Speaker A

goal is to um make it easy to develop agents but also we want the agents to be really good and really reliable right so um I would say our research efforts are kind of split between the two um there's

Speaker A

there's efforts that go around how do you think about um making it easier to build agents right without going into too much detail you can think about leveraging a lot of automation to do that right um and then the the other

Speaker A

part is around you know how do you make it easier to test the agents you know uh how do you make it easier to build the agents. What are the components like the core parts of things like agent SDK that

Speaker A

actually make it better, right? This this involves models and all of that. There's a little bit of both, right? Um and uh that ties back to much of the day-to-day, right? So there's there's parts about like thinking about these

Speaker A

different projects and how we how we move forward on all of these. Um but other parts are not, you know, different parts on like things like recruiting and more ops are different things, right?

Speaker A

What about like reading papers, going to conferences, talking to professors, and like within those categories, like can you be a little bit more concrete about like because I think like pretty broadly those are two very big categories that

Speaker A

every company's working on, right? Like how do you build agents? How do you evaluate them? Like what are some like very specific categories that your researchers working on?

Speaker A

That's a good part. Um uh so one of the things that um we at Sierra definitely want to do is you know in addition to doing research that benefits internal products um we also want to give back to

Speaker A

community right that's how top bench is a paper we're working on other papers that hopefully come out soon um and so you're you're definitely right in sense like there's a lot more papers now on archive or whatever um uh uh you

Speaker A

know different different channels that you get your papers from and it can be quite hard to like sift through these different different ones. Um, we do have like sort of like reading groups to to go through that. But, uh, again like

Speaker A

there's there's a lot of like a noise and and you want to sort of sip through and get the signal to noise ratio. I think it helps to be somewhat focused on, you know, the the sort of problems

Speaker A

you're trying to solve um that are relevant, right? Um, but also with a with an eye towards like things that that that can cause a big shift, right?

Speaker A

So, I mean, I don't I don't have like a an idea for like how do you pick the papers or anything like that. I mean, it's just like part of uh um what what you decide is important.

Speaker A

And then we have another Okay, we have Wow, we have a bunch of questions. I love it. Okay. Uh in the pink in the back, Christa, thank you.

Speaker A

Hi. Um I work on productionizing AI. So, my question was, what are some challenges that you've seen in deploying agents on prem versus on the cloud?

Speaker A

Um I I don't think we deploy agents on prem. So our agents are all on cloud. So I I don't know the answer. Yeah. The the difference between the two.

Speaker A

That's very interesting because customer support is so often like when they're legacy enterprises they have onrem like they want on-prem deployments. So is there like something shifting in the industry?

Speaker A

Um I think if you I I yeah sorry I I don't know the answer to that question but but my sense is like you you are handling requests right and as as long as like the the the data flows through the approved vendors

Speaker A

then it's fine for for most of these kids. Okay. Okay. More questions. Oh my goodness. Why don't we just go there?

Speaker A

Oh yeah. Um I was really curious about like Tow Bench particularly. Um I hadn't like heard of the paper before but like I kind of like took a look at it like during the talk. So I guess there like a

Speaker A

few like kind of like technical aspects of it I was curious about. Like I think this is the first time I'm seeing like kind of a multiu like like a multi-turn like Asian um setting like done as a

Speaker A

benchmark where like the user like the standard for the user is a language model. Um, yeah, I I was like kind of curious like what kind of like failure modes like you guys saw for like the user like stand in

Speaker A

language model. I think I didn't like see too many like examples of that in the paper. I was like also curious like you you kind of mentioned like simulating like the adversarial aspect of like the user where like they're

Speaker A

maybe not like giving you all the information like right away or like they're like kind of withholding stuff like even accidentally. Um yeah yeah yeah there there's actually a huge um you know I I actually suspect like we

Speaker A

can do a lot of research in that as well like how do you make a user realistic enough I mean like you can argue that a user is also an agent in some sense right and uh you would you would want to

Speaker A

like simulate at at the very top level like you almost want a entire world model to be able to simulate the user right so there's a lot of challenges there um you're right in that like the user being a language model causes some

Speaker A

issues we we are working on some of those like you know reducing issues caused by those for example um and uh yeah I think I think the other part of this is like sometimes uh you can also think of the

Speaker A

the flip side of this is because it's such a uh it's a setting where like you know the release of information from the user is um um is not like everything at once right so sometimes mistakes are fine as long as the agent can handle

Speaker A

them right um everything is plausible within the space of you know the the the possible things happening in the universe right so I could just be a bad user and say like hey I want to cancel my flight or whatever and then next next

Speaker A

step I can be like actually no I don't want to do it right that is plausible right so your agent should be able to handle that situation um and so as long as you think of it that way you know

Speaker A

some of those errors don't really uh cause such much uh that big of difference let's do two more is there one in the front here Christa Uh we'll do these two front lines for um in the RL context, do you think using

Speaker A

language models as your sort of initialization takes us away from purely experiential learning such that they have maybe like a ceiling because of the inherited kind of cognitive huristics of humans?

Speaker A

Um I don't have a definitive answer to that but that's a good point right um I for what it's worth like we were try we were doing this purely experiential learning for quite a while right as a community right learning from nothing um

Speaker A

and I think it's it's definitely like from the evidence that we've seen so far right having a basis of you know like base to start off with like which need not be read everything on the internet it could be something lower I don't know

Speaker A

the answer to like where that threshold is at which we can take off. But you could argue like part of it is like wired in our DNA, right? So where does that information come from and how do you have the equivalent? Um so so I

Speaker A

definitely think it's like practically it makes a lot of sense to start from language models and do it. Um the question of whether it limits us at some point uh TBD you want to pass it right there.

Speaker A

Hey, so I wanted to ask about the more realtime reinforcement learning that you were talking about. Um, I know that OpenAI and a couple other services have come out with this reinforcement fine-tuning, right, where there's very little data that actually is required to

Speaker A

fine-tune a model and you use this LLM as sort of a base that continually learns. Have you guys actually implemented that in production or experimented with it? Um, yeah. And what type of like data actually goes into it?

Speaker A

I know logs are something that is very useful where you can just extrapolate a lot of information from something very little. So, yeah, just curious about that. Um I think there's a part of this which is like how do you do the re the

Speaker A

overall thing in general and then one part was like specific to Sierra I think um for the latter I would say there's experimentation going on um for the former I would say um DPU is just like one way of thinking about it um but more

Speaker A

broadly right you you don't always want a human to go in and label what is preferred right um you also want to be able to like define reward functions on the fly or like you want the agent to

Speaker A

figure out like hey a bottle falls down and it breaks that's a bad thing for certain reasons but it's also good thing it can be a good thing situationally right how do you think about that and and how we learn um uh for agents right

Speaker A

so like when as a human you have this entire stream of consciousness but you're still learning in an episodic sense um but they're both compatible whereas in RL um normally you think of it as like ep episodic it ends there and

Speaker A

then like okay you just restart um so there's some shift that needs to happen there when you think about continuous continuously learning that's what I meant great I know there's more questions but uh unfortunately we're out of time so

Speaker A

that concludes the panel um however the Sierra team is going to be posted up uh in the corner over there so if you guys want to see a demo or learn more about what it's like to be working at Sierra

Speaker A

they're more than happy to tell you more about that and we have more than a few members from the team here uh but thank you everyone for coming thank you for Carly for the talk and uh yeah really

Speaker A

appreciate everyone being here for change engines. Thanks for having me. Great. Thanks.

Topics:multi-agent interactionAGIAI agentsSierra AIcustomer support automationnatural language processingreinforcement learningGreylock Change Agentsagent capabilitiesautonomous systems

Get More with the SozAI App

Transcribe recordings, audio files, and YouTube videos — with AI summaries, speaker detection, and unlimited transcriptions.

App Store Google Play

Or transcribe another YouTube video here →

Free tools: TXT to SRT · SRT Validator · Merge SRT · Subtitle to Text · All tools