Building Enterprise-Grade AI Agents: Lessons from Sierra's Arya Asemanfar

The process of building an agent is kind of like building a product, and that actually goes deeper than the surface. The analogy that I use is like Google in its early days. They not only built a great product in their search engine, but they also got really good at building internet-scale products. I think the same is going to be true for agents. Specifically, at Sierra, we're not only sort of building AI agents for our customers, we're also investing in building the tools and processes and ultimately a product that is the best way to build the best agents. But sort of like the first derivative of a great agent is a great product and building agents. So that's how you and I initially met. As part of that is in evaluation, which is a very, you know, important step in building the tooling for having a great agent. Today, we're talking with Ara Asamar, who leads product engineering at Sierra. They are building agents for companies for customer service, other ways of engaging with their customers, and have really been at the front lines of figuring out, you know, what does it look like to make agents work well for enterprise use cases. So we're going to get into a lot more detail today with what they've built, how they've gone about both thinking about agents and their process for working with customers and bringing this kind of custom AI functionality to life for their customers, as well as what they've built on the platform side, how they make their systems work from observability to evals, testing, and some of Ara's advice for other builders. With that, we'll jump right in. Appreciate you listening. Hey everybody, I'm joined here today with Aryafar from Sierra, and we're going to talk about what they're building with enterprise agents, and we'll let him tell you more about what that means. But Ara, why don't you just quickly give us an intro, like who you are, what you're working on, and maybe some perspective on, you know, why we're talking today, and we'll jump in from there. My name is Arya. I lead the product engineering team at Sierra. I'm one of the first engineers on the team, and I play a role in making our earliest customers successful all the way to now, where we've scaled the team and have tons of agents in the wild. As Ian mentioned, you know, we're building company agents, so it's worth feeding that apart a little bit though since it's fairly new to the world. Please. Yep. I think a lot of people are wanting that to depart these days. Yeah. So if you think about your experience today, nearly every business you interact with has some online presence, like usually a mobile app or website. For example, earlier today, I ordered lunch through DoorDash, and through a couple dozen taps and swipes, I found the right restaurant, picked what I wanted, got some lunch, and time to eat before this recording. With that framing in mind, I think what we're building at Sierra is kind of simple. We want to take that interaction and elevate it by leveraging AI-powered agents to make it fast, intuitive, and delightful. So if you think about this lunch example in the future, I should just be able to have the quick conversation like, "Hey, I've got a podcast recording at 12:30. I want lunch before that, so I'd love to have some tacos no later than 12. Thank you," and maybe through a quick back and forth be done with it. It's not just like basics like this lunch example. I think really any interaction that you have with a business online can and should feel a lot more straightforward than it does today. And we believe at Sierra that AI-powered agents can deliver that for consumers. No, that's awesome. And I think, you know, I've been aware of you all since the beginning, know some of your customers. You know, for people that are not as close, maybe even just starting with agents, right, there's a lot of buzzwords in the AI ecosystem. Agents are the current one, and I think a lot of the pite guist—can you just help us anchor, for people that are listening, like sounds cool, you know, what exactly do you mean? You know, what's different about agents versus RAG chatbots, for instance, that came before them? Let's start there, and then we can talk maybe like hear some examples about what you're actually building for people. I think it's a good question. Yeah, agents is definitely like thrown around as a buzzword. There's probably like a more academic answer, but I think practically for me, the difference is sort of can it take action, to have agency, to do something for you as the consumer interacting with it. So if you squint, like a RAG chatbot agent kind of just looks like a fancy search engine, right? You know, it's taking your question, giving you an answer. Maybe the answer is personalized and, you know, unique each time you ask to some extent. Search engines have started to do that already, whereas an agent actually has, you know, I guess coming from the word agency, has the power to do something, and usually on behalf of a consumer or a company. And so I think ultimately the ability to take action and have an effect out in the world is what makes something an agent. And when you think about it, if you define it that way, like if we get kind of more brass tax technical definition, you know, is this as simple as in that definition you can take a right action, you know, to another API, like you have access to a tool that will do something other than just look up data. You know, is that the nuance the way that you're describing it technically? Yes, but if you just try to do that in the sort of the simple, you know, simple way where you give access to the LM to make API calls, you're just not going to get the reliability and, you know, the behavior that you want. And so ultimately I think what ends up happening when you try to build an agent using AI that's going to work at scale, you have to create an architecture around it. This is probably true even for non-agents, right? If you're going to stand up a business, like technically you can set up a lemonade stand and, you know, have a business going, but to have that actually be efficient and scale and make sense, you got to create processes, and you got to, you know, figure out how to have this be repeatable, have it be, you know, profitable and all that. So similarly with agents, I think your technical definition of it has access, right? Right access is probably right, but in order to do so reliably, you end up having to create architecture around it. Yeah, I think we get a mix. People listening to this show, people like yourself that are actually building these things for real, you know, leaders at companies that are thinking about building these kinds of things. And I think that's one of the, you know, in practice we'll probably get into it more, but it's like cool, taking action in the world, but the architecture around it often involves some amount of self-reflection or, you know, self-critique, and there's, you know, these other steps that we'll get into. We love to hear more about how you're building. In a couple minutes, to just help anchor it even a little bit further, what are some examples of agentic behavior that you've built for customers? Don't need to give us customer-specific examples, you know, names or whatever, but just give us a sense of what does Sierra help people do in the world? So we initially started with customer service. So customer service being, you know, let's say, let's use the retailer example. You purchased something and you used to exchange it for, you know, a different size, or maybe you have a question about how to set up some consumer device. Since customer service is already chat-based, it's kind of a no-brainer, right? You know, often you might be waiting in line to talk to a human, but if you can get that experience both immediately as well as effectively, where it solves your problem, it's just a much better customer experience. What we've done for a lot of our customers is take their customer service and build an AI agent that can d—

engine but they also got really good at building internet scale products I think the same is going to be true for agents specifically at Sierra we're not only sort of building AI agents for our customers we're also investing in

building the tools and processes in ultimately a product that is the best way to build the best agents but sort of like the first derivative of a great agent is a great product and building agents so that's how you and I initially

met as part of that is in evaluation which is a very you know important step in building the tooling for having a great agent today we're talking with ARA asamar who leads product engineering at sierra. they are building agents for

companies for customer service other ways of engaging with their customers and have really been at the front lines of figuring out you know what does it look like to make agents work well for Enterprise use cases so we're going to

get into a lot more detail today with what they've built how they've gone about both thinking about agents and their process for working with customers and bringing this kind of custom AI functionality to life for their customers as well as like what they've

built on the platform side how they make their systems work from observability to evals testing and some of AR advice for other builders with that we'll Jump Right In appreciate you listening hey everybody I'm joined here today with

aryafar from Sierra and we're going to talk about um what they're building with Enterprise agents and uh we'll let him tell you more what that means but ARA why don't you just quick give us an intro like who you are what you're

working on um and maybe some perspective on you know why we're talking today and we'll jump in from there my name is Arya I uh I lead the product engineering team at Sierra it's one of the first uh

Engineers on the team and I play a role in making our early earliest customers successful all the way to now where we've scaled the team and have tons of agents in the wild as Ian mentioned you know we're building uh company agents so

it's worth feeding that apart a little bit though since it's fairly new to the world um please yep I think a lot of people are wanting that to depart these days yeah um so if you think about your

experience today nearly every business you interact with has some online presence like usually a mobile app or website um for example earlier today I ordered lunch through door Dash and through a couple dozen Taps and swipes I found the right restaurant picked what I

wanted got some lunch and time to eat before this recording with that framing in mind uh I think what we're building at sier is kind of simple we want to take that interaction and elevate it by leveraging AI powered agents to make it

fast intuitive and delightful um so if you think about this lunch example in the future I should just be able to have the quick conversation like hey I've got a podcast recording at 12:30 I lunch before that so I'd love to have some

tacos no ler than 12 thank you and maybe through a quick back and forth be done with it it's not just like Basics like this lunch example I think really any interaction that you have with a business online can and should feel a

lot more straightforward than it does today um and we believe at Sierra that AI powered agents can deliver that for consumers no that's awesome and I think you know I've been aware of you all since the beginning know some of your

customers you know for people that are not as close maybe even just starting with agents right there's a lot of buzzwords in the AI ecosystem agents are the current one and I think a lot of the pite Guist can you just help us anchor

um for people that are listening like sounds cool you know what exactly do you mean you know what's different about agents versus rag chat Bots for instance that came before them let's start there and then we can talk maybe like hear

some examples about what you're actually building for people I think it's a good question yeah agents is definitely like thrown around as a as a buzz word there's probably like a more academic answer but I think practically for me

the difference is sort of can it take action to have agency to do something for for you as the the consumer interacting with it so if you squint like a rag chat B chat bot agent kind of just looks like a fancy search engine

right you know it's taking your question giving you an answer maybe the answer is personalized and you know unique each time you ask to some extent search engines have started to do that already whereas an agent is is actually has you

know I guess coming from the word agency has the power to do something and usually on behalf of a consumer or a company and so I think ultimately the ability to take action and have an effect out in the world is what makes

something an agent and when you think about it if you define it that way like if we get kind of more brass tax technical definition you know is this as simple as in that definition you can take a right action you know to another

API like you have access to a tool that will do something other than just look up data you know is that is that the Nuance the way that you're describing it technically yes but if you just try to

do that in the sort of the simple you know simple way where you you give access to the LM to make API calls you're just not going to get the reliability and uh you know the behavior that you want and so ultimately I think

what ends up happening when you try to build an agent using AI that's going to work at scale you have to create an architecture around it um this is probably true even for for non- agents right if you're going to stand up a

business like technically you can set up set up a Lon lonate stand and you know have a business going but to have that actually be efficient and scale and make sense you got to create processes uh and you got to you know

figure out how to have this be repeatable have it be you know uh profitable and all that so similarly with agents I think I think your technical definition of it has access right right access is probably right but

in order to do so reliably you end up having to create architecture around it yeah I think we get a mix people listening to this show people like yourself that are actually building these things for real you know leaders

at companies that are thinking about building these kinds of things and I think that's one of the you know in practice we'll probably get into it more but it's like cool taking action in the world but the architecture around it

often involves some amount of like self-reflection or you know self-critique and there's you know these other steps that we'll get into we love to hear more about how you're building in a couple minutes to just help anchor it even a little bit further what are

some examples of agentic behavior that you've built for customers don't need to give us customer specific examples you know names or whatever but just give us a sense of what does Sierra help people do in the world so we initially started

with customer service so customer service being you know let's say let's use the retailer example you you purchased something and you used to exchange it for you know a different size or um maybe you have a question about how to set up some consumer device

since customer services already chat based it's kind of a no-brainer right you know often you you might be waiting in line to talk to a human but if you can get that experience both immediately as well as uh effectively where it

solves your problem it's it's just a much more a better customer experience what we've done for a lot of our customers is is take their customer service and buil an AI agent that can deliver that experience for their

consumers and that that you know spans from retailers to subscription businesses to um consumer electronics uh you know I think we you know there's a bunch of customers on our website if you wanted to take into some case studies

sure uh but it's it's currently largely in the the customer service space if you take examples you know it's like I'm going to help initiate a return right like these are you're getting all the way into these companies actual business

processes and other systems how do those Integrations come to life you know it just be cool to hear you talk about more like what are the parts of the CR platform and how you work with a customer to make those real when it's a

very you know unique process for each business that's a a good um insightful question I think that uniqueness is actually one of the pieces that we learned about early on is that really every company has a unique brand as well

as the unique set of processes that they want to uh you know embody you know if you think about even just like sort of taking the analogy back to to websites now everyone has a website and you know

especially retailers they can stand up retail e-commerce sites on Shopify very easily and it's always customized if you go back like 20 years none of that was possible you know there there was sort of totally tiny retailers that just had

nothing and when they did eventually get something it was sort of like the the template from you know a WordPress or something like that bringing that back to agents what we found is that our customers do want a personalized

experience where the the agent actually embodies their brand voice and has their policies uh enforced and I think that as a consequence one of the things that we've done is that our our uh our model for onboarding a customer includes what

we call an Asian developer who works very closely with each customer to understand what is it that they want to have their customer experience be and how would we make that come to life and so to some extent and each agent is its

own product and so you have you know like a a product manager and a developer who are responsible for producing this product on top of sieras uh platform you just said something that like this is one of the funny nuances of

talking about this topic you know each agent has to kind of have its own pm and engineer I think when some people talk about these topics like they're talking about a Brand's agent is actually a collection of individual agents each

with kind of a discrete function when you're saying what you just said what do you mean are you talking about the collective you know experience for one of your customers or each of the functional things like processing and

return you know how how much does each one have its own person or small group of people taking care of it and training it and tuning it we don't think about you know given a particular company their agent as being a you know

collection of sort of coordinating agents um I think at the end of the day it's just it's just one agent there is some sort of overlap between sort of the types of cases that an agent can handle from one retailer to another and they're

built on top of abstractions so there's there's some amount of Leverage that we get from that we we don't yeah it's not so much that somebody owns returns holistically across all but it's more that each agent has its own set of

processes so you know uh return policies or you know um the ability to suggest exchanges or other preferences for for loyal customers and that kind of thing that makes sense right I think it's helpful just clarify work phrasing

because you know we do hear folks talk about collections of Agents differently we know how you're talking about it one agent per customer with different functions that it does and each of those can be somewhat custom and based on that

customer's needs and desires what have you found about helping bring those to life you know I can imagine there's people that come in with all sorts of different expectations about what's possible what's going to be involved in training it what's going to be involved

in tuning it you you have clearly figured out how to guide people through that what have you learned about you know kind of that path of idea to reality and I'd love to hear you talk about like how you involveed the

customer in that because I think one of the curious things here is how people that aren't close to building this stuff ramp and learn to set their own expectations about how this stuff's going to work um I think early on when

AI was still fairly sort of new you know with h Chachi just being two years old now a lot of customers would show up and be kind of unsure about what to expect I think that's we've actually seen that

start to change fairly rapidly I mean if you think about it it's um only been a year I think customer expectations are starting to be much more uh grounded and what's happening in particular I think early on there was a lot of fear about

you know the AI doing crazy things or sort of having no guard rails um and then what what we've been able to demonstrate is with our early customers and early interactions with our new customers show that there's actually a

lot of control the the guardrails that we're able to put in place with because of our technology actually gives our customer a lot of control over how the agent behaves and how you know to what extent it does embody their brand and uh

guidelines so I think a lot of it is like being able to actually demonstrate that and show like hey this agent is for for uh your company um and so this is how we can behave um in order to match

what you want that process to get it there like sounds like you have some amount of you know onboarding and Alignment that you do with your customers how much are you involving them in that how much is your team

figuring those things out like I'm just kind of curious to get a sense of what that looks like for the people that are building these sorts of products how much that takes to get on the same page yeah it's a good question I mean there's

probably a little bit of self- selection that happens just by nature of this being used but I think a lot of the people who are getting engaged with us have a curiosity to be more involved and so they end up being fairly involved uh

is it's not so much like a fire and forget you know they don't send us a set of requirements and then disappear or anything like that it's like almost for each customer a codesign process and then in terms of sort of how involved

they get it kind of runs the the Gambit and there's some customers who they show up and they're like here's we we've architect the this is how we want the customer experience to be and we have uh you know a ton of put a ton of work into

that and others that you know let us pull it out of them right you know give us some uh insight into how you think about the customer experience and we have to you know ask questions and do a

little bit of back and forth to discover at the end of the day I think what is most useful is to to make it concrete right to to have sample conversations to build a prototype of the agent and

understand how it interacts let them test it um you know do an early pilot with their customers I think that starts to make it uh much more I guess Concrete in the sense that it's it's actually there and then you have something to

react to If you're only discussing in the abstract you probably go on forever so I think thinking about getting how to figure out what the the simplest use cases to start with or the ones that sort of help prove the value and and you

know sprinting to making that tangible I think it's been super powerful for us yeah and that all tracks right like kind of just classic product development stuff that you're talking about there yeah you're starting to get into like

what is your product actually do you mentioned some of the things you've done you know to give people confidence helping them understand guard rails you put in place for instance but would love to hear you unpack as much as you're

willing a little bit more about like the anatomy of what you've had to build like what are the pieces of your stack conceptually at least that help make agents do the right thing be trustworthy you know repeatable all the things that

need to be true to work at scale love to hear you talk about some of the details for other teams that are building this is actually one of the pieces that I'm personally most sort of motivated and passionate about I think I I realized

you know maybe last year the process of building an agent like I said earlier is kind of like building a product and that actually like goes deeper than you know might be obvious from the surface I think the analogy that I use is like if

you think about Google in its early days they not only Built a great product and their search engine but they also got really good at building internet scale products and they so they as a consequence they did that over and over

again you know early inventions like map ruce made that possible I think the same is going to be true for agents specifically at Sierra we're not only sort of building AI agents for our customers we're also investing in

building the tools and processes and ultimately a product that is the best way to build the best agents and so it's sort of like the first d ative of a great agent is a great product and building agents and that's obviously how

I obviously that's how you and I initially met as part of that is um in the in the valuation which is a very you know important step in building the tooling for having a great agent so I think it actually ends up being fairly a

fairly deep analogy where everything from the design process so if you're designing a web product you know maybe you use something like figma to sort of just Express what the UI looks like you actually need a tool to help help you

design what the conversation looks like with an agent um I think trying to do that with the existing tool doesn't quite work convers conversation is much more complex there's an infinite number of things that anyone can say at any

time I think that the having a an idea of like how do you how do you want to design the conversation how do you want to go requires new tools that that's sort of the design phase beginning all

the way to monitoring you know how do you monitor that your agents are doing the right thing uh you know if it's something like a you know using Web 2.0 ology if you just have like a htttp success rate that's one level which

obviously you need for an AI agent but you also have to have an understanding of the content of the conversation and what the customer's experience is and when the the experience is just natural language that gets a lot more complex

and so building the entire pipeline of design uh test build QA monitor um you know regression tests and po observability all those things fit into this uh agent development life cycle and making that sort of uh figuring out the

right pieces for all those steps I think has been both a lot of fun and extremely high leverage and you guys have built out this platform I know you've talked about it publicly as like agent OS is that one to one with what you're

describing right now people that are checking out your website yeah yeah exactly I think the Asian OS is all is all these things come together to yeah I mean I'd love to just like dig a Little Deeper maybe just like a baseline

question when people are thinking about building in this space one of the things I've been seeing is there's a lot of teams that have done something you know with a simpler set of prompts maybe they're running a chatbot with a basic

system message and like a rag process which just kind of like got invoked every time or you know they're building a feature that maybe does a couple steps um you know you all are getting into a much more complex open-ended you know I

I can imagine uh set of things that are happening on the back end different prompts and Tool calling and otherwise what are just like the core Primitives that you found to be helpful to organize you know your world all these different

functions that an agent can use i' love to just kind of hear you talk about like information architecture as much as you're willing how do you think about setting up and organizing these systems I think when it comes to interacting

with the LMS like obviously the base unit of you know going a request response with an LM it's prompt often you might just be able to get away with one prompt to you know to sort of have it one end user interaction I think what

we found is that to have more reliable and steerable interaction you have to use sequences and sort of constellation of these things sort of serving different purposes like one mental model that could be useful is you know llms

are you know at least today pretty good at some things and then as As you move sort of into more advanced reasoning or requiring more context not not so much a contact window but just like uh information Theory they then they start

to Veer off or if you need it to do sort of like logical reasoning you know you probably can get away with like a you know a chain of thought but then then you've added a bunch of latency so I

think figuring out how to decompose what you're trying to do into a reliable set of steps that LMS are good at today and then being able to compose that together in a way that can you know when you compose them reliably

produce the custom interaction that you want or the you know the product behavior that you want that ends up being you know much more effective process than trying to like prompt engineer prompt engineer one giant prompt into doing what you you want I

think it's um that level of like there's not going to be any silver bullets to like this oh if I just get these these props right then everything's going to click into place that in combination with like I think for us you know

because we're building not just one AI product but uh each agent is its own product as a necessity what we built is not only these these components but they're uh they form abstractions on top of which we build the agents and so I

think depending on whether or not you know for your listeners like if you're building one product versus a platform on top of which many products are built I think um try to figure out how you decompose it into a way that uh you can

get the reliability that you need is uh probably a useful advice maybe like a couple key pieces in there that I bet listeners would be interested to hear more about you know you talked about a couple things like you're calling into

customer systems and you want to uh know if the right thing is happening right you know are we are we actually returning this item is it all correct you know in the example I mentioned earlier I remember when you and I were

first talking about evals you mentioned the idea of like tool hallucination right you have a nice friendly customer support agent says like yeah sure I'll update your email address for you it doesn't have access to the tool to make

that call so you get into these weird kind of you know hallucinations as you try to take these actions in the world what does it look like for your team to like test these systems and gain confidence like what have you had to do

to figure out if those actions that are happening in the world are the right ones um and do that at scale yeah um this is one of the the definitely one of the bigger challenges that we faced I

don't think that we found any one solution there was a constellation of things or a collection of things that we use to help here we do manual review you know similar to when you call on to customer service even for a human they

might say this call is been recorded for quality insurance purposes uh so then that definitely is sort of the the ultimate back stop because we have you know we can have humans be uh you know the catch there then on top of that I

think there's you know for each customer there's certain things that you might want to watch out for and obviously llms can be fairly good at just like answering a basic question like you know did the AI you know promise something to

the customer and you know not actually deliver that I think we can those are because those are simple checks you can sort of do them at run time and so there's this concept of like a supervisor model which is in the zist of

the industry and then there's a number of things that you know I think we've done to help at runtime catch those things but they ended being for like lowlevel um I think not too dissimilar and structure from a supervisor model

But ultimately there's you know many forms it can take and a bunch of it is like just probably traditional software testing too like did this API action occur the other end of it is like I think that a lot of folks were you know

worried about AI actually know taking action in a way that was undesirable ultimately I think that's actually fairly like a well solved problem in technology through just the the patterns that have emerged for how you separate you know a web browser client versus a

server you know if you're using sort of any e-commerce website you can't just like give yourself a free item and so those kinds of things are fairly simple to architect just you know the API that the AI has access to has the same sort

of contracts as a web browser has to its back end it's not going to just be able to do whatever you want the the back end enforces the behavior the customer wants I think we're on a dotted line here from

you know the space of things how do you test you mentioned a little bit I think LM is a judge uses in your world but what have you learned about doing evals of these systems when you're trying to

iterate on them and know if they're behaving like you mentioned human review is a fallback or kind of the back stop that's always there but I'm curious like what you could tell us about your eval system like when you're iterating on a

prompt or trying out a new model or introducing new functionality and you have these you know multi-step flows that aren't totally directed and you know linear how do you all think about doing evals on that system yeah I think

um so there's some of the basics that like obviously your product um handles as well where you just need to have a data set where that's it's labeled correctly and you you know you have an idea of what should happen I think

trying to make it as discret as possible is useful so like if you have purely free form text responses it gets more and more complicated to really know whether one response is better or worse than another so trying to figure out how

to make it as as discret as possible is useful on top of that I think another thing that we've learned is that what changes between models is sort of nuanced like over a year ago then we had an assumption that new models would sort

of just be a super set of the old model in practice I think that they each have their own nuances some things are better some things are worse and just to trying to figure out how do you know quickly

determine in which ways is this other model better than the last one and how to you know accom those strengths and weaknesses it's kind of weird maybe akin to like moving from one CPU architecture to another which obviously has different

performance characteristics and I think it's constantly changing like I don't you know there's you know I think 03 just came out uh the other today or yesterday from open Ai and I'm sure it'll have its own know works I think be

having the data set in order to identify those things is important and then having the tooling to be able to quickly experiment so I think if you can't experiment like the learning Loop and ends up being really slow um you can't

push to production and and then you know get the data back it's got to be like a fast loop the faster you make it I think the faster it's not just about the speed of delivery but actually like

accelerates this the pace of learning and if you learn more by learning I mean uh a human not like machine learning if you learn more you'll just be able to get further I mean I want to make some

of this as practical as you're willing right I think I recognize some of this could be like secret sauce for how you all do things but you know this topic of evals I think we all know it's like

critical for the reasons you just described in a system like yours where you've got this platform element you've got customer specific implementation you talked about some things at a high level like you want to curate good data sets

or like you need to a to have ebil you can run in the system maybe just to start a little bit into that space how much is what you're doing platform level versus customer specific have you found that you've had to build like customer

specific evals and data sets just for them or do you have some like Benchmark that you start with that just operates at the platform level again say as much as you want to I'm kind of curious like how you how you organize those parts of

the system uh there's no secret nugget or anything that's going to be the lunch pin of anything um that we talk about here today in practice both are required I think in our experience we do have a platform level eval where some of the

abstractions that we've built need to improve and um get more powerful and so in order to do that we have to iterate on them so that requires platform level eval in addition it would be kind of Reckless I guess to have customer agents

out in the wild without some sort of assurance of of its behavior before it goes representing the company so we also have company level um you we think of them as like um it's not motion to eval but more of like a test like you have a

specific behavior that you desire to have happen let's make sure that that continues to happen even as the the you know the agent Behavior evolves as as you extend it so yeah I think and what would an example be like if if you think

about at that level you know the test that you'd run in a customer scenario versus a platform level eval like what's an example of one versus the other in your world yeah I think they're probably most akin to like unit testing and

integration testing I think our evals are closer to unit testing you know we have some discret thing that we're trying to determine from the LM and you know you can have an eval on that so we have some input and some output and let

just make sure that continues to happen the on the integration testing side which is you know closer to an agent Behavior it's a whole sequence of things right you know when you're having a conversation with an agent there's

dozens of if not hundreds of llm calls there's interaction with external systems those can break too right like just like any other software you got to make sure that those things don't break and so that that think thinking of that

more as an integration test like what is the end to endend us your journey versus some more discreet llm interaction so the first one is kind of like this prompt these behaviors on the other side unit test layer and then System test

like you're not I'm just this is interesting too A lot of people in this space are figuring out what is this like are you calling that an eval as well you're talking about it more like integration test but hey this whole flow

you know given this question at the top about making a return like does it happen at the bottom or not is is that how you organize things kind of like top of the chain bottom of the chain what

happened how do you think about what that test consists of yeah that's definitely one of them um I I hate keep saying like there's multiple answers but I think maybe someday there will be like a grand unified theory of testing or

something but I think in practice uh there's multiple Tools in place to help ensure the outcome so yeah what you described is is correct as part of it and then in addition there's you can get more granular and you can we you know

there are some cases where there's very specific things that we you know want to have extreme reliability on in which case it ends up looking like an eval or like a unit test but it's implemented as you know the equivalent of an

integration test and so I think the answer is it's all yeah yeah this I was having a conversation with a friend the other day about you know like how take like a a a PR with right you know like

the whole ecosystem 10 12 years ago didn't have that language now we all do and there's expectations around like what is that workflow what are the different things that might happen related to that workflow feels like we're looking for a similar version of

that with building products around llms you know there's this sense of uh a bunch of standard practices are emerging they don't quite have a clean name like a PR anymore yet but I think everyone's ending up in the same place I wonder

what the standard is going to be a year or two from now that's exactly what I was like I think alluding to earlier with the whole Google analogy I think I think you're right we are going to develop new standards they're going to

exist I think enough is changing with you know conversational AI that a lot of the existing things break down like this this happened before too right I you some extent like PR is necessary because code is constantly going to production

uh you know in the world of like CDs or software on CDs that go to goldm I'm sure they had you know code review to some extent but they also just like QA the hell out of the thing before they

made the goldm and printed the discs and so that was enough and so I think similarly with with AI agents I think A lot's going to change um I'm pretty like confident that something it's going to be different I don't know exactly what

it'll be I'm obviously hopeful that we'll be a part of it but it seems like inevitable it's going to have to have new standards still in the very early days and a lot of it's bespoke we keep going on the analogies here you know we

talked about like this is like the hand-coded HTML era of building websites you know yeah the web flow will emerge um right one other kind of tactical thing then maybe we can change topics but you did talk about how important you

know just kind of human review of data has been I think a lot of folks know this making it work in practice is always kind of tricky where you know or do we have Engineers doing this that are

really close to what's happening and understand the ioa the system and can make sense of complex traces or these domain experts that you know know what the right answers should be but maybe a bunch of the other stuff can get

confusing it'd be really interesting to hear you talk about like what do human review look like in your world and how have you made it tractable for the people involved to to it I think your observation around uh you know domain

expert versus technical expert is is correct I think the domain expert is probably the the more important one once you have some level of scale as the the technical expert you have you were sort of on a more stable foundation in the

early days you probably do need both or maybe even more in the early days might need need even more on the technical side for us as we've scaled I think the customer knowing what the right customer experience should be is more important

because ultimately you have to give that feedback if the behavior is incorrect because the behavior was not correctly specified um yeah the only person who's going to know that is the person who knows what should happen instead and

then in terms of like how do how do you make it work you know I think ultimately this goes back to my point about needing to build the like the first derivative is like we have to build the best

tooling um so the experience of doing these reviews has to be exceptional in order to get it reliably and get high signal you could try to scale it without that But ultimately I think having uh you know great tooling is high leverage

for scale the multiplier there is is totally worth it I could probably keep asking you questions I'm keeping an eye on time but you know on that path like you you mentioned just how important the feedback is what else have you learned

about building like a feedback loop you know how does that cool we found something wrong here domain experts looking at you know the way that a agent flow behaved what happens next to turn that into a fix you I'm curious what

that feedback loop looks like definitely one of the things that we've gone through many iterations on I think our initial intuition was probably insufficient it was too simple I think what we've learned is that having an opinion about the kinds of feedback that

you want to collect and how to act on those things in particular is important so like I guess to use some analogies I get some fan of analogies like in code review it's sort of just like or you

know Google doc that you might have you just have the ability to have a text box you just type like in your free form English your feedback in practice when an AI agent does something know suboptimally depending on the kind of

error you kind of want different feedback so yep if the way if the tone or the the phrasing the the agent used was not quite right just giving you know giving the correct phrasing is is the most efficient then you can just use

something like fine tuning to help steer that behavior but if the error is a little bit more how do I put it like it it has more to do with what is the the described Behavior like how how what

what have we informed the agent to do you end up needing a lot more detail it's almost like a spec as a customer experienced person you're you almost need to just sort of give a spec you found a bug in the

product here's here's how it should work and in that case it's like giving the detail of where do you get this data from or these are the business rules or uh you know first you should do this then you do that that kind of thing is

is much more useful having it just if you thought if you think about like the my first answer around just sort of giving the the next response that doesn't quite work because then you end up being like a little a little bit of

whacka Mo you go back and forth and it ends up being a worse experience so I think having an opinion about what are the classes of errors that you want to collect feedback for and what is the the

right information that's most useful so thinking about the endend journey like when a customer provides feedback what will you do with it and having that sort of you know influence what you end up collecting I can imagine some of this

starts to get customers specific too you keep asking you these questions all day I think for the sake of time like just high level thoughts on tactics you know maybe to wrap this section up if you think like give some advice to other

teams that are building building agents following kind of the work you all have been doing what have been like the highest leverage decisions that you've made or things that you've built that any other team in this space should like

make sure you go spend time on I think personally it's sort of a value of mine that I think has been effective here is to be fairly grounded in like customer specifics I think it's too easy to like

get in your head too much about what is the right solution to a particular problem in the abstract but keeping it like hyper focused on sort of One customer or one concrete example of how something should go and then that helps

you build a foundation from which you can now have more informed intuitive mental model for is like super powerful I think just because the space is so new we need to like build our intuition through concrete experience just as you

know I think we will build a m model very quickly of how something works and I think it's just because the stuff is so new just we have to start kind of start over like back to the basics

that's a good one and then what's the Counterpoint you know if I were to ask you like that's a bit of positive advice where have you made mistakes or you know wish you'd done something differently that other people should avoid or maybe

even just like spent more time on a topic than you wanted to right you know I just I think there's all kinds of ways that we learn hard lessons as product teams yeah I mean I think actually like

evals is a good one I think we built we we we've built evals internally but I think we built it later than we should have going back to what I was saying earlier around like the how fast you

learn I think we would have learned faster if we had evals earlier so I I think that's probably a mistake going you know in hindsight something I would have done differently the kind of test driven development analogy is emerging

more and more for people building in this space like start with your evals build from there we're getting close on time I'm curious what else like you know parting thoughts if you're thinking about what the future holds we've got

new models coming all the time new capabilities like you know for anybody that's thinking about building agents like what else should they be paying attention to in the next couple years or you know maybe staying away from either

one of those thoughts what's what's your parting advice for this audience I think one of the things that I'm excited about looking into the future is like the new product paradigms that will form sort of Beyond AI agents but just to AI you know

holistically I think people are going to discover new paradigms for interacting with products that are going to be fairly exciting like obviously there's chat um with with chat TBT but you already see a bunch of sort of popular

tech companies like figma and um others who are adding AI features I think no one has like totally cracked the uh figured out the the perfect solution for any of these things yet and I'm very excited to like see what people come up

with I think it's G look different than it does today stop just can of be like the faster version or the more automated version it might just completely change the interaction Paradigm like my my toy examples like you know Twitter invented

pull pulled the Refresh on mobile that like Paradigm is just a UI Paradigm but like it doesn't make any sense on web like on on desktop web but it becomes native to the to mobile experience and I think similarly with AI we'll discover

the industry will discover new product paradigms for experiences and I'm excited to see what it is and I saw you all started talking about voice agent recently like there's going to be I don't know I'm excited as well and we'll

be watching what you all create because I feel like you're on the front lines here and leading the way in a lot of ways so well done thanks for taking the time to talk with us and uh I look

forward to being in touch more in the future appreciate it thank you thanks for having me have a great day see you all right that's all we have for today's episode of deployed the AI product podcast I'm Ian Karen the CEO and

co-founder of free play I hope you enjoyed listening and if you did we'd love to hear about it your reviews mean a lot they help other listeners so give us a like share subscribe wherever you're listening to this and if

something stood out to you in the episode or if you've got feedback or ideas for a future episode or if you simply just want to learn more we're up to you at free play shoot me a note I'm

Ian fre replay. you can also find me on Twitter Karens thanks so much for listening