**AI Agent Builder Associate (AB-620) - Full Course - Pass The Exam! — Transcript & Summary | SozAI**
Source: https://sozai.app/transcript/ai-agent-builder-associate-ab-620-full-course/

Griffin Lickfeld walks through the Microsoft AB-620 AI Agent Builder Associate exam, focused on Copilot Studio: agent orchestration, knowledge, tools, and evaluations.

## Key Takeaways

- The AB-620 exam is entirely about Copilot Studio and its agent orchestration model.
- In Copilot Studio, the agent acts as the orchestrator that chooses which knowledge sources, tools, or agents to invoke.
- Evaluations are a major exam topic that is underrepresented in official study materials, so candidates should prepare for them separately.
- Choose the single most correct answer on the exam rather than an answer that is only technically possible.
- Copilot Studio's rapid evolution means some exam content, such as topics, may change over time.

## What the video covers

- The speaker, a Copilot Studio MVP and certified trainer, recently passed the AB-620 AI Agent Builder Associate exam and shares a study guide.
- The exam focuses solely on Copilot Studio, not other agent platforms like Agent Builder or Foundry.
- The speaker found the exam harder than expected, with some confusing new questions.
- Copilot Studio is changing with a new experience, so topics and skills content may evolve, though the covered concepts are intended to remain relevant.
- Agent orchestration in Copilot Studio means the agent itself decides when to call its added resources (knowledge, tools, other agents).
- The study guide covers four parts: agent orchestration, knowledge, tools, and evaluations.
- Evaluations were a significant portion (around a quarter) of the exam but were not covered well in Microsoft Learn modules.
- Exam tips: passing score is 700, choose the most correct answer, questions can be marked for review, and there is no penalty for wrong answers.
- The exam lasts 100 minutes with roughly 50 questions, including a 10-question case study.
- Knowledge sources, generative answer nodes, connectors, and custom API tools are discussed with their trade-offs, including credential and content moderation considerations.

## Chapters

1. 00:00 Introduction and exam overview
2. 03:47 Exam scope: Copilot Studio focus and agenda
3. 07:27 Exam tips, format, and cost
4. 19:18 Knowledge sources and generative answers
5. 32:00 Tools, connectors, and API integrations
6. 44:10 Connected agents and A2A protocol
7. 58:00 Solutions, environments, and wrap-up

Answers

## Questions about this video

What exam does this video prepare you for?

It prepares viewers for the Microsoft AB-620 AI Agent Builder Associate exam. The speaker emphasizes that it is focused entirely on Copilot Studio rather than other agent-building platforms.

How long is the exam and how many questions does it have?

The speaker had 100 minutes for the exam, with about 50 questions in total. These include roughly 40 standard questions and a 10-question case study.

What is agent orchestration in Copilot Studio?

The speaker explains that the agent itself acts as an orchestrator, deciding when to call the knowledge, tools, or other agents attached to it. This is different from the common view of orchestration as only multiple agents calling each other.

## Full Transcript — Download SRT & Markdown

00:00

Speaker A

I just took and passed the AB620, the AI agent builder certification from Microsoft. And I wanted to make this video to take everything that I learned and break it down all in super simple terms in one video. So then, by the end

00:14

Speaker A

of this video, you'll be able to understand all of the concepts that you actually need to be able to go and take and pass your exam. So, let's get into it.

00:23

Speaker A

This is the AI agent builder exam, but really it should be called the Copilot Studio exam because Copilot Studio is the entire focus of the exam. There are a couple of different ways to build agents inside of the Microsoft

00:37

Speaker A

ecosystem, like with Agent Builder or with Foundry, but this exam does not talk about those at all. It is only solely focused on Copilot Studio. And my name's Griffin Lickfeld. I'm a Copilot Studio Microsoft MVP, AI solution architect, and Microsoft certified

00:53

Speaker A

trainer. But you do not need to have that much experience or expertise in order to pass this exam. This isn't a social level exam. Not to say that it's easy, but you don't have to have built and deployed hundreds and thousands of

01:07

Speaker A

agents into production to be able to go and acquire or get this exam. And kind of some initial just overall sentiment that I guess I want to share about this is I would actually classify this exam as a little bit harder than I expected,

01:20

Speaker A

unfortunately. And I think really the reason for that is because this is a new exam with new exam questions. And some of the questions, in all honesty, were a little confusing to me. I felt like there were a couple of times where

01:33

Speaker A

instead of thinking, "What's the correct answer?" I'm stuck wondering, "What is this question even asking me?" or "What What is this thing talking about?" With that being said, I think it made the exam a little bit trickier for me. And I

01:46

Speaker A

will say Copilot Studio, as of this very moment, is coming out with a new experience with some new capabilities and an entirely new approach to agent orchestration. And so, I expect that with this new Copilot Studio experience that some of the content of the exam is

02:03

Speaker A

going to change. Now, one specific example could be the concept of topics. Topics are something that are from the old Power Virtual Agent days and are pretty heavily relied upon on agents inside of Copilot Studio. I had a ton of

02:17

Speaker A

questions that related to the concepts of topics or required the understanding of what topics are. And that in the new experience, currently there's no way to create or manage topics in your agents.

02:31

Speaker A

So, that could be one example. A second example is in the new experience there's a place to add skills to your agents. I had zero questions or zero content inside of the Microsoft Learn modules related to Copilot skills. Now, I say

02:43

Speaker A

all that to say that, you know, if you're taking the exam or watching this video and maybe the exam has been out for a little while that all of the concepts that we're talking about as a part of this video, I've avoided those

02:53

Speaker A

that I foresee changing. And so, with that being said, all of this content is absolutely still totally relevant and isn't obsolete now that maybe the Copilot Studio platform has changed a little bit. The final thing that I want

03:05

Speaker A

to say to share with you is this is going to be a presentation-style video.

03:09

Speaker A

You can actually grab the PowerPoint slides with the first link down in the description down below for completely free. We'll send it directly to your inbox. And so, then that way you can continue to shore up your learning as

03:21

Speaker A

well as follow along in case my head is covering some of the content that you need to see. So, with that being said, grab that as well, completely free.

03:30

Speaker A

We'll send it directly to your inbox. Let's go ahead and get into the video.

03:33

Speaker A

AB620 AI Agent Builder Associate Full Study Guide Presentation, I guess we could say. And so, like I said, this this exam is totally focused on Copilot Studio. It is not focused on any of the other agent building platforms. And so,

03:47

Speaker A

as much as, you know, we need to learn about different ways to build AI agents, it's really the Copilot Studio exam.

03:53

Speaker A

Here's kind of our agenda for the rest of this session. Broken it up into four parts, where one I want to talk about agent orchestration and what that means.

04:01

Speaker A

And if you've heard of this term agent orchestration, I'm going to take a different spin on it. And you know, people may be familiar with you know, an ecosystem of agents, having them call each other. But I I want to take a

04:11

Speaker A

different a different approach to what that means in the context of the AB-620. We're then going to talk about knowledge, we're then going to talk about tools, and then we're going to talk about evaluations in ALM.

04:21

Speaker A

Evaluations was something that I did not prepare for in any way. Throughout the entire Microsoft Learn modules in preparation for the exam, I got zero information about evaluations. Now, I knew what evaluations were and how they kind of worked, but I had zero prep for

04:36

Speaker A

evaluations before my exam. And I had several questions, at least a quarter of my exam were on evaluations in ALM. With that being said, that's how we're going to round out our conversation today.

04:45

Speaker A

Some quick Microsoft exam tips, if you have not taken a Microsoft exam before, really just the thing to call out here is you need a passing score of 700. You will definitely find in this points to some of those confusing questions that a

04:58

Speaker A

lot of them are going to ask you for the most correct answer. So, there definitely could be a world where multiple are correct. Don't overcomplicate it. What whatever you tend to think is the most correct, you know, obviously thinking correctly, but

05:10

Speaker A

if something feels the most correct, but you're like, "Well, technically this could be correct if this is true." If you're in that sort of scenario, don't go with that answer. Go with the most correct one. You can always mark

05:18

Speaker A

questions for review, and so then that way at the end you can come back and and click through all of your review ones.

05:24

Speaker A

There's no penalty for getting incorrect answers. I had 100 minutes to complete this exam. So, unlike some of the other AI focused ones that are very time bound, this one was kind of the full length. It was an hour and 40 minutes. I

05:36

Speaker A

had about 50 questions that I had to complete, including a 10 question case study. So, it was about 40 or so questions, and then 10 questions that all related to one business case study scenario. Ultimately, all of those had

05:49

Speaker A

to be done in 100 minutes, so about 2 minutes a question is kind of how you could plan accordingly. This is currently what it costs, but refer to Microsoft Learn documentation. And if you work for say a partner or anything

05:59

Speaker A

like that, there generally are some deals out there for you. So, let's get into agent orchestration and what exactly this means. Agent orchestration is the process of coordinating multiple specialized tools, capabilities, and AI agents to work efficient together

06:14

Speaker A

efficiently on complex tasks, ensuring seamless collaboration, task allocation, and workflow management. When people hear agent orchestration, what they think about is multiple agents in one system all calling each other. But I want to just kind of give you this idea

06:29

Speaker A

that one underlying principle of agents in Copilot Studio is the agent itself is an orchestrator. So, what I mean by that is with your agent, you can add knowledge, you can add tools. I'll call them resources. You can add resources to

06:42

Speaker A

your agent, and the model or the agent itself inside of Copilot Studio is orchestrating when to call each resource. That resource could be a tool, could be a capability, could be a another AI agent. And so, when we think

06:55

Speaker A

about agent orchestration in the context of Copilot Studio, yes, it means being able to connect to different agents together, but it also means this idea that you are adding all these resources to an agent and the agent orchestrator

07:08

Speaker A

decides when those things all can be called. That's kind of one important thing to think about. Now, one highlight I absolutely got a question on this around what are the three pieces of every agent. These are the three pieces.

07:21

Speaker A

It is knowledge, it is tools, and it is other agents. Now, sure, there's other things you can add your agents like skills, like topics, and different things and knowledge and tools. There's all sorts of different subcategories that we are going to get into in the

07:35

Speaker A

next couple of sections. But in terms of what will get you the correct answer on the exam, it is these three things. And there is a major distinction between these that's really important. Well, here, before I get into that. Knowledge

07:48

Speaker A

sources are the things you can ground your agent in, right? So, this is the business context that you can add to your agent. Think of it like the library of resources that you want your agent to go and read if it deems necessary, if

08:01

Speaker A

the orchestrator, if the agent feels necessary. Tools Tools enable your your agent to take action. Tools give you the action ability. If you want your agent to do something for you, like update a record, create a record, send an email,

08:15

Speaker A

those sort of things, then you need a tool. And an agent is its own separate agent that you can delegate tasks out accordingly. And there's different ways to have these different agents. When are these three things all used or

08:29

Speaker A

applicable? Well, knowledge could be used for when a user asks a question, the agent then searches its knowledge at runtime, which means when the question was asked, right? Like in the prompt terminal, and use the knowledge sources available to it to find and generate an

08:43

Speaker A

accurate, relevant response. The difference between knowledge and tools is knowledge has no action ability. It simply just has read access. If you're maybe, you know, a little bit more technical and understand some of this stuff, it has the ability to read, but

08:57

Speaker A

it does not have the ability to write. Tools is where you get the write ability, right? This is where you can now have your agent do things on your behalf. Now, yes, there are some tools like list rows in Dataverse, you could

09:09

Speaker A

add that connector, and sure, that is a a reading tool, but you're get you're getting too much in the weeds. You're you're too, you know, you need to step back just a little bit. These are kind of the higher level concepts. Tools are

09:20

Speaker A

when an ask when a user asks the agent to retrieve a record, update a ticket, send a notification, the agent calls a tool to make that happen. And then for agents, instead of building like one massive mega agent that does everything

09:34

Speaker A

for an organization, instead you can have smaller specialized agents that they can all call each other. In the same way you don't have one employee do everything, right? You have employees across your organization that do special things differently and have different

09:47

Speaker A

expertises. In the same way, that's how we want to think about agents. When we talk about agent orchestration, there's also something very important to discuss. And the reason why I've introduced these topics in this way is for this point. So, you can add

10:02

Speaker A

knowledge, tools, and agents directly to the agent itself. This is like in Microsoft Copilot Studio on the first screen, right? When you see knowledge and you add knowledge there, you're adding it to the agent. You're adding it at the agent level. In the same way you

10:17

Speaker A

can go to say like the tools tab and you can add tools. You're adding those tools directly to your agent. And so, you're giving the orchestrator, the agent, the ability to call on those things whenever you would like. This is an

10:29

Speaker A

indeterministic process, right? And what I mean by that is it's not super structured, right? It's up to the model or the agent to decide, based off of the instructions, when to call these different things. I should say instructions and tool descriptions, but

10:45

Speaker A

we'll get more into that. It's based on the model figuring it out, right? And that's kind of indeterministic. There's not a ton of structure, there's not a ton of guardrails. You don't have a lot of control over that as the author of

10:56

Speaker A

the agent. That is indeterministic. It's undetermined what's going to happen each time. Where, conversely, you could leverage topics and add these tools and knowledge sources and calls to other agents inside of topics. Topics give you some more of that structure, some more

11:11

Speaker A

of that flow that your agent follows, which helps make it more deterministic. To say, "Hey, only call this tool at this specific point in the conversation." In this concept, I didn't necessarily have any explicit questions on why should you add a tool to the

11:28

Speaker A

agent level of your agent. I didn't necessarily have that, but this concept is really important in terms of how we're designing and building our agents.

11:38

Speaker A

And understanding this is is critical to passing the exam from an underlying foundational perspective. We're going to get more into the different tools and knowledge sources and some of the nuances applicable to those here in a little bit. So, if you're feeling a

11:51

Speaker A

little confused, that's okay. But just know there's a very fundamental difference between adding the tool directly to your agent and calling that tool from within a topic. They they do two very different things. Also, we'll call this out when the time comes, but

12:05

Speaker A

some tools can only be called inside of a topic, dependent on the type of tool that it is. Um and some can only be called at the agent level. There There is some nuances that you will need to be

12:16

Speaker A

aware of. Let's talk about agent ecosystem. This is like the typical when we say agent orchestration, what people think. All right, so this is being able to call other agents. There's actually different agents or different ways you can connect agents to your parent agent,

12:32

Speaker A

and they behave in very different ways. And this is something that a lot of people actually do not know. There are multiple ways to connect agents to your agent in Copilot Studio, and how you connect them plays a critical part in

12:44

Speaker A

how it behaves and how it is managed. One is child agents, and the other is connected agents. Child agents are the simpler of the two. Child agents, again, are different than connected agents. And what child agents are is these have

12:58

Speaker A

their own set of instructions, they have their own set of tools, their own knowledge, and ultimately are added to the parent, but think of it as it can only be called by the parent. It's essentially a separate mini agent that

13:11

Speaker A

is only allowed to be used for the parent agent. It's not something like a conversely a connected agent is its own standalone agent that could be called by multiple different multiple different agents. Maybe I'm saying the word agent

13:25

Speaker A

too much. But a child agent only lives inside of the context of the parent agent. This means a couple of things.

13:32

Speaker A

One, it means that the child agents instantly get all of the parent context, which can make passing conversations between the parent and the child agent and variables between those more straightforward. It can also have a slight performance increase in terms of

13:47

Speaker A

it happening faster. They don't have their own independent publishing life cycle. So, what that means is when you publish the parent agent, the child agent is automatically kind of published with it and all packaged into one. If you have a connected agent where it's

14:00

Speaker A

its own standalone thing and you publish one, it's independent from this one being modified in any way, shape, or form. So, also in terms of say adding the parent agent to a solution, the child agents automatically go with it,

14:12

Speaker A

right? Um and and move up. And so, if you were to move it from dev to test to prod and follow proper application life cycle management, those child agents have essentially zero extra overhead.

14:22

Speaker A

But, if you're thinking, "Okay, well, why even build child agents?" Like, I got you. We'll we'll get to that here in a couple of slides. Connected agents are their own separate individual agents and you can still relate these to each other

14:35

Speaker A

inside of Microsoft Copilot Studio underneath the agents tab. But, when you go to create an agent underneath that agent tab, it's going to ask you if you want to create a child or a connected agent. These are the very distinct

14:47

Speaker A

behaviors between these. Connected agents have one nuance that you need to be aware of. I I believe I got a very specific question about this. And so, what the connected agent, really how it's different is it does not

14:58

Speaker A

automatically get all of the information from the parent agent when it calls it. You have to explicitly outline what information gets passed from the parent to the connected agent. And so, because you have to do this, it takes a little

15:11

Speaker A

bit more expertise. And the connected agent is its own separate entity. What that means is is it has its own relief site release cycle. If If update one, it doesn't update the other. But, it also means that the orchestrator is going to

15:25

Speaker A

take the prompt and figure out what it needs to do. And then it's going to say, "Ooh, I need to call this connected agent." And it's going to pass whatever information you like to the connected agent. The connected agent is then going

15:33

Speaker A

to say, "Okay, what is this?" and reprocess that information again to then figure out where it needs to go. So, I'm not saying that connected agents are bad, but I am just saying that they have kind of a separate cost consideration in

15:47

Speaker A

terms of Copilot credits or token con sumption. And because it has to process it twice, it may be slower in the actual chat pane, which dependent on the user experience could be a bad decision to make to to leverage. You know, if if the

16:02

Speaker A

performance or the speed of the answer is really important in your scenario, you should look to maybe leverage child agents instead. And so, just to kind of summarize what I'm saying, when should we build child agents? We should build

16:13

Speaker A

child agents when your team builds and maintains the full agent solution yourself. If there's multiple teams, you know, that would be involved in building these different agents that are all connected to each other, then we probably shouldn't be using child

16:26

Speaker A

agents. We should probably using connected agents. That if the components don't need to really live separately or need to be published separately or need to be migrated separately to other environments. If there's no need for different authentication settings or

16:39

Speaker A

model configuration and ALM process, right? If one agent needs to have M365 or Microsoft Entra ID authentication and the other one needs to have no authentication, you can't add a no authentication agent as a child agent to an Entra ID authenticated agent because

16:57

Speaker A

they're they're different. They have they they inherit the same authentic A lot of the settings they inherit the same settings. And then lastly, the capabilities are specific to this solution and won't be reused by other agents. You can't create another agent

17:09

Speaker A

and call that child agent that you had previously created. It's only within the context of that. Where connected agents, I'm sure you're getting the point, are a little bit different, right? They they have their own separate deployment configuration, their own separate

17:21

Speaker A

orchestrator, their own separate publishing life cycle, own separate team that owns it, all of those sort of things. When we talk about connected agents, there's a couple of different types of agents you can connect. And I'm not necessarily going to spend a ton of

17:35

Speaker A

time walking through all of these because I don't frankly think I got any questions specifically about these, but it was in the Microsoft Learn documentation and is considered a part of the exam. But understand that you can connect with connected agents, you can

17:49

Speaker A

connect more than just other Copilot Studio agents to your agent. You can connect existing Copilot Studio agents with connected agents. You can connect Microsoft Foundry agents. You can can connect Fabric data agents. And now when I think about that, I I definitely had a

18:03

Speaker A

question about connecting a Fabric data agent to my agent in Copilot Studio. You can connect Microsoft 365 agents. You can connect external agents using agent-to-agent protocol, which we're going to get more into agent-to-agent protocol in a later section, but a

18:18

Speaker A

little bit on agent-to-agent protocol. This is essentially say you have an agent in Copilot Studio, but you also have an agent built in a non-Microsoft ecosystem or platform, like your ERP system or like Slack or whatever it may

18:32

Speaker A

be. You can call your agents. You can even connect those agents using agent-to-agent protocol. This is essentially a AI-wide standard that anybody building these agents generally follows. And so we're able to, you know, leverage those standards to connect our

18:50

Speaker A

agents. We're going we're going to talk about a little bit more about agent-to-agent protocol. Let's talk a little bit about perfecting performance though as we wrap up this agent orchestration section. I mentioned this, right? When in talking about connected

19:01

Speaker A

agents, I got ahead of myself. Apologies. Some performance considerations is connected agents are going to potentially have tie higher token usage or Copilot credit usage would be the more correct term, as well as potential delays in performance because it's a separate entity that has

19:16

Speaker A

to completely reprocess all of those inputs and then ultimately make its decision where child agents do not have this. This is a huge point. I absolutely had like four questions on this. So, like 10% of my exam was on this single

19:29

Speaker A

concept right here. Say you're in a scenario where your agent is calling the incorrect resources, I'll call them, right? So, it's not calling the right knowledge sources, not calling the right tools, it's not calling the right agent, it's not calling the right topics, it's

19:41

Speaker A

not calling the right things. Well, there could be two problems as to why. And the first one being you have poor resource descriptions of your tools, of your knowledge sources, of your agents.

19:51

Speaker A

And also then that your parent agent may just be a little bit too big. Let's talk about this in a little bit more detail cuz like I said, this is this is a huge hitting point of the exam. The question

20:00

Speaker A

of how does our agent know what resources to call when? How it does that is based off of the resource's name and the resource description. Primarily the resource description in terms of the correct answer on the exam. So, what

20:14

Speaker A

that means is if you've ever added a tool or added a knowledge source, it asks you to write a description. And it will fill a pre a predefined one in there for you. Those ones generally aren't very helpful at

20:25

Speaker A

all. So, you need to update those. The model or the orchestrator is looking at that description to determine when it should call that resource, when it should read that knowledge source, when it should run that tool, when it should

20:39

Speaker A

call on that connected agent or child agent. If you are feeling like, "Man, I'm asking about X, Y, or Z and my agent's not calling the right tool or doing the right calling the right resource," you should look at

20:52

Speaker A

reevaluating and redrafting your descriptions and your instructions, but your instructions, I would say, would be secondary to making sure your descriptions are correct. Poor descriptions can lead to missed or incorrect resource calls. And think of in descriptions as the little mini

21:08

Speaker A

instructions for your agent. And this is I took this directly from Microsoft Learn as kind of an example that it was giving. Let me read this. When a prompt routes to the wrong child agent, the activity map shows which agent was

21:20

Speaker A

called instead. For Fabrikam, the prompt, can I track my package, unexpectedly calls a different agent rather than the order status. A closer look at the description reveals why.

21:32

Speaker A

Track and package appear in another connected agent's description, but not even in the order status's description.

21:39

Speaker A

So, the orchestrator found the closest contextual match to the wrong agent and chose incorrectly. I'm hoping that makes sense. All right, if you're feeling like your agent is not calling the correct thing, you should heavily look at updating your description. The other

21:54

Speaker A

point could be you have too much stuff connected to your agent. You have too many other agents and tools and actions and instructions and knowledge and topics that it's confused. It doesn't know when to go where. It has too much

22:08

Speaker A

on its plate. This is when we want to potentially look to leverage child agents or connected agents. It's when we start to build agents that are starting to get a lot of tools. Microsoft Learn gives you a rule of thumb of 30 to 40

22:19

Speaker A

choices. I'm pretty confident I had a question around how many actions should you have as a rule of thumb before changing, you know, how the orchestrator is set up. That number was 30 to 40.

22:30

Speaker A

Something to highlight here is this isn't like a hard limit. It's not like when you hit 40, it's like, "Hey, you need to make a child agent." It's kind of just the soft rule of the fewer actions or the more specific or the more

22:40

Speaker A

thin you can make your agent, the better it's going to perform. And so, instead of maybe building one everything on the agent itself, looking to leverage a bunch of several child agents to handle all of those tasks so that ultimately

22:53

Speaker A

the parent agent has less that it's responsible for. And so, that's how we can think about improving performance.

22:59

Speaker A

Let's move on to the second section of our four-part course here, and that is agent knowledge. Now, agent knowledge, this is one of the faster sections that we're going to be talking about cuz I'm going to assume that we know what

23:09

Speaker A

knowledge is at this point. But, there's a kind of two different There's more, but really two main pieces of knowledge that the exam talks about. Let me flip to this side cuz I realize I've been covering up some of those some of the

23:23

Speaker A

text on the screen. But, when we talk about knowledge, for the sake of the exam, there's more and you can add your own custom knowledge, but these are two that are really important to hit and they have a very They both serve a very

23:33

Speaker A

distinct purpose. Power Platform connectors, these are, you know, knowledge sources that you can add.

23:39

Speaker A

These are the connectors inside of the Power Platform. Really, the value here is this can help you get real-time knowledge at runtime. So, the moment somebody asks, "Hey, what is the account balance of this account?" It can leverage a Power Platform connector to

23:52

Speaker A

go and get that information at that point in time. And so, say in an hour that changes and somebody asks again, it will go and find the new updated information cuz it's doing that at runtime. And ultimately, one thing to

24:03

Speaker A

call out is that no data is moving into your M365. You're not replicating this data inside of Microsoft Graph as opposed to what we would call Copilot connectors. Think of Copilot connectors as a search and index pattern. What that

24:19

Speaker A

means is you can take data and you can add it to Microsoft Graph with a Copilot connector. It will take that and index it, and then now it has this repository of data that your agent can regularly call back to. And so, what this means is

24:34

Speaker A

is that this is no longer real-time data. Now, it's something that you can update on a cadence, whatever that cadence may be, but because this content lives in the graph index, agents retrieve it using the semantic search and can cite the specific items in their

24:48

Speaker A

response, but it's not real-time. And so, with that, this would actually be stored inside of Microsoft Graph. When you upload a PDF to your agent as a knowledge source. If you notice, it will sit there and be like processing for a

25:02

Speaker A

while, sometimes up to 15 minutes, sometimes up to an hour, depending on the size of that document. The reason for that is because it's being indexed.

25:10

Speaker A

It's being indexed and processed and stored inside of Microsoft Graph, so then your agent can then call it. The other important thing to mention, I absolutely got a question on this about Copilot connectors, is Copilot connectors need to be configured inside

25:28

Speaker A

of the M365 admin center as an admin before you as a maker can add it to your agent. So, I definitely had a question around, "Hey, you want to leverage a Copilot connector. What system should you go to?" A, Copilot Studio. B, Power

25:44

Speaker A

Apps. C, Power Platform Admin Center. D, M365 Admin Center. That wasn't necessarily the exact question, but like the answer is D, M365 Admin Center is where you can go to set up Copilot connectors that can be used as indexed

25:59

Speaker A

knowledge sources for your agent that will not retrieve real-time data, but they can have citations and sometimes be a little bit more performant as well in terms of getting answers faster. They all kind of serve different purposes.

26:13

Speaker A

And ultimately, where we're going here with knowledge is to talk about generative answers. Generative answers are a resource, I'll call them I won't call them a tool, but a resource that you can leverage inside of topics. only be leveraged inside of topics. But what

26:27

Speaker A

this means is is inside of a topic, say at a certain point in in a conversation, you want your agent to go and look at knowledge and think and come up with an answer and respond. When a user asks a

26:38

Speaker A

question, the generative answer node will then reason over the configured knowledge sources and generate a grounded response using the model to do so and share that to the user in the chat. Essentially, what generative answers are doing is you're effectively

26:53

Speaker A

saying, "Hey, what does our content added to our agent say about this?" It's only going to pull its output from the defined information that you've given it. Now, in generative answers, you can tell it to leverage all of the knowledge

27:05

Speaker A

sources on your agent, or you can have it only leverage very specific ones. So, say you're building an HR agent and you've uploaded or attached eight different policies all relating to different things to your HR agent, but you want this generative answer node to

27:20

Speaker A

only look at information around fishing and scams, right? And that's only two of the eight resources you've added to your agent. You can add a generative answer node and select just those two knowledge sources that you want it to search at

27:34

Speaker A

that point in time if that is your business requirement. And this is kind of the one of the main pieces when I was talking about the orchestrator level in terms of adding something at the agent level and adding it inside of the topic

27:46

Speaker A

level is cuz it behaves differently when you add it inside of your topic. When you configure a generative answer node inside of a topic and give it its own sources, it will first search through those sources to find an answer. In the

28:00

Speaker A

event that it doesn't find an answer, then it uses all of the additional agent level sources as a fallback. That kind of behavior is something that's really important for you to know. I definitely had a question in terms of what that

28:13

Speaker A

means and that when you are you leveraging generative answers, it will first search the knowledge sources on the generative answer, then it will search the knowledge sources for the agent if it doesn't find anything or, you know, if it if it finds something

28:26

Speaker A

but it it ranks it or has such a low confidence score in that. Now, the other thing that I wanted to talk about at this point in time is a different type of searching and that is Azure AI

28:37

Speaker A

Search. Think about this is just the pro code way to search knowledge sources. Information can be indexed and then searched using Azure AI. This is a retrieval knowledge tool. This is not an actionability tool. This is a knowledge

28:54

Speaker A

resource. I shouldn't use the word tool. This is a knowledge resource. But ultimately, the real value here is think of Azure AI Search as the big brother to just generative answers. This gives you the ability to search more information

29:08

Speaker A

faster and more performant. It also can improve the quality through rag. So, for example, if you have a very large PDF, I recognize there's no number associated to what very large means, but if you have a very large PDF and you feel like

29:23

Speaker A

you're asking your agent about it and it keeps getting it wrong or miss or looking at wrong information, the reason for that most likely is because it's not able to find the specific indexed piece of information correctly. Where Azure AI

29:37

Speaker A

Search, that is a big strength of Azure AI Search is its ability to find the correct indexed information. And when you're able to do that, the LLM or the agent is able to provide much better responses. And so, in terms of exam tips

29:51

Speaker A

and gotchas, things like that, this is going to be used when the context exists outside of Microsoft Graph. This is in Azure. It is outside of Graph, but it is in Azure, so it's inside the Microsoft ecosystem, but it's not inside of Graph.

30:04

Speaker A

And ultimately, it can help us answer agent questions on a large amount of data using semantic and vector search and rag. Generative answers allows you to call specific knowledge sources at specific times. And I I think maybe I

30:18

Speaker A

meant to put this slide before the Azure AI Search. So, sorry for the whiplash here. The kind of final piece we need to talk about in terms of agent knowledge is content moderation. So, content moderation in simple terms is this is

30:31

Speaker A

like the the risk meter or or the creativity meter that you want to add, say, "Hey, I want you to be super strict." Or, "Hey, I want you to be super creative." This is the content moderation piece. And in Copilot Studio

30:44

Speaker A

content moderation, it is that safety layer that evaluates both the user inputs. There's two parts. It evaluates both the user inputs, but then also evaluates the AI-generated outputs to help prevent harmful, unsafe, offensive, or policy-violating content from entering or leaving your

31:05

Speaker A

agent. This happens at two points in the process, right here. When the user submits a prompt and for the AI response is returned to the user. So, it's kind of like the final gatekeeper for when the user prompt goes in, as well as when

31:18

Speaker A

the final message goes out. You can actually have content moderation at several different points in your agent.

31:24

Speaker A

And I had definitely had a question about this. You can have it at the higher agent level. This is in the generative AI settings. You can set the content moderation level on, you know, if it's you want it high, medium, or

31:35

Speaker A

low. You can also do this for the AI prompt tools. These are prompt builder tools, which we're not necessarily talking about prompt tools as a part of this video. They were in the Microsoft Learn documentation for the exam, but I

31:49

Speaker A

personally didn't have any any questions on prompt tools, so I I don't think they're that heavy of a a thing you're graded on, but be sure you know what those are. Anyways, you can set the temperature or the content moderation

32:01

Speaker A

there. And then you can also set the content moderation inside of the generative answer nodes themselves. And essentially how it works is it works just like knowledge sources, where if you select certain knowledge sources in your generative answer node, it looks at

32:13

Speaker A

those first and then the agent level. It's the same way with the content moderation. It will execute that generative answer node and respond based off of the content moderation of that node or of that AI prompt tool. It's

32:26

Speaker A

also important to be said that even if you were to turn this to all the way down to low, that Copilot Studio as an orchestrator as a whole still has some unskippable content moderation in terms of the things that maybe you could turn

32:40

Speaker A

off or enable. There are some pieces that you just simply cannot cannot work around. Let's talk about agent tools.

32:46

Speaker A

Again, agent tools is where we get the action ability of our agents. And really when we talk about any sort of actions or any sort of tools inside of the Power Platform in general, it kind of comes down to inputs and outputs. And so

33:01

Speaker A

inputs and outputs, these are the things or the variables that your agent can use to execute these tools or execute these actions. And so ultimately what this means is you need to gather the information through asking questions or

33:15

Speaker A

by looking it up if it's available in the system if the user's logged in. You then need to store those in variables, whether they're topic variables or global variables. You then need to pass those variables to your tools so then it

33:27

Speaker A

has all of the parameters that it needs. The tool then returns some kind of output. And so that's number four. And then ultimately you need to look at that output, maybe transform it, maybe reuse it, throw it into an adaptive card, and

33:39

Speaker A

share it with the user. Like whatever that may look like. And I realize I'm covering up this right-hand side. But what this means is like the user provides an employee ID. You then store the employee ID in a variable. You then

33:50

Speaker A

use an HR connector action. You pass that variable in as a parameter. You then get back the employee record returned by the connector. You then take that, you clean it up, you transform it to extract the employee name,

34:02

Speaker A

department, and manager. And then you use those values in subsequent actions or responses to the employee. Like this is what that kind of can look like practically. And I just wanted to outline like if you're going to be

34:13

Speaker A

having tools, you're going to need variables, which means inputs and outputs. And so every tool as well has inputs and outputs, but also has authentication considerations. So every tool must authenticate with the system that it is calling. So if you're wanting

34:28

Speaker A

to call Dataverse or you're wanting to call Slack or you're wanting to call your SAP, or you're wanting to call Outlook, you need to authenticate in order to do that. And so, this is a really important question is, how is

34:40

Speaker A

access to each tool managed? There's a couple of different ways to authenticate, and some different tools have different nuances. I'm not going to be going into the details of each tool and each nuance. I don't think you necessarily need to know that for this

34:53

Speaker A

associated exam, but I still want to highlight kind of the three main ways this exam outlines that that you need to be prepared of. The first, and the default, is running as the end user.

35:05

Speaker A

This is delegated access. What this means is is every time you add a tool to your agent, you want that tool to run in the context of the user that is chatting with it. If I build an agent, and then

35:16

Speaker A

my colleague begins working with it, and it's connected to Dataverse, I want my colleague to see their Dataverse data. I don't want them to see mine. And so, end user authentication means when this agent goes and accesses that information

35:29

Speaker A

through the connector, through the tool, it's going to access it with that person's login, that person's credentials. Maker, and there could very well be a scenario like this, is to authenticate with the agent author, or the maker access. And so, what this

35:43

Speaker A

means is, say I build the agent, and then my colleague uses it. If I want to use the run as the maker, then every time they call that tool, it would be as if I am calling that tool. And so, it's

35:54

Speaker A

using my credentials to do it, not necessarily whoever the end user is. And so, like I said, there could be some business value in this. Say you have kind of some call it like metadata, or not sensitive data that is in a certain

36:07

Speaker A

system that you don't want to buy a bunch of licenses for, and it's not critical, and everybody should be able to see it. Well, maybe you just set up your account to access that information, and now everybody can access that

36:18

Speaker A

information through your credentials, through this agent. That's kind of like the idea. The last one here is as a service account. And so, this is like a non-human identity. It's essentially the same as running as the maker, but

36:30

Speaker A

instead of my personal credentials, it's running through a service account. When we talk about agents being deployed to like production and different things, we don't want them dependent on specific employees. What What happens if, you know, the example I always use is what

36:43

Speaker A

happens if I win the lottery, right? And I leave my company, my agent would break the moment my user got disabled because now this connector wouldn't have proper credentials anymore. And so, the service account is essentially the same thing as

36:55

Speaker A

running as the maker, but just running in the context of a non-human identity. Those are the different ways or types of authentication. There's different ways to verify that authentication. The most common is going to be OAuth 2.0. This is

37:10

Speaker A

essentially your Microsoft Entra ID access, not to be confused with tokens in terms of Copilot credits, but it uses that token to authenticate and understand what your agent can do. There is no authentication. You are leveraging tools with no authentication. That will

37:27

Speaker A

actually limit your ability to add certain channels like M365 as a channel to your agent when you go to publish it.

37:35

Speaker A

So, there is some dependency on terms of what type of authentication you want your agent to use and where your agent can even be published inside of Copilot Studio. And then API key authentication, essentially think of this as like custom

37:47

Speaker A

authentication. It's not built in and and piggybacks off of Microsoft Entra ID, but this is where you can pass an API key to the connector to give it access and it's essentially the green light from the the service that you're

38:00

Speaker A

calling to be like, "Yep, you have access. You have the correct API key." So, there is a host of different tools that we can leverage inside of Copilot Studio, and I want to kind of outline what all of these tools are fairly

38:15

Speaker A

briefly. These are all have different purposes and serve different business use cases that you need to be aware of.

38:22

Speaker A

They are connectors, custom connectors, MCPs, agent flows, REST APIs, HTTP requests, and then we're going to talk a little bit more about agent to agent protocol even though we've already introduced it. So, the first one being connectors. If you're familiar with the

38:35

Speaker A

Power Platform, these are the pre-built Power Platform connectors that you are used to seeing inside of Power Apps and inside of Power Automate. These are pre-built and managed by Microsoft.

38:46

Speaker A

They're essentially pre-built APIs that are safe and secure, and like I said, managed by Microsoft, so you can have confidence in using them. They are particularly strong for all of the Microsoft platform tools, but they also have hundreds for third-party

39:02

Speaker A

applications as well. These are the fastest way to integrate with existing business systems. So, if you have some kind of kind of question around, "We want the minimal setup as possible. What should we be looking to use?" We should

39:14

Speaker A

be checking if a connector, a pre-built connector, already exists. This includes common things, like I said, all of the Microsoft services, like Outlook, Teams, SharePoint, Dataverse, and then also things like Salesforce and ServiceNow.

39:25

Speaker A

Also, it's important to know that these connectors, when we talk about types of authentication, one of the nuances about this one is this one requires you to either use end-user authentication or maker-provided authentication. And then we kind of called this out as well in

39:38

Speaker A

terms of the knowledge, but these allow you to get the real-time access to data or run real-time updates and and actions at runtime. Let's talk about custom connectors. These are very similar to the pre-built ones, but these just

39:51

Speaker A

aren't created and managed by Microsoft. Instead, they would have to be created and managed by your organization. And so, these are connectors or built API tools that you can now invoke throughout your entire organization. So, what that means is is you should use this whenever

40:08

Speaker A

no pre-built connector exists, but can be used inside of agents, inside of flows, inside of applications when needed to access the same API. And so, this the really val- real value of this is this gets you a reusable enterprise

40:22

Speaker A

integration asset. And so, what this means more practically is if you have a custom connector, you could have a flow over here that uses it. And an agent over here that uses it. And an agent over here that uses it. And an app over

40:33

Speaker A

here that uses it. All managed with one single custom connector. So, it's more scalable and easy to use as opposed to all of those things connecting to that one service doing the same thing individually. You can manage it all

40:46

Speaker A

through one custom connector. And so, if you only need a simple one-off API call, the next thing we're going to talk about is REST APIs and the HTTP action. But, if multiple agents or multiple components inside of the Power Platform

41:00

Speaker A

are going to use the same API, a custom connector is the better investment. And I will say Microsoft Learn kind of poses it as like building a custom connector is like some huge technical debt and overhead. It's really not that crazy.

41:12

Speaker A

But, in terms of the exam and getting the correct answer. If somebody's wanting only speed and they're only going to build one agent ever, and they're wanting to call a service that's only going to ever be called by this

41:25

Speaker A

specific component, we want to stay away from custom connectors. Because custom connectors, the real perk is reusability across assets in the organization. REST APIs are kind of the next step down. So, these are no longer reusable across components inside of your organization

41:41

Speaker A

or across assets inside of your organization. But, you can create API. This is essentially essentially like a custom connector tool, but is only usable within your agent. It can be called multiple ways throughout your agent. It could be called inside of a

41:57

Speaker A

topic and at the agent level. So, it has some reusability across the agent itself, but it doesn't have the reusability across the environment or across the tenant. But, ultimately this allows you to expose application capabilities as a reusable agent tool

42:12

Speaker A

inside of your agent and reduce the need to build extensive topic logic for external system integrations. And so, most enterprise systems typically expose REST APIs that you can call. And so, this is a way that you can be able to

42:26

Speaker A

leverage that technology for your organization. This has some pretty big authentication considerations. And so, if there was to be a tool that you really need to know how authentication works, it would be the REST API. There's different ways to authenticate, but this

42:41

Speaker A

this would be this would be part of that. Then, the smallest level, the next level down would be the HTTP. So, think about this as the REST API call or the custom connector call, but there is zero reusability. So, this can only be used

42:55

Speaker A

within the context of a topic and it's essentially saying, "Hey, at this point in this conversation, we want you to call this service." But, there is no way throughout your agent or any other assets in the tenant to be able to use

43:08

Speaker A

that same action. There's There's no reusability towards it. Now, there could be a service that you want to call this one specific time and so, it wouldn't be worth it to set up a custom connector or to set up a REST API tool for your

43:21

Speaker A

agent. In that scenario, that's when you're going to want to just leverage the HTTP action. And so, this is often implemented directly within topics. And like I said, it has no reusability. And this is a good option when no connector

43:32

Speaker A

exists, a simple API call is needed, and the integration is not expected to be broadly used. If you can just remember that, you're going to get a question right. Here's kind of the if I were to, you know, put all of these into one

43:43

Speaker A

screen, right? We have our connector. This is the pre-built Microsoft managed Power Platform connectors that we're used to seeing in Power Apps and Power Automate. Then, we have the ability to create custom connectors that we can call from all sorts of different assets

43:55

Speaker A

inside of our environment. Then, we have REST APIs, which are APIs that we can call but only within the context of our agent, but there is some reusability there. And HTTP is no reusability and should only be used when the agent needs

44:08

Speaker A

to call this thing only once and it's not foreseeable for that to change in the future. Could also be advisable where say we're building an agent and right now it's just a proof of concept it's just doing this one thing, but next

44:20

Speaker A

month we're going to enhance it and make it do X, Y, and Z. Well, next month is it going to need that same API call?

44:26

Speaker A

Okay, let's go ahead and make it a REST API tool and add it to our agent as opposed to just the HTTP action inside of a topic. So, also making sure you're thinking about, you know, the long-term life cycle of your agent as you're

44:38

Speaker A

making the decisions of how you're calling these external APIs with tools in your agents. We are to move to another tool. We're almost almost through the tools. The next one is model context protocol or MCP tools. Really what this is is a standardized way to

44:54

Speaker A

call APIs or to call these endpoints that allow the agents to discover and interact with external systems, data sources, and capabilities through this.

45:03

Speaker A

Now, what is the real business value? Let me speak to the exact exam topics that you need to know. The thing about MCP that gives it a lot of really cool value is if you've connected an MCP tool

45:15

Speaker A

to your agent, I'll use the Dataverse MCP as an example. You can build an agent and connect the Dataverse MCP tool to your agent. And inside of that MCP tool it lists a bunch of different actions that the MCP tool could take. It

45:29

Speaker A

could be create row, it could be update row, it could be list row, it could be create table, it could be delete table, it could be update table, it could be create view. It's several in there, but the real business value of MCP tools is

45:42

Speaker A

you can add that MCP tool to your agent. Now, it's likely that you're not managing this MCP tool, right? For Dataverse, Microsoft is managing it.

45:51

Speaker A

Well, there is a world where say they make an update to one of those tools that are available to the MCP server, which has now been connected to your agent. You don't have to manage that.

46:04

Speaker A

All of that kind of ownership is now managed by the MCP owner. What that means is is you could connect, say, three agents to the same AP MCP. And say that MCP were to be updated for whatever reason, a new capability were to be

46:19

Speaker A

added, all three agents instantly get those upgrades or the the access to that tool through the MCP. As opposed to if you just added the Power Platform connector directly to the agent. What that means is is, you know, if Dataverse

46:33

Speaker A

comes out with a new tool, it doesn't get added to your agent. Now, you might be wondering like, "Whoa, I do not want my agent to just like be given tools.

46:41

Speaker A

Like, let's not do that, all right?" Well, like I said, the value is this gives you exposure to tool access and capability in a more scalable way. You don't have to go to all of these agents and turn all these things on or add all

46:51

Speaker A

of these things, but you can actually determine if these new tools get turned on or turned off by default. And so, this was actually something that I learned as a part of these modules that I was working through. So, when you add

47:03

Speaker A

an MCP tool, it will list all of the actions. Again, you cannot control what actions show and the descriptions of those actions, but you can enable or disable the actions that you want that MCP tool to call from your agent. Well,

47:16

Speaker A

there's a checkbox at the very top of that list that says enable all or not enable all. That checkbox determines when new ones get added what the default is. And so, by default that is turned on, but if you turn that off, it then

47:30

Speaker A

allows you to turn off any in the MCP server, excuse me, the MCP tool. And as a new tool were to get added, if that allow all is off, the new one gets added and is set to off for your Copilot

47:42

Speaker A

Studio agent. So, I hope that maybe makes sense. It's really just a a revamped, potentially more scalable way to leverage the existing connectors your agent already has through MCP tools. It also generally, I feel like has a little

47:56

Speaker A

bit better performance in terms of generative AI. It's not as strict in terms of the inputs it needs. It has the ability to kind of figure them out leveraging generative AI to do that as opposed to if you were to just add the

48:10

Speaker A

list rows action to your agent, you have to come up with the inputs and the outputs kind of yourself leveraging variables. And so I would recommend at least checking out, you know, leveraging some MCP tools in those scenarios. Let's

48:24

Speaker A

talk about agent flows. This is the second to last one. The next one is agent to agent protocol. But agent flows, these are Power Automate flows.

48:31

Speaker A

And we could go down the rabbit hole talking about agent flows. I didn't really have a ton of questions around agent flows, but I just wanted to call out that you can add Power Automate flows to your agent to run and execute

48:42

Speaker A

actions. And so you can add these inside of a topic, you can add these at the agent level. And so really what this allows you to do is execute a series or a process of actions in a very

48:53

Speaker A

deterministic way. And so when we talk about how agents, you know, are variable, they don't always do the same thing every time. If you're wanting your agent to do the exact same thing 100% of the time, you should be looking to

49:04

Speaker A

connect your agent to an agent flow and have your agent flow do that work. Agent flows are just Power Automate flows with the particular trigger of triggered by an agent. So they really are just I don't even know if it's fair to say

49:15

Speaker A

they're built off of the Power Automate system. Like they are in the Power Automate system. That that's agent flows. But one thing that is important to call out in terms of agent flows is that agent flows have a timeout

49:28

Speaker A

consideration. So that timeout consideration is 100 questions. I can tell you this number may or may not happen to be very important to know in terms of getting a question on the exam.

49:38

Speaker A

What this means is is if your Power Automate flow, if there's a chance where it could have run longer than 100 seconds, you should think about redesigning your agent flow and your agent and how they are related to each

49:50

Speaker A

other. Essentially, it's not your flow's fault, it's Copilot Studio's fault. Your Copilot Studio, your agent will only wait 100 seconds to get an answer back from the agent flow that it calls. And so, your flow will continue to run, but

50:03

Speaker A

your agent itself will fail. Um, it will say, "Hey, never received a response. Sorry, tough luck." So, there's a couple of things that we could do. One, maybe you make your flows smaller and more bite-size so that they run faster. And

50:16

Speaker A

so, then we don't have to really be concerned about a 100-second timeout. Two, when you go to build an agent flow, the trigger is when an agent calls the flow, and the final action is to provide a response to the agent. Technically,

50:29

Speaker A

the things between those have to be less than 100 seconds. And you could have your flow continue after responding to the agent. We really just have to respond to the agent within 100 seconds.

50:39

Speaker A

And so, there could be a world where, say, you know, you wanted to start an approval that obviously could take minutes, hours, or days, you could essentially add those longer-taking actions after the response is provided to your agent. Maybe you need the

50:53

Speaker A

response of those longer things. Again, maybe there's a different way There There definitely is a different way to do that outside of an agent flow, but you just You need to be aware that this limitation exists, and you have to kind

51:04

Speaker A

of design and work around it. The last piece to talk about in terms of agent tools, and then we're going to be on to the home stretch, which will go quick, is going to be agent-to-agent protocol.

51:15

Speaker A

And I already mentioned how we can call external agents that are outside of the Microsoft ecosystem with agent-to-agent protocol inside of Microsoft Copilot Studio. But a couple of gotchas that I wanted to call out here. And I mentioned

51:26

Speaker A

that it is an open standard, and there's kind of this idea of tasks. So, just like with connected agents that we talked about, right? You delegate this task to the connected agent, and then it's essentially fully on its own to go

51:38

Speaker A

and figure that out and work it out itself. Agent-to-agent protocol is the exact same way. And I think that's to be expected, right? You're taking this task, and that's kind of the terminology that A2A uses, as well as the exam uses,

51:50

Speaker A

that task, that thing you wanted to do, it's fully delegated. The target agent is ultimately responsible for all reasoning, execution, all security around that. You know, if you're using A2A protocol and you're calling an external agent, you have zero control in

52:06

Speaker A

terms of how that information is being secure in that external agent. I mean, this is maybe self-explanatory, but it's a very fundamental concept that you need to know for your exam. The final key concept is around agent cards that I

52:21

Speaker A

want to mention. Earlier, I mentioned it's an open standard. I I definitely had a question on this. There is an agent card that essentially displays the agent metadata, like the name, a description, how it kind of works, and

52:32

Speaker A

some other pieces. I didn't necessarily need to know what exactly is on that agent card, or like what it looks like, per se. But, what I did need to know is where you can locate agent cards. And Microsoft Learn modules also give you,

52:46

Speaker A

you know, kind of talk about what this looks like. Generally, they are located at {slash}.well-known/agent.json after, you know, whatever the the URL of that external agent is. You can find the agent card. Microsoft Learn also talks about the fact that there is no agent

53:02

Speaker A

card, then the people who own that agent should absolutely make one to be in compliance with A2A standards. And so, it it kind of was under the assumption that every agent you would want to call with agent agent protocol would have an

53:16

Speaker A

agent card and be abiding by this standard. So, all right, we're almost there. Evaluations in ALM. This is, to answer this question, what do you do after building your agent? And now, you might be like, hey, this is the AI

53:28

Speaker A

Builder Associate exam. Why are we talking about anything outside of building? Well, like I said, this stuff wasn't in the Microsoft Learn modules, but was absolutely on my exam and made up like 25% of the exam content. So,

53:40

Speaker A

let's talk about evaluations. Evaluations are the way that you can kind of bulk or maybe even automate the testing of your agent inside of Copilot Studio. There is an evaluations tab at the top where you can do that. That is

53:52

Speaker A

explicitly what we are talking about here. Running evaluations and analyzing the results is how you can optimize your agent's behavior and validate that your agent is working correctly. You can also run simple tests multiple times. You can have standard tests. Essentially,

54:07

Speaker A

evaluations take a few minutes to run. You can basically pre-script any number of prompts that you want to be sent to your agent. I found that each one you add to that list, you can add I guess I

54:18

Speaker A

don't know if there's a limit, but I've added a hundred before. Essentially, each prompt you add to that list takes about 30 seconds to run. So, just know, you know, if you're adding 10 test scripts to your evaluation, that will

54:29

Speaker A

take about 5 minutes to run and it will continue to grow as you as you add more to that. Ultimately, this gives you kind of the live processing of test cases and can give you immediate visibility into quality trends and potential failures

54:44

Speaker A

while the evaluation is running. And I just wanted to highlight what evaluations were before we kind of came to this slide. And I wanted to to kind of work through this exercise cuz I had several questions over this. And you

54:57

Speaker A

might be like, "Hey, this might be self-explanatory." But I I want to call out and and go through this exercise.

55:03

Speaker A

Let's say you run an evaluation with 20 prompts. What do you think could be the case if the error seems consistent?

55:10

Speaker A

If it feels like there's a consistent happening, what could be the problem? Well, we know that the problem is with something very specific. It could be a certain action. It could be an agent flow keeps failing or it could be that

55:20

Speaker A

the agent consistently is messing up a certain topic that it's not doing. What if the error seems inconsistent? What could be the problem? Maybe there's something wrong with the model that we've chosen or, you know, the use case

55:32

Speaker A

Copilot Studio, it's a sensitive use case and the Microsoft orchestration layer doesn't really want to talk about it whatever. If it's something that seems inconsistent, maybe it's a bigger issue than just a certain tool or a certain topic or a description. What if

55:45

Speaker A

the agent is regularly calling the incorrect tools across different test cases? What could be the problem? Well, if it's calling the incorrect tools, then it's probably our descriptions, but it's happening across different test cases or different scenarios. So, maybe

55:58

Speaker A

it is something larger like we need to rewrite all of our descriptions or maybe we should look to update our instructions and call out specific topics or excuse me, call out specific resources in different times. Maybe we should look at changing our model. What

56:09

Speaker A

if the agent is regularly calling the incorrect tool for a specific type of test case? Well, that probably means like we've talked about earlier that we probably just have the wrong description on something. Something about the model, about the agent orchestrator is

56:22

Speaker A

confused. So, that points us in that direction. What if one action, let's say to call an agent flow, is regularly failing across different test cases?

56:30

Speaker A

Probably something wrong with the flow itself. Maybe is it timing out, right? Like we kind of just talked about or is it missing an input? Is the service that we're calling inside of the Power Automate flow down or whatever that may

56:40

Speaker A

be? The only reason I bring this up is because one of the more confusing couple of questions I got was essentially an exercise like this. I was like, "Hey, you just ran an evaluation and this is what happened. What do you think you

56:52

Speaker A

should do?" Again, like I said, it kind of caught me off guard. I feel confident that I was able to do it right just given my score on the exam, but these are the things that we need to know. We

57:01

Speaker A

can't just kind of have the head knowledge. We need to really understand the deeper level concepts in terms of evaluating our agent and what that could mean and also all of the different pieces to our agents, not only how to

57:11

Speaker A

make them work, but also how to fix them in the event that they do not work.

57:15

Speaker A

Let's talk about agent lifecycle management or ALM or if you're familiar with just the Power Platform in general, application lifecycle management.

57:22

Speaker A

Something that I just want to highlight for the event that some people are not familiar with it, but inside of the Power Platform we have environments, okay? So, environments, these are these isolated places where you can store and

57:35

Speaker A

manage your Copilot Studio agents, your flows, your connectors, your data, your components, your apps, all of those different things. What this means practically is, right, you have a development environment, you have a testing environment, and you have a

57:47

Speaker A

production environment. And your developers are building the agent in dev, and then you move that agent from dev to test, where then people test the agent, right? But it's not being edited or developed in test. Once it gets all

57:57

Speaker A

the green lights, let's say it doesn't get the green light, then you go back to dev, you change it, you modify it, and then you remove it to test, right? It's kind of its own separate place that serve very distinct purposes. Uneditable

58:08

Speaker A

in test, let's say it gets the green light, then you move it to production, where then end users and other employees, or if it's something that's externally facing, you know, customers or different people are using. We don't just want to be building that thing

58:18

Speaker A

directly in production, right? And what if we make a mistake or or break something? We wouldn't want this thing in production to stop working. That's essentially what we're talking about here, um in terms of environment. So, there's separate workloads for

58:30

Speaker A

development, testing, and production, and can enable us to have safe testing without impacting production users.

58:36

Speaker A

Helps enforce governance, security, compliance, and access controls. It also supports solution-based deployments. I absolutely had questions on how we can move agents from dev to test to prod, and that ultimately leads us to environments, but one step further than

58:52

Speaker A

that is understanding solutions. And so, solutions, I know I just said environments were containers, but solutions are like the individual boxes that you pack all of your stuff in, right? And then you ship it off. The environment is essentially where you're

59:03

Speaker A

shipping it from or where you're shipping it to. The solution is the container you're shipping these pieces in. And so, solutions are the primary ALM mechanism in the Power Platform, used to package, transport, and manage Copilot Studio agents and any related

59:18

Speaker A

components like the tools, like the connections, like the agent flows that you've connected to your agent. This solution then you can export and then import into the target environment. Like move it export it from dev and import it

59:33

Speaker A

into test or export it from test and import it into production. This is what you can do. And there's two different types of solutions if you're unfamiliar with the power platform. That is unmanaged solutions and that is managed

59:45

Speaker A

solutions. Essentially, unmanaged solutions mean the components inside of the solutions are editable. And so when we think about where is development happening, it's happening inside of an unmanaged solution in the the development environment. When we move something to testing or production, we

60:00

Speaker A

do not want editing to be happening there. We do not want people to be able to modify the agent directly in production and potentially break it. So we're going to export those as managed solutions. And so then that way when

60:12

Speaker A

they're managed, they're kind of it's like if we were to use that shipping example, it's like box taped up, you cannot touch it. It is it is in there, it is locked away, it is uneditable in that sense. And so for the exam, all

60:24

Speaker A

solutions should be exported as managed. Okay? Now, I'll be honest, in the real world, there could be a scenario where we would want to move something in an unmanaged solution, but I'm not talking about that. For the sake of the exam,

60:37

Speaker A

anytime you are exporting an a solution, you want to be exporting it as managed.

60:42

Speaker A

And I'm using that terminology very particular, exporting it as managed. I definitely had a question around converting the solution and kind of coming back to what is the most correct answer. You can't convert solutions in that sense. You can't take a managed

60:59

Speaker A

solution and change it to a unmanaged or vice versa. This is something that is done when you export it. When you create a solution, it it gets created as unmanaged. And you add all your components, you create your components

61:13

Speaker A

and you're you know, you're adding them all into this box. But then when it's time to move that, you select if you want it to be managed or unmanaged. The default is it asks is managed. And so, when you export that, then it it is a

61:26

Speaker A

managed environment. You cannot, excuse me, a managed solution. You cannot change the state of that solution to unmanaged ever again. Now, you could re-export that same solution as unmanaged and have an unmanaged version and a managed version. But, you can't

61:41

Speaker A

you can't convert a managed solution to an unmanaged one or convert an unmanaged one to a managed one. I'm hoping this is this concept is maybe making sense. The last piece kind of in terms of moving components throughout a system is going

61:56

Speaker A

to be environment variables. Environment variables allow you to essentially store specific information that may change between dev to test to prod. So, where am I going with this? Let's say you have built an agent in dev and you need it to

62:11

Speaker A

call out specific information that would apply for its current environment, like the dev environment. When we go to test, we need it to now point to say something in test or or have some kind of different piece of information, right?

62:23

Speaker A

Or say something slightly different. In that sense, if we export this as a managed solution and move it to test, it's uneditable. We can't necessarily go into the agent and change it from pointing at the dev one to pointing at

62:35

Speaker A

the test one, right? In order to get work around this, we can have environment variables. And so, essentially, just like a typical variable inside of Copilot Studio, you can insert this environment variable inside of your agent. And so, your agent

62:48

Speaker A

will remain unchanged and pointing at that variable, but you can modify the value of that variable outside of your agent. That's essentially what this is.

62:56

Speaker A

Some common use cases are for API endpoints, service URLs, SharePoint site addresses. Like, if you were pointing at a at a sample dev one and now it's in production, you want to point it to the real SharePoint site, you can do that.

63:07

Speaker A

Some kind of feature flags, custom connector configuration. Now, with that, this is actually something I learned as a part of the modules. Not all environment variables behave the same way when their values change. I absolutely had a specific question on

63:20

Speaker A

this. There are a couple of different types of variables, but for the sake of this conversation, there are two. There are non-secret environment variables, and there are secret environment variables. And the name is self-explanatory. Non-secret is for storing non-standard or non-sensitive

63:37

Speaker A

information, where secret could be storing like an API key or a secret key or something that is particularly sensitive that you don't want just anybody in your Power Platform environment to be able to see. These two serve very different purposes,

63:50

Speaker A

specifically when the values change. So, when the value of a environment a non-secret environment variable changes, and then I remember, we're not changing the agent, we're just changing the value over here. When we change the value over here, we actually have to republish our

64:06

Speaker A

agent for it to be given that update. The change of the value of that environment variable is not immediately reflected live Copilot Studio agent. We need to republish the agent. Conversely, secret environment variables essentially calls that at runtime. And so, when the

64:25

Speaker A

environment variable value changes, does not need to be republished, it instantly calls that sort of stuff. So, that is kind of the one major distinction there.

64:34

Speaker A

And here is a just a not to recap, but you know, here is what we've covered, right? Agent orchestration, agent knowledge, agent tools, and evaluations and ALM and I am super confident that you are now more than prepared to take

64:50

Speaker A

and pass your exam. Like I said, I am a Copilot Studio MVP, so I have some experience building agents, but I don't think you need to have a ton of experience to go and pass this exam. Now that you are here, you are totally

65:04

Speaker A

equipped to pass your exam. I have curated all of the content here to do that. Really, you just need to know how to play the game, which you do now.

65:13

Speaker A

Thank you so much for sticking this far to the end. The best thing that you can do is share this with a friend to help them learn, or if anything, like the video and it helps tell Google in the

65:23

Speaker A

algorithm that this video helps other people with the AB620. So, I I truly appreciate it. Don't forget to grab the slides with the first link down in the description down below for completely free. Thank you so much for sticking to

65:33

Speaker A

the end of the video. My name is Griffin Lickfeld, the host of the Citizen Developer channel, and I'm excited to connect with you in the next one.

Topics: AB-620 Copilot Studio AI Agent Builder Microsoft certification agent orchestration knowledge sources evaluations exam prep Power Platform connectors generative AI

Study this video

- [Flashcards from this video](https://sozai.app/tools/flashcard-maker/?from=transcript&id=93955)
- [Quiz on this video](https://sozai.app/tools/quiz-generator/?from=transcript&id=93955)
- [Crossword from this video](https://sozai.app/tools/crossword-maker/?from=transcript&id=93955)
- [Word search from this video](https://sozai.app/tools/word-search-maker/?from=transcript&id=93955)
- [Study guide from this video](https://sozai.app/tools/study-guide-maker/?from=transcript&id=93955)

Free, made by AI from this transcript. No signup.


---
This is the markdown twin of https://sozai.app/transcript/ai-agent-builder-associate-ab-620-full-course/ — the same content, without the markup.
Published by SozAI (https://sozai.app). Reuse and quotation are allowed with attribution and a link back.
Machine-readable index: https://sozai.app/llms.txt · data API: https://sozai.app/api/