I Coached an AWS Engineer From Mid-Level to Senior (Live)

This is Aaron. He's a software engineer at Amazon with six years under his belt working on critical products like Alexa and AWS Bedrock. But like a lot of smart mid-level engineers, he's struggling with interviews for his first senior-level role. And there's a reason for that. Despite having a great resume and an impressive amount of experience to go with it, he doesn't know how to tell those stories in a way that makes him sound senior. Today, we're going to fix that. If you don't know me, I'm Steve Quinn. I spent close to 20 years at Amazon where I made it to principal engineer, and as a bar raiser, I personally conducted nearly a thousand interviews. I've seen everything there is to see in an interview. So today, I'm going to help Aaron level up his interview so that he can land his first senior-level job. Let's get into it.

Today we're speaking with Aaron. Aaron, thanks for stopping by the studio today.

Yeah. So you are interviewing for senior-level positions. Maybe you can tell me a little bit about yourself.

Yeah. So, um, I've been at Amazon for six years, three of which was in Alexa, um, in a science org under NLU, uh, building a lot of experimental features around using the device without the wake word.

So, try not to talk too long on my intro or I think giving an introduction is an art.

What you want to do is you want to give the intro like so many times.

Yeah. Yeah. Yeah. That you're not thinking about the individual words that are there. So I do think that it's a good idea to practice but, you know, you didn't get through your intro so I didn't know if it was too wordy or not.

Yeah. So you want to try again?

Yeah. Yeah. My name's Aaron. I've been at Amazon for about six years. Three of which was in Alexa. So we were working in a science org under natural language understanding building a feature for using Alexa without the wake word.

So we had models on device and in the cloud to determine device directedness. Um, and then we were working with the scientists to build out the actual engineering to make those things run at scale. My team was in control of the cloud service mostly and then we had a small component on the device as well. But essentially deciding when to turn the mode on and off. So monitoring the device state continuously.

You didn't want to be like watching a movie or a video and then all of a sudden Alexa thinks that you're talking to it randomly.

So yeah, we built that out and maintained it for two, three years. Then the LLM craziness hit and got swapped to Bedrock because the NLU org needed to have, they needed an evaluation framework to run like eval checkpoints that they were building and Bedrock didn't have enough headcount. So they did a swap.

Yeah, I started building the internal eval framework for Alexa scientists to use to evaluate their checkpoints against whatever data set they wanted metrics. So we built out that for about a year and a half and then the last year and a half we were building the external evaluation framework for actual AWS customers.

So we built an AWS managed service to run evals and then the last year or so we worked on agent evaluation. So we launched a new service instead of just doing model evals against some data set; you're running agent evaluations.

Yeah.

Got it. Okay. I'll give you some feedback for this intro.

Yeah. Super interesting. But it was more like a timeline of the teams that you worked on and what those teams did.

Oh yeah. Yeah. And so when I'm asking about an introduction, like I want to know about Aaron.

Yeah. Right. And so obviously a lot of your professional life is defined around the projects that you worked on, but I think there was a little bit too much context for like the organizations that you were in and the teams and the projects that were there, right? And I want a little bit more about you. Another thing is like there was a lot of like we statements there. So it was like we, the team did all of these things. And unfortunately, your prospective employer is not hiring the team, right? They're hiring you. And so, we want to be a little bit more crisp about like what your contribution was, what you were able to achieve in these other places. And so, what I might say is something like, "Hey, my name's Aaron. I've been at Amazon for six years," so I think that's fine. I've been on the engineering side of a lot of science and natural language understanding at the company both at Alexa and in AWS Bedrock. Then you'll be like, okay cool, he's worked around science for a while but he's on the engineering side and so then you'll be like, okay cool, like I understand that he's not a scientist or an applied scientist which is like a completely different skill set. So we know he's an engineer but then maybe we are like okay with, you know, some of the things that you shared before the call which is like you don't have a ton of experience around scaling systems out that you would associate makes sense with Amazon. Think about the introduction as like an opportunity to give a signal about like whether you're a good fit for the company that you're applying to.

Yeah, that makes a lot of sense. But I think there's a lot there. I don't think a lot of people understand the engineering around wake words or how to make the Alexa work without a wake word. I wouldn't have any idea personally, you know, I've been Alexa adjacent throughout my career. Like I have no idea how you would get like a non-wake word experience around an Alexa, especially around a use case like a movie. That would irk me so bad if Alexa started going off while I was watching. So I don't even know how to think about those things. So like immediately my interest is piqued because I know enough to know that that's a hard problem and that you are part of it. One other thing is like I think I might try to extract a single difficult problem within that space. So you did a good job with it on the Alexa side, right? So I was part of the solution for non-wake word functionality. If there's like a sub problem in there that is simple to state, you know, would get the interviewer to be like, man, I don't know how I would solve that problem. That would be really good.

Yeah, I think something around. So, we basically built a real-time rule engine. So, I worked on a real—So, go ahead.

Yeah, I worked on building a real-time rule engine to continuously query the device state and check is the movie device capability playing some functionality. Is there a song playing? Different states that we considered incompatible with you start talking and it thinks that you're talking to it.

So that's super interesting. But you gave me the solution.

Yeah. Yeah. I want you to post the problem.

Oh yeah. And so, but don't give the solution because so I'm an interviewer.

I, you know, I've been a software developer for decades. I can't help but try to solve a problem that's posed to me or that I hear, right? It's like when you hear like a leetcode question and then your mind immediately goes into solution mode. Now, the difference here is that you lived that problem in solution. If you give the problem but say hey I solved this problem instead of saying how you solved the problem and I can't immediately come up with the solution then automatically I'm thinking about like this guy's a pretty clever dude able to like work on these types of problems I can't even come up with like a high-level solution for this sort of stuff so what I would say is something like by the way the solution turns out to be pretty simple I have these like rules engines that like do some ladder of conditionals to determine what the state is and then based off of that like do the functionality that needs to be done. So you took the magic away from it in a sense, right? So what you should do is you should say something like the problem that we had to solve was that there are certain states where it's just really inappropriate for our...

personally, you know, I've been Alexa adjacent throughout my career. Like I have no idea how you would get like a nonwake word experience around an Alexa, especially around a use case like a movie. That would irk me so bad if

Alexa started going off like while I was watching. So I don't even know how to think about those things. So like immediately my interest is peaked because I know enough to know that that's a hard problem and that you are

part of it. One other thing is like I think I might try to extract a single difficult problem within that space. So you did a good job with it on the Alexa side, right? So I was part of the

solution for nonwork wakeword functionality. If there's like a sub problem in there that is simple to state, you know, would get the interviewer to be like, man, that I don't know how I would solve that problem. That would be really good.

Yeah, I think something around. So, we basically built a real time rule engine. So, I worked on a real So, go ahead.

Yeah, I worked on a building a real time rule engine to continuously query the device state and check is the movie device capability playing some functionality. Is there a song playing different states that we considered incompatible with you start talking and

it thinks that you're talking to it. So that's super interesting. But you gave me the solution.

Yeah. Yeah. I want you to post the problem. Oh yeah. And so, but don't give the solution because so I'm an interviewer.

I, you know, I I've been a software developer for decades. I can't help but try to solve a problem that's posed to me or that I hear, right? It's like when you hear like a leak code question and

then your mind immediately goes into solution mode. Now, the difference here is that you lived that problem in solution. If you give the problem but say hey I solved this problem instead of saying how you solved the problem and I can't

immediately come up with the solution then automatically I'm thinking about like this guy's a pretty clever dude able to like work on these types of problems I can't even come up with like a highlevel solution for this sort of

stuff so what I would say is something like by the way like the solution turns out to be pretty simple I have these like rules engines that like do some ladder of conditionals to determine what the state is and then

based off of that like do the functionality that needs to be done. So you took the magic away from it in a sense, right? So what you should do is you should say something like the problem that we had to solve was that

there are certain states where uh it's just really inappropriate for our functionality to fire like movie playback or something like that.

Yep. Right. And then you'll be like, it was actually really difficult to actually determine what states were appropriate and what context, you know, what what pieces of context that we needed.

Yeah. To to know whether to fire or not. Yeah. Say that with like fewer words, but you get what I'm saying. Like give them the problem. Don't give them the solution because ultimately a problem well defined is 90% solved.

Yeah. You basically have given them the solution once you understand once you've given them that problem framing. The solution itself is just like everybody's built a rules engine.

Yeah. like it sounds less cool if you just explain it right away. Yeah, it's a perfect time during your introduction to be like I was I delivered a solution or we I think it's okay to say we if it's a big big

initiative where it's obvious that it took a lot of people. If you can identify the sub problem that you personally contributed to, I think that's really good. Quick word from today's sponsor, Linear. Linear has been a sponsor for almost a year now and in

that time a lot has changed in how teams build software. Coding agents like Cloud Code blew up and they're being used to write more and more code. Writing code got fast, but reviewing it didn't. A lot of the speed that you get out of agents

just gets swallowed back up in review. Now, I spent 18 years as an engineer at Amazon. A lot of it reviewing other people's code, so I know the feeling.

You get a huge pull request with changes in some random order. The context is scattered across other tools and at some point you're either grinding through it or rubber stamping it. What people forget is that code review is how

accountability spreads across the team. When you approve a PR, you're putting your name next to it. You own that change now, too. That's the part that only a person can do, deciding whether the change fits the system and solves

the problem the customer actually had. So, a rubber stamp just puts your name on code that you didn't read. That's what diffs is for. It's a code review inside of linear right next to the issue that started the work. It opens up

instantly and breaks a giant PR into readable chapters. The Y sits next to the code so you can decide whether it solves the problem instead of hunting through tabs to remember what the problem was. Linear used to track the

work. Now it reviews it too. Check out diffs at linear.com/al. So let's move to the bedrock side of things. Right. So an AWS bedrock like what was a problem that you contributed the solution to or that you continue to

work on today? There's kind of two stories. There's the internal service and then now the external AWS service. So on the internal one that's actually where my best story that I tell came out of.

So no just uh this we're still in the intro. So is the internal versus external split important? Is it internal customers and external customers?

Yeah. Okay. Why is that detail important? Might not be super important. I mean there were different use cases like on the external service it was more customers are evaluating production like frontier models against whatever their data set and metrics are.

They're more evaluating what model do I want to use in production. The internal one was here's a docker container that the scientists have some checkpoint on.

they want to upload it and then run the metrics against. So I guess the angle is similar. It's more like what are you evaluating that was different?

What I'm getting at is that we don't want to provide details that aren't important.

Yeah, they might be factual. But if you say, oh, there's a split between internal and external, then I'm bracing myself for it sounds like I need to know the difference between the two. And if I don't need to know the difference

between the two, like I've wasted cycles trying to figure out whether that was a good detail or not.

Yeah. I think it's just noise. Yeah, essentially. Yeah. So, we want to be higher signal, right?

It's kind of an art, but like I'm just I'm just pushing back on this fact that like if none of your stories if it matters that you worked on the internal one versus the external one.

Yeah. We don't need to bother the interviewer with these details. Okay. So, what's the sub problem within Bedrock? internal team but doesn't matter that you were contributing to the biggest problem that I worked on within Bedrock was we had this

evaluation framework that was just for textto text models and then the science team was working on their multimodal checkpoints and our whole service was only built to handle data sets up to 600 megabytes and it was taking several

shortcuts throughout the flow where it would just call S3 with a simple get called to download the data set, things like that, and then you have a 500 gigabyte data set of multimodal videos and stuff like that's not going to work.

So, had to overhaul the entire thing basically. Yeah. Again, just pose the problem. Yeah.

Literally, don't even say the S3 or anything. I don't need the detail. Yeah. Yeah. Yeah.

On the off chance that the interviewer doesn't know what S3 is, then you have to be like, it's this big object store that's there. Then you gave like details like 600 megabytes or whatever versus like gigabytes and then so then now I'm

like thinking like okay well I need to understand the storage stuff. Again this is in the context of the intro but also for the other questions that are there.

The problem I solved was that when I got there within bedrock the evaluation framework which is super important for all of these LLMs it really only handled text.

Mhm. So turns out text is really easy to handle. file sizes tend to be smaller and we can make a number of different shortcuts inside of the framework that would allow for faster evaluation, faster execution. But what the

scientists are really after is multimodal evals. And it turns out when we talk about multimedia, videos are gigantic and all of those shortcuts that we took within text, they just don't work, right? And so we had to solve

problems on a number of different like dimensions where where the scale wasn't just going to work out, right? And so then I'm like, awesome. Yeah, I know that that's a hard problem and that you probably have a story about

overcoming that. You presented the problem. So, what they're doing, the interviewer is doing on the other side is they're doing pattern matching. Okay, Aaron's got a problem that he's describing on this side, a junior level problem, a mid-level problem, a senior

level problem. You don't even have to give the solution. You're just basically saying, I I work on these size problems.

And what they're doing on their side is like, oh, we have problems on our side.

Does he like solve the junior level problems? Did he solve our midsize problems? Does he solve our senior problems? Right? And so they're just trying to square up those problems and then match those across, right? Uh the solutions are going to be commensurate

with the size of the problem. So we'll just describe the problem so that they understand what it is. The solution is going to be different. They're just looking at the size of the problem and comparing those across. Does that make

sense? Yeah, that's actually really cool because they have this role that they're hiring for with certain size problems or expecting and they want to hear that you've handled similar or larger Yeah.

Yeah. problems. And again, the solutions tend to be pretty underwhelming, right? So for your side, you're probably like, well, I I went through all the code and anywhere where we make an assumption that it's going to be text or

anywhere where like we'll have some sort of overrun because of size, I had to go in there and make sure it supported something that was bigger. And then probably there's an assumption here that we have like a long live connection for

like downloading all this stuff and like there's probably a bunch of details. Yep. That are there and they're all super annoying, but the framing of that problem is the is doing most of the work. Yeah, that makes sense.

Yeah, for sure. Because Yeah, a lot of the solutions I don't know, they've been done before. You can Yeah, like a lot of people have done similar things. It's not some It's not rocket science crazy thing. Yeah.

But but the thing is, you know, take that problem that you did. You would not give that problem to a junior.

Yeah. Right. They're straight out of school. It's just like you just need to be around production systems long enough to know what to watch out for, what's going to be an operational problem. Even though it's an internal service that

you're, you know, you're serving internal scientists, it's still really important. The stakes are higher.

Yep. Right. Okay. So, uh, lots to work on with the intro. Yeah. But I think there there's just sort of small tweaks, right? So, we're not going to give them solutions. We're not going to give them a timeline like a

historical record of like the teams that you worked on and and all of that stuff.

I think just really a focus on the problems that you solved and and what your contribution was to that team.

Okay. So let's start off with what you perceive to be your strongest story. So when I was working on the internal evaluation framework, our customers, the AGI scientists wanted to run some eval against multimodal data sets and models.

We had made a lot of assumptions throughout the data flow that we were using text data sets. We were using yeah a certain smaller file size because it was only text and also we were it was tightly coupled to one form of inference

using a docker container and one location of data set storage. So it was like it's an S3 in your tenant account and you're using a docker container. And so some other requirements they also had was we have these data sets stored in

some internal data lake a different storage that they had built for you know whatever and they wanted to use that as an input source. They also wanted to run different inference endpoints. Let's say they really liked one of their

containers and they had deployed it on their production endpoint. They wanted to also run eval against their production inference endpoint. I had like three problems to solve. Had to generalize the data set format to be agnostic to where are the data sets

stored. Give ways to say is this a multimodal data set like define a new format for a multimodal data set. And then also with running inference had to abstract out a lot of our inference to be able to extend it to different

endpoints, different clients. um maybe you're running the Docker container user story still and then also had to overhaul how the data was flowing. Yeah, I had those three big problems. So designed a new data set format that was

able to be extended for like text and multimodal regardless of where it was stored.

Made the inference abstraction so that you could easily put new clients in. and then also added a client for their production inference endpoint which used some streaming protocol and then yeah for the data flow ended up so we were

using a training job a SageMaker training job and before we were just pulling the data in directly from S3 with a simple get call so I looked into a few different ways to mount the data to the training job um ended up just

going with S3 file mode because we didn't have to maintain the like chunking download code because we had a few options. One was like writing our own algorithm for chunking the download.

One was using something more expensive like FSX for luster which is this really like high power file system mounting thing. Ended up going with S3 file mode just because it was cheaper than FSX. We didn't need the performance there and we

didn't have to write our own algorithm. So that worked really well. And then yeah, the first use case went great and then I wrote uh like a doc and did a knowledge share on here's how we if we

need to make a new data set format or add an inference endpoint new inference endpoints new clients you know here's how we do it and then we ended up that service got transferred to another team uh just made more sense for them to own

it. I did a knowledge share with them and what's fun is I always get notifications when they read my doc and they've read my doc a lot of times and when I look back at the code package there's like a ton of new clients

right they've heavily extended it and they didn't have to change what I did like they're still using the way that I abstracted it which was really cool so this was for you say different clients so this is for different

endpoints so for different inference engines or like different clients in some other so there's like 3P clients like using anthropic or open AI there's internal inference endpoints got it got it okay I think it's a great story I think the thing that I I

immediately think through is like why is this hard so when you were like okay well the data is in S3 and so we needed to be more robust and be able to accept data from say like some data lake

somewhere right part of me is just like why don't you just define an interface and and yeah, you know, have like the S3 implementation and then the data lake implementation. So that's maybe what you ended up doing. Um, but like why is that

hard? I mean, I think one thing was I kind of had to rethink the entire service. It was it was very coupled, tightly coupled to one user story like the Docker container, S3 file, text data set. Yeah.

And it didn't have much thought for extending it. Maybe the problem in itself if you're starting from that is not I don't know it's not insanely complex to think about but I kind of I had to redesign the service on the fly

almost. What do you mean on the fly? The existing use case still had to work.

We had other people using the old text flow the existing story. So you couldn't break existing functionality. So I get that.

Yeah. So basically you join this team and they're like we need you to extend this so that it could be agnostic to the data source. we need you to extend it so it can work outside of inference engines

that were docker containers inside of you know ECS or whatever it needs to be much more robust in terms of we need to be extensible on some some number of dimensions so part of me is just like that's kind of common right like it just

kind somebody put something together really quickly and now a lot of people depend on it so we need to extend it for other use cases why is that difficult so the reason things are difficult usually are a confluence of

constraints that you have like the need to keep it up and running. You could have a tight time deadline associated with it. You could have people that disagree with like how extensible it should actually be. You know, there could be other sort of like

domain specific considerations that are there as well, right? Yeah. So, I think what you want to do is you want to add, you know, you said like there were existing customers that were there.

Were there people that were knocking down on your door just being like, "Hey, can you do this faster?" Right? Cuz you can see a situation where it's like, yeah, it was it would have been straightforward except for we didn't have 6 months to do

it. We had to do it in like two weeks. I didn't want to just like do another quick and dirty solution. So then the next guy that came in Exactly. would have to like decide whether to like overhaul it again or

just add yet another set of duct tape onto the That actually makes that's a good point because I don't have the exact date off the top of my head. They needed this done because they they had a deadline

for running an evaluation on these checkpoints and they didn't have a tool to do it.

Great. Yeah. Now I'm like, "Okay, cool. That's that sounds like, you know, if you have a bunch of time and you have a bunch of resourcing and you you know, there's not a ton of constraints. just like we'll

get it done eventually. It doesn't sound hard. Yeah. What makes it hard is and you can go back to your notes or you know think back on it but probably given like 3 weeks to do something like somebody had

signed a check. Yes. And then you had to go and deliver against that. Maybe you didn't have input into the actual date. They didn't ask you to scope it. And so the question for you is like, okay, well, I can just add

newspaper on top of newspaper to this thing and just like hack in this like specific data lake and hack in like the anthropic client or whatever it is, but you have this like need to just be like, oh well, if we do it in a generic way,

people can just build plugins. There's going to be new models that are coming through. Like it's not going to stop here.

That's a good point to think about it. It wasn't like my manager saying, "Hey, we should just make this extendable because I foresee this in the future or something.

It was yeah, it was we have this need, they have this date, they need to run an email on something. They might switch to a different like we might lose them as a customer if we can't get this done, but

then we also should not just hack together. Yeah. This one separate path that's not extendable.

Yeah. So what I would say is something like hey you know my manager came to me and he was like we need to be a little bit more flexible on the data we need to extend it to multimodal we need to make

it so that it works outside of these docker like a very specific docker setup and we didn't have a lot of time the extensions themselves like the underlying like code to just be like hey I'm going to use S3 file system instead

of like you know regular S3 gets that stuff is relatively unremarkable. But I had a choice. I had like whether I wanted to just sort of hack on or if there was an opportunity to do a little bit more. We can make it slightly more

extensible, then it would be solved. And so I decided to do that. It took a little bit more time to design something that was robust enough, but we were able to make the deadline. On top of that, I

I no longer work on this thing, but I do know that every time somebody reads my document, I get a notification. And I went back to the repo and it turns out people actually extended my code so it gives me the warm and

fuzzies. Yep. Right. That's a much better story. Yeah. I think I was just missing the the time pressure or the Yeah. the reason that we were doing it in the first place.

The the thing that makes it hard is the thing, right? So ultimately at the end of the day like you know we still I think we still use IDs. So you have your ID up, right? and you're like what do I

need to type in or what prompt am I supposed to give to like what am I supposed to be making that is ultimately typing code or giving a prompt to thing is not hard right it's like knowing what to build

and then making a choice as to whether like hey do I is it now the time to invest for the future and make it extensible or do I just need to do the thing that I need to do right in front

of me nobody has the answer to that I've gotten in so many arguments I tend to be on the extensible side which tends to take more time and more buying in and all of that stuff. I still remember a

senior principal that I always buted heads with and he's just like, "Dude, we don't know where we're going to be in a year.

Why make this crazy robust thing that Who was right? No one's going to use.

We were both right. We were both wrong." And he's just like, "You you don't even know the right dimensions to extend this thing on." Like now you're building this like list of objects and those objects can be anything, you know, and it's like

it's super extensible and it's like that shit's just going to break later. just kind of like setting the stage there and like setting the stakes and then just being like, you know what, I I took a risk which was to take a little bit more

time to build something that was a little bit more robust and it turned out that it was the right thing. So, you want to give some concrete numbers. Hey, we were supposed to extend it for three other clients or something like that and

I look back and you know there's like 15. It was like Yeah. 20 or Yeah, it was a lot. And so since you still work there, it's like go get the actual numbers and then maybe put the names of like the

specific model and version number in there as well, right? Just be like, "Hey, somebody just added uh Opus 48 or something like that." Yeah.

Cool. So, I think we've uh judged up that story a little bit more. Let's do a question uh of my choosing. Tell me about a time you had a conflict with a coworker. So this one is hard for me

because I think for a lot of my career I I go on the side of avoiding conflict sometimes. So if let's say I have a thought on a design and another team member has a thought on a design

and like if they both make sense and it's kind of a matter of taste sometimes I'll just say if you're really like passionate about this approach I'm willing to just like go with what you think. Mhm.

So, it's hard for me to think of a time where I really had a real conflict.

Like, what are some examples that you use? Well, I I actually think that your example, if you have a good one of that is is an example.

Yeah, that's a great one. Let's try to deconstruct the question. Why do people ask that question?

They want to see how you work with people probably like how do you handle disagreement?

Yeah. Do you just steamroll people and say that you're right all the time or are you willing to work with people that think differently?

I think the reason they ask it is because when you get a bunch of smart capable people in a room and you give them problems to solve, they are going to disagree on the solutions they're trying to double click on is to try to

make sure that you're not going to work with somebody that's super stubborn that won't ever change their mind. So, it's always, you know, everything is like a big fight. You know, sometimes it's good to steamroll people. Steamroll might not

be the right word, but maybe be a little bit more forceful. Sometimes it's good to acquies and just be like, "Hey, it doesn't matter." But we need to demonstrate that we make progress despite differences of opinion.

Y like that's the biggest thing. If every time you have a disagreement with a co-orker, it's like table flipping and HR violations and all of this other stuff, it's like maybe I don't want to work with this person. What I would do

is just be like, okay, there was a big difference in opinion about a particular design decision. I think it's a good idea to just be like, okay, well, this design was solving this particular problem. Typically, what will happen is

the battle lines are typically drawn in the same places. So, it'll be like there's the fast and dirty, the fast and cheap dirty solution and then like the right solution for the future is kind of what we were talking about earlier. You

know, what you want to do is you want to frame the problem. You want to say I wanted to do it this way because X Y and Z. A thing that people miss out on is like they wanted to do it because of A,

B, and C. Right? So you can be like a rational person can also take a difference of opinion with you and it it's not like this guy's stupid.

He wanted to do it the stupid way. I wanted to do it the smart way. We did it the smart way and like you know then everything was good. So we don't want to do that. The result needs to be how were

you guys able to make progress despite this conflict? Right? Was it new argumentation? Did you go and get new data? Or a perfectly valid way would just be like, you know, at the end of the day, we looked at it, the the

solutions are pretty different, but uh they're functionally equivalent. Yeah. Despite the fact that I wanted to do it this way, it was a matter of taste in my head. I was it wasn't like I'm going to let you win, but it was like, you know

what, I'm just going to commit to this other side because it didn't matter. What was the story there? What was the framing?

Trying to think of a specific one. Now, what they'll say is like the difference of opinion kind of gives people a view into how what level you're at.

Oh, right. So, we need to be careful not picking a design that's like super simple, you know, if it's like tabs versus spaces, it's just like, okay, yeah, that's that doesn't matter, right?

Yeah. What if you had a disagreement with a VP or a director? Like, what level would you think that that person was? I think what you want to do, especially if you're targeting a senior or highle mid, is to to come up with something where

it's like, okay, definitely this Aaron guy is uh a senior engineer, right? So maybe a disagreement with the senior engineer would work.

Yeah, it could also be so I I was telling you that the battle lines are drawn. So there's the quick and dirty solution versus the long-term solution that's going to take way too long. A lot of times um when we're thinking about

people that are making decisions that are higher level than you, the difference of opinion could be one where like you on the ground floor, right?

You're the one like writing the code, checking it in, deploying it, doing the operations on it, disagree with somebody, maybe a principal engineer like me who's maybe more of an armchair, like he knows the high level, but he

doesn't know the actual things that are going down on the on the ground floor.

Yeah. And so the difference of opinion there is like one of visibility. There's one in my head that's like a current problem.

I would have to generalize it a bit cuz it's still that's fine. Happening. One thing really quickly is because you're still employed. If there's a story that would be a good interview story like this one, you get to write the story right now

with your actions. Yeah. Right. And so now I'm kind of interested, right? Because we can just go and Yeah. So, I'll try to uh generalize it, but well, by the way, one last thing. It's good to generalize it because it needs

to pass that test of somebody that's technology adjacent, right? So, don't give me a whole bunch of like LLM internals and names of projects and stuff like that. Just like pretend I'm a pretend I'm a non-technical dude. So

throughout agent core and bedrock we have a bunch of different services that rely on obviously running inference. Through the last few years or so it kind of feels as though the hot spots within the AI like what is the up

and cominging service or what is the one that people are really investing in kind of changes. You mean like the frontier models within Amazon? The different platforms that people are using for like inference and running evaluations or guardrails,

things like that. It kind of moves through service to service. And there's one inference platform right now that we've been using for a while. A ton of teams have been using for a while. All the services are tightly coupled with

this one. And now there's this new inference platform, internal inference platform that's hosting a lot of more recent models. there's beginning to be this rift that now all these different services have to for this model or provider go here for this model provider

go here. We've been uh trying to build a common library to help with that. So recently there's been issues with the newer inference platform in a different org saying we actually don't want you to integrate with this inference platform

for whatever product it is. But the the products that we're maintaining are still like they're still serving customers. They probably should be updated with all the latest inference and such and they're kind of just saying deal with it. And so I've had to try to

escalate to product managers and like my senior manager um and it's going higher up the chain than that to find some common alignment because as you said like from the ground level from me on the ground level it's very hard to

maintain right now because I don't know which way they're going type thing. Okay. So I think we need to clean it up a little bit.

Yeah. We just need to make it super super simple with the internal services. There's basically one major inference platform. Now there's a new one that's popping up.

Yep. And there's a whole bunch of like library and tooling and dependencies and use cases that are supported by the old one that are not yet supported by the new one. Okay. So I understood up until that point. Where are you in this

squabble or this in the squabble? I'm maintaining several of the services that use the old one.

Okay. So, you maintain some of the services that use the old one and now C like we want customers are asking you to move them.

Yeah. To to support on the new one while still maintaining support with the old one. So essentially using both because they kind of have a subset of functionality with different providers.

So they would use both of them. Clients would use both of them simultaneously under the hood like their inference platform. So we're we have an evaluation service for example this is like an inference service on the back end and from a customer's

perspective they shouldn't have to care where it's coming from but for us we have to maintain so to make customers happy you would have to do extra work in supporting both of the platforms.

Yep. So where's the conflict? The conflict is that the new platform is saying like for certain old services they don't really want to support maybe are building their own thing because it would be a big lift for them

to go and support the first platform the legacy. Yeah. Okay. So I can understand why they don't want to do it and so I can understand where you would want to do it because you know it's better for clients and

customers. So what you're escalating like the story that's in progress right now you're escalating this.

I mean from me on the ground it feels a bit what can I do without asking my senior manager or product managers for help because I don't feel like I have influence into this situation.

Okay. Like what's the amount of work that would need to be done on their side?

Not much at all. They just need to let us do it. Like we would do it.

Okay. But they would have to support it from here on out. I don't fully know what the reasoning is be like. Yeah, we're basically saying let us do the work to add support for this new thing and they're kind of saying we're still

figuring out the road map or we're still we're not sure what we want to support yet so like just hold off. But then I have these dates to like from my leadership to support this and this you know the superset by X date.

So this isn't really a conflict. Is there a person on the on the new platform that you're having a conflict with or is it just kind of like organ team to team?

It's more team to team. Yeah. So maybe not. So this could play out well cuz I think if you were able to resolve it from your level.

Yeah. Then uh they'll be like, "Yeah, this guy's definitely senior if not higher than that." Cuz there's a lot of organizational dynamics that are here.

Basically, there's a team with a road map and then you have a set of goals and then those are in conflict with each other. So you need to get them to do the thing. You know this is happening in

real time. What I would advise is doing something like do they know that you would just go and do it for them?

Actually they definitely do. They definitely know that we would just go do it and they don't even have to change anything.

And how do you know that they know? Via my product manager that is talking to their PM.

Okay. Yeah. and they're pushing back because it's more of a road map or they're not sure if they want to essentially open that door of integrating with like showing customers we support this legacy thing as well.

Yeah, it's they're worried about like support for other things. Yeah, it feels like a product image thing almost.

Okay. Do they know about the customer delight that would occur if my product manager has told them multiple times how he feels? Yeah. That customer delight would be much greater if they don't have to care where it's coming from.

Yeah. When you escalate just making sure that it's just like, hey, we're just going to do it for them and it customers are going to be better off for it.

There's not a lot of downside. They just need to let us do it. But it sounds like it's gonna work its way up and then maybe resolve at a higher level and then come back down.

Exactly. Yeah. You know, this is like a sort of like proto influence. Yeah. Like example, but you're still in the middle of it. So, it still hasn't resolved. But I think the problem here is like you you have to

work through your product manager or through your management instead of directly with a counterpart on the other side.

Right. So, if there was a touch point there, I think it would be a better story. So I think for me it would be more picking some design decision like what what level of design decision complexity do you think would be good

for conflict? Like I'm thinking about examples where it's oh this person wants the source of truth for this data and this system design to be here. I had it here but oh thinking about it they both probably work fine. We just change the

APIs a little bit. Like is that too easy? you know, no, I I think it's good, but again, I think it'll be due to framing, right?

So, this might be definitely a midstory, but I'll give you an example. So, I'll just use like SQL and NoSQL as the as a design decision, right? So, I thought that we wanted we should use SQL for our backing store. Uh the other

engineers wanted to use NoSQL. I was like, well, we have like a bunch of like range queries. We don't know where the uh indexes are going to be. like they're going to start off here, but we might want to build indexes on a whole bunch

of other stuff. We might want to query on like a wide variety of things we don't know yet. We don't have like natural keys. Uh we might have like relational data. So like we have a table for one thing and a table for another

thing and a table for another thing. They all relate. We're going to need to make queries across all of these. But the team was basically like, "Dude, just use Dynamo." Yeah. Yeah. Just which is like probably what 99% of

Amazonians would say. But I'm like, "No, like we actually have a relational use case here." Who's right? Nobody's right.

I have my reasons for wanting to do this. Hey, dude, we have this big star schema. Like, relational databases have great performance. It's just that we're going to have to fight uphill to use a relational database at Amazon.

Yeah. And the other side is just like, why are you rocking the boat? Like, it takes 3 seconds to make like a Dynamo table. We can set it up. You're, you know, you're anticipating that we're going to have

this complex schema. We don't have that complex schema right now. And uh we don't need any like weird approval to go and get a relational database.

And you can change the schema whenever with Yeah. Dynamo. Yeah. Yeah. And you get global tables and DAX and all that other stuff, whatever. So that would be a conflict story. Who's right? Nobody's right.

Yep. So the thing that we need to make sure of after I've set that up is we're able to make progress. So it could be that there's an escalation that's involved.

maybe that we bring in an objective third party and and then we make our case. It could be like, hey, it's functionally equivalent and yeah, they're right. Uh, it's a two-way door decision. We can just go and migrate it

over to to SQL afterwards or vice versa. That's how we want to make sure that it's a success for that story is that progress was able to be made. Does that make sense?

Yeah. Obviously, it resolved in some way. So, we don't want to lie and just be like again, you know, then everybody stood up and applauded because it was such a an amazing thing, right? But we don't want to come off as being too too stubborn.

We want to come off as being like a reasonable person. We want to make sure that we're not like vilifying the other side. Depending on the situation, it might be a disagree and commit situation where it's just like, yeah, I didn't

agree, but you know, it seemed like everybody else wanted to do it that way, and this was not the hill that I wanted to die on because the consequences weren't as bad. So, I I strongly disagreed with the decision, but I

decided to lean into it. Or it could be like, yeah, this is a hill to die on.

The consequences are unacceptable, right? Uh if we get it wrong, and so yeah, I'm going to make a big deal about it, right? But it'll be dependent on the the situation. The big thing that I think that's really important with

interviews and the stories that we bring is like we just want to demonstrate that we would be a good fit for that company. An initial draft of my book was like, "Hey, you know, within the delivery section or shipping

software section, it was like, hey, you should you should bring out stories where you like work nights and weekends and, you know, grind yourself to get all of this stuff done." And then a bunch of people that read the book were just

like, "That sounds really unhealthy and toxic." Yeah. Right. And then I'm like, "Huh, okay." Okay. So, then I like rewrote it and then I was basically like, okay, you should make sure that you like establish really great boundaries at work and

stuff like that, but then it's like that's a weird thing to bring up during an interview.

Yeah, you wouldn't do that. Then I'm like, should I tell people to like lie about how much effort that they're putting in, you know, to go and work at these companies where delivery is like really important? And what I

came down to is like, no, you just want a good fit. So, if you care about work life balance and you're trying to go and work at a company that has no work life balance, it's not about what story

you're going to tell. Like, that's just not a good fit. Maybe don't work at that company. Like, what if you get the job?

Yeah. You're going to be sad, right? I think there are a lot of people maybe, you know, in their 20s like they don't mind like sacrificing their relationships and they don't have a life. They're totally happy to sleep

underneath a desk. those people need to go find the companies where that's expected and that would be a good fit.

And so I think for you, we're talking about this in terms of like a conflict story, you tend to be a little less confrontational, but I don't think that precludes you from coming up with a good story, you know? I think you just need

to maybe you would be a much better fit at a company that uh was a little bit more easygoing. Yeah, I do worry a little bit that like there are certain company cultures that you just get completely steamrolled and

yeah, they're kind of looking for people that don't mind wrestling in the mud for extended periods of time and like breaking a couple eggs to make an omelette, if that makes sense.

Yeah. It's just not my personality. Yeah. And so, you know, I think it's a like when you answer this conflict question, I think you want to just be like, yeah, the story should demonstrate that like a lot most of these hills

aren't worth dying on for you. Yep. Right. And that, you know, if it's functionally equivalent, you'll just go their way. But I do think that you want to when you're going through that, not saying, "Hey, I just fall over." But I

evaluated it and I was like, "You know what? It's really not that important." Does that make sense?

Yeah. Exactly. I I was about to say not just saying I I do whatever the person wants me to do.

Yeah. Whatever you say, boss, I'm just going to go and do that. Yeah. Cuz that shows a lot of bad things, I think.

All right. Well, uh thanks so much for uh spending the time to talk with me today. It takes a lot of guts to get on camera.

Yeah. And be vulnerable to other people. Yeah. I think you're going to do really great on your interviews.

Yeah. Thank you. This was awesome. Yeah. Thanks, Steve. My name's Aaron. I'm a software engineer at Amazon and I'm currently looking for high mid-level to senior engineering roles. Walking into interviews, I sometimes felt like I was telling stories differently every time. Didn't

have a super clear path for each story. And it did cost me a few times not showing exactly what they were looking for. I think the biggest thing that clicked for me from doing this podcast was framing the story better, especially

to someone that maybe hasn't worked at Amazon, doesn't know any of the internal things, getting to communicate better to build that problem statement, build the story better. for my next interviews. I feel a lot more confident in my stories

and I'm definitely going to think about how I frame them based on the company's values and what they're looking for and trying to pattern match better. I would say for someone doing these kinds of interviews wondering about coaching, I think a great thing

from it is it's tough to learn by yourself with these things. They're they're more complex, nuanced questions, and having another really experienced person's opinion is very helpful in shaping your stories better.