Speaker A
this is the neelen Norman group ux [Music] podcast I'm theres fessenden the end of the year is often a time for reflection on our professional and personal lives we often consider you know how we spent our time did we
Explore how behavioral science enhances UX by choosing better user impact metrics for real behavior change and meaningful outcomes.
Key Takeaways
- Behavioral science provides a deeper understanding of how to motivate real behavior change beyond traditional user outcomes.
- Choosing precise, behavior-focused metrics leads to more accurate measurement of user impact and business success.
- Behavioral science is not inherently manipulative and should be used ethically to support user goals.
- Integrating behavioral science with UX research enhances product design and user retention.
- Measuring both leading (behavioral) and lagging (outcome) indicators is crucial for evaluating product impact.
Summary
- The video discusses the importance of selecting meaningful success criteria beyond superficial metrics in UX and product design.
- Behavioral scientist Connor Joyce explains how behavioral science complements user research by focusing on deeper behavior changes that lead to desired user outcomes.
- Common misconceptions about behavioral science, such as it being manipulation or dark patterns, are clarified.
- The integration of behavioral science helps define precise behaviors that drive user outcomes, improving both user satisfaction and business goals.
- The conversation highlights frameworks like design thinking and jobs to be done for understanding user goals and behaviors.
- The episode stresses the challenge of measuring true user outcomes and the importance of leading and lagging indicators.
- It covers the role of usage and satisfaction metrics alongside behavioral metrics to ensure product success.
- The discussion touches on the ethical use of behavioral science to avoid manipulative practices.
- Practical advice is given on gathering fine-grained behavioral data using tools and aligning teams around impactful features.
- The episode encourages UX professionals to adopt a behavioral science lens to create products that genuinely help users follow through on their intentions.
Chapters
- 00:00Introduction and Year-End Reflection on UX Success Metrics
- 02:54Behavioral Science Lens in Product and UX
- 05:42Behavioral Science vs User Research and Defining User Outcomes
- 08:21Design Thinking and Broader Perspectives on User Outcomes
- 11:02Linking Leading and Lagging Outcomes to Business Success
- 13:38Challenges in Measuring True User Outcomes
- 17:12Role of Usage and Satisfaction Metrics
- 18:54Ethical Considerations and Avoiding Manipulative Practices
- 22:05Aligning Teams and Gathering Behavioral Data
- 24:31Final Thoughts on Behavioral Science Impact in UX
Full Transcript — Download SRT & Markdown
Speaker A
cultivate growth in the areas of Our Lives that mean the most to us or perhaps are there relationships or behaviors that we'd prefer to stop doing to make room for growth that we actually care about these are deep questions but
Speaker A
arguably the most important ones and often ones that most ux professionals or product folks don't clearly answer as much as we think we do so if I had to propose a resolution for our industry as we near the new year maybe it's this
Speaker A
let's choose success criteria that aren't superficial and focus on user outcomes that act matter to dive into this I interviewed behavioral scientist Connor Joyce who's recently authored the book bridging intention to impact in this episode we discuss what Behavioral Science really
Speaker A
is and some common misconceptions about it how product and ux teams consistently overly rely upon problematic metrics and tips for choosing more precise criteria and metrics to get more accurate perspectives of real user behavior and enact real change in the people were
Speaker A
meaning to to serve here's Connor I'm stoked to have you here because of your book obviously that's huge news congratulations um and I'm excited to talk about it because I feel like it's a much needed topic given everything I
Speaker A
know about ux professionals given everything I know about Behavioral Science so definitely interested in unpacking all of that but to get started to give our audience a sense of how like who you are what brought you to writing
Speaker A
this book I'd love to know a bit about you so your Behavioral scientist what does that mean what do you do yeah yeah it's it's it's a great question it's one that I get get pretty often and and I am a I'm a behavioral
Speaker A
scientist I'm I'm also a user researcher I'm also a product developer I like to say that Behavioral Science is as much a minor to other jobs as it is a major in itself in other words it is I bring my
Speaker A
Behavioral Science skill set to a lot of the different roles that I've taken whether it be in full-time roles or through consult and and and startup work and Advising it's uh being a behavioral scientist is really just bringing the
Speaker A
toolkit of understanding human behavior to be able to help create behavioral change in whatever the user is trying to do and within the product space that is what most of the different disciplines are trying to do is we want our users to
Speaker A
do something because if they do that thing hopefully then they'll purchase our platform or retain on the platform or whatever business outcome that is being pursued and so for me coming in with this Behavioral Science lens I just
Speaker A
have the specialty of thinking through well how do we actually get people to follow through with whatever intention that we're setting for them with the product experience that's super interesting and I'm excited to talk about this more because I think by and
Speaker A
large ux and Behavioral Science have a lot in common I mean we're doing very similar things we're carrying out user research we're building user experiences and Behavioral Science is really the Deep dive into how we can motivate people to take certain actions and
Speaker A
perhaps change behavior and I think even the concept of behavior change is pretty amazing uh to talk about so I'm excited to get into that um what do you think is a common misconception maybe that people have about Behavioral Science because I
Speaker A
do think maybe some of that may play a role too and how we evaluate our impact when it comes to behavior change a lot of ways I can answer that question and I'm going to say two quickly and then I'm going to talk about
Speaker A
a third that's closer to the work that I actually do one is that Behavioral Science isn't manipulation it can be but it isn't by default manipulation there is a lot of times when somebody wants to do something but they can't and it's
Speaker A
because Behavioral Science as core says that we are not perfect economic beings we have faults we fault into Temptations we're human and so we can't always accomplish the goals that we set for ourselves and Behavioral Science can be
Speaker A
used to help with that and in my book that's not manipulation Behavioral Science is also not dark patterns which in the design space is a manifestation of manipulation or is a manifestation of using Behavioral Science techniques in a
Speaker A
negative way but there are also great techniques that one can use to create great features that help a user accomplish whatever goals they have so I always like the level set and say Behavioral Science it can be used for
Speaker A
bad and it's I'm not going to say it isn't or cannot but it does not guarantee it's going to be used in a manipulative or a dark pattern type of way so that that's that's wanted to start with that just to clear that out
Speaker A
because that's a conversation that I get commonly for me what I how Behavioral Science I don't even say is different than user research I think that the place that they they really come together and where Behavioral Science brings in that that great evaluation or
Speaker A
what am I trying to say the augmentation to user research is that Behavioral Science helps Define a level deeper than a user outcome user outcomes are the Bren butter of good design good research really understand what that user wants
Speaker A
to accomplish in their life and and if you do that correctly you can build something that again ideally suddenly it will satisfy it and then they will retain onto the platform on the flip side there is generally a need to better
Speaker A
Define how do you actually accomplish that user outcome and there's a few Frameworks out there again design thinking can do that when you're going through the Double Diamond process or really trying to distill the different solutions that are possible similarly
Speaker A
jobs to be done in outcome driven Innovation those help you think about what exactly is the person trying to satisfy and how are they going to go about doing that but defining the specific behaviors the ones that yield a
Speaker A
change to that user outcome that is where I think Behavioral Science plays a really powerful role and it is that is where it can be different than where traditional user researcher or or user design ends and it goes a level deeper
Speaker A
but that level deeper is a great place to play because it sets you up with a whole set of new metrics that now you can be striving to how are we actually going to design something to change that
Speaker A
behavior with the confidence that it will yield the ideal both user and business outcomes that someone's aiming for that's really interesting um because I feel like there's two really important pieces here there's the user outcome that you're mentioning and there's also the concept
Speaker A
of things broader than user outcomes and I think I think it's even in the term user outcome right so there's often a lot of criticism in the realm of ux about like these are people these are human beings they're not users right
Speaker A
some people often criticize like the only people who call their users users are you know the drug the you know Pharma industry and uh Tech and so I I understand the sentiment in a way where where users kind of fixates on a very
Speaker A
specific context of use literally like looking at a tech piece of technology but we're looking kind of big in a way we're thinking about outcomes perhaps in a person's life maybe objectives motivations they may have um and and I
Speaker A
do think we'll we'll get into this as well right we're thinking a bit more broadly at some of these deeper core drives in a way so we've certainly talked about design thinking in the past and we are now also going to talk a bit
Speaker A
about that in your book uh that's a pretty big topic um I'd love to ask a bit about you know design thinking is wonderful and amazing and it's such a great framework we we teach it at n ourselves one of the challenges though
Speaker A
is cool we we have this research we have these great designs we'd like to propose how do we know whether we're actually moving in a direction that is making a positive impact and it's hard to really know that we're doing that I mean there
Speaker A
are metrics there's qualitative research there are there are ways we can gather that Insight but it can be hard to communicate that and that cracks me up because when I think about ux we care so deeply about users we care so much about
Speaker A
good communication and we are so bad at it ourselves like uh it's kind of ironic in a way so I would love to know a bit you know how why are we so bad at this like you've written a whole book about how
Speaker A
we're so bad at this not not offending ux people but just like product people ux folks like this is a huge huge challenge for so many of us enough that it's driven you to write a book so could
Speaker A
you share a little bit like what are some of the common issues with how we evaluate our impact when we make these designs so one is the difference between leading and lagging outcomes we'll touch on that another ties back to what I was
Speaker A
previously talking about which are these behavioral outcomes in other words how much is a product changing behavior and then the third one is is really just having a solid feature def to be able to talk about how research is
Speaker A
influencing it so I'm going to take these piece by piece but I want to lay out those those those three pieces so the first one's leading and lagging outcomes just like we have it for indicators of any sort of key
Speaker A
performance indicator or or um okr and and more business lingo behaviors are the same way what makes dual lingo so powerful it's because it does both create outcomes at the lead way I want to go and keep my
Speaker A
Streak by keeping on or by fil by completing a lesson or if we look at pelaton I get a reward because I completed my class the apple rings when I close it I get a I get a nice
Speaker A
notification that says it so all of these products they do a really good job encouraging somebody to pursue whatever actions are needed to satisfy that leading outcome just get a task done get a lesson done get a workout done Etc but
Speaker A
those leading outcomes tie into lagging outcomes in dual lingo case it's learning a new language in the in pelaton or the Apple watch it is losing weight or getting increased Fitness or whatever the person's attempting to do with their workouts so that's the first
Speaker A
piece is that that nuanced separation I think is very important to say we need to satisfy somebody so every time they use that product they are going to be feeling a little bit better so that in the long run through usage of
Speaker A
the product it's actually going to do something for them it's going to help them learn that language it's going to help them lose weight or ACH accomplish their Fitness goal or whatever the product's intent is that that difference
Speaker A
between the leading and lagging is an important piece the next one is to do that you have to measure those behaviors like I previously said a lot of the times teams that I work with will end their definition of a feature by saying
Speaker A
it is going to satisfy this outcome and and I'll say well how we like well that's the design part we're going to figure that out with the design but no that's not the design part yes you need to design the solution but you actually
Speaker A
should first Define what is this behav what behavior is being changed by whatever solution is going to be created and only with that level of definition are you then able to design a solution that will maximize those measurable
Speaker A
behaviors in a in a fitness example this would be measuring how many calories are burned each workout and then designing a feature to increase those calories burned over time in in um the Dual lingo case and I'm not a dualingo user myself
Speaker A
but let's just for the sake of the example it would be every time ensuring that person advances ever so slightly in their ability to learn that new language you would need to be able to learn what behaviors are tied to that so that that
Speaker A
could be measured in whatever solution is designed so you've got the leading and the lagging outcomes to really do that you need to be able to define the behaviors that fits into that last piece is to truly Define the behaviors you
Speaker A
really actually have to have a definition for the feature how is this feature fitting into our overall product to help that user achieve those behaviors that then connect to those outcomes and that's that is the thesis of my book is is what I call the user
Speaker A
outcome connection it's specific behaviors when changed lead to a user outcome I ideally both a leading and a lagging user outcome it's helping them do something today that will make them feel better tomorrow and then in the end
Speaker A
it's going to drive business success because that person's going to go oh this weight loss app it's actually helping me lose weight or oh this language learning app I'm actually learning this new language and so they'll they'll retain they'll tell
Speaker A
their friends which will increase organic marketing and they will be able you they will be a great Target to upsell them or or ultimately they're just loyal customers yeah I think that's a really important Point um and for those folks who are
Speaker A
listening who maybe aren't as familiar with the design thinking framework one of the key components at least one of the key components we talk about at neelon Norman group is the importance of especially the first kind of third of
Speaker A
that process the first third of that process involves a lot of understanding and defining the problem and to your point uh what you were mentioning earlier Connor about how often are like okay this is the user outcome or we're
Speaker A
going to design you know a feature and then we're going to evaluate based on the design in a way you're saying hey hold on we can't really evaluate our success by designing and then choosing an evaluation criteria we have to figure
Speaker A
out what is our outcome what is our evaluation criteria for that outcome and then the design follows afterward right and and it kind of needs to be something factored into why we're even making these decisions in the first place so
Speaker A
thinking about uh you know we kind of have this exercise in the design thinking class about you know what does this person need and we introduce this activity uh by showing an image of a little girl kind of reaching for a book
Speaker A
um it's a very popular exercise um but it's a little girl tiptoeing reaching for a book on a top shelf and so by all you know objective observation you might say well she's tiptoeing she needs a way to reach the book so like she needs a
Speaker A
step stool or if we want to just jump to what really needs it's the book on the Shelf that's what she's reaching for okay those are both valid answers right that's in a way we're saying that's the feature right that's the feature that
Speaker A
she needs she needs the step stool she needs the book but then when we really unpack that why does she need the book well the book is not the outcome right the outcome is what she's going to do
Speaker A
with that book is she going to read the book is she and then when she reads what's it for is it for a book report or is it for uh just she likes books and she's excited about it right those are
Speaker A
some of the deeper questions that really good user researchers like they know that that's that's the Nugget of wisdom that's going to help them make a really impactful solution and so I appreciate that idea of the you know the leading
Speaker A
and the lagging outcome because maybe the leading outcome is she reaches for and gets the book but the lagging outcome might be she learns she improves in school or whatever that may be so I feel like that's a really important
Speaker A
point to hammer Hol in how we evaluate success what are some issues in how people choose you know how what they evaluate or what they're measuring I guess to put it since I I say success criteria not just thinking about metrics
Speaker A
but naturally metrics is where people kind of gravitate so I'm just curious if you could kind of talk about it from the perspective of like choosing the wrong metrics like can you give an example of what what would be like a wrong metric
Speaker A
when thinking about some of these you know outcomes to to really answer that question I'm going to take it one step back and then I'll bring it back to the question is it starts with making sure you're choosing the right behaviors that
Speaker A
are connected to that user outcome because if you choose the wrong behaviors like if we say like a weight loss app really there the more science is showing weight loss is not exercise it's eating behaviors you can exercise a
Speaker A
lot but if you're still overeating it's not going to have that much of an impact so you can build the best weight or exercising app out there and it still may not be what is needed to to achieve
Speaker A
that outcome so if you choose the wrong behaviors then it's it's it you could again you could maximize every success metric but it may still not actually impact that outcome so it starts with choosing the right behaviors and that
Speaker A
process is not easy so and that's the biggest thing is people say well this this doesn't sound very easy and it's not but Building Product isn't easy and if you think it is then you're probably over you're over tooling on building for
Speaker A
usage and not actually building for impact in the book I talk about two processes on one side you can use things like ethnographic studies where you literally go and watch people how are you trying to solve this problem today
Speaker A
you can do user interviews you can do a lot of the the really the the the good work that many user researchers are out there doing right now is actually already very close to trying to do behavioral analysis you just need to
Speaker A
take it a step further and really actually kind of ask them how are you currently solving this today what would it look like to try to solve it doing this Behavior the other resource is that generative AI actually does this really well I've been
Speaker A
pleasantly surprised by the perspective taking that the systems are able to do they if you go in there and you say my user outcome or my goal is I want to meditate more often how do I go about doing that give
Speaker A
me 10 suggestions it's going to give you 10 decent behaviors that could be changed to help you meditate are all of those the right ones no that's research is still needed testing is still needed it's not a silver bullet it still is
Speaker A
challenging but generative AI will do a great job pulling in especially if you use it where it's giving you resources like within chat gbt search or within perplexity or one of these they're going to give you a lot of ideas that are
Speaker A
sourced that you can go and experiment and see with our real users do they think that having a dedicated spot in their house where they meditate will help them and if so we could build a future that helps them do that during
Speaker A
the onboarding experience of this app so it is it's that those types of questions that focus on finding the right behaviors then once you find the right behaviors to Circle back to your question of how do you define the
Speaker A
success metrics it's just about building the closest metric you can to that behavior now having that person build a dedicated spot in their house for meditation you may not able to have that guaranteed you event level or in other
Speaker A
words like technically build a metric that says whether or not that happens because it's happening outside of the digital environment but you can build a proxy you can maybe have them take a picture when they're setting up that
Speaker A
their their profile for their account that shows their dedicated spot and if they don't never upload a picture you can say well they probably never followed through with that Stu you could ask them collect attitudinal data have it be at the end of a meditation session
Speaker A
did you do this in your dedicated meditation spot and even you know if we're going to get wonky you could try to find it in some some way within the product to be able to measure that maybe if the gyro sensor on a phone is laying
Speaker A
it down you at least know that person put it on a table or something you know we get creative is what I ultimately say is once you've defined those behaviors if the behaviors take place in a digital environment You're Gold all you need to
Speaker A
do is throw in whether it be M partical or segment or another event level collection system to get that very fine behavioral data you know what that person is doing in any of your surfaces and you can measure that behavior down
Speaker A
to the finest of levels if it is happening outside of your environment you do have to get creative but it's all about finding proxies is you just try to find what data can we pull in what could we have the user do that would show us
Speaker A
this whatever that might be and then in the worst case scenario you ask the user and then they will always be able to tell you whether or not that behavior was taken now granted they don't always tell you anything and that's why
Speaker A
attitudinal data can be hard is they don't have to tell you but nonetheless hopefully that paints the picture of how choosing the right success metrics starts with defining the right behaviors and then finding the metric that matters which would be like that event level and
Speaker A
if that can't exist then it's building a proxy that gets as close as possible to that I think that Insight is super helpful and I also appreciate what you said about and I think the phrase you Ed was
Speaker A
over tooling on usage data which is so interesting to me because I think usage data is something people really really zero in on or we immediately point to say analytics and granted like you're saying if if the interaction takes place
Speaker A
in that digital space perhaps there are going to be certain analytic data you know you can reference like are people is the objective to be more productive okay maybe there are some pieces or proxies of that um so an
Speaker A
example might be is this person sending more emails are they creating more calendar events one might argue that's productive is me having 50 calendar events in a day productive debatable right maybe it's not maybe it's not but but that's a
Speaker A
really important question right is this the best proxy and and it does introduce some really philosophical questions like what even is productivity or if we are creating say a mindfulness app to use your example is someone achieving more
Speaker A
mindfulness that's that's a really difficult question to answer and if you were to take a metric to Define that maybe you could use some like you said some attitudinal survey data we could ask someone like how are you feeling
Speaker A
today um do you feel like you are more mindful now than before right and it's it's also a bit hard to not ask it in a leading way either right to make sure people are answering honestly and reflecting so so it's it is hard like
Speaker A
you said it is hard to get this data it is hard to build a product and it's even harder to get accurate representation of true user outcomes right that and really seeing are we making that measurable difference and and in a way I I'm also
Speaker A
really glad we're having this conversation too because like I said there is this fixation with analytics partially because it is easy to get um and I think it's certainly a better place to go than not getting any sort of
Speaker A
measurable data but uh but I like this conversation and that hopefully it can in kind instigate this creative thought of you know where else can I look right or what other sources of data can I perhaps start Gathering to ensure that I
Speaker A
am looking at a a realistic success criteria right so yeah this is amazing um now one other thing to take this idea of research right you you mentioned the importance of research in evaluating features perhaps research like ethnographic research um and it
Speaker A
certainly sounds like a combination of both qualitative and quantitative you know uh analysis are are useful here because it can give you in a way like a sense of triangulation you could say okay I can see this qualitatively uh I see people are
Speaker A
carrying out these particular tasks or have these particular concerns or have these particular barriers and I can also you know validate that perhaps with some of these other survey data type things um do you see any particular issues with
Speaker A
how people run this sort of evaluative research like whether their features that they're building are actually making that impact I think the the most common mistake that I see in the evaluative research is not matching the evaluation technique to
Speaker A
what is necessary and it happens on on both sides I have seen we have to go in and test if this feature is working to the point where we have statistical significance and we know down to the finest detail when the product team was
Speaker A
like Hey can you just give us a pulse and let us know if we're heading the right direction so way too over scoped I've also seen the other the flip side of okay well let's let's get this this
Speaker A
feature in front of five users and just ask them ask them if they think they' Chang their behavior from using this and the team's like we're about to invest in scaling this across you know the full product we we really want to know the
Speaker A
direction of this so to your earlier point I believe all good research has both a mix of attitudinal and behavioral metrics and to your earlier point I do believe the biggest mistake in modern product management is the over focus on
Speaker A
usage I also believe usage has a role to play it is a indicator as is satisfaction you could build something that is tremendous at changing Behavior but if nobody likes it they're probably not going to use it and then it's going
Speaker A
to sit on a shelf somewhere and it's never going to actually achieve its its goal so it's within the evaluation my gold rule is try to do as many levels of outcomes that you can in my book I talk
Speaker A
about five there's usage which we've talked about there's usability which is satisfaction ease of use the ones that a lot of ux researchers kind of their bread and butter there's behavioral which we've talked about a decent amount here this is are you changing those
Speaker A
behaviors there's user outcomes that is any of those leading or lagging outcomes and then there's business outcomes and this is the piece that I also see that gets that is not that common in a valuative research but it is
Speaker A
are we sure that when we satisfy that user outcome people are actually going to do what the business desires again retention upgrades increased loyalty whatever that might be and so it is about in my opinion to to the ACT to the
Speaker A
actual question that you said of mistakes I would say mistakes is over focusing on one metric and then not scoping the evaluative research to really be create the level of evidence that is necessary for the decision to be made on both
Speaker A
sides again this is sometimes overs scoping sometimes unders scoping and then my remedy for it is to try to include as many layers of metrics as possible really try to get those five levels of of success metrics as I call
Speaker A
them and then as you're designing the study designing the evaluative piece understanding what is the team actually asking for and then scoping the research in a way where I've you heard the name minimum viable research recently which I'm digging because it's
Speaker A
like what is the minimum research that we can do here to create the level of evidence that is necessary for this team to move forward and if you do that and at the same time you scope in as many of
Speaker A
those levels of metrics as possible I think that that is the path to help to not just helping the team make the decision but building their trust that research is playing a role where they're able to contribute to the product
Speaker A
development and the business outcomes the concept of minimum viable research is one I'm also very excited about but it's funny because when I talk to some researchers like you could sort of see this look of mild horror on people's
Speaker A
faces like what less research we already don't do enough research and it's like I I understand both sides of that coin cuz yeah there are plenty of organizations that especially now in the age of AI are just like why would I do research when I
Speaker A
can just ask chat GPT to give me like the top user needs and it's like I think especially in the age of AI That's super relevant and super helpful because maybe now there's even less desire to pay for
Speaker A
this when there's a belief that well I can get this from AI but if you could say hey if you do just a little bit more and it's not going to cost you very much then you are going to have even stronger
Speaker A
more impactful insights and not only that like you're saying it's going to help give the team a sense of wow I'm going to have a lot more Direction here or wow the impact of the product I'm developing I I get to see that I get to
Speaker A
see that qualitatively by watching a usability study and and seeing someone their face light up because something is working properly or the opposite I can see someone really struggling and that's giving me you know lighting a fire under
Speaker A
my team to help us want to improve this make it even better so minimum viable research I feel like is a really important component um but then the other piece of this is uh you you mentioned those five levels of success
Speaker A
metrics and how as long as you have those layers you're you're keeping a well-rounded perspective and the thing that came to mind immediately uh especially when when talking about zeroing in on things like usage is Campbell's law which if you don't know
Speaker A
about Campbell law it's basically the more important a metric is in decision- making the more likely it is to be manipulated or gamed in some way so some examples might be you know maybe you finished a customer support call and
Speaker A
someone's like hey you're going to get a survey you know really help me in retaining my job if you could uh answer the survey and then because now you the caller are like oh no this person might get fired if I don't answer this or if I
Speaker A
don't answer this positively it may kind of skew your answers to be a bit more positive than maybe it actually was right so uh that may be a particular problem especially when we fixate on a singular metric right if if a person
Speaker A
gets penalized because of a singular metric or a person is incentivized maybe a bonus structure hinges entirely on did we up our number of users who signed up or did we increase uh usage time overall then that may encourage some of the more
Speaker A
nefarious behaviors kind of the ones you were saying earlier the dark patterns the manipulative um or receptive types of patterns right that gets incentivized now because that's how I'm going to get my bonus at the end of the year so uh I
Speaker A
think you're right having this multi-layered perspective helps to avoid some of the ethical dilemmas that may come from you know overly emphasizing a singular metric and perhaps even gets us closer to better proxies of of actually creating more impactful you know designs
Speaker A
that people like using you know creating that organic marketing that you mentioned earlier 100% I I I agree with all that and I would even just you know highlight I like that Campbell's law uh I think the only time the last time I heard it I
Speaker A
should say was in nng course that I was taking so it was good uh good reminder because it is such an important piece and it's why I think you usage is such a it's it's again I'm not saying it
Speaker A
should never be used because it is very you need to build something that is used but it's such an easy metric it's the default metric that people just want to clamor on to it and then they maximize it and then the VC for startups or the
Speaker A
market for public companies say o growth we like growth we know what that means and so then they maximize it more and then everybody's just building for usage and then when retention starts to drop they'll go well wait a second people we
Speaker A
might even still be growing but now we're not retaining people and some point we're going to saturate this this pool of users what is going wrong and that's the first time that the team asks well is what we're building actually
Speaker A
working for our users and that's a very very late time to be asking that question absolutely yeah like like you're saying very late we've already disappointed in a way lots of people right and and that's not the time to be
Speaker A
defining these things um certainly earlier in the process is better um so I have a couple more questions one of which is is about this impact mindset which is a mindset you talk about in your book could you share a little bit
Speaker A
about what that mindset is and how people can start to cultivate this sort of mindset in their day-to-day work yeah the impact mindset is is we we've been talking about the impact mindset it is starts with the user
Speaker A
outcome connection that is again defining every feature based on the specific behaviors that it's going to change or is changing how that connects to to a user outcome and then how that will eventually create a business impact that user outcome connection why
Speaker A
I believe it's so powerful is it starts by giving definition to a feature and I've already talked about why that's great for for helping create metrics it's also great because it aligns the team around what needs to be done as
Speaker A
many times I've talked to teams and say why exactly are you building that feature and people talk they look at each other and they go well this customer asked for it or we think it we think we need something that's going to
Speaker A
do something over here and then I push them a little B more and realize well a definition would be really helpful for creating alignment within this team user outcome connection is the main foundation of the impact mindset because
Speaker A
it also creates a def it creates a need for evidence because within that user alcome connection each connection needs to be validated there needs to be evidence that says that if you change your behavior it will satisfy that
Speaker A
outcome if you satisfy an outcome it will create a business impact and then you also need evidence that says our feature is actually changing that behavior so there's three assumptions that in are inherent to defining a feature using
Speaker A
that user outcome connection and that's the next part of the impact mindset is you need to go out and validate those assumptions that's what I call the feature impact analysis and it's the bulk of the book because it is a pro a
Speaker A
step-by-step process for how to increase experimentation and research within a company by defining your features with the US outcome connection and then going out and either finding or creating the evidence needed to be able to ultimately validate that the user outcome
Speaker A
connection is correct so the team can confidently invest in a feature knowing it is changing Behavior it is satisfying both user and out and business outcomes beyond that there is the fact that the more research you do you want
Speaker A
to have a home to put those in insights in and so I call that an insights Hub some people like to call a research repository there's more conversation I'm not the only person that's talking about this but what I my addition to that is
Speaker A
that it shouldn't just be for research it should be where the home of where features are defined too so again it's where you say here is what we believe our features are doing here is the evidence that are is directly tied to
Speaker A
why we believe that that is true and then the last piece is if you do all of this it is leaning the towards an evidence-based decision-making culture because instead of creating a feature just because one loud customer suggested
Speaker A
it or because a product manager is leading with their intuition You're Building features because you are confident that you're going to be changing behaviors that you know do create impact that you you desire absolutely and I guess one last
Speaker A
question I have for you is do you have any tips for people who are looking to start who may be a bit apprehensive of stepping on like leadership's toes if they maybe pick the maybe leadership has actually zeroed in on usage right I know
Speaker A
you mentioned usage is a good thing so obviously it seems like tip number one is don't throw everything out of the window but but what other tips can you share uh for you know getting the rest of your team to start embracing the same
Speaker A
mindset start retroactively that's what I did the very first time that I I thought about these ideas when I was in my masters and then I wind up at a company out of my masters and I wanted to test these ideas out and I started to
Speaker A
pitch them and I and I got little support at first to that point the company I was at was really focused on usage and satisfaction not as much about these outcomes so how I found succcess was I went to a feature that we had
Speaker A
already launched instead of trying to get this baked in proactively I went and found a feature and I asked the team to do this type of analysis where we would go and Define the behavior that feature was supposed to be changing and then
Speaker A
measure those and lo and behold it was working we now had Behavior behavioral data that showed that the feature was successful at accomplishing the behavioral change that we hoped for and that became an opportunity for our marketing team to reference that
Speaker A
during sales it was at a B2B company so we were able to have our marketing team go out and tell potential customers we are building features we are confident are working at accomplishing why you are purchasing this software and that was
Speaker A
that was valuable to the point where the team said well maybe we should integrate more of this into the proactive feature development process and I was slowly able to integrate some of this thinking into how we actually built product from
Speaker A
the ground up and so that would be today the the two recommendations I would say is start retroactively and don't think you need to have everybody on board you really just need two people you need one person to do the work of defining all
Speaker A
these features and then you need a data scientist or a data engineer or engineer overall to go and build the new metrics and if you have the right systems in place it shouldn't be that much of an effort to do it but you'll just need
Speaker A
somebody that has some technical knowhow to be able to to to to gather some of these metrics and so start there and let that organ that seed organically grow because by asking the right questions it's you're going to find other people
Speaker A
who want to answer the questions that's a really good way to put it because I it's rare that someone like hears a great question and it's just like how dare you ask that question it's more like wow I've never thought of it that
Speaker A
way right or hm maybe we should learn right so so I agree I think maybe instead of framing it as we have the wrong metrics like don't go that way but instead start asking questions like do we know if people actually do this right
Speaker A
um that's actually how we started asking questions at n as well do we know if people are applying the skills that we teach how do we learn that and and this is kind of an ongoing experiment this is
Speaker A
something we're we're constantly trying to learn about and um it may not be easy to get the exact answers like you're saying but maybe even a close proxy but certainly asking the right questions is a great place to start so Connor Thank
Speaker A
you you've given me a ton of food for thought time has flown by I know we're even a little overtime um but just to close um if anyone wants to learn more about you and your book could you share
Speaker A
a little bit about where people can find more information about you and how to get this book yeah so the book is bridging intention to impact you can find it on any major retailer Amazon Peach pits Barnes & Nobles um me I'm on
Speaker A
LinkedIn and that's where my my main home I also write on medium and I write for a collection of Publications but if you want to just see what I'm up to what I'm publishing uh the the home is for
Speaker A
LinkedIn and then I have a company desired outcome Labs so if there's uh any interest in working with me or or working through some of the things that we talked about in this this show um you can reach out to me through through that
Speaker A
that channel also awesome well thank you so much Connor it's been an absolute pleasure talking with you today it's given me a lot to think about as well and uh I hope you have a great rest of your day thank
Speaker A
you theres you too that was Connor Joyce you can find more information about him and his book bridging intention to impact at the links in our show notes if you want more insights on on user Behavior measuring and communicating the
Speaker A
impact of your work and many more related ux topics check out our website and you'll find thousands of free articles and videos you can also take halfday and full day courses from people like myself and other amazing ux experts
Speaker A
go to www.group.com that's n n.com finally if you like this show and want to support our work please leave a rating and follow or subscribe on the podcast platform of your choice this show is hosted and executive produced by
Speaker A
me theres fessenden all editing and post- production is by Chrissy Richardson that's it for today's show until next time remember keep it simple
Topics:behavioral scienceuser experienceUX metricsbehavior changeuser outcomesproduct designdesign thinkinguser researchbehavioral metricsethical design
