Two Sides of the Same Coin: How LinkedIn Aligns Customer Learning and Community

Hello, my friends. It's all about when you know more, when you learn more, you accept and get new thoughts. And that's what happened with me and my cats. So I'm converted.

I was really loud about the cat hating previously. So it's been a journey to like fully embrace the love, but I love them so much. Friends, I am so excited to chat with you today. I have obviously the most exclusive time slot, which is the last day and the last time slot.

So I am choosing to look at this as like, if you are in this room, you care a lot about the intersection between learning and community. So that means you're in very good company. I have been in this room actually most of the day, listening to my peers who are thinking through this and figuring out how do we do this. Some have talked at very broad levels.

Some have talked at very narrow levels, all of which has been incredibly important. Simply here to talk about our journey. So, oh, hey there, who am I? I'm Stephanie Hurst, I already said all of that.

I have been with LinkedIn 14 years. The journey though, there has been an interesting one. I came up through this CS organization, focused on making our customers as successful as they could possibly be. And then moved over to lead the learning organization for one line of business.

And friends, it has been a wild ride since then. Absolutely wild. We added lines of business. We divested lines of business.

My organization has moved other organizations like five different times and like, we are now in marketing, which is, we really love being in marketing. We're really excited about that. But what I'm trying to explain to you is that like, we've been through it, if it can be gone through, we've probably been through it in some way. But where we are landing, I am so excited about because it's going to lock us in to do more powerful output than ever before in a more clearly articulated way as to how that affects the bottom line.

And that's what we're going to talk about today. LinkedIn is genuinely trying to make the world a better place. We are trying to connect all the world's professionals to make them more productive and successful. These are my people.

And these are the people I'm trying to make the world a better place for that's my family. So, you guys will all tell them that I showed them off. So before we like dive in, you know, hop into the phones and answer the questions. I just am trying to figure out who is in the room.

So, you know, who, like organization by organization, do you even have an education, yes or no? Like an education team. So, as we're kind of pulling in, seems like universally. Wonderful, probably at the right conference.

Great, good to know. Try the next one. Okay. You do or do not have a customer community team.

Ooh, this one's a little bit more, a little bit more of a horse race. Yes, I do, no, I don't. That makes a lot of sense. Because I think like, innately we understand that if you train someone to do something, they are kind of likely to do it, right?

But community seems like a really good idea. And how do you get the outcome from it? A lot of us struggle with that. We'll talk about that a little bit today.

Okay, interesting, great. All right, next question. For those of you who work at companies who have both teams, do they share the same goals, leader, definition of success? Or I would even back that out to just say goals and definition of success.

I think like people who have come to this conference are probably closer to that happening, but this is a very clear industry trend. And something that I think those of us who've run learning organizations have really tried to kind of put a finer point on what's missing. Those of us who have run community have tried to put a finer point on what's missing. And as it turns out, it's the dichotomy between the two.

So let's go ahead and like get into it. So you guys now know, like who is in the room. There we go, let's, did I do it right? Did I do, yes I did, okay, great.

So historically the way that we have thought about the two different silos and how they have worked, education is like pretty straightforward. It's coursework, it's certifications, it's webinars. We care a lot about activity. We care a lot about engagement.

We measure that engagement and we show it off. We had one bajillion people come to our webinar. Isn't that really important and exciting? Historically, although it was not the case at LinkedIn, but historically my industry peers have often reported up into like product as an organization or marketing.

We have just now finally made our way over there, over to marketing. And then community historically has focused very much on activity. Like this is the sense of connection, of course. And this is the way in which we talk about success historically has been very much based on how much activity is going on and like what kind of like traffic drivers.

There's nothing wrong with any of this, but what I am trying to point out here is they are two totally separate things. They are two totally different goals. And if you have two totally different goals, you are going to arrive in two totally different places. And so that's what we're gonna try and figure out is like how do we just get the one goal?

So here's how we think about the two sides of the coin when it comes to learning and community. So learning without community is a lecture without classmates. So picture it. You've been accepted to the most exclusive university of your choice, educational institution of your choice.

You are beyond excited to join. You even got into the class with the professor that you were hoping to hear and learn from. You show up on the first day with your first day of school outfit, obviously. And you walk into the enormous lecture hall and you sit down and it's a little echoey in there.

You have no one else sitting next to you. Now your professor who you're super excited about comes out and puts on quite a show. Very clear that this professor knows what they are talking about, right? But you don't have anyone next to you to lean over to and be like, wait, wait, wait, what did she just say?

You don't have a pal to leave that lecture hall with to go argue with at the coffee shop over something that was really boldly stated, right? So technically you got the information and maybe you even learned, but you weren't necessarily inspired. In fact, you might have had this feeling of like, ooh, creepin', why isn't anyone here? What's happening?

It feels very lonely. It feels very lonely. And there was like this feeling of, we're missing the emotional element, okay? Community without learning.

That's a whole different vibe. That is chaos without a curriculum. Okay, so same campus. Guys, we're still very excited to be here.

And we've been invited to the most exclusive party on campus, obviously. And so you've gone and all of the peers that you were hoping to meet are there. We are chattering on about all of the things. The vibes are immaculate, absolutely immaculate.

However, you do know that you're going into class, back to class on Monday morning. And my question to you is, did you learn anything at that party that will be of value to you in order to move your educational career forward? Maybe. I don't know if my college experience, the answer was like confidently no, okay?

Confidently no. Been a real good time, folks. And so, you know, like let's put the two together, obviously, is what we're here to talk about. So here's what this looks like at LinkedIn.

And this is like that one unified goal. And we try, we do not always do this in a really great way, but we try really hard to do plain speak. Talk to me like I'm a five-year-old. So I'm about to walk you through in the very most basic terms.

What we are trying to do on the team is get the most users possible. I'm using users as kind of like a broad term here. We can argue over that, admins, does that count? Blah, blah, blah, we're just gonna say users.

Product users. My team looks after a whole bunch of different lines of business. We look after the hiring solutions business, the sales solutions business, the premium business, and yes, elephant in the room, we are responsible to educate on LinkedIn learning, but we're not LinkedIn learning, and that gets real messy real fast, okay? Regardless, what we're trying to do is get the most users possible to their goals.

This is critical. It must be their goals. Without this, don't bother doing it, okay? And the way in which you need to think about this is again, this very plain speak way.

So if I think about a recruiter who has bought LinkedIn Recruiter, the thing they're trying to do, regardless of industry, regardless of role, regardless of all the nuances that we fight over in the personas and everything, is they are trying to have more conversations with great candidates, full stop. It's the only thing they're trying to do. If we've got sellers who are on Sales Navigator, the thing that they are trying to do, no matter where in the organization they sit, is just have more meetings to sell their wares. Like that's it.

Which means my job is to aim as close to that moment of like this is what I'm here to do as possible, metrics wise. And yes, we're gonna talk about measurement. It's probably the most important thing, okay? However, getting them exclusively to their goals is charity work.

If you cannot contribute measurably to the bottom line of your business, okay? And there's a saying that goes something along the lines of we need to do well in order to do good. We need to do well in order to continue to do good. The budgets ain't getting bigger, folks.

Scrutiny is at all time high. Automation and request for automation is at all time high. This has to be on lockdown. Or we can't get the job done, okay?

And so what does this actually look like in practice? So I brought one single program with me today. Certainly I'm happy to talk about like the whole kit and caboodle of everything that we do offline. But I wanna get as simple and clear as possible.

So this is just like one very simple program that represents all of the elements of two sides of this coin, okay? So this program, this is our Drop Start program. The original hypothesis for this program was like disengaged users. Disengaged users, we don't know why they're disengaged, but they ain't using the thing.

And we need to go after them and we need to bring them in and we need to give them something, anything, in order to kind of re-engage them, okay? But because we don't have like super strong signals as to why they are disengaged, which makes obviously crafting a program to like suit their needs very difficult, what we did is tried to look at this through like a lowest common denominator type of way, which is like, what are like the only, if you did nothing else in these products, literally nothing else, if it was just these things that you did, would that be enough to like spark the inspiration? And so this is a very simple guided path. It's four steps.

You don't even need to do all of the four steps. In fact, all of our like outreach to these customers drops you it simply into the first step. And if you did that, that would be plenty, okay? So four steps and those four steps include the very first step of that is a short, short, this is 30 minutes, human-led workshop.

This human-led workshop is not an end-to-end workflow experience. This is not a stand and deliver. This is a real talk chat. This is like a, hi friends, glad you're here, open your laptops, we are here to work.

We are here to do the work that you bought this thing to do. We're gonna talk real talk. If you have the real workflow questions, there's like other webinars for that. Please go there, right?

This is real talk. This is our internal facilitator as our LinkedIn experts doing this. We have a reward, we're arguing over this right now. We do issue a certificate.

Does it matter? Does it not? We're running hours on that. We'll figure it out.

Maybe it does, maybe it doesn't. But of course, what matters a whole bunch is that through this program, you get into an exclusive spot in the community to do that coffee shop argument piece, right? So that this is where like the two sides of the coin come together is like you are getting the baseline information that you need, but then really where things come to life is when other people say the way in which they do those things. And that's the thing that needs to scale in a big way.

And so that's how these two things come together. So when we did this program, we were super excited because we measured all the things and as it turns out, roughly 65% of people who attended only the first thing took immediate action in the product. They re-engaged, super exciting. And so that was within seven days.

So we wanna keep that window very, very narrow. And so like normally we would be like, great successful program moving on, but we felt like maybe we could take it up a notch. And so what we started to have a conversation about is like, why is this thing so much more effective? So just for context, 65% of users taking action within seven days is roughly 10 points above and beyond what the rest of our curriculum normally does, what the rest of our assets normally do.

And we watch this very carefully. And so like, this is like more effective than some of the other things that we're doing. And so like, why is that? And like one of our hypothesis is that like, we're not doing the dog and pony show.

We're not doing this like very polished thing. We're doing real talk with our customers with a real life human being. And so like, if that's the hypothesis, how do we push that further? What if a customer held the session?

Okay, so we changed nothing, same program. Guided path, literally identical path. The one and only thing that we changed is that we did a short customer led workshop. And boy, did we argue over this in all the ways.

We were 110% sure they would show up. Like the people would come, like very, very popular, obviously when a customer is gonna talk about what they do. But we were not at all clear as to whether or not we would actually get lift in product adoption from doing this. I mean, we don't, we're kind of turning the keys over to the customer.

We don't necessarily, I mean, obviously like we coached them, we worked with them, but like we also don't have the ability to like guide them and say what they're supposed to say because they're genuinely there to say the thing that they want to say. And we want them to do that, but like that's really risky. And so we were like, ooh, cross your fingers folks, here we go, this is what we're gonna do. And sure enough, they came in waves, unsurprising altogether.

But what was very, very surprising. Is that above and beyond already that highly effective program, we created another 7.3 percentage points of lift. That's not 7.3%, that's 7.3 percentage points. It went up again in a very clear and measurable way.

And so then you gotta really start thinking to yourself, well, why is that? Why is that? What is happening in the world? And like, I assume all of you all day long have done nothing but hear about how AI is changing about things, hear about humans in the middle and what do we do about that?

And what's the strategy, et cetera. I can tell you like kind of what we're thinking and I'm sure this is happening in all of the products that you all support. We used to teach a very sort of scientific method, okay. So if you got into our product, we would teach something like click this button, write these three words in search, click these three filters and this is the result you will get.

Like the instructor could show it and like you would look at your screen and they would like be identical, right. So we were teaching two plus two equals four, we were teaching science. But our products have shifted. Our products are now AI based, which means you could put a very elegant prompt into our product that is designed to do exactly what you need it to do and like that's the journey just to even get to that point.

But you could do that, you could put it in the product today, you put the exact same product prompt, excuse me, in the product two, three weeks later, get totally different results. I mean like that's AI, right, that's what we're doing. Which means we are no longer teaching the how to or even the sequencing. What we are trying to do is teach that they can trust the output even though they do not have any logical evidence that two plus two equals four.

And if you think about that, you are then trying to tackle a very different problem. This is no longer about 101, 201, 301, this is about getting out of the way of the truth and helping the human beings walk through this journey of embracing AI. The only way I know how to do that is to let the humans lead and in our case it was literally like LinkedIn needs to stand aside. Like we're experts know a lot of things, but our customers even more so, right?

And so what does that really mean? It means that credibility is the new currency, full stop. Navigating trust in an AI era is going to be everything. It is not going to be the fact that you can do something.

We're all gonna be able to do the thing, if we're all gonna be able to create training, we're all gonna be able to create communities, we're all gonna be able to do all the things. But will we trust them? Will we trust the output? Will we trust what we're seeing there?

That's a very different question, right? Like the whole trust but verify situation, how do you do that? Well right now we're thinking about exclusively operating at the intersection between education and humans. So that's what we're working on.

However, it doesn't matter if you do all of those things, if you can't measure it. And like I would even go so far as to say, correlation's not gonna cut it. I think getting close on correlation is so exciting because you're like, I did the thing and like 17 steps later this thing went up. And like I did that because we all know the things that we do every single day do make an impact.

We like, we know that in our guts. We are hopeful that those are the things that are happening. And when we see evidence that it's gonna be happening, that it does indeed lift, we're super excited about that. We wanna like tell the masses.

So let's start here. Be very, be aware, be aware of like the vanity, metrics, spiral track, right? I think we're all experiencing this. I'm sure I went to a session yesterday where it like, there was this clear like declining graph of just like straight up traffic, right?

So we've usually previously talked a lot about registrations and attendance and CSAT and all those things. I'm not saying that those aren't valuable, but what I am saying is people are getting their information from totally different sources now. They are going to chat GPT and asking questions. Now obviously chat GPT is getting their answers from us, like from our content, right?

Like it's, it doesn't magically know that. It's feeding off of the things that like are out there in the ethos, which means we're sort of like not getting credit if we only look at vanity metrics, right? We are only considering trends and that isn't ultimately what we're going to try to do, right? Remember we're going to go back to like our single source statement or single strategy statement.

We're going to try and do that instead. So here's how we do it. There's like implied impact and there's like causation. In fact, you got to get to causation no matter what you do.

You like, you just have to get there. You got to figure it out. So like, I'm going to take some like really slow baby steps here to really talk to you about the journey that we've been on. I'm going to literally show you the mechanics of exactly how we do it.

It will look different at your organization, but like these are the things I wish someone would have told me. It shouldn't be a secret. Like we should be able to show off the value of our jobs, right? So on the left hand side, we use a metric that we're very proud of.

It's called actioned learner. The learner took action, okay? Actioned learner. This is a correlation metric and this is what we use as our operational metric.

Action learner is awesome because it gets us out of some of these like historically really yucky cycles, which is like, how often do you update your content? Is it monthly, quarterly? Like I don't. I look at my content and I figure out what content is working and driving action and I look at the content that is not working and not driving action.

And that's how I look at my catalog and for reference, like our catalogs are roughly three-ish thousand assets. So that really saves us a lot of time of like zeroing in on the thing that actually needs to, does it need to be updated or does it actually need to be retired? Does that not even matter anymore, right? So this gets us in fast.

Then you gotta get to causation. The one and only way that I know, and I'm talking about this with such authority, like as if I actually personally know this, I wanna be very, very clear. I have some incredible partners who have done some incredible work and what they have assured me and what I totally am bought in and believe in is that really the one and only way that we can get to causation is A-B testing. We have to have a holdout group, we have to have a treatment group and we have to then measure the difference between the two.

Because then what you're gonna do is zero out all the other things, the yeah-butts. You're getting rid of the yeah-butts. Yeah-butts, the economy. Yeah-butts, AI.

Yeah-butts, well if those things happened to the same people but one person got the program and one didn't, it was the program and that's A-B testing. So let's like really very detailed go through this because this matters a lot. So here's one of our courses. It's like a very old historical course.

This is an end-to-end workflow course. This is recruiter essentials. If you want the more formal walkthrough, this is what you would do if you were a recruiter and user. So what we've done is we've taken this asset.

We are on SkillJar, we are on GameSite Community, so we are the how, they tease us all the time. We are the house that tags built. So if that means anything to you, especially SkillJar users, we tag everything. We have like a quadrillion tags.

So what we do is we take this course and we say this course is built specifically to drive these actions. So like this is a workflow course, so it's more than one action. But you could do this if it's exclusively built to do one thing, great, tag it once. So the idea here is like if this course is built to get the user to take it and do the thing, well then all we gotta do is look in the product tables, really easy, right?

But it's simple but not easy. You're gonna look in the product tables, you're gonna look for those actions, but then you're gonna put a very clear constraint on it. And that constraint is time. The less time, the better.

We use seven days. Okay, so we are looking for it the moment that the person engages with the course. We are looking at the seven days that happened after that. We are looking in the product table and we are looking for anything to happen within those product usage tables.

And wouldn't you know it, this user did indeed do one thing. Now that's your choice. Did they need to do all four? Did they need to do one?

I mean, that's up to you all as to how you would set something like that up. In our world, we are simply trying to get them to use the thing. And so one's fine because we believe in that one. We believe that one would inspire them to maybe do more.

So this is that operational metric. This is correlation, but it's tight. It's short, right? It's almost inarguable, but not quite.

Okay, if that all happens, ding. I like to think of this like light bulb goes on. Action learner, there they are. And so we can operationalize this.

We can look at action learner rates. We can understand whether or not, you know, this is like the jumpstart program. Like the action learner rate on this one, 65%. Our average action learner rate is like 57%, right?

Ooh, this is doing better. Why is that? And it gets us into the questioning. But if you really wanna do it right, and you really wanna get to revenue, this is the healthy, you know, in our statement, this is the healthy LinkedIn bottom line.

This is the one and only way that we can tie it together. You have to do A-B testing. So it's the same actual setup. You're gonna do a holdout group of some kind, million arguments as to how you can do this.

And then you're going to do a treatment group. So in this case, I'm just showing individuals, okay? So like picture individuals from similar companies, similar industries, similar geographies, et cetera. One took the course, and actually, in this case, we're doing a program.

So like one engaged with the program and one didn't. Okay, we're still looking at seven days. That matters still a lot. We wanna keep that very consistent.

But guys, the thing we never ever wanna ever even whisper, sorry, that was not a whisper, is like the age old, but aren't people who go to training, the ones that are curious to begin with. So they're probably also the ones in your product, like trying already. And like, yes, as it turns out, that's totally true. A lot of our very enthusiastic trainees, honestly, are using the product just fine.

They're there out of a sense of curiosity, but that's a real problem when it comes to showing lift, whether or not you made your business healthier, which means you have to look backward as well from that moment. You have to look at what happened seven days beforehand. You have to look at what happened seven days after, and you have to compare the difference between the two. So in this case, we are looking at this sensitive KPI, whatever this might be.

Now, I would highly recommend that the thing that you pick out to look at is a thing that your C-suite cares about. Chances are your C-suite cares about money. Can you find a KPI that you can attach a dollar value to? Easier said than done, again, totally simple, not at all easy.

But if you can do this, honestly, it just becomes spreadsheet work from that point forward, and you can make incredible decisions. So in our case, let's say it's in mail. I was talking about a recruiter. What they're trying to do is engage in conversations with other members.

And so you would do that via an in mail. So that's the closest KPI thing that I have to the customer's goal. And I can also figure out roughly what that's worse to link 10 to cause. So in our holdout group, maybe that individual was rolling at 20.

They were sending 20 in mails. And then seven days later, they're probably still sending 20 in mails. But I do want to be really clear that even though there's no lift here, this is what we would expect to see. It doesn't matter if this is a number.

This could be 10. This could be 20. It doesn't really matter, because what we're going to do is look at the exact same relationship in our treatment group. And as long as there is a difference between these two numbers, you can prove lift.

This is business impact. This is the thing that your leaders actually want to know but don't know how to ask you for, because they don't know your worlds. They don't know the terms. What they do know is they have a business to run.

They need to run it profitably. And so if you can come and talk with them in a very black and white way, you're going to have a lot more successful conversations. So that means you also have to be a storyteller. No bigs, guys.

You just got to run a community. You got to run some learning. You got to get yourself some customers. You got to do all the things.

And now you got to be a storyteller too. Oh, you should be a data scientist, for sure. And so now you're going to go tell your story. So here's how we tell ours-- this is at least the framework that we use to tell ours.

We do actually start with the vanity metrics. If they didn't engage, that is a problem. And then we kind of look at, OK, within our program, what's doing the lifting? What's doing the working and what's not?

So for example, in the jumpstart thing, what I was talking with you about is we know that workshop is doing a lot of heavy lifting. We are less clear on the certificate. I'm not convinced it really is adding or it isn't. I think in other context, maybe it could.

Maybe it couldn't. But if you can get real clear on which asset is doing which piece of work for you, it makes experimentation so much easier. It means you can get clear hypotheses and then work toward them. So that's where we use Action Planner, for example.

Then once you've worked out the kinks in the program, then get yourself to the point of A-B testing. That's when you want to invest the time, the money. Go buy Starbucks gift cards for measurement pales. Slide them over there.

Make some side deals. Do whatever you need to do, partnership wise, to figure this piece out. Because if you can figure it out, what it then gives you-- whoops, well, we've moved on. Let's see.

Here we go-- is the story you can tell your C-suite. We definitively added dollars to the bucket. I'm 100% sure that we did. I can answer in a very scientific way exactly how we did it.

And depending on what you are trying to accomplish, we could do it again because it is repeatable. I could figure that out. So that's why all of these elements matter, which actually then puts you in a position to say, show me the money. We all run, honestly, cost centers, unless you are doing some sort of revenue recognition model, which is totally a possibility.

That is not what we run. We're cost centers, which means I have to be able to say we, as an organization, cost you x amount of dollars. And we give you y amount of dollars, and we'd better be in the black on that, full stop. But depending on how clearly I can tell that 2 plus 2 equals 10, 2 plus 2 equals 20 story, I can actually confidently say yes to practically anything.

Here's the money it would take. Here's the assets it would take. Here's the whatever it would take. And this other thing would have to go away.

That's never going to go away, but I can do it in a more dollar-and-sense way. And so here's my recommendations. We tried to bring an example of exactly that today. Start small.

Do not boil the ocean. Do not try to take your entire academy and your entire community and somehow jam them together and figure it out. And oh my god, the UI, and I don't know how I'm going to work this. And what about identity?

And who's signing into what? And are we ungated? Don't do that. You will be overwhelmed before you even start.

Start with a simple, tiny little program, a clear hypothesis. Find a way to measure it. You've got to find a way to measure it. Start with correlation, but get to causation.

And finally, tell your story. Tell your story confidently. You are not a support organization. You are an engine to deliver value, literal dollars and cents, to the bottom line of the organizations that you work for.

And I've sat in this room all day long, and I have listened to leaders, my peers, say that story again and again. And again, we've all ended on the last slide that said this. We are responsible for the health of our business. We just have to be able to tell the story.

That's it, my friends. Let's go to questions. [APPLAUSE] Thank you, Stephanie. All right.

Time for some questions. OK, the first one right here. Can you hear me, though, Stephanie? Yep, I can hear you.

OK, awesome. If I don't own both community and education today, just own learning, what is the best first step to start proving the value of connecting them? Yeah, I think my answer to that would be-- and actually, I should talk briefly about this. When we started the Jumpstart program in-- well, in both lines, several lines of businesses, the community team actually did live in another portion of the organization.

So we actually started that through just a partnership. Hey, this is how we're thinking about it. This is a problem we both share. Let's go tackle it.

We have more recently fully integrated and redone an entire org structure to support this model. But it was actually this program and proof that helped us say, you know what? We could do it a lot bigger, better, faster, stronger if we were simply all under one roof. So I think it's a matter of proof of concept.

Start small. Amazing, thank you. For the Jumpstart program, how did you identify who to target and how did you target them? Market the program to them.

Yeah, Jumpstart has evolved pretty dramatically. So as I mentioned, when we first began Jumpstart, it was around this disengaged user. So we had a business problem that needed to get solved. We identified that this set of disengaged users is basically the highest risk to the business, and it's got to get tackled.

So that's how we began. But as we iterated and it gained learnings, what we also figured out is this is actually the world's best onboarding program. Because as it turns out, people don't want to do like four hours worth of onboarding. I know we're all shocked to learn that, right?

Like what they want is enough information to be dangerous. And they want to get in there and they want to see success. So that's like when we started making some decisions of like what if we just centered this as the central thing? So to answer your question very directly, we had marketing partnerships.

We have Gainsight as well. We sent Gainsight campaigns that included the Jumpstart program, welcome, come join us, that sort of thing. So that's how we got it out into the world. And then the more confident we were that this was a needle mover, the more places we put it.

So now it is in all onboarding checklists. It is in our provisioning emails. It's absolutely everywhere. What changed operationally when community and education came together?

And what had to be true internally to make that model work? Okay, that's what changed operationally, right? That question? Yeah, that one.

It's the echo, that's what it is. Now we know. What changed operationally when community and education came together? So many things.

And also not all that much. So we've always had a very, very strong partnership with community. In fact, we've had several different versions like when we all were different lines of business, community actually was in the same place with the lines of business, then we separated off. This is the one we were going through at Friends.

Like we've tried a whole bunch of different ways in order to figure this out. But I think operationally what matters a lot is making sure that you are centering your structure's goal by goal. So the way in which we have centered our structure and that affects our operations is that the hiring solutions business has one single adoption goal. All of these people should aim at it.

And the sales solutions business has one single goal. All of these people should aim at it. And that makes figuring out the operations so much simpler. You keep your experts.

You're always gonna have an expert on the team who knows a lot about community. You're gonna have an expert on the team who knows a lot about structured learning, but you gotta get the goals right. Great, thank you. The next one is, have you had any success running the A-B test within community specifically?

Yeah, great question. So I can only tell you how we've hacked it only because we had to. And I think that's what we're here to do, right? Is you take what you got and you do your best with it.

So what that looks like in our world is the learning piece had the most structured data. We had a way to get to actioned learners. And so what we would do is we would run a program in community, but if it was a live program, what we would do is the registration went through Skilljar. So that at least I could tag the registration in Skilljar.

And that's clearly a community program. It's a round table event. We're talking about things that are not even necessarily product training, but we do wanna know the effect of it. We can't do that often, otherwise you're gonna muddy up the data, right?

But there are cases that we wanna try out and figure out are we moving the needle on things? Is the juice worth the squeeze? Is this worth our time and energy to do? And that's kinda how we hacked it.

Great, yeah, thank you. The next one is from Taylor. We have community and learning under one roof and ready to ensure they are combined working together. How do we help the customers feel that any practical ideas for learning within their community of peers?

Yeah, this is so exciting. I'm so excited for you, Taylor. Where are you? Let's talk, let's talk about all the things.

This is super exciting. I actually think this is an easy one. Pick one as your front door. We have chosen the community.

The community is our front door. If, when in doubt, send them to the community. We will suck them in from there and we'll try and do our best to get them into the right program from there, but make it simple. Our customers don't wanna know how we're structured.

Our customers don't wanna know that we have a community team over here and that we have a learning team over here and then what about the help center? Our customers are just like, you know what? Where do I go? And it's our responsibility to sort afterward, but I think there's all kinds of things you could do there that could be really exciting.

Thank you. How did you decide which behaviors count for an action learner and how do you avoid picking vanity metrics? Yeah, great question. Easy to avoid picking vanity metrics because they have to be, the actions have to be product actions.

They are not learning actions. So that part's really easy. What's not easy is deciding on the product actions and this looks different. I will tell you we have wildly different action learner rates depending on the line of business.

Is that because one line of business team does more effective learning than the other? No, it actually has more to do with our data structure and over here I can measure it this way and over here I have to tag it this way and it's just not super clear. But what you are trying to do, and again I would encourage you to go back to the very, very simple speak, like the plain speak, what at the end of the day are your users trying to do? What one thing are they trying to do?

Start there. If you could tag everything with that one thing and only that one thing, you would still get an incredible amount of insight using this technique. And then maybe start layering on things. For us, the simpler we go, the clearer the insights are.

Yeah, great. What were you using for reporting to actually do the causation A-B testing? What were we using? I assume what you mean is like what technology we're using to evaluate, maybe, maybe.

This is, we are in an extremely fortunate position that we have a measurement team. We have a measurement team that has access to our data tables and they know how to run SQL queries and they know how to run Python. I don't even know all the magical things that they know how to do. Truly, honestly, what I know is what I'm tasked to do and what they know is that if they know what we're tasked to do, they can get us there.

And so I would be totally lying if I told you I knew how the nuts and bolts worked there, but I can tell you I asked one million questions in the process and I still do and I'm sure that's what my reputation is with that team. Perfect, amazing. This was the last question. So thank you so much everyone for joining today's session.

Thank you so much, Stephanie, for sharing these amazing insights.