Scaling CS Without Scaling Headcount: How GitLab Runs a Pooled CSE Model with Gainsight

Great to be here with everyone. I will do my best to keep my energy high. I know that we are after lunch and I appreciate everyone attending. My daughters, by the way, they're adults and maybe that's why I'm not naming them, but 22, 25, Grace and Ella, and yeah, very proud of my family as I'm sure you all are.

So looking forward to talking today about Scaling CS without scaling headcount and how we use GainSight to help us do that. Okay, no Slido. We're gonna start the old-fashioned way and I'd like a quick show of hands for anyone who currently is running a named CSM or CSE model. Great, keep them up.

Keep them up, please. If you have been asked to cover more with less, keep your hand up. Fantastic. That's why we're all here today.

Again, I'm Ally Frisiano. I run our Scaled Success function at GitLab. I do not have a slide on what GitLab does, so in a nutshell, we help customers build software and I run the Global CSE in our Digital Success program. We pivoted from a named CSM function about two years ago.

It was quite sudden. It felt sudden to us. We had about a quarter to launch it and we've been running it for two years. So my goal today is to give you an actual playbook not just a framework, but an actual playbook that you can take back to your organization if this is something that you are looking at as well.

All right, let's get into it. So let's talk about our journey. Here's where we started and it sounds like maybe where many of you may stand today as well. We, in this specific book of business I'm talking about, let's call it our middle third.

We were running and so that was mostly our higher spend mid-market and our lower enterprise customers. We were running a named motion. We had about a one to 30 ratio, kind of typical CSM account management based. It was white glove.

We had recurring cadences with customers when they would show up. It was free. We were offering QBRs and things were moving along quite smoothly. Let's just say it worked for a while until it didn't.

We had three things that really happened at once within our organization. So number one, our volume was increasing. We were adding customers faster than we were adding headcount. In addition, our complexity was increasing as well.

We had a more mature product. We had harder questions from our customers, different pricing, tiering options. So complexity was increasing. And then priority shifted.

The business asked for different outcomes from CS and frankly, to be blunt, we were not getting the funding that we needed to keep up with the threshold that we had committed to. So the model broke. The model was not going to sustain and we had three choices to make. We could hire, which is really hard to do if you're not getting funding.

So that obviously wasn't a real option. We could raise the threshold and sort of walk away from this middle band and put them all into digital or we could build something different. And what we chose was to build something different. So let's talk about what we built.

We moved away from named ownership and moved to a model where nobody owns an account. Everything is in a shared pool. So again, this is our middle third of our customer base. Work comes in and our CSEs, customer success engineers, pick it up and work it like a queue.

So I'm gonna stop when I use the word queue because oftentimes people say that sounds a lot like support. And I wanna be really clear that this is not support it is not break fix. We have a mature support organization like many of you do but it is a very systematic and operational way for CSEs to work a pooled model. Every engagement is designed to drive an outcome.

So whether that's adoption, expansion, maybe migration guidance, advancing their maturity, value realization. We go into it with an outcome in mind so we're not just reactively answering questions. And there's a lot of discovery that we guide our CSEs to do to make that happen. We do a ton of just in time enablement.

An example of that would be our latest feature releases at GitLab we release on a monthly basis. So perhaps that is something that we are just in time enabling a customer based on their needs and the engagement that they have raised with us. It is free to customers. So there is no skew, there is no carve.

And we do have some engagement limits that apply. We affectionately like to call those customers frequent flyers. And if there are frequent flyers, we do have sort of a published policy on how many engagements customers can have. I'll tell you that is the exception to the rule.

Most of the customers that we engage with want to get in, want to get out. There may be some asynchronous communication but nobody's really abusing the scaled model in our experience. Okay, so I want to dive in first to the second one. I want to talk about the role of the CSE profile and why it matters.

None of this works if you hire the wrong people to do the job. This is a different job than the traditional CSM job that I described in our original assigned model. So let me say it again. This is one of our biggest lessons that we learned.

It only works if the people are right for it. There are four things that we hire for or we look for in a successful CSE. So we're looking for specialists, people who understand the work that our customers are doing. We build software.

We need people who understand what it means to build software. Many of our CSEs have actually built software, been SREs, worked in that world and are now interested in getting customer facing experience. Products proficiency, pretty straightforward but we need people who are fluid across the platform. Nobody is ever going to know everything and we don't expect that.

We have a very wide technical many use cases platform but we need them to go fluent and we need them to be fluent and we need them to be able to go on the off ramp quickly because sometimes the conversation is going to change course. Speed dating is the term that we use internally to describe our motion and I think that's one of the most under weighted skills to look for when you are building a model like this. Not only do we need the product chops, the technical chops but we also need somebody who can establish trust quickly because this is not a long term relationship. This is getting someone to trust you and build credibility quickly on what may only be one 45 minute phone call.

So again, we want to drive to outcome. Yeah and then outcome focus. So just going in with the outcome in mind is the guidance that we give the team. Make sure there is a concrete next step.

Don't leave things being vague, I'll follow up. We actually capture that specifically in GainSight so we can see what are those outcomes and those are sort of the critical four pillars that we look at when we recruit for these roles. Okay, there's a lot on this slide. So let me try to talk you through what is most important.

Here's our workflow. How does it work? It starts with sales. So we have a process where a sales rep and we call them account executives, an AE or a renewals manager, they are our main internal stakeholders, can raise a case in sales force.

That flows into GainSight as a CTA, a traditional GainSight CTA and goes into the CSE's queue which I mentioned before. That is where they manage their work. The CSE picks it up, they sync with the account team, they sync with the customer and that's either asynchronous or via a live call. One of the critical pieces I wanted to mention about this workflow is that the activity flows both ways.

So the data syncs in GainSight and then the activities also sync into sales force. And what I mean by that is some of the value that that provides for us is it gives sales the opportunity to inspect exactly what is happening with this engagement. And I think most of us in the room know that if sales doesn't trust the model or doesn't have the confidence in what is going on in engagement, they're gonna lose trust real fast. So the architecture of having the data flow between GainSight and sales force and back is really mission critical to I would say our internal brand if you will and helps drive that awareness and visibility within the sales organization.

One other thing I wanna mention that we use in terms of a workflow is this thing here is all about internal. This is sales force to GainSight and back, bidirectional. We also use Calendly. So via our digital campaigns, we allow customers to book with CSEs directly.

So we will send a campaign, we'll include a Calendly link, it will, a customer will schedule directly with a customer success engineer and that will automatically create a CTA. So again, if I hadn't made it clear, it's a queue, it's CTAs and it makes everybody really as efficient as possible. Okay, so some immediate learnings. Phase one was meant to be messy and it was.

We had about a quarter to launch this and really we probably had about two months to kind of put a wireframe around what we were gonna do and how quickly we were going to do that. We had to fill this gap, we had to scale our coverage approach. So we were reactive by design and looking back those two years, it is kind of funny to say this out loud, but we did whatever sales wanted us to do. Whatever they asked, we did.

We weren't trying to be elegant, we were trying to be available and make sure that customers that needed us had what they needed. As you might imagine, within months of launching, that case data got real valuable real fast. And basically we saw patterns that emerged as we would expect in our most common use cases. For us, that's around CICD, security, AI, migration assistance, et cetera.

But you all have those specific use cases that you can probably rattle off, but the nuance of what a customer actually wants to talk about in these different topics was all visible in our cases. And even back in the day, two years ago, we were able to start to use AI to review that and codify what our offerings should be. So the point I want to land is that we launched a catalog of our top 10 offerings and it came from the work. So it came from the data in the cases, as opposed to us just deciding what it was going to be.

And we found that real valuable. So I know others, peers in this space, designed their catalog first. I would just encourage you. We all know that it's important to understand customer feedback and what customers actually want, but really do it, especially in today's day and age, when you're able to capture that information and quickly analyze it with AI, it really paid off.

Okay, you've all seen this before. Progress is never a straight line. Let me be honest with you, we iterated a lot. What we launched in May of 2024, two years ago, looks a lot different from what actually runs today.

So our routing has been rebuilt. Our capacity model has been adjusted. Our catalog we continue to iterate on. So I want to land the point, if you are considering this, to ensure that you know that iteration is an operating mode.

It is not a sign of failure. And especially in a scaled high volume framework, it is really important that you can iterate and iterate quickly. We had tons of failures. We would run campaigns and we would have no engagement.

We had data system breaks. We had salespeople mad at us. I mean, the whole gauntlet. And it's far from perfect today, but it has improved as we've moved along.

So plan for iteration, give your team permission to do it and be willing to say it. We tried it, it didn't work, and we're going to move on. OK. So with all that, let's talk about what was actually delivered.

We have run through about 2,500 cases since our initial launch. That represents 1,500 unique customers, unique companies for us. What's interesting for us is that is not all the customers in this book. There's nowhere close to 100% engagement.

I think in some of our highest quarters, we might have reached 40% engagement. And I'll talk a little bit about the digital foundation that we use to buffer that. But we're very proud of these numbers. That is real scale.

And we were able to do this without adding the additional headcount. So that's the part I want everyone to take home. In addition, the majority of the inquiries and the conversations that we were having are adoption themed. So customers are asking us how to get more value.

So that sort of ties back into my initial point of, this is not a support model. And when I say that, I want to land that customers understand the value that they're getting. And so we're not misguided. We're not sending customers to the wrong place when they should be going to support versus coming to a scaled CS organization.

And the data shows that. Most of our engagements are actually adoption values outcome focused, which is a real win for us. So in terms of our data, we definitely-- I talked about engagement. We were lucky if we ever hit 40% of our customers engaging with us in our total book.

But what we can confidently say is that the customers that do engage with us perform better. And that's through adoption, telemetry, and renewal rate. The renewal rate of the engaged cohort is higher than those who don't engage. So there are other factors, I think, that influence that.

But it is an encouraging statistic for sure. Let me sort of move on from this slide by talking about scale is not lower quality coverage. It's different coverage. It is optimized for outcomes over relationships.

I love that point because for those of us who have been running named models, relationship-based models, scale gives you a different approach where it is less about, did you have the meeting? Did you have the call? Are you doing the quarterly cadence? Versus, did you meet the customer when they needed you?

And were you able to drive that outcome? All right. Let's talk about metrics. I know that you all are familiar with leading and lagging.

The question I get very often is, how do you know it's working? So when we talk about leading, we're going to tell you how is the model working day to day? Nothing super surprising here, but we are looking at case creation volume. So how many cases are actually coming in?

Within those cases, we are looking at timeline activities. So you all know Gainsight really well. Case is coming in. It's a CTA.

And then the timeline activities under the CTA capture everything that the CSE is doing to progress this case. That may be, I'm still chasing the AE. They opened a case, and they're not answering. Or the customer has rescheduled on me six times.

We've actually done a lot of coaching on what determines what should go in a timeline entry and what should not. But regardless, we can see how busy the CSEs are by looking at those timeline activities. Another leading indicator that we're looking at is the source of the request. So when you launch something like this internally, it's critical to ensure that the rest of the business understands, and they know how to get value.

And so the data will show you which sales organizations, which regions are engaging, which are not. And that can help drive what we call our internal road show to make sure that people understand the value. The majority of our engagements do come from account executives and renewals managers, but it's not closed off to anyone. We have our solutions architects sometimes for edge case reasons engage with us as well.

And then I think the last one for us is really most important. So again, we're looking at some of the performance metrics for engaged versus not engaged. But we're also measuring ourselves on the engagement rate for at-risk accounts, because scale is defense. Scale really-- it is very hard to play offense.

We try with some campaigns and looking at metrics, et cetera. But at the end of the day, the most important thing we do with our scale motion is consider it a defensive motion. And so we are prioritizing at-risk accounts. And we are looking at the data to determine, is it adoption risk?

What is the telemetry showing us, license utilization, et cetera? Or we're simply looking at forecasted churn and contraction. We have a systematic way internally to look at what have our commercial counterparts indicated is the commercial risk on this customer. And we need to hold ourselves accountable to be engaging with those customers, or at least be trying to.

You can lead a horse to water. You know what I mean? You can't force the customers to meet with you. OK, so moving on to some lagging indicators.

Our closed success rate-- so we have many, many more cases open than actually closed successfully. These are statuses that we use within the CTAs. And understanding why things are not closing successfully continues to feed the model and continues to help us operate. Renewal rate, expansion, and growth-- I've already talked about those.

That's pretty obvious. Let's look at how the commercial performance of those customers that are actually engaging. And then CSAT-- we also use Gainsight. Every time a case is closed successfully, we fire a CSAT survey.

And that is fascinating data as well. I can't remember which session it was, but I think it's important that we all remember getting the data directly from the customer on their satisfaction level is so important. And I know that sounds so obvious, but I think so many of us get caught in this false narrative of, I think this is what the customer wants, or I think the customer is happy. And it's just a reminder that your good old-fashioned, old-school CSAT is still really valuable.

We have another Gainsight process in place. If we do have a detractor score, then we have a CTA that automatically opens for the manager of the CSE so they can follow up and see what the concern was and figure out how we can rectify that. OK, so three lessons. Some of these I have touched on already.

I'm hoping you can think of this as a playbook that you can take home as you're thinking about potentially launching scale in your own organization. So one to many is fundamental. That's maybe our internal term for our digital foundation. Pulled CSE does not scale far enough.

You are never going to engage with all of those customers. So that digital foundation underneath, which serves all of our customers, not just in this middle third book that I'm talking about. For us, that is webinars and labs, free webinars and labs, our LMS, which is GitLab University, and then also our lifecycle emails that we're using Journey Orchestrator for, so those regular digital campaigns. CSEs are the human touch layer on a broader digital foundation.

We talk a ton-- if we could have even less CSEs, right? Human last is a phrase that we talk about a lot. It's kind of one of our guiding principles. So second lesson learned that I want to just repeat myself on is hire for that right profile.

Technical, fast trust, outcome focus, retrofitting traditional CSMs doesn't work all the time. So be aware of that. We learned that lesson, and I wish I had known it earlier. I'll leave it at that.

Hire for the job you actually have. And then three, internal alignment is critical, and it is not automatic. So our sales organization, our buy-in was earned case by case. It's not earned by Slack message.

It's not earned by all hands. And two years into this, you guys, it's still not earned. Like, we are constantly communicating value success stories. It is a constant PR campaign that we never want to turn off because we will see dips and behaviors change.

So that go-to-market and in our world, the CRO org alignment is really mission critical, so everybody understands the model. Strong ops and tooling support is critical. This is a cross-functional effort. My next point is about Gainsight specifically, but there is so much more tooling that touches this process than just Gainsight.

I talked about Calendly. I think we use Zapier for our workflow integration. Obviously, everything is running out of Salesforce. There are a lot of tools that are critical to our scaled that we don't control in CX.

So ensuring that we have that cross-functional partnership is really, really critical. And then finally, our Gainsight admins. I have to give a shout out to Keith Mattes, who is in the room here with me today. We give a wave, Keith.

[APPLAUSE] We are lucky enough to have two Gainsight admins at GitLab. Unfortunately, our second one, Licket, is not able to be here, but he was a very integral part of this process. And our admins are able to help-- I'm a business leader. I don't know the back end of Gainsight.

That is not my expertise. That is not what I'm bringing to the table. And with Keith and Licket and many other partners in the organization, when we faced this coverage crisis quite quickly, we're able to come together, help us design, what are we actually trying to do here, and then how can the tool support us. So thank you, Keith and Licket.

All right. This is kind of funny, because as I was thinking about putting this talk together, the majority of this content is on our scaled motion, which we ran for two years. But as we know in CS, the pendulum swings a lot. And ironically, I felt that it was important to share that we have swung back.

And now in our pooled free motion, we have actually moved back to assigned. Assigned CSE in our world today is not what it was two years ago. It's not even close. Besides all of the influences of AI, what we have learned in the pooled model has really influenced how we have launched this new assigned 1 to 50 model.

We're making it much more impactful and efficient. And we are assigning that top echelon of that 1 third bucket. So that is just rolling out. We are continuing to run the pool.

So this is the model that we just covered. The good news is we are able to expand it lower into the long tail. So we are now looking at almost 90% of our ATR, of our-- let's call it our install base-- having some sort of CS resource. We spent a lot of time with over 50%, really not having access to any human.

The jury's still out, guys, on how this will go, because we are going deep into the long tail where the volumes are getting really high. What did we learn in the last two years is that we didn't get overwhelmed with engagements. In fact, some of our biggest challenges were driving those engagements. So now we're opening up the aperture, and we'll see what happens, because we are going-- we're going pretty far downstream in our long tail.

And then I just thought we have also introduced a first order onboarding. So our company, like your company, has always had an onboarding motion. We have decided to invest in this free sort of scaled resource for onboarding as well. It's a big company priority for us, new logos.

We could do a whole session on that. But the point is we have evolved. From two years, we were a purely reactive, we'll do whatever sales tells us to do. We've learned a ton along the way, and now we have matured into these three separate motions.

So hit me with your questions. I enjoy Q&A more than I do sitting up here listening to myself talk. Well, first of all, round of applause, everyone. Thought that was amazing.

So many great insights. Thanks for bringing the questions, and there's a ton. So the first one is, how do you manage the cost of a CSM, in quotes, conversation with customers without devaluing the scaled model? It's a good question.

So say more about the cost of a CSM. Are you saying, how do we not-- I'm going to interpret this as how do we not cannibalize our paid model? Raise your hand if that's not. Yeah, go ahead.

That is. OK, yeah. We get really precise on that. We have engagement limits.

GitLab, we are famous for our handbooks, so we have all sorts of public-facing documentation on what it is and what it isn't. What's been really interesting-- I mentioned the word frequent flyers. We've had maybe seven or eight customers that are trying to take advantage of the free service. But overall, this is where myself, leadership, or first-time managers get involved to really explain the differences.

Sometimes it's just talking about offense versus defense. So yes, maybe Keith is your CSE, and you have enjoyed your engagement with him. But by letter of the law, Keith can really only have three meetings with you a quarter. You need more.

Let us introduce you to your paid-- this paid model. It has been the exception for us to have to cut someone off. I genuinely can't think of a time where we had to say, company ABC, no more. I mean, whoever wants to say that to a customer.

And if they need that much engagement, it's worth digging deeper into what is actually causing all of that pain around their needs. Yeah, that's a really great point, laying down the laws, per se. So this next one is, how do you handle accountability-- oh, it just moved-- and quality, consistency, and a pooled model? So AI-- sorry, it's the answer for everything-- is a game changer, just being able to quickly surface the data.

Much like all of you, we have great gain site dashboards. We have a CSE command center. So we can dive into the number of cases that each CSE is managing. What are the status of those?

What are the activities associated with it? How many are at risk? I mean, sometimes, to be honest, it's analysis paralysis. But I would rather have all the data at my fingertips.

And we have first line managers that we really rely on for in terms of accountability. In this sort of model, especially with the involvement of our AE and RM counterparts, there's really not an option to not get the work done. If we've been asked to do something, we're expected to do it. If anything, it is more of the inverse of a customer has even raised their hand directly, or an AE has said, this is mission critical.

And then it just stalls. It stalls out. That's customer time. They're not necessarily on our time.

So this-- Yeah. Just to clarify, you do have a customer who can engage. It's not just your account executive that you can engage. We do.

We call that open the front door. And we have accelerated and decelerated that motion. But yes, we run programs out of Journey Orchestra with Calendly links, where you can book with the CSE directly. And that will create a CTA.

And honestly, that's usually the most meaningful interaction. So when we do run those campaigns, what a salesperson thinks that a customer needs in terms of assistance may not be what a customer actually needs in terms of assistance. So I think the more you can open the front door and hear from the customer directly, the better. There is a lot of-- there can be more expectation setting that has to take place when the customer is coming to you directly.

But at the end of the day, it is a win to talk to a customer. And it is a win to have a customer say, hey, GitLab, we need your help. We want to speak with you. If it's support, we'll get over to support.

But the engagement is really valuable. Nice. Again, thanks for all the questions. There's just some popping up over others.

But we have plenty of time to get to a lot of these. What are the rules of engagement for the AE or RM to raise a case so it's not just firefighting? I'm going to interpret firefighting as like, this customer renews in 15 days. I need you to do something.

So no. It doesn't work that way. Again, I hate to hark on the data too much. But we're not successful.

We're less successful in quarter when we are firefighting, if I'm interpreting that correctly. So it's that PR tour. It's those success stories. It's constantly through all of our internal all-hands, slack, et cetera, everything that we do to communicate internally.

And we are a noisy environment, let me tell you. It is hard. You probably all feel the same. But I think we are gold medalists when it comes to a noisy slack environment at GitLab.

Keith, would you agree? Basically, and the leadership partnership is key there too. So we have also evolved this year to more of a pod model. So the CSE first line manager is aligned to the same RM manager and is aligned to the same sales regional director.

So that just helps that human to human connection of, hey, guys, this is the way scale is most impactful. But don't call us 30 days out. You need to be looking at your data. And we don't have an automated way to do that.

We're doing that through human interaction and relationships right now. Next two are great questions. Align a little bit, but definitely we'll have different answers. So how was the transition for clients for a managed to pool-scaled model?

How did the clients handle that? And you touched a little bit upon that. So this is the great surprise. It was so much more painful for our revenue organization than it was for our customers.

That's sort of a hard thing to say out loud, potentially could lead you down a road of questioning what you were doing before. Customers are busy. I don't know. Customers are maybe getting what they need out of the sales relationship.

But let me just say, when we were-- two years ago, in March 2024, we had email templates. And we had escalations of processes of what are we going to do. It wasn't that noisy. It doesn't mean some made noise.

We were actually-- in some of our non-USGOs, there was more noise. It's always interesting to see globally how different customers react. But it was not as noisy as I was expecting. [END PLAYBACK] A lot about our revenue teams and how just anxious and nervous that we get about how the customers are going to react.

Do not take my resource. For sure. And for those that maybe did say something, how did you respond to that going to the next question? We are always-- again, let's go back to the term frequent flyer.

We're going to go above and beyond. We are never going to put some T's and C's in front of a customer and say, no, sorry, you're cut off. I mean, if a customer has some absolute bleeding need to talk to a technical adoption-based resource, and we as an organization can't get them to the right places, whether they need PS help or something through our university, then they were sold wrong? I mean, something is off.

That is the exception to the rule. So just continuing to dig on customers-- our customers, at least in this, were not specifically like, wait, you're taking something away from us, it had always been free. So it would be different if we started messing with SKUs and renewals. I would say it was just a lot of discovery about why are you so upset?

And then obviously, we make exceptions. If we need to put a CSE on a customer for six months, we're going to do that. What's interesting-- and it could be a whole other session-- is while we were transitioning to scale, we were launching our paid model for high touch. So we were just in this wild flux of change and CS, and we had free high touch.

So those are some harder conversations, is converting that free high touch into what we call our paid success tiers. I'd like to talk to you about that after this. [INAUDIBLE] Let's do it. Yeah, it sounds like not many people fought back about it.

Obviously, some legacy customers are going to have something to say, just because we all get used to what we're used to. So how did you handle that? I mean, it's very similar. We did what we needed to do to make them happy, period.

And it worked out. I guess, take the point away that your internal audience is probably going to be much louder, at least for us, than our external audience was. Because again, the model that we built allowed customers to quickly speed date and get to the outcomes they needed. We didn't cut them off.

We asked them to come into the program a different way. And they were happy. And they were happy because we had the right resources helping them. So in a lot of ways, we were able to flip it.

We would have customers say, this is great. I'm not interested in a monthly cadence call anymore. But I'm thrilled that I know that I can get in touch with Keith when I need to. [VIDEO PLAYBACK] What metrics convinced your leadership that the pooled approach was working?

Well, if you think about there was a cost to serve, let's talk about how we got here. So there's the success. What did we measure? Let's talk about metrics that help us measure success.

But the reason we got here is because we no longer had the funding to run the model that we had. So the metrics that our leadership looks at are a lot of the leading and lagging that I showed you. So we have a regular internal cadence that I'm reporting at my level and up. What is our engagement rate?

How much of the customer base is engaged? But they're really looking at that renewal rate. And I would be dishonest if I said some of the pressures in the market didn't influence our growth to return to 1 to 50. So there's real value in the pooled motion.

And I would say even outside of our book, in our market specifically, just a ton of changes. We're changing from a seat model to consumption and understanding that we need humans involved. I've heard people call our CSEs speed dating FDEs. That's kind of like sacrilege.

But we need humans that are involved that can help our customers through all of this chaos that we're all experiencing. And the business knows that. The business is betting on it. What does proactivity look like for your team in this model?

Have you found that reactive ask from sales? Have slowed that down. Reactive asks are super seasonal. So you get close to the end of the quarter, and they slow down.

Everybody's trying to close the quarter. Before we launched, we put one third of the book into this official scaled motion. We were running scale as a little mini pilot over here. It's not like we had never experimented with scale.

What's interesting, before we put all this horsepower behind it, we had CSEs using outreach to kind of almost be like mini BDRs. Be proactive. Go out there. Look at your book.

Look at your telemetry. I don't think we had the operational rigor around it, probably not the maturity around it yet. And we weren't seeing the results. So we are still, except now that we're in our 1 to 50 assigned model, for our whole two years, it really wasn't up to the individual CSE to go attack, if you will, play offense to a certain part of their business.

They were receiving incoming requests. We probably have time for about two more questions. So if you see something on the board that intrigues you or have one, I suggest jump in real quick. So what was the communication process, like transitioning named accounts to a CSE model, and how do you protect trust?

Yeah. We were anxious. We were super anxious about that. So in terms-- we had a pretty rigorous process with existing CSMs, because remember, we had humans.

We had these 1 to 30 ratios that we were running. And we needed customers to know that Bob was no longer their person, and that we were introducing this new model, and this is how they could access it. And here's the person that they could go to if there was questions. So I would say just-- it was either a cadence call at the end of a final cadence call, we're introducing a new model to you.

I think we had a blog post about it as well. We had emails. We tracked the communications to the customers. We were able to say, have these customers received it?

You guys know, just because a customer receives an email, that doesn't mean anything. You've sent it, but what are the chances they read it? We tried as hard as we could to be extremely transparent. It is one of our values of what we're changing, why we're changing, and how customers can get maximum value out of the service.

Does it make any sense to have-- if there's an opportunity to have an iterative interaction, it's like, oh, I helped them last time. I have an opportunity to figure out how to take this one out. We have logic in place, routing via Gainsight, where I think it's-- Keith, you may know. I think it's 60 days.

If the customer has been with a certain CSE in the last 60 days, it reroutes to them automatically for familiarity. Yeah. So how are you determining which accounts sit here, skip one gig, and then also how do you remind people what kind of [INAUDIBLE] Yes. So first question, who ended up in the scale bucket was just blunt force ARR spend.

You spend between x and y, and you were in the bucket. We didn't get fancy. We would love to with propensity to expand, et cetera. It was blunt force.

And how we made the decisions of the right really goes back to the profile we hired for. So it was a tough time. It was a time of transition for us. And most of them did, but not all of them did.

All right. One more question. Can you be more specific about how the CSM is different from a CSE? Yeah.

I like sports analogies. So I'm going to go back to my offense and my defense. A CSE is mostly working a reactive queue. They're kind of a Jack and Jane of all trades.

They have as much information as possible to be successful, but they're speed dating. And they have to be able to off-road quickly in conversations. A CSM is a dedicated resource. We are now running that at 1 to 10.

Those are account relationships. There's a lot of account management in there. And with our paid model in CSM, we also have a CSA paid model, customer success architect. We have deliverables that we have to meet.

We have accelerators. We have workshops. So it's NetIRR. It's not a service, but it is a SKU.

It's renewable. So the stakes are higher when you pay for it. And there is literally a laundry list of when you pay for this, these are the deliverables you get. Are they just detailed and technical?

Is it CSEs, the CSMs, and the CSAs? The CSAs are. The CSMs are a mix. I see you.

[INAUDIBLE] No, that's hard. So the question was, are the AEs aligned to the model? And that's something we're struggling with right now. We would like to make that cleaner.

But you may be an AE, and you might have all of us. You might have a CSM, depending on your book of 10. And that's hard and can be confusing. We just train them.

I mean, it's revenue. I mean, it's pretty simple. I mean, there's a field in Salesforce, obviously, that indicates you either have a human in there or you have the scale field in there. But we did succeed, I would say, in training the field on.

They're spending between x and y, and you're not able to sell them success tiers. They're in the pool. Well, I'm sure Ally is happy to continue the conversation after this session, but really appreciate your time and the insights and everyone who joined. Thank you.

You guys are a great group. Thank you very much. Thanks, everyone.