Automate This, Not That: A Framework for Knowing the Difference

Thanks, Roz. So it's really great to be here today. As Roz said, I am the head of AI at Front. For those of you unfamiliar with Front, Front is a customer communications platform.

It's a little bit different in that people use it for multiple teams, right? So 90% of our customers have customer success and customer support or customer operations, multiple teams using Front so that they can collaborate. But today I am here, I mean yes, I am working on AI, but the 15 years before I joined Front, 18 months ago, I was an operator. I had started a couple of companies.

The first one was a food delivery business where we had a thousand customers and like including enterprise accounts like Netflix and Google and so I ran customer operations there and then right before Front I spent nine years running a SaaS business. Actually before AI was cool, we were classifying customer conversations for companies that got a lot of it. So like Slack was our first customer and I also had the customer success teams rolling into me at that time. So the discipline that you are all in is one that I care a lot about.

So yeah, let's dive in. Let's start with the question. Think about your book or the books of your team members. How many of those accounts are folks genuinely ahead of?

And like for myself, speaking for my previous teams, the answer was like three to five, right? And those were the biggest accounts, those ones we cared the most about. For everyone else, we represented a huge percentage of the book. We just didn't have the time to necessarily be very proactive.

So we would talk to them when renewals were coming up or if something happened and it was inbound. And you know, this is, we understand that that is not the best way to do it, right? But it's not really a hustle problem. It's not like the teams are being lazy.

It's just a constraint of how much time they have to really do all of these things proactively. And you know, what we're trying to talk about today is how things are different now. Some of the constraints are fundamentally different because of AI. So from this research study earlier this year around what we call the coordination tax.

And what was happening was we went out to over 600 companies, B2B companies, and across six industries, and we asked them a bunch of questions. And one trend that came out was that for every hour teams are spending actually solving problems, they're spending three hours coordinating. So what that means is like, you know, trying to find information, asking other teams for information, all that. And you know, that's really backwards, right?

In that you want, you know, your customer success professionals to really be spending time on their relationships, solving problems for their customers. And you know, so this is something that we call the coordination tax. And only 14% of companies, one in seven, were actually spending more time solving problems than, you know, chasing systems around. And that's something that really has to change.

So the reason that this coordination tax stays invisible and people haven't really been able to tackle it is because everything that the teams measure is usually a lagging indicator. So renewal dates, last touch, NPS, this tells you what happened, but they don't tell you what's happening right now or what's about to happen if no one acts. The signals that predict turn, predict expansion, predict the relationship at risk, they exist. They're in your customer conversations, they're in your product usage data, your CRM, on LinkedIn, your support queue.

You know, the information is like, hey, a global admin that changed jobs, the account that's at 38% utilization, going into renewal in 60 days, the gone call from last quarter that had a signal that no one caught. And so the information is there, but the problem is that humans can't actually have the hours to find this. So, you know, my CMO always tells a story. So I'll go back and tell a story about my last startup, Idiomatic.

So, you know, at that company, we had these big customers that I talked about, and I thought our customer success team was like pretty on top of it, right? They looked at product metrics, they looked at what was coming up, account scores, etc. And then one quarter, we lost three major accounts, and it had such a big impact on the business that like I found myself in front of the whole company trying to, you know, calm people's nerves and explain like what happened. But what actually happened was these invisible signals that we're talking about.

So like one company brought in a new chief customer officer, so our champion was still there in his role, but the chief customer officer had a different way of doing things, a different tech stack that she was familiar with. And even though we actually had six months to renewal, we didn't really understand that, we didn't see that, and we lost opportunity to really build a relationship, and we didn't do it, and we lost the account. Another customer was gearing up for IPO, and gearing into IPO, you really want your financials to look really good. And so we got these low key questions around, you know, renewal terms and like, you know, various clauses, and our support team like didn't just answer the questions.

But we didn't really capture the fact that this is something we should be worried about, and then, you know, that they turned. And then the third customer started traveling, actually competitive product, and we should have been able to see it inside, I mean, we did see it inside the systems that were integrated with, but it was not something that anyone really paid attention to. And we should have used that as a signal that, hey, there's some dynamics going on here, we have to maintain this. And so none of these customers called us, none of them really sent a warning email, the first signal we got was the intent to churn, and we didn't really have a system to find it, right?

So those are the types of things that we really want to be able to prevent. All right, so most AI advice starts with technology. Here's what AI can do. Now let's find workflows that it can automate.

We think, or my experience is that that's kind of backwards, and we should start from the customer. And the question is, what is in the relationship with each account, where judgment of the CSM actually creates value? Where does a human being with memory, with empathy, with the ability to read a room, have to be able to do that and no automated system can recreate. And once you know that, the automation decision, the decision is about what you focus technology on, become a lot clearer.

So here's the framework that I use. So there's basically three buckets. There's, and everything that your team does should fit in one of these. The first bucket is things that you should fully automate.

This is research, signal detection, system maintenance, meeting prep, CRM updates, order change monitoring. It's like basically reading everything available in all the systems you have in the internet and synthesizing that information. And AI should be doing this because as I talked about earlier, this takes an extraordinary amount of time for humans to do. And this is where humans are teams frequently under invest.

And they're spending human hours doing these things that, you know, humans don't really need to be doing. The second bucket is assist activities. And this is more like the triage layer where AI has done the research and surfaced what matters. And the CSM's job is to triage and make a judgment call, right?

What accounts, given this information, require my attention today, what signals are important to react to? So this is where, yeah, the grunt work is done by the machine, the judgment is done by the CSM. The third bucket, which is called own or protect, is just what humans should really, really be investing their time in, right? This is the relationship itself.

This is the QBR, the renewal conversations, the executive escalation, the moment where the customer is deciding whether your partnership is worth continuing, there should be no AI in that specific room. That's yours or your team's. And it should be because the presence, your presence, your memory of the relationship, that is what, and the ability to read what is unsaid is what allows you to actually keep those accounts and expand them. And so the goal of doing this bucketing is to get as many hours as possible into the third bucket and as few hours as possible in the first bucket.

So the uncomfortable truth about this framework is that when I talk to customer success teams and ask them, like including the ones at front, and ask them to map their work against these three buckets, most of them discover that they're spending a lot of their time doing the ones on the left. And what that means is that, you know, they are 10 p.m. the night before QBR, and they're like trying to do research to be prepped for this. And these are things that, you know, AI could be doing so that they can focus on the actual relationship.

Now, again, this is like really not about laziness or, you know, a commitment. This is just a bad process and it's the lack of infrastructure to really automate this work. And, you know, like I said earlier, we just really need to find a way to prevent people from spending high value time on low value work. So most teams have actually tried to fix this.

With AI and a lot of them have been a little bit disappointed. And the reason why maps a bit to the framework. So the most common mistake is automating without knowing which bucket someone is in or a certain work is in. So a team will buy an AI tool.

They will point it at their ticketing system and say, hey, let's surface some renewal risks. But the AI is reading one system. It doesn't know about the conversations that are in the CSM's inbox or in the support queue. And it just like misses signals that are really important.

71% of companies in our survey said that they had at least one major AI failure in the last three months. For 25% of them, they're saying that they're seeing these failures daily. And the most common failure was that they're saying that AI created more coordination work and not less. And that's what happens when you try to automate work that isn't actually in that first bucket yet.

And the context and because the context that is required for AI to do that work properly isn't in that system. And so when you try to point AI at the things that require judgment or the things that require relationships, you get chaos and really results that are very underwhelming. So here's the question that unlocks the framework in practice. The question you really want to ask is not what can we automate?

But it's actually if research and analysis were free, right? And like I know, like, you know, companies like getting excited, trying to charge for AI, but like functionally relative to human time, it's basically free. And you could read anything you want, you know, search any sort of data, and the prep happened automatically before anyone spent time on it. What would your teams do differently?

And the answer to this question tells you actually what belongs in which bucket for your specific team. And it also tells you what your CSMs could actually stop doing and what they would finally have time to start doing. And this is actually one of the core questions that we found it very useful to do internally. So if you build the intelligence layer that makes this framework real, it looks like three things working together.

First, you have proactive account health. It's not a dashboard you check, but rather a system that automatically sits through all the signals and surfaces risk and expansion signals automatically. The account at 38% utilization going into renewal gets flagged before it becomes an emergency. The next the second thing is what is the next best action?

It's not just what's happening across your book, but what to do about it. So AI has done an analysis and your job is to make a judgment call about, okay, what should I do about this? And then the third thing is to have everything in one place. So I'm sure your CSMs, certainly in our company, they are switching between their CRM, their email, their calendar, the product analytics, call recordings, and a million Slack channels to prep for a single customer conversation.

And the intelligence layer only works if the context is connected. And so you really need the last six months of relationship history sitting right next to a renewal date, next to open tickets, next to tomorrow's QBR prep. And that's what we that's kind of the solution that you really want. So the 14% of the companies are spending more time solving problems than coordinating aren't doing that because they have better AI.

It's because they actually applied this framework and they actually pointed automation at the work that should be automated. They built the assist layer to triage everything before the inbox opens. And they also protected the relationship work by removing everything else from the customer success teams' plates. It was interesting at sales kickoff earlier this year, we showed our proc vision to our account management teams as well as our sales teams.

And the response was unlike anything else that we had shown. And the reason for that is that it wasn't abstract. These were the problems that they were dealing with every single day. And they realized that they were reacting to whoever was loudest.

They were, yeah, as I said, working late at night, prepping for tomorrow's meetings. And then they were finding out churn in the moment rather than being able to see it happening. And the companies that are able to break out of the coordination tax have been able to put this into practice. All right.

So let me paint the picture for what this actually should look like. Today, if you have a CSM with a $2 million book, she opens her inbox. She triages what's coming in. And then, yeah, she has a QBR that she hasn't started prepping for.

She has a renewal in 45 days that is at risk but hasn't had time to dig into. And she has 20 accounts that she hasn't meaningfully touched this month because no one called and no one escalated. Now, three of these 20 have signals that she actually doesn't know about. One has an economic buyer that change roles.

One has seat utilization that's dropping for eight straight weeks. One has an open support issue that's sitting for 11 days. But that customer hasn't escalated to her yet. And she'll find out about these things eventually.

The question is whether it's before or after they tell her they're not planning to renew. And in the model we're building towards, she already knows when she opens her inbox because it's sitting there. And because the intelligence layer did all this reading overnight and her morning isn't about triaging, it's about deciding which of these signals that she's actually going to act on. And it's trying to make it so that that bucket at the end, the relationship piece, is her whole job.

So many of you that I've been talking to today, like, actually spend your day increasingly in cloud, cloud co-work. And this is something that, you know, one of the great things about these systems, and I'm sure OpenAI has one of them as well, is that it can just connect to your various systems, right? Your CRM, your email, your backend systems. And what I do is I do a lot of what I did here was I literally told Claude everything in that intelligence layer earlier.

I was like, hey, please build me a view that shows up once a day in the morning that does these things, right? And, yeah, kind of the remarkable thing is that it can actually just build the software. And I mean, in this case, it's a skill where it and I have it scheduled to just run every morning, right? And it will actually pull, like, go through and read the emails, pull the product analytics, and then tell me, okay, these are the folks that you should consider talking to.

Here on one click, you can kind of generate the drafts. Of course, there are there software that will do this. But I think my point here is that you don't actually have to buy bespoke software. You could actually start experimenting with some of this type of behavior just in the platforms that you probably use already.

So rather than just giving you, you know, a framework, right, of three buckets, let's just talk about what are the three things that you could literally do on Monday or tomorrow so that it kind of moves you forward. So the first thing is that to map your workflows, right? Look at what you're spending your time doing, what your teams are spending your time doing, take the time workflows and really try to classify them into, okay, is this something that should be fully automated? Is it just research and analysis?

Is it something that requires some judgment? Or is it something that requires relationships? And then kind of, like, build a version of that map for your organization itself. The second one is to name accounts.

So, like, try to find kind of like in my story of the three companies that I lost, customers I lost, try to figure out, okay, what went wrong in those cases? And then what could have happened in an alternative world where research and analysis were free? And then that will also give you ideas about what you would want to put in practice. And the third one is probably the most important.

And this is the question that I had asked earlier, which is, what would you do if research and analysis were free? And, you know, ask your teams because they'll actually give you different answers that, and this is going to be different for every single organization. And it tells you, like, which buckets, what processes are under built within your organization. And, like, what your teams are wasting their time on.

And this is going to be different for every organization. And that's where the coordination tax is really hitting the hardest, right? And so if you do these three things, then it gives you a chance to kind of switch the balance so that your team is spending their time when they want in ways that are valuable. So to close, right, the coordination tax is very real.

The information gap is real. The reactive model that most teams are running is not a character flaw or indictment of them. It's really a rational response to the constraints that have never really had a solution. But that constraint really is changing.

And so when the cost of research and signal detection drops close to zero, when your CSMs are able to walk into every conversation already knowing everything, the job doesn't get automated. The first two buckets finally work the way they're supposed to. And the third bucket, the relationship bucket is are the things that only humans can do. And that becomes the whole job, right?

And actually, one of the things that we're seeing is that as customer success professionals get more productive, the ROI on the investment actually increases. And they actually should increase their effectiveness and their impact to the organizations rather than being something that they fear that AI automates away. So that's the reinvention. The framework of the three buckets is how you get there.

And yeah, so before your next playing conversation, I would advise, you know, try to map the work to the buckets and find out where your team is spending protective time and start there. [Applause] Thank you, Kevin. As a reminder, you guys can still ask questions. Just go to the app, click on room 126.

You can also upvote any questions. So if there's any there that you really want to get answered, feel free to upvote those. So starting off, your three bucket framework, fully automate, AI assist, and own is a useful lens. Where do you see us leaders tend to get this wrong?

What do they over automate or under automate most often? So I think one thing that companies do is they literally take their existing workflows and try to get a system to automate them end to end. And this is like AI is great at some things and it's terrible at other things. And then when you basically get it, have it try to do things that it's not great at, it kind of falls down.

Right? And so I think it's not really a matter of over automating or under automating, but it's not being discerning about what AI is really good at and where judgment and the relationship context is really important. Right? And so basically rather than trying to reproduce your current workflows, chopping it up and then taking that first bucket and then pointing AI at it, that's probably the best advice I could give.

Personal question. Well, not personal, but I guess my question. Thinking about AI, being the head of AI, how does your company like do all AI requests get funneled through you or does everyone on your team have access to different LLMs and they're kind of all doing their own thing and figuring out what works best or how do you make sure that the AI that's being built in your company is actually impactful and everyone's not duplicating efforts? Yeah, that's a good question.

So certainly is not being funneled through me. Right? For this to work well, basically people have to be enabled throughout the organization. We do a couple of interesting things where we have like AI champions program.

So people that are fairly well informed about what's going on in the industry and probably are building with it and they get access to the tools earlier and then they also kind of inform our choices around what tools we adopt. And then on top of that, we also have actively encourage team members to showcase the things that they're doing. So we will create time at all hands for folks to play videos or demo their workflows. And then one of the things, the journey that we're going through right now is trying to make it so that you don't have five CSMs that have five versions of meeting prep.

Right? And so we're trying to make it so that like our organization is on Claude. So if people build skills, like to be able to basically share those skills right now, I don't know that the skill sharing mechanism is very good, but you could, they're just text files. You can literally just copy and paste them, upload them, and then people can install it into their own instances.

But yeah, that's a journey that we're going through. Yes, I think having a skills repository somewhere is kind of the next version of making sure that everyone's aligned on what people are using. For teams that are still largely reactive today, what's the single best first step toward building a more proactive AI assisted motion? So one of the things, I mean, this is maybe something like my therapist says to me, but like you should just start, right?

And find a place to start rather than saying, hey, I need to transform my entire organization overnight. You know, if you say, do that exercise and look at the deals that we have lost with the benefit of hindsight, what signals would have been helpful that would have allowed us to act differently? Like start there, right? Say, all right, we are going to have AI go into whatever system, right?

Prox system, email, whatever, and do the analysis every night and then populate a spreadsheet with a certain score. That's a scope problem that you could literally do like in a day, right? And I think once you start and once team members have fluency about what is possible, then it's kind of a snowball effect and folks will kind of like individual CSMs will see this themselves and then you will get innovation that comes from or organically from the team. Do you have cloud directly connected to a product database to a credit usage as a risk or renewal up?

If so, what are the security implications of connecting to a product database? Also, did you define good and bad usage via skill or are you letting cloud make its own conclusions? Okay, so two separate questions. So the way we do it, not saying this is the best way to do it, but we have a business intelligence system that like all of our product data, usage data lives in Snowflake and then we have a system called Omni or Looker or Tableau or other versions of this and our version, our BI solution is called Omni and so that basically has a bunch of tables that every or it has data that everyone has access to.

Everyone within the company has access to scope to their role and Omni has MCP server which we then connect cloud to and so you're able to basically ask questions, right? And you can ask questions like, hey, of the companies that are in my book, right, and it's inside that system who the CSM is for every company. What are companies that are coming up for renewal in the next 60 days that have less than 70% seat utilization, right? And it will use MCP server to basically run that query and figure it out, right?

Now, what are the security implications? At some level, because these BI systems are already available to our individuals and that their access is scoped to them individually, it's really, it doesn't really create any additional surface area, right? Like, they could have done this manually. It's just that they may not have had the technical skills.

They may not even know that these tables exist but they know how to ask the question, right? And so that's really the gap. We're not actually changing in this model the access controls that individuals have. We are just making them incredibly fluent in writing SQL when they wouldn't have been able to do it.

Now, the final part of the question was how do we define good and bad usage and are we letting cloud use its conclusions? This is actually the thing that humans are really, this is where your retro of past lost deals is so important, right? Because it allows you to figure out what the story is. Okay, how was it that there was high product usage but they still churn?

And then you're able to kind of describe the signals that are actually the most predictive. But it's very important to define these. You really, I mean, you can have cloud take the first step but it is so less, so poorly informed relative to your actual deal history. In a perfect world, what are the main sources an LLM or AI tool should be connected to in order to actually be able to truly automate tasks?

And do you feel like security or IT teams are being smart by being cautious about connecting the systems or are they impeding progress? This is my question. I definitely spend some time arguing with security. Though I realize this is recorded so I should be so upset.

So the model that is working for us is to find systems that have permission models. Like if they have MCP servers or API access where it gives you exactly the permissions that you have as a human, right? And by and large, this is the way most business applications are moving. Certainly, front MCP works that way.

But the tools that I think are really useful for CSMs are certainly email, your CRM, right? Salesforce, your product analytics, so in this case, we have Omni. We also have it give it access to our code base actually because through GitHub, so that allows basically all CSMs to become technical because they actually understand what is actually happening. And that last one is very dependent on your business, right?

We're a software business, so that turns out to be really helpful. I do think that security and IT teams are starting to come around. I do think that telling them, hey, just giving us access, giving our LMs access to the systems that I have access as a human, that seems to be a really simple model that satisfies their various considerations. What use case of AI has had the biggest impact on retention for you?

So we actually did some really interesting analysis as AI assisted that told us what are the three vectors that are most important for retention at front? So like I said, front is customer engagement software, and we see, okay, if people have automated rules that route conversations to the right people, that's one vector. There's another vector that is if they are collaborating on conversations, that's really important. The third one is if they have some sort of like they use analytics to kind of monitor what is happening across the board, that really matters.

And these insights, like you get like a PhD in data science in your LLMs, right? And so you can actually just have the LLMs kind of figure these out, do the statistical analysis, and then it really simplifies the problem for the CSMs because we're like, okay, we will just run plays to drive these three behaviors because we know that these three behaviors highly correlate with retention and expansion, right? And so that intelligence has actually really been helpful for us. We have a huge tech stack.

How to convince our leader to define the bucket before they implement another automation. The thing is that this exercise of bucketing does not take that much time, right? Like I bet on your plane ride home, you could literally just do it in 30, 60 minutes. And I think if you just tell the story of the success, like it just kind of stands to reason that the tasks that are in the automate bucket, AI will do a much better job on.

And so I think, you know, you do that exercise and then just make the case to it. I know that was not a very useful answer. I think on the huge tech stack part, again, just getting started is better than doing everything, right? And so if you find the parts of your tech stack that are already like half MCP servers are scoped well and it's very easy to connect it to cloud, then just start with that, right?

And then what I suspect will happen is over time, as the world moves in this direction, the companies that are the their systems are very closed will become less attractive, right? So over time, I imagine that more and more systems will be accessible easily in the way that that's useful. How are you measuring your team's AI efficiency? Yeah, this is a this is a strong debate.

I spent a lot of my time on the engineering side of the world as well. And a bad way of doing this, and again, this is a debate we have internally, is like lines of code written by AI, right? And you can just have one engineer saying, hey, generate 10,000 lines of code, and we'll do that. And that doesn't help us at all.

So what we do is we step back and say, okay, what is the net goal, right? And so in product development is how quickly can we actually get features out? And so we will do things like we will do product road mapping for a quarter based on how we would traditionally develop. And then we'll say, all right, if we are as going to have an impact, we will be able to finish this three month roadmap in one month, right?

And the teams are able to really compress the timelines are effective. And the ones that aren't like, sometimes it's like they're not buying in. But sometimes there are blockers that we just didn't understand, right? And so, like, for example, we have found that it's actually really easy to write code and build features, but it's hard to stand up the infrastructure behind it.

And so that's a bottleneck that we have to as an organization invest more time in to make it so that it's easy to stand up infrastructure, right? And so, yeah, I mean, I would just say, that's an analog from the engineering side of the world. On the CSM side, it really comes down to, you know, what do you want people spending time on probably the relationship things. So if they're able to spend much more time in the third bucket, that's probably a win.

The bucket exercise, is that something that CS teams do like quarterly or biannually? I think it doesn't it shouldn't change that much, right? So if you like, you get by far the most value doing it the first time. If you find that it the the automation efforts aren't working, then it's maybe worth revisiting to see whether or not the first pass at it had some errors.

Do you think this framework changes when considering traditional named CSM coverage versus pooled digital CS coverage? How? That's a good question. Like, I think conceptually, it probably makes the two a little bit more similar in the sense, in that before pooled and digital CS, you always had to have systems, right?

Otherwise, it would have been overwhelming. But I do think that I guess the difference is maybe the motions in terms of outreach may be a little bit different, right? Whereas for like kind of named CSM coverage, that action may be going and talking to the customer for pooled and digital, it would be more campaigns. But I think it stands that so I actually think the name CSMs get more leverage of this because they're probably doing more of it manually before.

Teams are all testing different paths to the same outcome with AI. How do you unwind and enable champions to ensure the team is enabled for consistency? Yeah. So I actually don't think getting consistency early on is the right path.

It is actually remarkable the amount of creativity that comes bottoms up. So I think what it is is it's like you give people access to tools, you encourage them with some cases where it works really well. And you I mean, we occasionally do like kind of show and tell type of things to give like social karma to the people that are doing this to encourage other people to try. And that literacy then causes more innovation.

And, you know, as managers, this is an opportunity where like if you see something that works really well, then you then say, all right, well, how do we turn this into something that everyone does? Right. But I think the premature standardization may actually cut off the ability to get something that's actually impactful. Yeah.

Yeah, I like that. That's what I was talking to someone else about. With AI, like what are the parts that you want to make consistent and standardized versus kind of what do you want to give your team free reign to do so that you don't cut off creativity? So do you believe the action plan risk mitigation strategy should be generated by AI and then reviewed by humans?

Or do you consider this a task that should remain 100% human led because it involves critical judgment and decision making? So one of the things that I think is most fun playing with Claude is trying to get it to make the judgments the way I would. Right. And so like it's so the short answer is that I think you should be having AI like tee up, tee things up and recommending the judgment.

But the way I actually do this is very similar to like onboarding a teammate, which is I will give high level instructions, right? I'll say, okay, here are the things that I consider turn risks, right? And this is what I've seen. And then it will take a pass at it and it will produce something.

And then I will critique it and I'll say, okay, I think this should be higher risk than that one for these reasons. And then I will ask it, hey, can you try to rewrite your skill to take this into account? And then it will do that. And then you just iterate over and over again.

And so I think when working with AI systems, the thing is that you want to repeat as you want to try to get it to do the things that you would want to do. And you want to review the outcomes over and over again. And it will make it so that the system gets better. But also, you're able to impart more and more of your judgment there.

And yeah, so I think that's more the path rather than saying, is it AI is a human? Perfect. Don't forget to give feedback to Kevin's session after this, please. You can submit the survey in this session afterwards.

So thank you all for attending. Please, I'll see you in the foyer for some drinks and refreshments. And Kevin, thank you so much. It was a pleasure.

Thanks, everyone.