There Is No Future-Proof: Building Digital CX as AI Rewrites the Rules

[Speaker 2]
I see some familiar faces from the community session that I did a little bit earlier, but by way of reintroduction for the new folks that are in the room, I'm Sam, I am VP of Education Strategy at Gainsight, so I oversee the SkillJar and Gainsight CE product lines, so product strategy and go-to-market strategy. I am joined today by my friend, customer education extraordinaire, Adam Avramescu. Do you wanna maybe introduce yourself and just let's start with like a bit of a share on your journey with customer education and into, as of late, I guess, digital CX more so.

[Speaker 1]
Yeah, yeah, happy to, and thank you for extraordinaire. No, you know. Really, it's a lovely word.

So, yeah, indeed, most of my experience has been in building and leading up customer education programs. I started doing this now, I guess, an embarrassing long time ago, back in the world when SaaS and customer success were still kind of new words. I like to say the first time that I got a job in a customer education team, in a customer success team for a SaaS company, I was evaluating the job offer, and I called my dad, and I was explaining this to him.

He was like, what's the job? Well, I'm gonna be on this customer success team. They're hiring for a customer education role, and he said, customer success, that sounds fake, don't take it.

So since then, though, I have been really enjoying it. I've been building and leading teams that on one hand are customer education, but are more holistic than that. So very rarely just what you think of as narrow customer education with just trainings and webinars and what have you.

So it's always been more expansive to include things like documentation and communities and in-app. And that's what eventually brought me into the world of scaled customer experience to digital. My most recent couple of roles were at Slack, through the brief life as a public company into the acquisition by Salesforce, and then what brought me out to Europe, because I'm currently in Amsterdam, to a company called Personio, where they originally hired me to build and lead, again, a very holistic customer education team, which eventually morphed into what we called scaled customer experience.

So I was VP of scaled CX there, most recently left earlier this year, and now I have been consulting and really enjoying the opportunity to see what people are doing across the board. Awesome.

[Speaker 2]
Okay, so you mentioned the shift to scaled CX, and I'd love to, I guess, get a show of hands from this room, because I see some familiar faces again, but some of you are in communities, some of you are in customer education, some of you might be in scaled, some of you may even be in more traditional, I guess, like customer success, and so I'd love to get a sense from the room what, where you lean. So customer education, raise your hands.

Amazing, love it. Okay, community? Okay, only a few.

What about scaled or digital focus? So like in-app engagement and the like?

[Speaker 3]
All right.

[Speaker 2]
All righty, amazing. And what about just like traditional CS? Any folks in the room?

Awesome, okay, good. You had another question that we wanted to ask as well.

[Speaker 1]
The AI native. Yeah, I'm curious, who works for an AI native company?

[Speaker 2]
Okay.

[Speaker 1]
A few. A few, nice.

[Speaker 2]
Okay, cool. That'll just help us kinda tailor some of the questions that I had planned. I wanna make sure that this is relevant for all of you in the room.

Topic, actually, before I get into that, questions, if you have questions, we're gonna save the Q&A for the end. If you pop into the app under track four, you'll be able to drop your questions in there, and I'll make sure that we reserve some time at the end for audience Q&A. As we mentioned in the description for this talk, this is an unscripted conversation between Adam and I.

We go way back, and so expect a fun time. Very unscripted. But the purpose of this discussion is to talk about agentic and the shift to agentic.

Everyone is talking about this shift. You heard Chuck talk about it this morning in the keynote, right? We're all trying to make this shift right now.

But the fundamentals, we believe, really still matter. Because even in this world where we're shifting to headless everything and we're shifting to agentic, not everyone has this figured out. And at the end of the day, these fundamentals that we've always aimed for around, I guess, digital, but customer education, community, et cetera, are still extremely important.

So we're gonna dig into that a little bit. The first question that I have for Adam came from our community. So if you follow Adam on LinkedIn, you would have seen him poll our customer education community on LinkedIn.

The question that came from Clea, what have you been most surprised by and most disappointed by in your own AI experimentation?

[Speaker 1]
Yeah, so if we kind of hit the positives and the negatives, I would say surprised by in a positive way is there are things that for years I would have thought were completely inconceivable. Things that might have taken months or weeks, let's say maybe months isn't an exaggeration, weeks, now can take days or even hours. So as an example, I can now create a solid on-brand product walkthrough video just by clicking through the product and have that then output with high quality 11 Labs voices, a great product tutorial video that can then be updated as the product gets updated.

That's something that would have been completely inconceivable. I can create a first draft interactive on-brand course module from an outline in again, a day if I want to. These things are really, really powerful.

Even when we think about connecting data, now what is available through an MCP that before would have taken so long to basically get into a Tableau dashboard that is completely inflexible and is very rigid in terms of what I can report on now is possible and just wasn't possible before. So those are some really, really positive things. What I've been most disappointed in I guess is I still feel like we are running towards this dream where we say because so many things are possible with AI, shouldn't we be able to have a completely end-to-end automated no human in the loop experience, whether that's a production pipeline or a customer experience.

And I feel like whenever I am creating something, I always have a decision that I need to make, which is maybe you've all had a moment like this where you start creating something with AI and you get to like a pretty okay first draft and then you keep tinkering and then you keep tinkering and then you keep debugging and then you keep debugging and then you're like, yeah, it probably would have taken me less time just to do this manually. And I always have to look at, okay, am I doing this because I'm enjoying the tinkering or am I doing this because actually getting AI to do this replicably, that's a word, and I said it correctly.

[Speaker 2]
You did.

[Speaker 1]
To get it to do it over and over in an automated fashion, in a scalable fashion, then it might be worth putting in the cycles to continue debugging it, but in a lot of cases, maybe the juice wasn't worth the squeeze. So yeah, I think in some cases, like some of those things that really seem like we should be able to do that, like should doesn't always equal reality, especially when you think about the labor costs, like my time or the cost costs in terms of token spend.

[Speaker 2]
What about, so let's dig into like what is noise and hype versus what is actually moving the needle? So as a consultant, you're obviously seeing this across multiple teams of varying sizes. What are you seeing that still leans on, again, the assumption that this should be easier, this assumption that AI is gonna be able to support this particular workflow versus what's actually moving the needle in teams, making a true impact?

[Speaker 1]
Yeah, I mean, maybe I'll start with some of the things that I think are working well. So one is, again, around production time and production pipelines, especially when we're talking about content. And actually, maybe I'll even zoom out before that because the reason I think content is important is not just because I'm a customer education person and my life has been about creating content for a really long time.

Obviously, I'm a big content booster, but because when you actually look at what works well from an agent perspective, one thing that I've been leaning into more and actually when I was at Personia, it was my last major project, was about implementing an agentic AI support system. And there, we were doing kind of a combination of a build and a buy, where we were looking at how do we create a support experience that starts in-app, it is contextual, there is basically a hub that you start from so you're not confused as a customer about where to go to get a support question answered. And then at the same time, make sure that that was integrated with the agent that we were using to get the product itself to do things.

So a bit of a question there around what's the more confusing versus more clear experience? Do you create two separate agents that somehow sit side by side and ask the customer to do the mental work of figuring out which agent to use for what? Or do you create one agent that can basically traffic control between the two of them, even though that might be a little bit opaque to the customer?

We chose the latter. That was more complicated for us to implement. But as a result of doing this, within a couple of months of having this up and running, we were already seeing over 80% resolution rates, which, and this is a B2B company, so especially if we compare this to what we were seeing before and the impact that it had on our support team in terms of needing to answer a, at that point, days-long backlog, it really had an impact.

And it had an impact during our busiest season. Because as it turns out, HR is pretty seasonal. And when you're getting into January, a lot of things reset.

Our support ticket volume spikes. So that worked really well. But the reason I think it worked really well, and now I'm realizing how long of a detour that I took, is primarily because the content that we had underneath it was already strong.

The success of an agent is so much the context that it can pull from and solid content that's already been developed and vetted. So I guess that's already one thing that I see work well with AI, but another thing that I see work well is AI to actually improve the content pipeline. So I'll keep this part short, is to say examples like I was giving before, being able to generate product videos or course drafts or things like that, again, from well-positioned templates.

But we're talking about what, what is hype?

[Speaker 3]
What is hype?

[Speaker 1]
A lot. I feel like a lot of the time, there are so many things where either, back when I was in my full-time job, an executive would come to me, and I see it in consulting as well, where I have an initial discovery meeting, and they're like, why can't we just, why can't we just do this? Why can't we just create a completely self-healing AI-first documentation system that keeps pace with product releases?

Well, okay, like, why can't we just? AI is really good right now, why can't we just do it? And the thing that I've started to realize about why can't we just questions, even though I don't really like why can't we just as a question to begin with, is it's often framed as a question about scrappiness.

Like, why can't we just do this now? But actually, I think why can't we just is more of a question of prioritization and foundations. We can do a lot of things, we can just do a lot of things if we as an organization are aligned behind them, if we have the right resources, if the data is ready to be ingested and modeled, and in some cases, if we can absorb the cost of throwing tokens at the problem.

So I think that's maybe a little bit where hype and reality conflict with each other.

[Speaker 2]
Sure, let's talk a little bit more then about these, like the alternate why can't we just, well, what does it actually take to create those experiences? In an ideal world, right, if we wanna get to a place where we can answer to these executives and say, yeah, we can actually do that, what does it actually take? We've talked in our prep around data hygiene, number one, jobs to be done frameworks, content architectures.

Can you talk us through a little bit of what you've seen lay the foundation properly for us to be able to get to a place where some of this stuff becomes possible?

[Speaker 1]
Yeah, I mean, I think you just hit a lot of the key points, and those of us who were around before AI, those don't sound very different from what was already necessary to do those things. It's that last mile that we can now do a lot faster. So yeah, where should we start?

With jobs to be done frameworks. Sure. If we don't understand what our customer is trying to do with our product and who they are, then what ends up happening is we can create a lot of content quicker, but that content is not necessarily relevant or consumed.

So I think one trap that I've seen teams fall into, and I've fallen into myself, is basically creating systems to be able to create a lot more content and to create it quickly, but then you look at how the content is actually being consumed or how it's going to be updated over time or the costs to keep it updated or to keep it vetted, because we know garbage in, garbage out when it comes to content, that if we're not doing it with already a stronger hypothesis about which personas are we developing for, what are the most frequent tasks they're trying to perform with our product?

What are the most frequent support questions that they're asking and really already designed with that in mind, then yeah, that's basically a recipe towards creating a bunch of content that's either going to not be consumed or is going to end up going out of date. Sort of the same thing with data hygiene. If we're looking at the wrong data or if the data's not out of date.

I'll give another example of this, actually, from the AI support world, which is that when you're just looking at answering a typical support question where it can pull from the documentation, well then, having the content might be enough, but we usually want to push that forward because that can get you to about, again, like 80% of the questions that are being asked, especially if it's very how-to or things like that. But then you have a whole layer of other questions that start being asked, and these are about troubleshooting or these are about how do I, X, Y, Z use case. And especially when it's troubleshooting, when the question is like, why can't I see something in the app?

Why isn't this working the way that I expect it to? Well, then you actually need a data pipeline that operates in near real time. Otherwise, what the agent is going to do when it gets to that question is it's going to either direct you to the documentation, which tells you, in theory, what might be happening, but not actually how to solve this problem, or it might be basically giving you an out-of-date feedback loop that then requires escalation behavior to your support team or your CSM, depending on what type of question is being asked.

[Speaker 2]
Do you see community playing a role there in terms of community content, posts, threads, where some of that might be getting answered differently than the Help Center article? And so in a world in which the support bot is pulling from all of these different digital resources, do you see the accuracy increasing, I guess, and the resolution increasing as a result of having access to more content?

[Speaker 1]
That's a double-edged sword. So with community content especially, there's a question of how accurate is what's being posted in the community and how much is it enhancing versus polluting the data. So we've actually had examples like this where we've tried to implement either federated search, that includes all of these different channels, or in some cases tried to ingest all of these into an LLM so that they could be queried, and we have had instances where, including a massive volume of community content, well, some of that might be out of date, some of that might be super edge-casey, some of that might have been written by someone who didn't actually know what they were talking about, and in some cases can actually degrade the quality of the answers being given versus the kind of word of God that you published. But at the same time, if you don't find a way to include some vetted content from the community, then all you're drawing from is that word of God content, which doesn't include all the clever hacks and workarounds that customers found that actually can solve a problem. So no solid answer to this, except that I would look at putting some parameters on it, like weighting certain pieces of content higher than other pieces of content, maybe looking at content freshness when it comes to community posts, maybe looking at something that's been marked by as a verified answer.

Right.

[Speaker 2]
What about, so you mentioned the 80% resolution with your, was it at Personio? The in-app support agent? Yeah, that was Personio.

How does that change your perception of the role that Academy plays in this world? Because if an agent is able to answer the majority of the how-to questions, then what does that leave for an Academy?

[Speaker 1]
Well, where's the agent getting the information from? Just the help center? Possibly.

Possibly. And again, that's a, it's a great source of content. I would never say don't use your help center to feed your AI support agent or to feed any agents that you're working with.

By the way, the other thing that I would call it, that we've been talking more about native agents owned by you as the vendor, we aren't even talking yet about customers who are accessing your content via MCP or something like that. So I'll take a quick 30-second detour here to say that, again, one fundamental that hasn't changed is meeting your customer where they are and keeping them in mind from that perspective. So we're sort of moving from a world, this is gonna kind of touch back on Academy, where we used to expect that we would put an Academy and a help center and a community into the world and customers would find them and good things would happen.

[Speaker 2]
Magic.

[Speaker 1]
And magic.

[Speaker 2]
Just magic.

[Speaker 1]
And then question mark and then profits.

[Speaker 2]
It all just works.

[Speaker 1]
It all just works. And of course we know that's not true, right? Like if you create any of these properties, then it requires constant marketing and constant awareness for your customers to know that they exist and what they do.

And over time, we tried to resolve a lot of that by putting all of those under a single pane of glass. One site that you could go to that had all of those things together and it was very easy to navigate between them. And by the way, that's still helpful, right?

Because we still have a lot of customers. With every company I've worked with, with the companies I've been consulting with, they still have a lot of organic and referral traffic to their actual help sites, to their actual Academies. That hasn't gone away.

Even some of the biggest AI labs, when I say that, whose names you might be thinking about, are still relying very heavily on having content that is openly accessible on the web. And why is that? Well, partially because of answer engine optimization.

So if I'm going to use Cloud or ChatTPT or insert your LLM of choice to find information on a certain product, they are using, they're scraping the web to find that information. So, and I promise I'm going to get to the question about.

[Speaker 2]
I know you'll get there.

[Speaker 1]
The one that you asked, yeah.

[Speaker 2]
Like our detours.

[Speaker 1]
It's important to have that information in there so that those answers get picked up by consumer LLMs. So that's already one important reason that that content should exist somewhere, whether it's the Help Center or the Academy. But the other thing that they're doing is they're providing product agnostic information. So let's take Cloud Academy as an example of this.

One of the reasons that Cloud Academy exists and is open on the web, available that way, is because there is a large and growing population of people who don't just need to know how to use Cloud, but need to know AI fluency. And they need to learn that. And they might not be in Cloud to learn that.

Well, the same is true of a lot of our apps. Customers may not be in our apps to learn how to do certain things. And customers may be not even in our apps, but now getting information via MCP.

There's one client that I work with right now, a smaller, let's say more startup-y, AI-native company, where they basically have just built into their MCP the ability to query information from their help resources directly from the MCP, because they realize that customers who are primarily using their product through Cloud or through Chat GPT and not through the UI of their product still need help. And they can't just surface an in-app notification.

They can't just put a side blade in there that has contextual help. Really long detour. Maybe answering a different question that nobody asked.

[Speaker 3]
We'll summarize.

[Speaker 1]
When we get to, I guess, when we get back to that question about what is the role still of an academy in that content, well, one is, I guess, what I just said during my detour is that the academy has a lot of information that isn't necessarily about how to resolve basic troubleshooting issues or questions about basic product functionality. You wouldn't expect that to be in a help center. And in fact, having that formatted in help center format when it's consumed by an LLM often dilutes the quality of the answer, because often when you're drawing from that content, what the LLM is expecting to ingest is short, topical, discreet articles with very structured headers, bulleted lists, all of that.

And well, that kind of breaks down when you're still thinking about working with your human audience, who are not necessarily expecting to draw from it that way. They're expecting something that takes me from zero to 60 or I guess zero to 100 if we're doing kilometers per hour.

[Speaker 2]
So academy, we've got, it sounded like role-based is what I kind of understood from that, right?

[Speaker 1]
Role-based and product agnostic, why would I use this to do my job?

[Speaker 2]
In a world in which, all of our roles are getting significantly disrupted by AI at the moment. And so I think there is a place for certifications, especially as organizations are in the middle of massive transformations towards potentially AI native services, potentially to AI native, just like agentic. There is, in my opinion, a need for certification in this world because we need to help people think differently about our product.

And when you're creating a new category or you're creating something net new that didn't exist before, there's a lot of value in a certification program or at least having like strong learning paths that teach people the new way of doing things in this new world. And anything else in the academy?

[Speaker 1]
Yeah, I mean, that last part that you put on there though, I think is so important. Like a lot of the times an exec might come to you and say, we need a certification. And then I think the next most important question is, what do you mean by certification?

What is this meant to do? Because you could go down a rabbit hole of building a super high stakes, formal proctored certification that awards a legally defensive credential. And even in the age of AI, that is still going to probably take you months to stand up correctly.

But more importantly, might have nothing to do with customers are actually looking for. I think certifications don't create demand, certifications respond to demand. But yeah.

[Speaker 2]
And then, but it goes back to what you were saying earlier around jobs to be done, understanding tasks, understanding like, is it an admin certification that requires proctoring and high stakes? If not, do you wanna go down that path or does it just make sense for you to build little C certs? Like some sort of badging or credentialing program.

[Speaker 1]
Like we would call it a certificate or a skill badge or a learning path. And those often work very well if they're role-based, right? Like especially thinking about point in time.

I think these work really well when we are talking about certain points in the customer life cycle, like onboarding, where the customer doesn't have a schema for what your product is supposed to do. They're saying, hey, you know what? I just bought or my boss just bought this product.

I think it vaguely does this. What is it actually supposed to do? And what is it actually supposed to help me do?

That's where learning paths and certifications, especially outside of the product itself can be really helpful because it's not just about learning where to click. It's about learning what this product should do.

[Speaker 2]
Another area that I think Academy still provides a lot of value is instructor-led training or peer-to-peer, like group live learning experiences. Because those of you that were in the room when we had our community conversation earlier, you heard Chuck also talk earlier about human connection being so critical in this new world that we're entering into that none of that goes away. It becomes even more critical.

And so having a place where you can bring together different customers, different users to learn from each other and from an expert is so, so important.

[Speaker 1]
Yeah, I mean, we have this tendency, I think, to look at AI-native companies and say, oh, we're so behind. What are they doing that we're not? And they do have a leg up on legacy companies in that they don't have to refactor their code base to be AI-native.

And I think they take as priors things that those of us who are doing an AI transformation don't have to account for. But a lot of them, especially early stage, are still doing these exact things. They are still having weekly office hours, weekly customer webinars, covering the onboarding topics.

They look pretty much exactly what a SaaS onboarding webinar would look like. They just update faster. So yeah, I think they'll still have their place.

And the reason they have their place is because there is still a large perspective, a large contingent of customers who expect to have some sort of human contact, who expect to be able to ask questions. And I think the bonus is also learning from other customers and seeing what other customers are asking.

[Speaker 2]
Totally. All right, so we've talked about the Academy. Let's talk a little bit about community.

There were a few people in here that are community managers or community-focused. If we have the Academy, we've got some sort of in-app hub. We've got a support agent handling ticket deflection, knowledge base, et cetera.

What is the role of the community then if you think of this channel mix?

[Speaker 1]
Yeah, I mean, I think the community still captures something that none of these other channels really capture that well, which is user-generated answers to long-tail questions. I think that still is an important part of the community. And once those answers are vetted, sort of like we talked about, finding a clever way to feed them back into the sources of truth without necessarily publishing those as these are our word-of-God answers, because again, that's usually a fine line.

Product ideation and co-creation, still a really important part of communities. I think customers expect to be able to give feedback, have that feedback heard, and be able to close the loop around what that feedback meant. And that doesn't necessarily mean answering every single product request and closing it out with a roadmap, but it does mean, at the very least, being able to aggregate what are the top customer requests and tie them to the releases that you're making as a company.

AI makes that easier, because you can now aggregate and monitor those and tag them to when did a, like let's have an agent, for instance, scanning our release notes, and from that, tie that to the top product ideation questions that were coming in the community. So the community manager or the PM, depending on who owns that, knows to come back and close the loop. And then finally, I think in-person events and local communities and user meetups, those are still very alive and well.

In fact, they're growing in a lot of cases where it adds back a human element to what is being perceived as an increasingly faceless AI world.

[Speaker 2]
Okay, we have a lot of folks in the room that are digital and scaled focused. And so one thing that I do wanna click into that I think will provide a lot of value to the folks in attendance is the content production lifecycle and this concept of we're not only training humans anymore, we are also now training agents. And so what does the new content production lifecycle need to look like in order to accommodate both of those audiences?

And how do you make the content production lifecycle faster to match the innovation lifecycle that's happening on the product side?

[Speaker 1]
We haven't talked a lot about experimentation yet, but I think both as an answer to that question and I guess generally thinking about process, like this is one thing that are, I'm saying in capital letters, our current situation, the reality of our current world is forcing us into. But it's something that I've always been a fan of and I think you have too. Because we can easily say, oh, this should be able to be done, but the devil is in the details.

And so when we're thinking about creating content for humans or we're thinking about creating content for agents, well, both of those require certain tests, certain evals, again, depending on, I guess, whether you're talking about humans or AI, to see how that content lands, whether that content is properly absorbed and referenced, have you created a golden set of questions and answers that you can test every time your product updates or every time the model updates, again, to make sure that you are keeping pace with what's being released. But the other thing that I think helps in terms of experimentation is when we think about creating agents or when we think about creating skills, this is a related but not the same thing, thinking of discrete tasks that can be picked up and automated. Because I think often the problem is that we try to create this whole holistic system, we spend six months creating it and testing it, and it doesn't survive contact with the customer, it doesn't survive contact with reality because the space has already moved on.

So I think one thing to say is, what's our hypothesis about what this content is supposed to do? Which audience is it for? And can we progressively test our way into it?

With agents, that is usually going to mean running it against a defined set of evals where we have, again, structured questions or answers, or we have human in the loop to review how well those pieces of content are doing at their jobs. And with humans, I think it's still the way it always is, putting it in front of groups of customers and making sure that we have good ways to monitor qualitative and quantitative feedback. So qualitative, of course, that's the feedback that they give us on surveys or sometimes through humans at the company and making sure that we're keeping a good pulse on that.

Often that is unstructured, but we can seek it and we can aggregate it. And by quantitative, I mean looking at things like drop-off rates in courses. Thumbs up, thumbs down reactions is sort of like qualitative, quantitative, like in the middle, things like that.

[Speaker 2]
One thing that I find very interesting in the content conversation, I've just noticed a lot of teams, as we're talking about AI discoverability, making content available in markdown or in some sort of format that's gonna be readable by agents, I think the common thought process is to just make the content that we developed for humans available to agents. It's just like flip the switch, make it a markdown file and away you go. And to go back to your point earlier about the jobs to be done framework, just the same way that we design for humans and design for the tasks that they need to accomplish and the skills that they possess, the levels of proficiency that they have, we kind of need to do the same thing for agents, right?

Like their job to be done is actually different than the human orchestrating the agents doing the job. And I don't think a lot of people are thinking about this yet or talking about this yet. Like what's your take on that?

How do we design for this?

[Speaker 1]
No, I agree. I mean, I have, well, yeah, I've encountered this firsthand. When we were at Personio, when I was at Personio, for instance, and we were first going through this transformation, we were testing how well agents could read and ingest our help content.

And we had optimized at the time for much longer form documents and courses because our hypothesis that we were operating under for humans is that if you have to jump and toggle between multiple different help articles, you are eventually just going to drop off and get bored and get unengaged. So we should attempt to kind of answer as much of the question or as much of other things that you may not have thought about already within the same longer form article. And then we started testing that with agents and it was terrible.

Any agents that we had pointed at, it could not consume that content correctly. And it wasn't until we started making the articles much more discreet, much more short form and very structured in terms of the headers that we used, for instance, that then those could be consumed and referenced more easily and making sure that as much as possible, we had structured data in there. For example, who is this article for?

And I think the same is generally true for courses where now lesson sizes are starting to get smaller, but also thinking about setting up templates for the way that you create courses and validating those against both AI best practices, which are, by the way, very similar to accessibility best practices. So these are published and documented and working those into the skills that you have to transform raw content into course formatted content.

[Speaker 2]
So I think this sort of covers one of the questions that came in, any best practices to create LMS content for SkillJar to be searched or surfaced by agents, like the separate format, the headers, the already kind of touch on this.

[Speaker 1]
Yeah, I think that's a good way to put it. And then I think it's also thinking about, yeah, whether you're using SkillJar or whether you have a third party system also in there that might require certain templates to be used.

[Speaker 2]
I'm gonna go to another audience question here that's sort of related, but how do you tell whether your team is ready for something like agent ready content or customer facing AI or just excited by it?

[Speaker 1]
Yeah, okay. So I guess two things that I would look for at least are one, like we talked about experimenting our way into it. If we have a hypothesis about what our content needs to do to be consumed by agents or, yeah, to roll AI in front of customers, what's the minimum viable product that we can experiment with to see whether this is actually ready for prime time?

And don't stop there, right? Like if you're gonna propose that, have a plan for what you're going to do afterwards. But I think the second question is as you start experimenting, inevitably what's gonna happen is you are going to run into roadblocks.

Maybe the agent is going to start hallucinating. Maybe you run into issues with real time data pipelines. Maybe the agent gets lazy as we know they do sometimes and comes back with, oh, I looked at the first three bullet points that you sent me and didn't look at the others, why?

So start building those in as you iterate, but also look organizationally at where might we need to put in additional budget or additional resources or actually have like an AI systems architect attached to these projects to start unblocking things after we get through some of the quick wins. I think those are the things that I would look at.

[Speaker 2]
Maybe just add, well, just tack on to what you just shared. This is why, so I was a consultant as well, doing some advisory work just like Adam prior to joining Gainsight. And one of the things that I always focused on with my clients was their operating model and their internal processes and how they operated as a team.

Because I think if you don't have the right operating model and for clarity's sake, I'm a huge fan of the product operating model and how product teams work because of the experimentation that you mentioned, because of the way that they approach, I guess like formulating a hypothesis and then validating and solution prototyping before you go all in on something to make sure that whatever it is that you're building as a solution actually makes a measurable difference to the outcome that you are trying to drive, right? Everything being focused on a Northstar metric. What is it that we're trying to accomplish for the business?

If you don't have the right operating model in place, then any efforts that you make to transform and AI-ify your team, there's that word again, I said it earlier, I'm making it up. They will fall flat, I think, because you need to be able to, to your point, like have the feedback loop to say, is this doing the thing that I expected it to do? Was there a hallucination that suddenly came up that I need to catch?

Is there an error somewhere along the way that I need to address? This was evidenced in the program that our incredible digital customer success team built for our own customers with SkillJar. So it all comes down to operating model for me.

It's so critical to have the foundations correct before you start layering things in because it'll be too, by the time you notice that there's a problem with whatever you've implemented, it'll be too late.

[Speaker 1]
Yeah, that's fair.

[Speaker 2]
Okay, another question, which I'll cut in half because what's your favorite type of potato is, looking at you.

[Speaker 1]
In joke? In joke? Yeah, that's true.

I didn't expect to answer that. I live in the Netherlands now, so I'm going to go with the Roosevelt potato.

[Speaker 2]
Oh boy, say that again?

[Speaker 1]
Roosevelt.

[Speaker 2]
Roosevelt. I don't know, I'm not pronouncing that correctly, but okay. Over a beverage.

[Speaker 1]
I don't even know if I'm saying it correctly.

[Speaker 2]
Do you have any great examples of knowledge bases is using AI to create more tailored content for customers?

[Speaker 1]
I think here we have to stretch our definition of knowledge base because I'm actually not seeing a lot of, let's say, help center style knowledge base that is doing true on the fly creation. There are certainly some third party solutions that allow you to query what's in a knowledge base and from that, kind of ingest somewhat tailored answers. So, for example, one of the clients that I work with right now, they are using Mintlify as their help center and it's not the articles itself, but rather it's the search within the help center where you can ask a question and it will draw from the help center to create a fairly customized answer.

That, I'm seeing, works pretty well. Actually, again, as long as the content is well structured and accessible. I'm also seeing people kind of do this outside of traditional help centers.

For example, maybe this is like a pseudo-competitor alert, a frenemy alert, like Pendo, I think, is doing a good job of bringing their knowledge contextually in app with basically a side blade where the content is being populated, again, contextually based on what area of the product that you're in. But I'm not seeing a lot of companies doing, let's say, first-party, on-the-fly, customized help center content generation. I think probably because of token costs and the amount of manual work that it would take to validate it.

[Speaker 2]
Yeah, let's talk about token costs for a second. I just wanna go there because you and I were talking about this earlier. Like, I think there's this, again, misconception that, well, if we just, if can't we just do this, let's just do it, but the costs are so exorbitant and I don't think a lot of companies are thinking about token costs in the context of, like, if, right, but the teams are not.

[Speaker 1]
Yeah, true.

[Speaker 2]
So how do you think about this blend of, like, I wanna identify as much as possible, but while remaining somewhat contained in terms of spend because otherwise, like, the table, I guess, like, it'll just get unruly at some point.

[Speaker 1]
I don't know. Last time we talked about why can't we just, someone got up and exited the room, so I don't know. No, I think, like, one thing that I've seen, at least, in terms of this is basically tiering to what extent you want AI involved, and so basically having a pre-evaluation layer versus, let's say, let's completely AI-ify everything.

So, for example, if you are thinking about self-healing content, one thing that I've seen people do is they basically look at the release notes and they use AI to do an evaluation first of whether we should do a full self-healing loop on it or not, like, should we even think about this, and that is not necessarily related to the size of the release itself from a marketing perspective because the things that your product marketing team might get excited about are not necessarily the things that are going to have the biggest user impact, right? Product marketing gets really excited when you do a whole new product release, but in terms of support tickets or in terms of customer sentiment, a smaller change that you make that might go across multiple product areas could end up having a way bigger impact on customer sentiment, so being able to scan for some of those things and first tell you, like, where might we be seeing issues that have wide-reaching impact across a lot of our help center, a lot of our documentation, a lot of our courses, and saying, oh, there, those are the areas where we actually might want to make an investment in having an agent run against that, so basically, yeah, as much as possible, not just saying, like, everything goes through AI, but pre-sizing it.

[Speaker 2]
Just like any other vendor relationship that you're evaluating and you're trying to quantify ROI, like, at the end of the day, it's like, maximize ROI, feels like.

[Speaker 1]
Yeah, basically.

[Speaker 2]
Okay, I don't want to stand between everyone and the happy hour because we're already seeing the numbers dwindle in this room, so.

[Speaker 1]
We're not taking it personally, but we thank you all for staying to the end.

[Speaker 2]
Thank you so much for staying until the end. Look, Adam and I are going to be at said happy hour, so if you want to chat more about any of this, please come find us, come have a drink with us. We are happy to dig into this some more.

We are both, obviously, also on LinkedIn, and so if you have direct questions for either of us, you can reach out to us directly, and we'd be happy to chat. Thank you so much for staying, and thank you so much for joining us. Thank you.