Dashboards Don’t Drive Decisions: Combining CS Data and AI Agents to Surface Retention Signals That Do

[Speaker 1]
So good afternoon, everyone. We know that we're standing in between you and lunch. So what we promise is no fluff, maximum signals, and practical takeaways, or sorry, a practical framework that you can take back to your teams.

This is a roadmap for the next 40 minutes or so and how within BMC Helix we are using AI to filter the signals to drive decisions, combine both structured and unstructured signals to provide the leadership with better customer intelligence, how we can leverage Gainsight MCP to enable our leadership to self-serve and reduce the customer success practitioners' efforts, and finally, look at the design principles that we used.

But before we get into the mechanics of it all, let's just do some quick introductions onto who we are and why we're here. Adriana, would you like to introduce yourself?

[Speaker 2]
Yeah, so hi, everybody. I'm Adriana Diaz-Festula. I am the Digital Customer Success Operations Manager.

I've been doing this work for about seven years. And before that, like many, many people, I believe Zoe Marshall is one of them, I was a teacher. So seven years of that, and now I'm seven years being in the CS space, definitely a little bit different, not where I thought I'd end up.

But in all that time, I started off as a CSM and then had just a wonderful opportunity to start in ops. And I haven't stopped doing that. And I've been in love with the digital scalable aspect of CS ever since.

And I like to think I do a pretty good job of it, like many of you. I work on campaigns, automations, just anything and everything at scale. I don't love, I didn't love being a CSM.

I did not love that one to one. So I'm really, really happy where I am at right now. I am from the US, from Florida, Tampa, Florida.

That's really, that's like an hour from Disney, just for reference. But I am living in Stuttgart, Germany right now. So if there's any Germans out there today, please get in touch.

I need some friends. And so I do have a fun fact, because we all love those. I am a passionate cook.

I cook a lot. And a lot of my cooking includes onions. So rather than suffer, I do wear onion goggles.

They're specific. They're not just science goggles. They're like sealed around.

And if you are like me and you cry with onions all the time, I couldn't recommend a better solution. Ian, tell me about yourself.

[Speaker 1]
And she really rocks it on Teams calls as well.

[Speaker 2]
Yeah.

[Speaker 1]
So I'm Ian Taylor. I'm a customer success director within BMC Helix. Yeah, I've been in the industry for a long time now, covering multiple different areas.

The last eight or nine years has been, I guess, with customer success of some shape or form. What I'm currently doing is not only working with our customers as a customer success manager, but one of my other remits is to actually drive the AI adoption within the customer success management team globally. And this is really why I'm talking about this.

So I'm based in the UK in the Midlands, which is actually quite ironic considering how much I love the water. I still manage to find time, even though it's in the Midlands, to open water swim all year round and go sailing twice a week. So for those of you that don't know about BMC Helix, we help organizations transform digital service management and operations with the use of automation and AI.

So to understand really where we wanted to get to and why we had to rethink our approach, we needed to have a real honest look at where we were starting from. And I'm sure the challenges that we faced in Helix aren't unique. I'd like to think that some, if not all of you, face the same challenge, some of these challenges at least.

So that is having that unstructured data, manually having to search through mail, through Teams, through your transcripts, and SharePoint for every customer that you're managing every week with no single place to look. And on top of that, some of the key areas like your ARR, your renewal dates, your license subscriptions, and also your renewal dates. So all of that was referenced in Gainsight for us.

But again, it was having to be manually re-referenced with every report. There's also no real link back to any prior updates. So open risks could either be rediscovered, restated, or quietly just drop in between reports.

And finally, we had the issue of timeline entries. So there's that swivel chair. Once we consolidated all that information, we then had to manually put it into Gainsight.

And there was no, and it was subjective to each different CSM and practitioner. So there was no potentially, no real consistent link back to the signals that actually produced them.

[Speaker 2]
Oh, OK.

[Speaker 1]
So, yeah.

[Speaker 2]
All right, so I think there was supposed to be a Slido here, but that's OK. Would it be OK if I just have you guys, oh, there we go.

[Speaker 1]
Yeah, here we go.

[Speaker 2]
Oh, lovely, perfect. So log in to your Pulse app and go ahead and ask this question. What signals, what customer signals does your executive team most often ask for or act upon?

Mm-hmm, yeah, like to renew.

[Speaker 1]
So hopefully, you've been able to find it. It's on talk track one and the poll.

[Speaker 2]
Risks, OK. Does anybody look at sentiment? That's one.

It's a bit controversial now. Churn rate, uh-huh. But what signals are prompting that risk, I guess, is my question.

Adoption, yeah.

[Speaker 1]
There we go. They're coming in thick and fast now.

[Speaker 2]
Yeah, now. We have to give them a second. Yeah, client feedback.

OK. So I mean, the list, it really goes on and on. And honestly, our customers are giving us hundreds of signals every day as they're talking to our CS professionals, as they're talking to the rest of the environment, and as they are using our products, right?

Not all of those signals are valuable. And this is a lot. This is a lot of information.

And they're not all going to tell us what we really want to know, which, as the first couple of entries pointed out, was risk and renew and likelihood of renewal. So one of the first things we had to do when we wanted to create this agent that was able to give our leadership some actionable next steps is we had to identify what are the signals that actually drive renewal, and what are the signals that our executive leadership even cares about? And so we were able to do that.

[Speaker 1]
Oh, yeah? Yeah, can we just go back onto the?

[Speaker 2]
Oh, I see. OK, cool. And we were able to do that.

And when we did that, we distilled them into these signals. So Ian, can you tell us a little bit more about these signals, what they mean?

[Speaker 1]
Yeah, so what we did was we really looked at starting where the leaders had the real questions and not just the data. Basically, which accounts moved from healthy to at-risk this month or this week, and why? And importantly, what we were going to do about it.

So leaders actually act on the questions like this rather than a 40-tile dashboard.

[Speaker 2]
Yeah, yuck.

[Speaker 1]
So as Adriana said, to bring this really into structure, we, sorry, to bring the structure into noise, we broke it down into four different categories. And that actually showed really well on the responses on the Slido. It broke it down into health, adoption, risk, and growth.

[Speaker 2]
Yeah, so OK, lots of signals. So you know that we just narrowed it down to the ones that our executives were talking the most about and asking most questions about. But not every signal is a good signal.

Not every signal is a credible signal. Customers are weird. They do weird things.

They have holidays. They have their own lives, right? So here's a couple of things that we learned that make a good signal that you want to be able to consume into any AI agent or any executive presentation, really.

First one, for either signal for health, adoption, risk, or growth, you want to make sure it's tied to some kind of KPI. That is what's going to make it actionable. You also want to make sure that it is product-specific.

If you are like us, we have a huge portfolio of different products. And we even have things like versioning, right? All of that information, that structured data, is very valuable to the executive team.

And it's going to change those decisions that they need to make. Now, those two things together, those two requirements, make it actionable. And at the end of the day, if what you're presenting to your leadership is not actionable, then it's kind of useless, all right?

What you're wanting to do is create minimal back and forth between the leadership and your CS professionals. You have a little bit about that. Go ahead and remind me, what was the story?

[Speaker 1]
Yeah, so we were getting our requirements are quite dynamic. So we were getting leaders just coming back to the, reverting back to the CSMs all the time, asking for more context. And it became a bit of a status line.

You know, it really, really, instead of just having that one touch, leaders getting the right information at the right time, it was always the leadership coming back, back and forth, back and forth. And that really, really drived struggle. It took a lot of time, both from the leadership and from the CSMs, from taking actually, you know, having that time with our customers.

[Speaker 2]
Yeah, definitely. I mean, imagine you're a CSM. Like, you know what your goal is.

You know what you're trying to do. So that back and forth was really just a huge time suck. And I think mentally, it was like an emotional suck there.

All right. So remember, we created this agent to be specific to our executives. That was our ideal audience.

I wanted to share this snippet of a context file that we were able to provide the agent, because I wanted to show you how specific you can really be. So with any agent, I'm sure you're familiar, you can give it context files, which means that's going to help it understand how you want it to think. What we did was actually quite clever, if I do say so myself.

We took recordings that our leadership had had, high-level leadership meetings, and we put all of those into an agent. And we asked the agent, hey, what are the themes that my executive keeps coming up with? What are the things that he or she keeps harping on or keeps asking about?

And it was able to go through several meetings, plus, of course, we just asked, of course, right? And give us an idea of what it was that they were really asking for. So instead of having to suppose, well, on Tuesday, he asked for this, and on Thursday, he asked for this, we just put it all in there.

And with that, we were able to give it a context file. So again, this is just a quick snippet showing you what our leaders really wanted to pay attention to. And it's something you can do immediately, even if you don't build an agent exactly like ours.

This is just a really, really cool shortcut.

[Speaker 1]
And I think the important thing to say on that as well, Adriana, is it's not a static file. It's constantly getting updated.

[Speaker 2]
That's right.

[Speaker 1]
Yeah? And what we do is we use that as a knowledge source for the agent. So it is constantly, instead of having to update the instructions in any way, it's just using that as a knowledge source so you can continuously get the updates.

[Speaker 2]
All right. So remember, the name of the game here with any agent that you're going to build and anything that you're presenting to leadership, really, is that it's actionable. So here's kind of a high-level, 1,000-yard overview of what our agent does.

It took our unstructured data, like emails, chats, meetings, with our structured data, which is, again, mostly in Gainsight, so ARR, versioning products, and some usage, mix those together, put it through our context files, and then the output was a very specific formatted way of understanding that information. So our format was operational, tactical, strategic, as well as just a very high-level executive summary at the top. Now, we have all this information.

Where did we put it? It did not stay on the agent. It actually went to two places.

It went into our weekly timeline. So this is a weekly activity. And again, weekly sounds like a lot, but it's not because an agent's doing it, right?

So it went into our weekly timeline, and we actually have a backup file. And this is all totally automatic. We're not having the CSMs create the backup file.

We have the agent just doing that automatically. And of course, the posting to timeline is courtesy of the Gainsight MCP. All of this to say, we really got back a lot of time for our CSMs, our CS professionals, and for our leadership.

They could take all of this data, and it was in their language that they cared about. And they're not having to clarify and go back and forth and all of that. It was like translation.

Yeah. All right, so Ian, I want you to go ahead. He's going to talk to you about a little bit more of the details and the nitty gritty about this agent.

[Speaker 1]
Yeah, so the agent's job was simple, but it was rigorous. Over the next few slides, what I'll do is state how the agent validated the systems, consolidated all of the signals across all of those touch points, synthesized that firehose into those four pillars that we spoke about, and outputted it into a consistent audit-ready briefing across all managed accounts, which provided the SLT needs. So validating the systems, Adriana's already mentioned this.

What I did was I built the agent to go out and validate the customer name into Gainsight, making sure that we've got the right one. If there were multiple candidates, we had that human in the loop. So the agent would stop, ask the CSM or whoever was running the agent to validate which account we were talking about within Gainsight.

Once that was done, it would surface the key metrics or the key data, so things like we've mentioned before, that account name, the ARR, the license information. If there wasn't, and we had a bit of resiliency into the system as well. So there was always going to be a backup.

And that was in the form of a SharePoint site for every one of our customers. So if it was a new customer, or if that folder structure didn't exist, it would ask the CSM whether it wanted us to create that folder structure. All the time through, just going through, making sure that everything is there and everything is in place, but putting the agent being there, making sure that the human validated everything.

Adriana also mentioned about how we provided the three different levels of insights. And that was against the operational, the tactical, oh, sorry, the operational, tactical, and strategic. So when we're talking about operational, we're talking about what's happening at the moment, what the immediate impact is, whether there's any escalations or whether there's any changes.

When we're talking around the tactical, it's looking about the progress against the projects. It's looking at any of the initiatives, including any blockers and next steps. And then finally, on the strategic, really, really aligning to those business outcomes, making sure that future requirements and business cases are also included.

Why did we do this? It provided that combination of activities within commercial, product, renewal, and the entitlement context. And it connected that day-to-day activity with the business impact.

This allowed us to move the CSMs from a reporting activity to explain exactly what matters, why it matters, and what actions are needed. So that structured insight can then flow into Gainsight and those executive dashboards that we spoke about. And this was the real thing that made things a lot, lot easier for us.

It was that integration into Gainsight utilizing the MCP. What the MCP does is it's the connective tissue that lets the AI agent read and write back to Gainsight directly. So instead of an analyst exporting a CSV for the agent to chew on, it allows the MCP to just provide that more natural language conversational piece.

And I'll explain that a little bit further on. What it also allowed us to do was, instead of having to write multiple different APIs, we just had one connection. And just that configuration alone saved a huge amount of time in authentication, that configuration, and the maintenance, just in case those APIs had to be changed.

It's what it's allowed us to do is it's allowed us to read it. Instead of it's moved us from reading a dashboard export and summarizing towards querying Gainsight live and being able to write back to the timelines. And it's closing that loop between Insight and the system of record.

So let's talk about the Gainsight agent itself and what the MCP can actually provide for you. I've used it as a sub-agent to that main weekly update agent that we've got. So it allows us to have that conversation, that native conversational sort of exchange that you can have.

It provides that account intelligence through simple conversation. It asks the information without having to navigate either multiple screens or reports. But importantly, it gives us that broader context of within Gainsight as well, whether it is accessing health scores, getting those timelines entries, looking at the CTAs, the success plans, the key contacts, and the relationships that we've got within Gainsight as well.

We've injected into Gainsight some of our really important customer or key indicators as well. So we've got those product versions, the ARR. So we're able to extract all of that information.

And what I did was, what I needed for the agent was to actually have a stretched output. So I built just that JSON output to extract the account name, the display name, and all of the really important key metrics that we were looking for. So why did that matter?

It really helped that reduction of having to search through different layers of Gainsight to pull the information that we required. And it allowed us to get that picture very, very quickly. It's given our CSMs the ability to get access to that trusty context that we needed for the preparation, update, and decision making for the senior leadership.

So Gainsight has provided that structured data, which we can then combine with that unstructured data from our Microsoft ecosystem. The final destination, I'm sorry. Obviously, I've had to redact this quite a bit.

But essentially, what it does is it's allowed us to have that consistent structure for our senior leadership. It's in exactly the same format for every CSM. So there's no subjectivity to it.

We've got the account overview, the executive summary, the retention and adoption plans, the key risks, and importantly, the next steps going forward for that. It's a single source of record as well. So the latest account narrative is captured wherever the wider audience, or sorry, the wider team is already working.

And it leads to less administration. So we don't have to do that ourselves. The agent is automatically writing those timeline entries into Gainsight itself.

And what I really like it, and I know it's a really small thing, but what I like about it is whenever the MCP agent is writing anything to a timeline entry, it's tagging itself. So you know exactly where it's coming from. So importantly, the insights aren't just generated.

It's embedded now into our operational cadence and workflow. And it's that repeatable process that we've now been able to do to turn that account activity into that visible, actionable intelligence. Reliable, so that was basically what we did and what the agent's doing.

This was really the design principle about how we actually started to build the agent. But reliable agents need clear principles. They can't just work on good technology.

I'm sure you've heard this plenty of times at the moment. Make sure humans in the loop. Get your practitioners and your leadership trusting what the agent is going to produce.

Think about the structure and the signal over duplication. So it's consolidating all of those relevant emails, meetings, updates into one clear narrative. Ensure that the agent is accountable and providing that human quality insight.

So be specific with your names, your dates, and the numbers and outcomes, not just vague statements. And ensure that the context insights align to those KPIs that we were talking about earlier as well. So every statement should be linked to a measurable business indicator with a clear next step.

These principles make the agent output easier to trust, validate, and act upon. And this really helps with providing useful intelligence with human accountability, clear evidence, and measurable business relevance. So you've heard how we filtered down the noise, consolidated the structured and unstructured signals, leveraged the Gainsight MCP, and looked at the design principles that we've used.

So where are we now? So the agent has allowed us to resolve the accounts through the Gainsight sub-agent, confirm that SharePoint folder, then query all of our Microsoft ecosystem, all in one valid pass, and all automatically. The structured information, those key indicators that we needed, like the ARR, the renewal dates, the diversions, are populated within the account information.

Headings, and every other section must be traced or is traced to those signals that we're aligned to. All open risks are still tracked, and any missing account fields are, they're backfilled from previous reports. And if there's nothing there, oh sorry, they're backfilled from previous reports prior to anything that's been tagged as missing whenever we're validating against that context file.

And importantly, those outputs are automatically being generated with the timeline entry only being, oh sorry, with both of the outputs actually only being entered once the CSM or the practitioner is completely happy with what the output is showing. So the timeline entry is gonna be exactly what the CSM wants the leadership to see. So let's look at the reality then.

What did it actually take? So moving from a proof of concept to production actually takes quite a bit of planning. Initially, oh sorry, just think about cost as well.

So although Gainsight doesn't charge for anything extra for the consumption of the MCP or the use of the MCP, we still have to use those tokens and those token credits. So think about every retrieval, every analysis, every update, they all consume tokens. So usage patterns and models are really important.

Think about the agent models you're using as well. We're not going into real deep research here, so use things like Sonnet instead of Opus or Cloud Sonnet instead of Opus. It's really gonna save you quite a few tokens.

But importantly, think about measuring cost over the time. If a CSM, and I know I've done this, spends hours running these reports, gathering their account intelligence each week, automation quickly offsets any of the consumption costs from using Gainsight. The other thing we wanna think about is engage with your internal IT teams early.

Make sure that you can get approval to actually use the systems. It took us probably about a month to get our internal IT team to look at and finally approve our use of Gainsight MCP. And at the moment, it's only allowing us to write using Microsoft Copilot.

We've got this running in Cloud, but we can only use the interaction with Gainsight as a read-only within Cloud, just because MCP was, at that time, our agent of choice, or sorry, our AI infrastructure of choice. Build a long-term governed architecture, but also making sure that you're proving value quickly as well. So provide that interim proof of value.

Show a snapshot of an export to demonstrate the outcomes to your leadership. That starts showing the value of it a lot quicker. Look at starting small as well.

Provide that minimum viable product. Why does that make it important? It's so the leadership doesn't have to wait to see for the full architecture to actually show the value that's actually being seen.

So the bottom line is plan for time, access, and credits, but keep the momentum going with an early proof of value.

[Speaker 2]
Yeah. Yeah, guys, so thank you so much, Ian, for going through that. I know it was a lot of detail to go through, but I mean, that is just how much work it took for us to make an agent that was worthwhile and really made leadership happy.

And of course, we're still updating it and learning and iterating. And one thing you didn't mention, you did say have a good relationship with IT, but I would say have a good relationship with your ops as well.

[Speaker 1]
Absolutely.

[Speaker 2]
Okay. So what is the payoff of all of this work? Really easy.

You get on-demand answers. You get fewer interruptions for both leadership and the CS ecosystem. And you get flexible insights.

So of course we have this template, right? But you can always just ask different questions and it's gonna give you that information as well because it's looking at all of this data that you have from your two main sources, structured and unstructured data, right? And again, it makes CSMs validators.

They don't have to be superstar writers, okay? All they have to do is say, mm-hmm, mm-hmm, mm-hmm. Yeah, that's true.

All right, accurate. Moving on. Bottom line is this, the leaders are gonna get faster, more detailed, more actionable customer intelligence.

And the CSs get everything put in and recorded in that timeline. It's always referenceable. And one thing, Ian, that you pointed out that I kind of forgot was it's personality agnostic.

It's not like, ah, Juan does such a good job, but, you know, George over there never gives us detail. It is exactly as detailed as our leadership and our company needs it to be, which I just love. All right, so what the future holds.

Like I said, we are not just one and done. But I do have to come clean about something that I, as the digital CS ops, very sad about. But this is not an agent that is for scale because it is depending on all of that unstructured data.

And our scale customers don't tend to have a lot of those email back and forths. We're not necessarily having meetings with them. So a lot of the data that we have with them are mostly support tickets or very early on in their journey in onboarding where they do have a bit more handholding.

So the future, I would love for it to be able to be used for our digital and scaled accounts. We'd love for this to just be one of many agents in the future. So this is really great.

This is a very specific use case. For us, it's that weekly update, right? It was very time consuming, but very important for our leadership and for our business.

We'd love to have other agents under it that use that same well of data and information, but from a different perspective or have a different output in mind. And of course, we'd love to 100% skip the dashboard, but it's not possible right now. But it does, it helps.

It helps, it makes it so our leadership isn't looking at the 40 different tiles. They're just looking at like five, maybe, right? And of course, it's just easier for our leadership to just ask it a question.

I mean, it couldn't be easier, couldn't be easier. All right, so we did promise that you were gonna have some takeaways and things that you could do yourself. So this is what I would like you to do Wednesday if you're going back Wednesday.

I'm not, I'm still vacationing. When you start off, you want to design the outcome or the output first. So in our example, that was that template and the format in which we wanted an answer to be given to us.

Second thing you're gonna do on Wednesday, so right before lunch, you're gonna inventory all of your signals and inventory all of your data sources. You have a ton. They might not all be accessible, but see what you have, see what signals you're actually getting.

So for example, if you're not getting usage, you're not getting usage, move on. Inventory, look at the other stuff that you're getting. Of course, the easiest stuff is emails, transcripts, all of that.

And of course, we've said this a few times already, but keep it small, keep it human. Make sure that there's always a human in the loop who is just the validator. Never trust AI to just do it right the first time.

And that MCP, making sure it's well-governed and you have that structure, which I know a lot of y'all are struggling with, is kind of that balance between governance and then letting people run with it. It's really important that that is there. All right, that was it.

Guys, thank you so much. We have a few minutes for questions, if you have any.

[Speaker 3]
Yeah, thanks, guys. That was an awesome session. Let's jump straight into some questions here.

So first up, in the digital CS journey, how do you maintain a balance between the human intervention and the agent while handling escalations?

[Speaker 2]
How do you retain the balance between human intervention and the agent while handling escalations? I mean, you need a human to intervene ultimately, right? But what the agent does for us is it allows the human who is going to intervene to get a full picture and not just whatever this small escalation is.

They're going to be able to quickly see the whole of that relationship in the last, let's say 90 days or so, I believe you said it was, to handle that. So we really don't need humans to analyze these emails too deeply anymore. I mean, if the email is like very short and curt and uses words that we know signal risk, the agent's going to pick that up.

What I'd like the human to do with their time is to deescalate it and put things into motion and put a strategy so that the situation calms down and you're back in the green.

[Speaker 1]
It's actually identifying those risks early. So it's identifying the risk so that the practitioner, our CSMs, can actually get in front of that. So hopefully we're identifying the risk before they get escalated.

[Speaker 3]
Great. Next up, have you had an account that bypassed your risk signals and churned? And how did you respond?

What lessons did you learn? And did you use those findings to evolve your risk signal model? Or do you accept that there will always be edge cases?

[Speaker 2]
I'll let you answer that.

[Speaker 1]
Yeah, realistically, there is always going to be edge cases. This is why we get human in the loop as well though. We make sure that the agent is giving you 90% of the data that you need, right?

There's always, and I've still seen it, people get rushed. Look at, and that's why we've got areas within the agent to say, this is missing from the context file. Always have someone in the loop to make sure that what the agent is providing is what you're gonna put out because they know the account better than any agent will do.

They still have those customer engagements. So that's, yeah. I mean, effectively, that's why we made sure that there was always that touch point with the customer just so we don't necessarily miss any of those signals.

[Speaker 3]
Absolutely. Probably got one or two more in here. What volumes of accounts do you have roughly?

Just trying to get an understanding of scale here.

[Speaker 2]
About, no.

[Speaker 1]
So for our, I guess for our- The covered accounts. Our covered accounts, it's primarily about eight, but then we've got a scale customer. We do have some scale that they'll be reaching out to maybe 20 customers or something like that.

So, but in reality, whenever we were doing this, our CSMs were maybe taking two or three hours per account. If you think about that on a Thursday or Friday evening to try and get all of that information consolidated, trying to remember where you were, that's an awful lot of time. So it's gone from hours to minutes, and that's been able to provide us a huge amount of value by just using this agent.

[Speaker 3]
Great. I think that's all we've got time for, guys. And yeah, thanks again to Ian and Adrianna.

Let's give them one last round of applause. Thank you.