Many companies aim to make data-driven decisions, but it’s challenging when insights are buried across multiple feedback channels and data sources. This session will explore how Learnship solved this by leveraging Gainsight CS, its survey tools, and AI to connect user feedback with behavioral data and uncover insights that drive key decisions and boost customer satisfaction.
0:00
Welcome at the second day of the post. Thank you very much.
0:07
First of all for coming here today at the second day of post and also to this
0:13
closing session of the day. I know it's been a rough maybe two last days and I
0:19
'm
0:19
super, super happy that you're fine at the moment also to come here and be here
0:24
and relax it's gonna be a really really nice session. Before I'm gonna
0:30
introduce
0:30
our wonderful speakers I want to call out that there's gonna be a Q&A session
0:36
towards the end of this presentation so make sure if you have any questions to
0:42
go to the pills app and go to the poll or end the Q&A section and put them
0:48
there and there and you can also vote there for questions that you think
0:53
that should really address to the speakers. So without further ado I want
1:00
to introduce you to Angela and Amy. Welcome to the stage.
1:06
Angela is a global customer success from Leanship and Amy is a product owner of
1:15
Gaze. I've got some success team. Hi everyone. How many of you were at the
1:22
party last night?
1:25
So how many of you remember the band saying come closer? Because you'll miss it
1:31
So come closer. We've got something to share and back there you're not gonna
1:36
see it.
1:36
So let's get started. Welcome to the last call bar. That's where we are. So the
1:44
final
1:44
call, the last call. I'm Angela Felicis Simo and the Vice President of Global
1:50
Customer Success at Learn Ships. Some of you might have heard me this morning
1:54
and
1:55
this is my colleague Amy Lane who's our product owner for Gaze site but she's
2:00
also a CSM team lead for the Americas as well. And we're here to talk about
2:06
data
2:07
alchemy. Basically how have we been able to take data and turn it into
2:14
something valuable? And I have to say I'm not sure what's worse to have the
2:20
after lunch slot or the slot that keeps you from heading home at the end of the
2:25
day. But let's continue. So Amy maybe you want to go first. Sure. So I'm Amy
2:34
Lane
2:34
and I've been in the tech industry for EdTech industry for 25 years. I joined
2:40
when I was four. So if you're doing the age math, use four. I as Angela
2:47
explained
2:48
currently I own the product for Gaze site within our company. I get to decide
2:54
which projects get worked on which is the highest value for our you know for
3:00
different teams to be working on. I also get to lead a CSM team which gives me
3:04
access to be able to see both sides of the implementation world and then the
3:09
application world which is fantastic. I also have an additional job on the CEO
3:14
of three teenage girls and two spicy cats. So when they say travel I say yes
3:21
please. I also really love finding anything creative but creative solutions
3:27
are you know my favorite thing to find and then be able to turn that into a
3:31
great story. Hopefully at the end I'll tell you my very favorite story if we
3:35
get
3:36
our timing and our pacing right. Back to you Angela. Yeah so as I said as I
3:41
said before I had a customer success and I've been in the EdTech industry for longer
3:46
than I care to remember and longer than I care to admit. So don't try doing any
3:52
maths there. I'm passionate about customer success and I've been in customer
3:59
success for around 20 years before it was even cold customer success and I got
4:05
into customer success because I was a customer before I joined. So that gave
4:11
me some unique insights as well and what I'm passionate about is showing how
4:17
our
4:18
solutions deliver business value and have real business impact on our customers
4:23
and to give you a little bit of background about learn ship. So we provide
4:29
language, business skills and intercultural training for over 2,000
4:33
corporate clients. We have around 250,000 learners around the globe so it gives
4:40
us two levels of working in our company. We have customer success that own the
4:46
customers and we have learning specialists that own the learners so
4:51
that's given us some unique challenges and we'll come on to some of those but
4:56
first of all let's talk to you about alchemy. So the dictionary definition of
5:03
alchemy is a seemingly magical process of transformation, creation or
5:09
combination in particular with attempts to convert base metals into gold or to
5:14
find a universal elixir. Now I'm not going to be able to convert base metals
5:21
into gold but I am going to be able to show a little bit of magic so it's not
5:27
engine oil it's gin, just a little tipple here and there and apologies if
5:36
you're at the back I did warn you that you might not be able to see but just
5:42
with a little bit of magic I can turn this from blue to pink which kind of
5:51
demonstrates what alchemy is. Just a little bit of magic you know we said it
6:00
was going to be a fun shot and we'll come back to those later. So you know
6:10
who we are you know what we're here to talk about right alchemy and as I said
6:15
I'm not going to make gold out of lead I'm not going to make water wine from
6:21
water but what we are going to talk about is how have we made vast quantities
6:26
of
6:26
data as you can see meaningful and useful. So let me start with a little bit
6:33
of a background about who learnship is and the challenges that we've faced so
6:38
it started with two companies. Global English came from a SaaS model digital
6:44
self-paced learning very much a licensed space model easy to know the start and
6:50
end of things and then we were acquired by learnship we were found which was in
6:58
2019. They were much more in the traditional way of delivering learning
7:06
it was virtual but it was train a lead face-to-face but online. More
7:12
important to what we're talking about today we had very different data profiles
7:17
two very different databases two very different ways of how we gather data and
7:23
and this has been a big challenge for us that we've been able to solve and Amy
7:29
will tell you a little bit about that later with with gamesite. So we couldn't
7:35
marry this data together and data is key we need it to inform our customers we
7:43
need it to be able to demonstrate value but we also need it for our company as
7:47
well to be able to make decisions. So just a little bit of a story and it was
7:53
you know something that stayed with me for a long time so before I joined the
7:59
edtech industry I worked for a logistics company and I was responsible for
8:05
training and documentation of our systems and I used to every month gather
8:12
all the data of how many courses we'd run how many people attended how many
8:17
hours we ran and I would proudly go to my boss and say look at these numbers
8:22
they're magnificent and they would say oh yeah that's great then I got a new
8:27
boss
8:27
and that's where my light bulb moment came on because the new boss said to me
8:33
those numbers are great but what impact are they having on our business is it
8:40
making our business any better than it was before otherwise we're just wasting
8:45
time. So let's look at some of the challenges that we face in a bit more
8:52
detail and some of that's how we gathered data from learners and feedback on
8:58
our
8:58
solutions so we were running before game site around 750 surveys a year
9:07
these were all set up manually they would take around one to one and a half
9:14
hours being generous to set them up to evaluate them but we couldn't
9:22
calculate data across all the surveys we could only run them at a fixed point
9:30
in
9:30
time and that was not necessarily the best time in a learner journey to be
9:35
able to gather that data it could be that someone had just completed a course
9:40
or someone had just started but we had a fixed point in time we ran them in 10
9:46
languages brought its own challenges because we had a high cost of translation
9:52
translating all the responses we had multiple tools somewhere proprietary
9:59
embedded within our platform we use survey monkey we used email we have
10:05
things coming in one-to-one it was a challenge and we couldn't measure MPS we
10:14
just were not able to do that MPS on a customer by customer base doesn't
10:20
necessarily mean a lot unless you're getting a lot of responses we had
10:25
custom information at customer level but we couldn't aggregate that across our
10:30
customer base it was very useful for demonstrating outcomes to customers but
10:36
not useful for demonstrating outcomes across our business and being able to
10:42
use that data so we started implementing gain site in at the end of 2021 and we
10:50
actually started using gain site in anger in 2022 around halfway through the
10:56
year and we started with a slow pace just getting people used to using it and
11:04
then we thought well what can we do differently so we had a challenge everyone
11:11
recognized where this comes from yeah mission impossible and so our mission
11:19
should we choose to accept that was to find a way to measure NPS and business
11:25
impact across our customer base at all levels that would be meaningful not just
11:31
to our customers but to every part of our business and that's kind of one of
11:38
the key points there and that's something that I'm gonna hand over to Amy to
11:44
tell
11:44
you more about right thank you Angela and I can say mission accepted now but
11:50
that mission when I first got it was quite daunting and so we're gonna talk
11:54
to you a little bit about how we kind of overcame the hurdles what was the
11:58
plan and then really kind of deliver on our on our title of delivering some
12:02
nuggets of gold to you so first order of business when you've been tasked with
12:07
mission impossible is to build your dream team and the most important
12:12
questions when building our dream team you can see here we needed to know who
12:15
was already using our feedback options then we asked who could use those
12:20
feedback
12:20
options if we looked at it differently gathered it differently was able to
12:24
present it differently and then third who would we need technically to make
12:28
this dream happen when we looked at the answers to those questions we actually
12:34
build quite a big team 11 experts from different departments came together from
12:39
nine different I'm sorry 11 experts from nine different departments across six
12:44
countries and four time zones so having a team of experts come together was was
12:50
what was required but it made it first of all pretty challenging because we had
12:55
the one find a time slot where everybody could get together where nobody was
13:00
getting up in the middle of the night or the very early morning so challenging
13:05
exciting everything you know started to move in the right direction when we
13:10
built our team now I know after two days of seeing this slide you're probably
13:14
really overseeing the bouncing ball but just give me one more one more time to
13:20
use it and we'll all be done with it right so once you've got your perfect
13:24
team right and then you have to architect the perfect plan nobody goes
13:27
into any kind of high situation without their team in a plan and so this was
13:32
our
13:32
plan we use these these bots right who who who who who do we want to ask now
13:39
that we have additional capabilities using Kain site CS we decided we wanted
13:44
to go big or go home we wanted to be all learners across all of our 16
13:48
different products and hey why not challenge yourself even more and go
13:53
ahead and let's add 800 trainers and a thousand program owners and a few line
13:57
managers and a food line yep few of them too next was timing right what was
14:04
that
14:04
optimal timing that we could get to in order to increase our responses and
14:10
having gain site CS automation was key to making that part happen for us now
14:18
understand we have learners who are on different journeys who've started at
14:22
different times all within one account so the ability to not have to have that
14:27
static time set that was key to our success standardization right we knew we
14:36
had to go from customized questions customized timing down to something that
14:43
was standardized so that we could realize our big data dreams so we had to
14:47
look at you know we wanted to get an NPS question in there but did we want to
14:50
add it to our addition did we want it to add it to our already existing
14:56
business
14:56
impact survey questions that really delivered that high value for our CSM's
15:00
and to their customers and we also knew that translations was going to be key
15:05
invites questions had to be dealt with finally we all know changes really hard
15:12
really hard so how are we going to win the hearts of minds of our customer
15:16
success team so they were going to be wanting to go change the minds of
15:20
our product owners program owners sorry so you can see by our little gift here
15:27
that having access to on them having on dent on demand access to feedback was
15:34
really the win so what does that allow us to do now from a CSM perspective if
15:40
someone comes to our CSM's now and they say we have a really good opportunity
15:45
to
15:46
market to our learners who are in Japan the CSM can say great let me go get you
15:51
some quotes from learners who are from Japan in the Japanese market we have
15:56
someone who says a program owner comes in and they say we have an upset learner
16:01
then you can say well let's go see what that learner's feedback was and finally
16:06
we have a huge upsell opportunity but we have to have ROI data for it tomorrow
16:10
no problem I have a chart for that so I'm happy to report that in May of this
16:19
year we change from mission impossible to mission possible we now run three
16:24
different surveys that you can see there and we have over 21,000 response rates
16:31
and actually when I look today because that was you know last week when I was
16:35
preparing these slides we've added 9,000 more to it so every day we're being
16:40
able to capture what's where you're able to capture the feedback at the exact
16:45
right time in that learner's life cycle what is that actually equate to when it
16:50
comes to mission status in our opinion it's winning right we're winning by
16:54
removing costs associated to our surveys and all of the man hours that
16:59
Angela discussed and you know the additional payoff of growing data every
17:05
single day so our savings continues to grow on top of that automation and that
17:13
feedback being a huge win for us now we have that big that big picture data so
17:19
overall NPS scores we have results in a graph format that's easy to share we
17:25
can drill down to scores and feedback based on many many different factors we
17:29
have not had this at any point in time until we implemented CS and and we're
17:35
able to move forward with marrying our databases I think just on that one as
17:40
well I mean that has been hugely well received by by the CSMs and and the other
17:48
teams and as soon as they got that it's like yep I'm sold on this it's true I
17:52
think that that actually opened the door to other departments taking notice
17:57
of what Gainsite could do for them and and really now we almost have to fight
18:02
them off like yeah great idea next quarter we've got a whole year's
18:07
priorities planned yeah yeah for sure so what does this mean right from a CSM
18:15
perspective CSMs get to give a bigger fuller picture now so they can take that
18:20
NPS score they can take the information from our business impact survey and
18:23
they
18:23
can marry it together and tell a better story they can also come to a program
18:30
manager or program owner who is you know maybe unhappy or heard of an unhappy
18:36
learner and they can have a more rounded response to that by basing it on the
18:42
whole of the NPS for their learners so it just expands the insights we can
18:47
bring to our customers and it helps us manage those tricky spots we're also
18:54
using Gainsites text analytics which allows us to bring something to every
19:01
department because there's something in there for everyone in fact our training
19:07
department is using this these keywords to measure whether their initiatives to
19:12
keep our trainers happy in the last two quarters are actually working so we're
19:17
really looking at text analytics to be a real real driver for change not only
19:22
that our learner level but on our product level even at our trainer level
19:26
and lastly we have attempted to do this before take all of that silo data pull
19:32
it together find some type of insight and quite honestly it took forever it
19:39
was expensive when it came to man hours and the insight it was able to deliver
19:44
was just a sliver and so to say the least we've never done it again right so we
19:51
're
19:52
pretty happy to be able to do it at our fingertips but I know I promised you a
19:56
little bit of magic some alchemy in the in terms of a turning data into gold
20:02
and
20:02
so the next level we took our took our insights to nugget the gold is really
20:10
adding that catalyst to chat GPT a survey analyzer kind of pouring in our
20:15
learner behavior data and our survey feedback data into chat GPT survey
20:21
analyzer and being able to see insights that we hadn't seen before those n
20:25
uggets
20:26
of gold now we're able to see whether our engagements are actually working we
20:31
're
20:31
able to bring bigger and better insights into our conversations at the EBR QBR
20:36
level so there are a lot of different ways to turn data into gold if you know
20:44
the right formula now be honest did anybody panic when I said learner
20:48
behavior learner data it's all right you don't have to raise your hand
20:55
Angela panicked when I said we're gonna put this slide up there no you said no
21:01
right I mean a is a little tricky when it comes to to customer data and I just
21:06
want you to know we're careful about it so what what have we learned so far and
21:12
honestly we're just scratching the surface of what we're learning with this
21:16
experiment so I love that we can we can actually explore assumptions right what
21:24
were the assumptions that we came into this experiment with we actually had an
21:29
assumption that maybe NPS tracked up and down along a learner's path the
21:35
further
21:35
that they got in the path that their NPS go up or down and what we've learned
21:38
is
21:39
that there's actually a pretty low a relative relativity to that or
21:44
correlation so we're exploring in an and an assumption just right at our
21:50
fingertips we didn't have to do you know a big data gathering using man hours
21:56
to find that out I also love that we can actually ask clarifying questions so
22:01
in
22:01
this question I asked wait are you telling me that trainers are less
22:07
important than course content and organization and it comes back and it
22:11
says yes learners are happy because of the course content be more over a hat
22:18
being happy with their trainer and to be honest with you I love that I can ask
22:23
clarification questions without being worried about being judged or being too
22:29
bossy you know AI is now my my best way to go out and get the answers to the
22:35
questions that I have for some assumptions and not have to worry about
22:39
any other thing around asking it to a co-worker or anything like that so just
22:47
a couple of examples the next layer right priceless when it comes to comment
22:53
analysis I know we Angela touched on it but being able to have comment analysis
22:59
without translation in the middle is gigantic for us because of the
23:05
globalness of our audience second we see it as just a jumping off point analyze
23:12
it give us some insights and then tell us ways that we can use it what
23:16
marketing
23:17
strategies could we use now we understand these insights and then
23:21
understanding them give us an example right here you see an example that a CSM
23:27
a marketing leader even a product owner could take and use right there from
23:34
from that data from asking AI without a lot of man hours thinking that type of
23:39
thing going into it and finally when we run out of questions which is hard to
23:44
believe that you would ever run out of questions but when you do and if you do
23:47
I love being able to say as a CSM what questions should I be asking what did I
23:53
not think of because we all know the AI is you know the new rabbit hole of
23:58
information so I promised you a story about a rabbit if you'll indulge me we're
24:06
gonna switch gears for half a minute here I'm gonna tell you my very favorite
24:11
story and it's actually a true story so in the 80s when I was a kid I would go
24:16
to my aunt's beauty salon because we all know the best place to hear good
24:21
stories
24:22
is at the beauty salon so I would go and just sit and listen and a lady came in
24:28
and I came from a small town and she said to the to my aunt she said you know
24:33
how much I love my dog right and she said yes I know how much you love him
24:38
and she says you know how he's kind of a handful and we've had some issues with
24:43
the neighbor and the way that the dog tries to get over the fence and get to
24:47
the neighbor's pet rabbit and my aunt says yeah what have you done about that
24:52
and she said well actually a couple of days ago after the neighbor had
24:57
threatened
24:58
us that if something were to happen to their rabbit that he would call the
25:02
authorities and have our dog removed something happened and my aunt said oh
25:07
no what was it and she said well was late at night you were watching watching
25:12
a movie and the dog door opens and the dog comes through it and I looked out of
25:18
the corner my eye and saw that he had something in his mouth and it was pretty
25:23
dirty and pretty floppy and the size of a rabbit and so upon investigation I
25:30
hate to tell you this but the the rabbit was dead and my aunt says oh no what
25:36
did
25:36
you do she said well we hatched a plan because we really love our dog and we
25:44
did not want to lose our dog so I convinced my husband that this would be
25:48
the best case scenario they took the rabbit they washed off the dirt they
25:54
calmed it even they even fluffed it up using their blow dryer with the plan to
25:59
sneak the rabbit out to its cage in the middle of the night so it looked like
26:04
it
26:05
died of natural causes right really it planned so they follow through on it
26:11
they
26:12
get up the next morning and they hear their neighbor screaming and they have
26:16
to go out there to see what's wrong and they say what's what's wrong and the
26:21
neighbor says my rabbit it's dead and my the lady said then I had to go oh I'm
26:28
so sorry to hear that that and she says no you don't understand it died two
26:34
days
26:34
ago and we buried it in the yard and now it's back looking like it just died in
26:40
its cage and the lady had just said and my aunt said what did you say and she
26:45
said I just said oh it's that's so strange I walked back in my house to tell
26:51
my husband that our dog was not a rabbit killer just a grave robber right so
26:58
thank you for indulging me on that story and I think it's always a good
27:01
reminder to make sure you have all of the information before you hatch your
27:07
plan you go out there and you actually execute it and that's what big data
27:10
gives you right is all of the information before you move forward with the plan
27:15
the moral of the story is yes absolutely and in our chief technology officer
27:21
actually agrees just in more tech terms without the dog and the and the the
27:26
rabbits involved but we all know real-time access to big data right big picture
27:33
data is enabling teams to make better and faster decisions so in conclusion
27:41
thank you to gain site CS and to AI because we can say mission accomplished
27:48
and with that we'll say cheers yes in case you thought it wasn't real I think
28:02
I'll hold on to mine for questions just in case you guys decide to be tough on
28:06
us it's real it really is real so you can kind of imagine that it was so
28:14
excited to be here a track leader for this so thanks Angela Naby we'll go now
28:21
to the Q&A's of you there's 15 questions oh 16 questions in it so if you take
28:28
moment also to to read them and maybe also up for them so it makes it a little
28:32
bit easier also for us to check and see which one you really like to address
28:37
here so let's start with the first one and what measures are you taking to
28:43
increase survey responses and what's the motive what if a for you customer to
28:49
answer them other than goodwill it's it's it's a good question NPS surveys were
28:56
not really encouraging people if they respond they respond they will receive
29:01
the NPS survey every 90 days or so depending on what's what's happening
29:07
with the business impact survey though on the other hand that's something that
29:11
's
29:11
critical to us for our EBRs so when the CSM is doing the end of year review we
29:17
want to be able to demonstrate that what their employees have learned is having
29:23
an impact on their business so our business impact survey is not a happy
29:28
sheet we're asking for examples of how have you been able to put into practice
29:34
what you've learned how many hours are you saving per week because your
29:39
language
29:40
skills have improved and that you're better able to do that give us examples
29:44
of that and that's where the analysis of the data comes in and the text
29:50
analytics
29:51
featuring games site the AI analyzer has been hugely powerful for us to get
29:57
those messages for either a customer or for our business but on how do we get
30:03
people to answer the business impact survey well there we work with our
30:07
customer and we ask them also because we're doing this to enable them to show
30:14
their business that they've been beneficial so we're often working with
30:18
HR teams or L&D teams and we try to move them instead of being reactive but
30:26
being proactive in adding business benefit and they will also send reminders
30:31
and encourage their their learners their employees to respond to the
30:37
survey and we also asked their line managers as well on for objective
30:43
sides of things and that works really well and you know we're getting with the
30:50
NPS survey the the response rate is lower but we're still you know across
30:55
200,000 learners we're getting good responses but the business impact survey
31:01
we're typically getting 30 40% response rates on that
31:05
Wow that's awesome that really isn't really high rate and yeah to your
31:12
customers to help me to help you right this inside you need your customers for
31:17
that let's go to the next one what is the most impactful question you have in
31:22
your NPS survey well we only have in our so I said we've got two surveys in the
31:27
NPS survey I think we've got two questions three into so so we asked them the
31:34
standard NPS question translated into the various languages and then we're
31:40
asking them what is more of a customer satisfaction question is the next level
31:45
down we asked them you know how you found your learning content the learning
31:52
platform your trainer the digital part of your learning and then to give any
31:59
other
32:00
feedback more an open question which which has been really useful just so you
32:05
know our NPS is currently around 50 which is pretty good our trainer led I
32:12
think is around 62 or something so we're quite happy with the feedback we're
32:19
getting with NPS but for the business impact survey from both learners and
32:24
line managers from learners it's how have you put into practice what you have
32:30
learned so how are you using our solutions to enable your company and for
32:37
line managers we're asking what impact has it had on your business so have
32:43
things improved and putting that the two lots of data together really helps us
32:49
tell a real story and where we're really able to quantify ROI on that
32:57
Awesome thank you for the ozone I don't dare to pronounce your last name sorry
33:04
for that what actions do you take for detractors NPS's good questions so we do
33:11
a follow-up on NPS and we send them an email to say either we're happy that
33:17
you're a promoter you know thank you for your feedback or we're sorry you had a
33:22
problem and we ask them if we can follow up with them and if they respond then
33:27
we
33:27
will follow up but we also have our trainer management team they're
33:32
responsible for making sure that we have the best trainers that we can have so
33:38
they will read every detractor response and especially if it relates to trainer
33:43
quality they will correlate that with who the trainer was and provide feedback
33:48
to those trainers to find out how things are going and I think we're
33:53
honest with our accounts also right when we go to do our EBR and QBR's we want
33:59
to
33:59
share with them some of the feedback we've gotten if the if the NPS is low
34:03
because a lot of times that may be tied to how the program is being managed or
34:08
the expectations that were set so I think it adds to a layer of
34:12
transparency when we actually do share that at the client level also and
34:17
definitely at the business impact level yeah go ahead oh someone bumped up now
34:26
what what other metrics beside the NPS do you measure how many questions are
34:31
there in your survey and what are those yeah so I mentioned with the NPS survey
34:36
there are around three questions one's a yes or no question so if they have a
34:41
trainer then they will get an extra question with the business impact survey
34:45
there are around 15 questions in total and we're not really measuring
34:54
everything there but we were looking at the percentage of people improving in
35:00
in different parts of language acquisition and how they're doing that and we
35:06
measure measure CSAT as well we do and I think that we're we're quite clever in
35:11
the way that we're building a story around that business impact survey data
35:16
we're asking you know how much time do you think that you've saved is language
35:20
learning important right so trying to continue to build the story that helps
35:27
us then renew and grow that account and quite honestly to tell you the truth
35:32
the very favorite questions are those that solicit comments right those that
35:37
say tell me how this has been you've been able to use this at work how has it
35:42
better your life those are really the nuggets of gold that put it in
35:48
perspective for us what we do as a company right is language training but
35:52
really what we're doing we're enabling people to be able to communicate with
35:57
their coworkers and be able to have you know a vehicle to navigating their
36:04
career that they may not have had so just those nuggets of gold really come
36:08
I think from those comments or from those questions that's the list of
36:11
comments yeah and I think the most magical responses I've seen in my my
36:15
time when you see people say I got promoted because of this I was able to
36:22
get another job and there was one example from a learner who was based in
36:29
Dubai and he said when I started in this company I was a tea boy I was taking a
36:37
trolley round everyone in the office serving tea and my boss I was interested
36:44
so my boss said oh do you want to take language learning he said yeah yeah yes
36:49
please I would like to do that and after a couple of years he ended up as a
36:54
project manager so it made a difference to someone's life that they'd gone from
37:00
being a tea boy to being able to manage projects that just makes my heart
37:05
well yeah these are great great great stories and maybe sometimes you would
37:11
wish that you could use them as in the testimonial on your site or something
37:16
like that these these are very impactful and meaningful so I was and again this
37:22
you're very popular what is your response right you already I think on
37:27
that but what do you do to increase it even though it is quite high yeah and I
37:34
think you know we work with the customer we work with our program owners to be
37:39
able to do that our job at CSM's is to make our program owners stars in their
37:44
company so we you know we're not just doing things at them we're sharing best
37:49
practices and enabling them to tell the message in their companies as well yeah
37:55
agreed so next one why not randomize the time of
38:04
surveying it's if it's timed at a targeted period in the customer journey
38:10
wouldn't if falsely inflates pasta it depends so this is language learning so
38:20
surveying someone if from a business impact perspective if they're only two
38:26
months into their language learning journey you're not going to get much
38:31
feedback on that you know when we did it as we did it before because it was
38:35
kind of randomized you'd say oh I've not had enough time yet I can't say if
38:42
there's been any improvement because I've only been learning for two months but
38:48
the MPS is kind of randomized because it depends on learn or events as I said
38:53
it is every 90 days but there might be something that happened or whatever
39:00
that would delay that going out so it is more randomized would you say Amy
39:04
yeah I would say that I also think that we're after seeing if there's a trend
39:08
right we're after seeing if over a life cycle of a learner do we see those
39:14
dips in those falls in those MPS scores specifically and then for business
39:20
impact survey we have a different timing for that because that's trying to
39:25
gather the kind of the fill of the whole program right but we need it in time
39:30
for
39:30
renewals you know because we're talking to CSM's right there's a tricky timing
39:34
between gathering the information you need to make this story that actually
39:38
gets you the renewals so that would be my answer there yeah I'm sorry I get a
39:45
little bit distracted by this question earlier I would say who asked that yeah
39:53
who asked it's a you Jill it's anonymous for a reason maybe all right let's go
40:04
to the next one how often do you circle back to the text analytics to see the
40:09
traits do you want me to answer that because I have a pretty good answer good
40:13
right yeah Angela and I have known out there for 20 years so I feel okay with
40:18
jumping in on this one over her here so text analytics we have a very specific
40:24
use case for it right now and as I mentioned slightly in our in my slides
40:29
we have trainers and trainers need to be happy in order to deliver a good
40:33
service right and so we have a trainer management team who's very focused on
40:37
measuring their NPS and so they set a goal to increase their NPS so every
40:43
month they're measuring that NPS and they've put into place different
40:46
initiatives to try to help move that NPS score and they're looking at text
40:52
analytics within like the first seven days of that monthly NPS going out to
40:58
see if they're seeing key key words and key phrases that match the initiative
41:03
that they launched that that month so there's a direct tie to those I think
41:10
that anytime anybody comes to us we've got product coming to us we've got
41:14
marketing coming to us we open up that dashboard and we look at it so I hope
41:19
that answers the question absolutely do thank you Amy let's go for the last
41:26
question for for today and also for the whole post event could you share an
41:31
example of a critical decision that was influenced by insights generated from
41:35
AI and how did these insights differ from what you may have gleaned through
41:42
traditional methods I think it may be for us a little early on kind of big
41:50
decisions from AI because we've only really started with that in the last six
41:56
months yes and six months or so but I think what it's enabled our product team
42:03
to do is to look at where should they spend their time you know we're all
42:07
facing economic challenges and we don't have this big pot of money or big pool
42:15
of people to be able to work on absolutely everything so being able to
42:20
analyze that data and see what the feedback is gives us the opportunity to
42:27
decide should we focus on this part of our product or is it more important to
42:32
do something else instead so it's really kind of helping guide the way and that
42:37
's
42:38
where Amy was saying you know it wasn't just nine teams providing input into
42:42
what
42:43
we do our eleven teams departments they look at that feedback as well so they
42:50
all have access into to gain sight and I think that's been really useful in us
42:55
being able to grow how we use it and keep the investment going yeah it's an
43:02
easy
43:02
return on investment story to tell when you have different departments knocking
43:06
on your on your door on a weekly basis that says hey I heard that you can see
43:10
things that that we haven't seen before and you can turn around and you know
43:14
create a dashboard for them and they can they can use that in a way that they
43:18
haven't even thought about using it yet right they've got their basic ideas but
43:23
it's just like asking chat GPT to give you some additional ideas right we're
43:27
just scratching the surface yeah well those were the last question
43:33
thanks yeah thanks Amy and I give a big round of applause for the speakers
43:39
all right this was the first session from the last day of polls I was a
43:45
good it was a big big big big event and it was as you can hear my voice also I
43:55
'm
43:55
getting a little bit tired right now but it was it was amazing having you here
43:59
thank you for that if you could I know not today or right here in this room but
44:06
if you could review us and give us some insights from what you have seen
44:11
through
44:12
these two couple of days that would be amazing we are yeah if you can tell
44:20
yeah it's a little bit old so thank you and have a safe travels home