When organizations talk about AI transformation, the conversation usually starts with tools, models, platforms, and capabilities. That framing is understandable, but it is also incomplete.
That is not where Jonathan Thayer starts.
In a recent episode of The Learning Experience Ops Show, Jon Thayer, Chief Data and AI Architect at K2 United, offered a quieter, more grounded perspective, shaped by more than two decades within the same organization. Over that time, he has moved from product engineering to analytics to AI architecture, and along the way, he has learned something many teams only discover after painful missteps:
“AI is the easy part. The hard part is creating an environment where people can actually use it well.”
What followed was not a conversation about hype or futuristic promises, but a thoughtful exploration of how organizations really change, why most AI initiatives struggle to take hold, and what leaders need to focus on if they want progress without chaos.
Listen to the podcast on your favorite directory:
AI Is a Business Transformation First
Thayer’s current role did not exist a few years ago. It emerged organically as his organization realized AI was no longer a side experiment or a standalone initiative, but something that would increasingly shape how work actually gets done.
Still, he is clear about one thing. This work is not about chasing technology for its own sake.
“It’s really business and digital transformation,” he explained. “AI is just part of it.”
That distinction matters because when teams lead with tools, they often end up forcing the organization to adapt to the technology, rather than the other way around. Thayer advocates starting with the work itself: the friction people feel, the decisions they struggle to make, and the problems that slow them down. Only then does AI earn its place.
This mindset shows up clearly in how his team evaluates platforms. Data access matters more than flashy features. Monitoring matters more than novelty. Governance matters more than speed, not because innovation should be slow, but because uncontrolled innovation breaks trust quickly and is hard to recover from.
Why Agile Still Works and Where It Breaks Down
One of the most practical parts of the conversation centered on agile ways of working. Thayer is a longtime advocate of agile, but he does not treat it like a rigid doctrine.
Agile works, in his view, because it balances structure and flexibility. AI complicates that balance.
Traditional software behaves predictably, while AI systems do not.
As Thayer put it, “You can change the model and it behaves differently. It’s not deterministic.”
That reality changes how teams need to think about testing, quality control, and deployment. You cannot validate AI once and move on. Instead, you have to monitor it continuously, watch for drift, and pay close attention to how real users interact with it over time.
This does not mean agile no longer applies. It means teams need new skills, new tools, and more realistic expectations. The process can stay lightweight, but the thinking has to get sharper.
Citizen Developers and the Guardrails Problem
One of the biggest shifts Thayer sees coming is the rise of citizen developers. With modern AI tools, people no longer need formal engineering roles to build useful systems, and that opens the door to a level of speed and creativity organizations have rarely experienced.
It also introduces real risk.
“When anybody can build something,” he said, “how do you encourage that and also keep some level of governance so it doesn’t turn into chaos?”
This tension runs through the entire episode: speed versus safety, freedom versus responsibility, innovation versus control. Thayer’s answer is not heavier process or centralized approval, but clearer boundaries.
Instead of telling people how to build, his team is working to define where governance becomes necessary. Touch financial data and oversight is required. Work with HR data and the same applies. Deploy an agent that can take automated action and that crosses a clear line.
Everything else is open for exploration.
“I’d rather give people guardrails than rules,” he said. “Have at it, just stay in the lane.”
It is a philosophy rooted in trust, but reinforced by thoughtful constraints.
Measuring Success Without Killing Momentum
Ask most leaders how they measure AI return on investment and you will hear familiar answers: time saved, tasks automated, headcount avoided.
Thayer is skeptical of that approach.
You can measure everything, he noted, but it comes at a cost. When people have to justify every minute saved, experimentation slows down, risk-taking disappears, and momentum fades.
Instead, he looks for passive signals. Usage patterns. Adoption trends. Changes in demand for new hires. Output that stays steady or grows without adding capacity.
“We’re doing the same amount of work or more,” he observed, “but we’re not asking for more people yet.”
It is not a perfect measure, but it is a practical one. AI’s impact often shows up indirectly, and trying to isolate it too precisely can undermine the behaviors that make it valuable in the first place.
When Knowledge Becomes Cheap
Near the end of the conversation, Thayer shared an idea that lingers long after the episode ends. He believes the value of knowledge itself is dropping fast.
AI is exceptionally good at knowing things, and asking the right questions is no longer the primary bottleneck. What matters now is judgment.
“Understanding what the business needs and what problem you’re actually trying to solve,” he said, “that’s where the value is.”
This shift has implications far beyond technology teams. It reshapes how we think about expertise, leadership, and learning. The skills that matter most are not memorized answers, but the ability to frame problems, break them down, and decide what matters next.
Ironically, these are the most human skills we have.
A Quiet Kind of Leadership
Jonathan Thayer does not speak in absolutes or promise certainty. What he offers instead is something more useful: a steady, systems-oriented approach to change that respects people and acknowledges complexity.
His advice for anyone stepping into AI leadership reflects that mindset.
“Use it yourself,” he said. “And talk to people. Inside your company and outside it.”
No grand strategy. No magic framework. Just curiosity, humility, and practice.
In a moment where many organizations feel pressure to move fast or risk falling behind, Thayer’s perspective is a reminder that speed without judgment is not progress. AI will continue to evolve, and tools will come and go, but the real work lies in building organizations that can adapt without losing themselves in the process.
Listen to the full conversation with Jonathan Thayer on The LX Ops Show or wherever you listen to podcasts.
Podcast Transcript– Jason Gorman with Jonathan Thayer
Jason Gorman (00:00)
Today on the Learning Experience Ops show, I am joined by Jonathan Thayer. Jon is the Chief Data and AI Architect at K2 United, where he has spent more than two decades evolving from product engineering to innovation to analytics and now AI architecture. Really perfect guest for the topic of this show is just really to dig into ⁓ operations and the way things that are working in learning operations have worked.
do work, may work in the future. John has a master’s degree in software engineering from UT Austin and also a BA in psycho biology from Wheaton College. I do not normally mention where people have been to school, but I’m mentioning it because John and I knew each other at Wheaton. ⁓ You arrived in my sophomore year, I think, and ⁓ we think we lived on the same floor.
And then we graduated the same year because I took some time off in the middle there. ⁓ So we’ve known each other for a long time. We had no plans to go into the same kind of work or same field. And I think when we reconnected years after graduating and talked about what we did, we’re like, that’s kind of weird. We’re doing similar stuff. So it’s been a real pleasure to.
to talk as our ⁓ lines of work have just kind of evolved in parallel in some ways over the years. Welcome to the show, John.
Jonathan Thayer (01:39)
Thank you.
Jason Gorman (01:40)
So I want to start with a lightning round. This is a new thing I’m trying out, a bit of a beta. And the idea here is I’m going to ask you a question. And just give a quick sentence or two on it. And if we need to, we’ll come back to your answer and kind of unpack it later. But just kind of want this to be a little bit more rapid fire. Yeah, you game? All right, good, good.
Jonathan Thayer (02:02)
Let’s do it.
Jason Gorman (02:04)
What is a quick elevator pitch about what you do?
Jonathan Thayer (02:09)
⁓ Would you like to know my company or me? Sorry to ask a question back, both. ⁓ So the company, we actually started off as an e-learning company, ⁓ moved into cybersecurity and hosting for federal space just because it kind of made sense. But really at our heart, we were an e-learning company.
Jason Gorman (02:30)
And you, what is your role?
Jonathan Thayer (02:33)
⁓ I’m right now the ⁓ chief data and AI architect. I’m here to give my intro. ⁓ I’m leading the AI part of our digital transformation, business transformation in the organization, which is a new initiative.
Jason Gorman (02:46)
new for all of us. ⁓ Okay, next question. What do you love most about your work?
Jonathan Thayer (02:51)
⁓ It’s all new. It is all really new and fast changing. I thrive in that kind of environment. So that’s really what I love.
Jason Gorman (02:58)
And kind the opposite ⁓ of that coin is this next question, ⁓ which is, what are you most anxious about for the future? And I guess for the sake of the podcast, we will limit this to work-related topics. Yeah.
Jonathan Thayer (03:13)
Fair enough. ⁓
Really, ⁓ our company is really people focused, both our employees and the people we serve. And I’m worried about both. I think AI is really changing things and it could have a great positive or negative impact. There’s just a lot of uncertainty in that whole area.
Jason Gorman (03:27)
Mm.
Okay, next question. What is something that you do in your process at work that you think you’ll have to change eventually because of AI, but don’t want to change?
Jonathan Thayer (03:52)
⁓ Probably not for me personally, but just work as a whole, ⁓ the creative elements. There’s just a lot of human input. We do a lot of creative things and AI is going to draw off. It’s going to draw off a lot of that work.
Jason Gorman (04:08)
Hmm, I think my kids would have the same answer. They are constantly ⁓ talking to me about AI, AI theft of art ⁓ in particular. Okay, thank you. All right, let’s go to the next one, two left. What’s the skill you’re trying to pick up right now?
Jonathan Thayer (04:26)
⁓ Right now my big focus is AI application, the lifecycle development. ⁓ It’s just a whole, not entirely different, but it is very different way of developing software systems, applications.
Jason Gorman (04:33)
Mm-hmm.
And maybe back to Wheaton, but perhaps further back or more recently, what was your favorite subject in school and does it still influence you?
Jonathan Thayer (04:50)
I’m not sure if Wheaton applies in this case, ⁓ just because again, going from psychology ⁓ for my master’s degree, I’ll lean back on the data science work. mean, I’ve been doing that for the past few years and AI is a big part of that now. It’s always fascinated me.
Jason Gorman (04:53)
Mm-hmm.
great. All right. So I am curious since you’ve been in your role for how long or at this company, how long have you been at K2 United?
Jonathan Thayer (05:23)
Uh, 24 years.
Jason Gorman (05:26)
24 years. So sometimes the way that I frame this question is, what ignited your passion in this work? But you have been in this one place for a long time. And I’m curious, sometimes an early job like that can be, it was just the job that was available. But I’m wondering if you, if you,
Jonathan Thayer (05:38)
Yeah.
Jason Gorman (05:49)
Was there a reason that you sought out this area, just kind of generally, like e-learning? You said it started out as an e-learning company. Was there a reason that you chose this field to work in?
Jonathan Thayer (06:02)
⁓ Honestly, no. I ⁓ actually had been working in biotech and I got laid off and really needed a job. And this happened to be the company and ⁓ it was… ⁓
Jason Gorman (06:08)
Yeah. huh. huh.
Jonathan Thayer (06:17)
I interviewed, got the job, and they just happened to be doing e-learning. And I was like, all right, let’s go with it. I was more interested in, at the time, was web application development, early 2000s, right? That’s what every company was trying to do. They just happened to be building a learning management system and training to go with it.
Jason Gorman (06:35)
Everybody comes into this industry through a totally different door. It’s quite interesting. mean, so, but it wasn’t just a job for right now because you’ve been there for 24 years. So what is it that has kept you there? What’s been so captivating about the work?
Jonathan Thayer (06:46)
Right.
⁓ really two things, you know.
Personally, like I mentioned before, like change, which seems to contradict the fact that I’ve been with the same company for so long. But we started with ⁓ only a few people and ⁓ we had to do a lot of different things and quickly. And my role changed and what we did just to survive as a company changed. So it’s always kept my interest. ⁓ And then on the more social side, ⁓ the owner of the company in particular has always set sort of a cultural focus.
Jason Gorman (07:02)
Mm-hmm.
You
Jonathan Thayer (07:28)
on people. ⁓ treating employees well, as well as ⁓ not just having a business to make money, but providing a positive social impact. And you combine all of those things together and it’s just a good mix for me, which has kept me ⁓ with them as long as I have been.
Jason Gorman (07:46)
And as I mentioned, just at the top, you’ve done a lot of different things there. ⁓ Product engineering, innovation analytics, ⁓ AI architecture were the things that I’ve mentioned.
I’m really curious about just across, you’ve just mentioned you had to wear a lot of different hats. It was a small company. It’s grown a little bit. You’ve changed roles inside the organization. You’ve just recently changed roles into this one that’s really focusing in on AI architecture and innovation internally. But I’m really curious just across the arc of your time. ⁓
at K-2 United, and I guess any job, what ⁓ are the mindsets that have endured across those in terms of doing your work or processes that you have that you have reliably deployed across the different kinds of work that you’ve done? ⁓
or any tools that you’ve used just over the arc of your career.
Jonathan Thayer (09:00)
Yeah, tools are tough because it’s such a long span. We’ve gone from ⁓ web application development on its own has radically changed from what it was even 10 years ago. As far as just mindset, that’s maybe a little easier, at least with a size company that we are having that entrepreneurial mindset, being willing to adapt, change, learn.
Jason Gorman (09:03)
Mm-hmm.
Jonathan Thayer (09:25)
The challenge of course is you you mentioned on the other side what kind of process and things like that. Balancing that is a real challenge because as you grow you’ve got to add some of those things but that slows you down. So trying to figure out what that good mix is. You know I think if I were going to say one thing maybe just a I don’t know if you call it a process more of a philosophy just adopting agile.
Jason Gorman (09:29)
Mm-hmm.
Mmm.
Jonathan Thayer (09:49)
has
been something that’s really helped us. ⁓ And not enforcing it from a, we all do Agile the same way, ⁓ but most of our teams now across different groups in the company are on Agile and everybody has their own flavor. And they make it work for them. So I think just from a process, a work philosophy, ⁓ it ⁓ works well, at least at the size that we’re at.
Jason Gorman (10:15)
And as, so now you’re in this position to really think about AI for your organization.
Maybe we can focus on agile. Have you found that what you’ve needed to do to run projects in an agile way has been challenged by or is challenged by new technologies that are rolling out like, I don’t know, every day pretty much now? ⁓
Jonathan Thayer (10:43)
Yeah, yeah. And maybe I shouldn’t say ⁓ we’re at the start of this process. So I’m more looking at what the potential problems are versus stuff we’ve run into. But one of the big changes that I see and how we do projects is going to be, ⁓ and I’ll borrow this term from actually, it was our HR manager who ⁓ introduced me to term of citizen developers.
Jason Gorman (10:50)
Mm-hmm.
Hmm.
Jonathan Thayer (11:08)
And
that’s kind of different in that now you don’t have to have somebody who’s a, a programmer. I’m going to build something. Anybody in the company can do that with AI. So how do you then let people loose to do those things, but then roll their work?
that might need to be taken to the next level into, let’s turn this into an automated AI tool, or let’s integrate it with a chain of agents, or integrate it into a larger AI system. Whereas before, you kind of had the requirements people, and they could pass things off. You get product design, and then you get the builders. Now it’s a big blend. And I think that’s going to be a challenge, even from an agile standpoint, the people doing the work are going to be different. ⁓
Jason Gorman (11:40)
Mm-hmm. Mm-hmm.
Jonathan Thayer (11:50)
And how do you both encourage that and also keep some level of governance over it to keep things from just being chaos?
Jason Gorman (11:59)
Uh-huh, yep, I see what you’re saying. And so it sounds like, I think what I’m hearing there are sort of two, a couple of challenges. One is that you have individuals innovating, which is great on the one hand, because you want people doing more in their jobs. But on the other hand, there is a need organizationally to systematize those things to some extent.
And what is that? Is that accurate?
Jonathan Thayer (12:29)
Yeah, I think that’s a great way to put it.
Jason Gorman (12:31)
So are there things that you’re trying that are capturing those good, successful practices as individuals are inventing them?
Jonathan Thayer (12:48)
⁓ Not yet.
That’s this is this is where it’s ⁓ we’re in that strategy and planning phase. The first thing that we actually did was we put out a company survey and we got about half of the company responding and the questions it was a very short survey. The goal was to really find out what are the what’s the feeling about AI so we can find those early adopters and then from there we’re hoping to pick these these early adopters these champions and start using them to help other people.
Jason Gorman (13:01)
Yeah. Yeah.
Jonathan Thayer (13:22)
And then also from there, start collecting and generating these ideas and we can decide where those ⁓ dividing lines are between you just run with us on your own and we’re not going to get in your way to, now we actually need to, let’s put this into some kind of project process. And those thresholds, I think are going to be where we work on governance. And it’s going to be, hey, you know, because of the type of data you’re accessing, hey, you want to integrate with our financial system. Well, we probably want to govern that.
Or are you going to be working with HR data? Well, we want to govern that. Or you get to the point where you have an AI agent that might be taking actions. Well, then, OK, now we probably want to govern that because we don’t want things to get too out of control. So we’re kind of at the point where we’re trying to figure out what those lines are because we do want to track things. And we want to help people out and help other provide the people and technology infrastructure.
But ⁓ we don’t have that in place yet. So that’s really our big challenge. Right now we’re just in the planning phase.
Jason Gorman (14:25)
And I’m curious, just with an agile workflow,
Do you sense or have you seen any incompatibility between ⁓ sometimes with certain kinds of technology deployments? They have to be done in a very, they have to be done especially with AI in my experience.
you have to have pretty consistent inputs and then have a pretty consistent understanding of what the outputs of those things need to be. There is almost like a rigidity in that that needs to be there. And you can have a lot of creativity within that rigid process, but you really need to understand what’s coming in, what is it that we are.
doing with that information and what is the consistent output? Are there any challenges that you’ve seen with a more agile process to try to figure out where AI fits in that process? Or are structurally there any problems between how agile works and what’s necessary when working with AI?
Jonathan Thayer (15:42)
I don’t think it’s gonna be a conflict with Agile necessarily, but this kind of gets into some of the things that I’ve been doing a deep dive on more recently, which is sort of the AI application development, AI deployment. Because part of that, just an example.
And this doesn’t necessarily go to Agile, just software development. You’ve got a quality control process. How do you then verify, hey, these are the requirements, this is what was built, does it do what it says it would do? Well, AI doesn’t work like regular software.
it’s not deterministic. You can get different inputs. You can change the model, and it’ll behave differently. ⁓ So how you do those, both the design and the development, and then, like you said, the quality check or that testing piece is different. And it’s going to require different tools and different skills. ⁓ But I don’t think that’s necessarily a change to the agile part. It’s more of the perspective of the people that are building and then testing the systems.
And then there’s a deployment and maintenance where AI, depending on how you built your model or what you’re doing, AI can drift. So you’ve got to monitor it. And you’ve also got to monitor, is it behaving the way you’d expect it once it’s out in the wild, once you’re having users, whether they’re internal or customers, actually use the AI tool. But that’s not really, I think, an agile problem. That’s more of a, you know,
That’s a software development, it’s an application development lifecycle challenge.
Jason Gorman (17:13)
How do you do that kind of monitoring on a large scale about what the AI is outputting and whether or not it’s hitting your own predetermined standards for quality to avoid that drift? I would imagine across a large team and externally with users, there must be too much for a person to review.
Jonathan Thayer (17:15)
you
Yeah.
Yeah, absolutely. And again, I haven’t gotten into this. The point that I’m at is looking at what platforms we want to probably choose for our company to build on. Because once we start building, there’s going to be sunk cost, both in time and effort and training and learning how to use the tools. ⁓ Azure has AI foundry that has built-in quality check and benchmarking and monitoring. ⁓
Jason Gorman (17:47)
Mm-hmm.
Jonathan Thayer (18:01)
We’re also looking, ⁓ we’ve been using Databricks for a while, which is another AI machine learning platform. And these platforms have built-in tools where it’ll monitor ⁓ a lot of different aspects of the performance of the AI, both in how quickly it responds, it’ll integrate user feedback, the thumbs up, thumbs down that you see. ⁓ It will let you…
tune different models and then check to see the performance difference between them. ⁓ Even what they call serving endpoints where, for example, you have an AI agent, a serving endpoint can have two thirds of the customers or users get pointed to one version of the AI model and then another third to the other. So you can do A-B testing and then compare the results and then choose the one that works better, that’s more accurate. ⁓
And then something else I ran across that I haven’t tested yet but I’m excited to try is ⁓ using AI to test AI. So the idea is you have, instead of a customer, is this right or wrong, you actually use a different AI model or different LLM to say, does this output make sense? And so you can automate your testing by linking together multiple AIs and having AI evaluate AI.
Jason Gorman (19:21)
Can you give, I’m curious to get like just a concrete example. So if a customer, what is a customer interaction with your system look like? What is that back and forth? What kind of data is being generated there? then how, it sounds like you are still working on how to set these things up in general, but it would be interesting to kind of hear a specific example and then talk a little bit about some of the challenges.
and opportunities that you see with AI around those interactions.
Jonathan Thayer (19:57)
Well, right now we’re not actually, we haven’t, we’re not planning on building anything customer facing. So the first step we’re really going to be looking at automating some of our internal process. So there’s a lot of security monitoring and patching and vulnerability management, things that we do, just operational things for IT. And then also parts of our contracting process, our contract and order management, vendor management, those are things that we’re
We’re in the planning stage to build and test. ⁓ So right now, we really don’t have any of these tools in place. That’s why I’m doing the platform evaluations to really see what works best for us. ⁓
Jason Gorman (20:35)
And
so what are the important things you’re looking for in the tools that you’re looking at? What do they need to be able to do to meet your needs?
Jonathan Thayer (20:45)
For the platforms, the biggest one is data access. Most of AI is, or for our organization, the real value is going to come out of, can you get to your data and use it? just as an example, just as a personal assistant, we’re using ChatGPT, we ran into a challenge where it wouldn’t connect directly to our data because we don’t have the enterprise version.
Jason Gorman (21:10)
Hmm.
Jonathan Thayer (21:11)
So
say you build an agent and you link to knowledge, for example, a contract review. Here’s our playbook for how you write contracts. Well, anytime you change it, you’d have to upload it to the chat GPT agent that you built. Whereas if you have the enterprise version, you could point to here’s where it lives on the network folder.
And then any time you update the playbook, which includes things like, here’s the name of our bank, here’s the name of our contracts manager, here’s our company address, here’s our tax ID number, just things like that. When you change those things, they automatically get updated in the agent. ⁓ So the platforms I’m looking for, one, data access. ⁓ The other thing is monitoring. We have to be able to, who’s using it, what are they using it for, what data are they accessing? ⁓
Jason Gorman (21:51)
Mm-hmm.
Jonathan Thayer (21:58)
We also want to make sure we can link into ⁓ data loss prevention systems. So is somebody uploading something they shouldn’t? That’s very important from a governance standpoint. And then with Chat GPT and Copilot and some of those digital assistants, it’s not to the point where you’re building AI applications yet. ⁓ But for that next step, where say you’ve got multiple agents you’ve built in Chad GPT that you want to start linking together, you want to automate.
Jason Gorman (22:06)
Mm-hmm. yeah. Sure.
Jonathan Thayer (22:28)
the platform that we move those to, one, we’d want to be able to use different models. So if we wanted to change between OpenAI or Claude or… ⁓
llama, we could just change them out and see what is more cost effective and what performs better. And then the monitoring, like I said at the end, know, is this, is it working right? Is it doing what we expect it to do? ⁓ So really those are, those are the big things that I’m looking for in the platforms. And right now we’re more at the building individual agents. So we’re not building systems or applications. ⁓ But I’m looking at, you know, if we want to really launch into this, it’s probably going to be three to six months before we have
Jason Gorman (23:03)
Mm-hmm.
Jonathan Thayer (23:10)
agents that we want to automate and then chain together and accomplish more complex business tasks.
Jason Gorman (23:17)
And do those business that even you mentioned a few of them. So it sounds like these things could apply to finance, to HR, ⁓ could be to any of your actual data work. any work the teams are where they’re interacting with data, does it apply to any of the program creation work that you’re doing?
Jonathan Thayer (23:46)
Yeah, in fact one of the great use cases both on the software and the instructional design, the training development, ⁓ you know we haven’t built them yet but we’re planning out and actually some people are have started to build their own personal agents we haven’t built a system ⁓ but let’s take requirements right.
Do you have using agents to ⁓ make sure you’ve covered everything that needs to be covered in good requirements development? And that applies both for software and instructional design, right? There are certain minimum things you want to make sure are defined. Well, AI can scan through your requirements and find the gaps. ⁓ Next step beyond that.
If you’ve got good requirements, it’s already been validated, well, go ahead and put it at your agile process. Instead of people copy pasting stuff into, for example, we use JIRA, you have it create a ticket for you. Now this is ready to develop. ⁓
Jason Gorman (24:34)
Mm-hmm.
Jonathan Thayer (24:42)
Beyond that, once you’ve got your requirements, you can also create your test cases. So you can have AI look and say, well, generate my test cases for me so I don’t have to write these. And then your quality control people won’t have to go bug the developers to say, how do I test this? ⁓ And then beyond that, we already know the writing code, building images, building content. Those are things that AI from a generative standpoint can handle. ⁓
Really, on both sides, I see from just the development lifecycle, AI can help from the unstructured stuff, the how do I write a good requirement all the way through to building the tools, building the product.
Jason Gorman (25:22)
And so are you looking for, so you were talking about trying to find a technology that will help you kind of ⁓ govern all of these different agents, how they’re deployed, get reporting on them. Is it possible to have a single system that does go across all of those functions from finance to program creation? Or do you imagine that
Jonathan Thayer (25:46)
Yeah.
Jason Gorman (25:53)
You might have a couple of different ways that you’re governing ⁓ the deployment of AI technology depending on whether or not it’s happening in program development or finance or HR or whatever. How does that look like from a kind of organizational standpoint?
Jonathan Thayer (26:09)
Yeah, I think really it’s the first part of your question. I don’t think it’s going to be a one size fits all. There’s no one application out there that you could say this does everything for me. ⁓
Jason Gorman (26:16)
Yeah. Yeah.
Jonathan Thayer (26:21)
It’s more of what are the parts and pieces? So there may be, here’s the part that manages helping you build the agents. Here’s the part that helps you test them. Here’s where your data lake, your data warehouse, that’s its own technology. ⁓ Data governance and the governance part of it, there are multiple platforms out there that do that. So I think really I’m thinking of it as that you have these pieces and then you assemble them together into here’s our platform. And then.
It’s not one piece of technology, it’s multiple, ⁓ but they do work together and they provide a larger business solution out of it. But yeah, don’t think there’s one place to go to do it all. And you may not want to do that because then you’d probably get locked in.
Jason Gorman (27:05)
and then they’ll jack up the prices on you. When you’re locked in, that’s when they increase their pricing is my experience. ⁓ I mean, this then sounds like a…
Jonathan Thayer (27:09)
Always. Always. Yeah.
Jason Gorman (27:18)
Like there is no obvious roadmap for you to follow, right? Just from based on what you’re saying that you have a lot of creativity and leeway in terms of how you might imagine how all of these systems connect or not, how they are governed in terms of the rules of your organization, but also how individual humans are, you know, if there’s a leader in a finance, they might have responsibility for a set of things. If there’s a leader in
know, learning operations, you know, building programs, they might have responsibility over a different piece of the system. Those things all might roll up or might not roll up into a single view for senior executives. But that’s what you’re working out, it sounds like.
Jonathan Thayer (28:03)
Yeah, yeah, it’s ⁓ more of a, when I say platform, it’s less of a, I’m going to install this on my computer or on a server. And it’s a framework, it’s a model to operate the business by. And I also want to emphasize, I’m really focused on the data and AI part of it.
Jason Gorman (28:10)
Yeah. Yeah.
Jonathan Thayer (28:21)
But the rest of my team that I work with, ⁓ we’ve got human resources in there and we’ve got IT and we’ve got security and we’ve got, so we have representation. We’ve got the finance ⁓ side of things because ⁓ it’s AI, but really ⁓ business and digital transformation, AI is just a part of it. ⁓ And a lot of it is just ⁓ creating an environment where
the technology can get adopted and people feel comfortable and use it successfully. I don’t know how to better put it. And AI is sort of the big thing right now, but ⁓ you could apply, you can say it’s automation or you go back to the old days, we’re moving from paper to digital.
Jason Gorman (29:10)
Yeah, I it’s I appreciate you saying that somebody I asked a question to somebody recently about, what is the what’s the most important thing in the transformation? And they said it’s actually not about the technology, it’s about the fundamentals of what you’re trying to do. And, you know, I think it sounds like you’re saying a different a different version of that, which is.
This is a fundamentally, it’s a business transformation project and that AI is involved heavily in that transformation, but that you are, it really needs to be driven by the essential needs of the different functions first. And the AI then needs to be serving those needs. I think sometimes I, I’m not gonna speak for everybody.
I can get so sucked into some new thing that AI can do that I feel like, all right, well, I need to be pushing, need to be changing something I’m doing to accommodate this new opportunity we have that the technology is giving us. But that’s definitely wrong. ⁓ I always end up in a bad place when I do that. And it sounds like, it just sounds like you’re saying a similar thing that you. ⁓
you’re really focusing in on the business needs and then you’re working hard to see what the technology is capable of. But I’m curious, as you do this, as you roll this out, I what you described in terms of how you’re going to roll this out across the organization and what the opportunities are, it sounds like this is an enormous project. There are a lot of things that need to be changed. This isn’t a like,
We’re gonna put this in place and then it’s gonna run and it’s gonna be fine. But how do you think you might measure the success of a new way of doing things once it’s deployed? How do you measure the success? How do you measure the return on all the investment that you’re making in it?
Jonathan Thayer (31:06)
Yeah, I know.
Yeah, yeah, and that’s really tough. Because you can, if you want to get down into the micromanage, you can say everything that everybody does. You want to say, how long did it take before? How much did it cost? And how much are we saving now? You get down to that level of micromanagement and you kill your agility.
Right, because now people are spending all of their time, you know, trying to track the work that they’re saving time on. And you’re also discouraging people from doing it, right? Because it’s like, well, I do this, but now I’m going to have to come up back with an ROI. So we threw that out. You know, I’m looking for, you know, ⁓
What can I learn just by watching passive? Passive watching, right? mean, everything is digital. ⁓ One of the reasons it’s so important, all the AI tools that we’re looking at buying off the shelf tools, we have the ability to see how they’re being used is you can judge a lot just by who’s using and how often and where they’re using it for. And I’d rather do that than have to have people provide some ROI report ⁓ or dig into time sheets and see, you know, did we save half an hour here or half an hour there?
And then I think broad measures. ⁓ We’ve already kind of seen just so far in what we’ve adopted for AI, which is really just using it kind of like we’ve talked about as just a digital assistant, right? Building individual GPTs or agents. ⁓ We’ve already seen a decrease in the number of requests for new ⁓ employees. So in previous budget seasons, everybody was always asking for more. ⁓
and either because they’re anticipating they won’t need it because they’re hoping AI will fill it, or it’s already doing some of that. ⁓ We’re actually doing the same amount of work or more, but we’re not asking for more people yet. So some of those, like I call them passive measures, just sort of these other things that you may not be able to say, absolutely, AI did this. ⁓ But you can look at the overall organization and its performance and you can say, you know, it probably did, ⁓ or there’s a good chance. ⁓
Jason Gorman (33:18)
Huh. Yeah.
Mm-hmm.
Jonathan Thayer (33:36)
But it’s a great question and it’s not an easy problem to solve.
Jason Gorman (33:39)
You just said a very big thing there about that you’re not hiring, well, that maybe you’re not hiring where in the past years you might. And I…
Jonathan Thayer (33:44)
Which part?
Yeah.
Jason Gorman (33:53)
It’s interesting. On the other hand, you’re in a job that didn’t exist three years ago. Right? So that work was invented. And this is something that I feel like, like you said, there is so impossible to tease out exactly what the ROI is and how to measure progress and how to know what exactly the tradeoffs are. Because I think in order to realize these gains, there’s more work that needs to be done. Trying to measure
Jonathan Thayer (34:00)
Yes. Yeah.
Yeah.
Jason Gorman (34:22)
Was it worth it? And how much did we save or make because of those changes we implemented? Yeah, I can understand the complexity and possibly impossibility or unnecessary drag that you could put on your organization if you asked everybody to go in and constantly measure themselves and their work as they’re doing it. ⁓
I’m kind of curious just because you’ve been…
My understanding when we talked before is that you’re coming into this new role just kind of organically grew out of the need that your organization realized it had and that you were able to step into it. You’re obviously very enthusiastic about trying to solve these problems, which is an important thing to have. And you have the right technical background. You know the organization inside now.
Jonathan Thayer (34:58)
Yeah.
Jason Gorman (35:13)
And I would imagine as you’ve been doing this, like there’s just, I think for all of us, there’s no playbook about for how to do these things exactly right. I mean, it’s so new, but you have been on the job. You’ve been learning on the job. So I’m kind of curious, you know, if somebody listening to this is in a position where they’re responsible for
re-architecting the way that their organization works using AI. ⁓
What advice would you give them if they’re just getting started in terms of how to do that job effectively?
Jonathan Thayer (35:52)
First thing, this is get hands on with, you use it yourself. ⁓
I think because ⁓ if you don’t have an idea of kind of what its capabilities are, you won’t understand. use it as deep as you can because again, you can start with just, I’m asking Chat GPT or Gemini questions, but then you can build agents and you can start understanding what instructions and knowledge and prompts and then dig a little bit deeper. The second thing is I would say, start talking to people. Talk to people in your company, get a feel for not everybody’s gonna have the same level of understanding.
A lot of people are going to be skeptical. A lot of people are going to be fearful, not wanting to adopt. And then talk to people outside the company. I’ve been getting more more involved with local AI groups here in Austin.
And it’s good tech hub, maybe not as good as Silicon Valley, but it’s growing. And hearing what other people are doing and where their successes and failures helps, ⁓ one, give you good ideas. Never thought of it. Give it a try. New technology. ⁓ And also some of the pitfalls. It kind of helps you level set. You go in assuming, maybe you assume AI can do everything. Hearing about other people’s failures, where it didn’t work or where it’s not ready, ⁓ that’s huge. So I’d say,
⁓ talk to people, talk to everybody. I think that’s just a huge help.
Jason Gorman (37:18)
And thinking about the future. I mean, it’s just this, so it’s such a hard question to ask. I’m just gonna ask you your predictions. But I always feel weird asking this question because…
It’s hard to know where we are right now, but I’m going to ask it anyway. You know, we know that things are going to keep changing, but what do you think?
might change in the next, I don’t know, year, two, three years ⁓ with technology that could maybe be another large fundamental shift that we might have to navigate through.
Jonathan Thayer (37:52)
I think, ⁓ I’ll steal this, I forget what the paper that I read that mentioned this. ⁓ Basically, the value of knowledge is going to go down to close to zero.
Right, because AI can do that. That’s what it does really well. It knows things. And if you know how to put the right question, it comes back with the right answer. ⁓ It’s all of those other skills that you have to learn that judgment, understanding what the business needs, what the problem is you’re trying to solve. That’s where the new value is. ⁓ So I think I could see within the next year, and maybe we’re already seeing it, a lot of that knowledge value for people’s jobs is going to change. It’s going to go down. So just knowing something isn’t enough.
⁓ And I don’t think AI has to advance much further to get there. I think we’re already pretty close.
Jason Gorman (38:40)
Sometimes we take inspiration from surprising places and we have kind of ⁓ things that we’ve learned or skills that we’ve picked up in the past that we use in our work that aren’t explicitly about what we’re doing, but they’re useful. And I’m curious, you you said that maybe physio or psychobiology, ⁓
Jonathan Thayer (39:02)
Psychobiology.
Jason Gorman (39:03)
doesn’t have a lot
of transferability into what we’re doing. But I’m curious, there, as those other things that are not knowledge become more valuable, are there things from your past skills that are maybe kind of untraditional to the workplace that you think are going to become more valuable as we head into that future you just described?
Jonathan Thayer (39:28)
Yeah, I mean, if you go back to maybe the psychology and biology, thinking of systems and how they sort of build on each other, right? You’ve got these components. does that, right? You’ve got these, I solve one problem and then I pass it off to the next thing that solves the problem, pass it off to the next thing that solves the problem. And I think from just ⁓ what skill helps me. ⁓
not just AI, but a lot of the different things that I’ve done. It’s that problem-solving mindset. How do you look at a problem and then decompose it into things that you can do to solve it step by step? And AI, I don’t want to say it can’t do that now. It doesn’t quite do it. ⁓ So when you go from the, our business has this thing that we want to do. We want to build a new course. Well, where do you start?
I may be able to give you the first instructions, but you still got to kind of put that, what’s your need to the plan? So think being able to break that down, having that skill, ⁓ the challenge I see is, you can learn a little bit of that by reading, but a lot of it is you have to practice it.
Jason Gorman (40:36)
Mm-hmm. Where there…
I don’t know, where did you learn that skill? mean, did you get that working with mice in the lab at Wheaton? Where did you first start learning those skills that where you’re like, you never thought they could be transferable to what you’re doing now, but they feel like they might be.
Jonathan Thayer (40:44)
Ha
Yeah, think ⁓ doing research helped with a lot of that. And even after Wheaton, my first job, I worked for ⁓ three, three or five years doing biomedical research for a company outside of Boston. ⁓ And a lot of that is they come up with an idea or take somebody else’s idea and then how do you test that? And then how do you verify it? And then how do you execute? think walking through that over and over again. ⁓
Jason Gorman (41:03)
Yeah.
Mm-hmm.
Jonathan Thayer (41:26)
It’s that hands-on approach versus just sort of the theoretical, philosophical approach versus a hands-on. I think you have to get hands-on to build some of those skills.
Jason Gorman (41:37)
This has been a great conversation. just have one more question for you.
Jonathan Thayer (41:40)
Yeah.
Jason Gorman (41:40)
⁓ After we’re done, I’m going to ask you for your own question offline. But after each conversation with somebody, I will ask them to give me a question for the next person. And so the last person I talked to, her name is Jen Briselli. Jen is a experienced designer and a consultant. And ⁓ her, it’s a great question. Now, let me ask it.
Very curious for your answer to this one. What kinds of rules are you breaking in the service of, she said learning, but I’m gonna say in the service of making your company more successful.
Jonathan Thayer (42:13)
Yeah.
Probably.
trying to think of, I know if it’s really rule breaking, maybe norm breaking, given our size, ⁓ I’ll do and then ask for permission later. And I think that’s a dangerous thing to tell everybody to do. ⁓ You’ve got to have a certain amount of judgment, discernment. You don’t want to wreck the company doing something, but if you wait for consensus and buy in from everybody before you take a step, it’s going to be too slow. So.
Jason Gorman (42:35)
Mm-hmm. Okay.
I love that. And I will say, I feel like I’m gonna like edit a thing where she’s talking about this and you talk about this because actually in that conversation, which was just yesterday, she specifically said that people should do this. She recommended exactly the thing that you said. I love that.
Jonathan Thayer (43:14)
Well, and I’ll do a pitch and this is one of the things, know, we’re very, we’re in it. We love innovation as a company, but we’re very risk averse, right? We move, sometimes we over deliberate. I think that’s where I’m looking at the governance piece of less telling people this is how you do things and more of just don’t go outside these guardrails, please, right? Like have at it, but you know, stay in the lane. ⁓
And that’ll be the safety part, because I think that is important. Let people go, let them take risks, but some things you want to make sure don’t go too far.
Jason Gorman (43:50)
That’s great. Well, I so appreciate you coming on the show. It’s really always a pleasure to talk with you, John. Yeah, it’s really fun. Thank you so much for joining.
Jonathan Thayer (43:57)
This was fun. Thank you.
Thank you.