Adam Larson: 00:05

Welcome back to Count Me In. I'm Adam Larson and today I'm joined by Clyde Calhoun, founder and CEO of Root Idea, an AI advisory firm. Clyde's career path has taken him from nuclear weapons R and D all the way to the world of AI governance, and he brings a wealth of insight on how organizations can keep up with the breakneck speed of AI innovation. In this conversation, we'll dive into the real challenges companies face around AI decision making, the crucial difference between merely managing AI tools and actually governing the decisions they shape, and practical strategies for setting up effective guardrails. Clyde shares stories from the front lines and actionable advice on avoiding costly risks while building a culture ready for the future of work.

Adam Larson: 00:47

Let's get started.

Adam Larson: 00:53

Well, Clyde, thank you so much for coming on the podcast. I wanted to start at the beginning. You know, you've had a credit career. You've, you know, been you told me you were like in nuclear weapons, R and D to supply chain sales and marketing.

Adam Larson: 01:05

You know, how does the path that varied eventually lead you to the world of A. I. Governance?

Clyde Calhoun: 01:11

Yeah. You know, the funny thing is that I have had the opportunity to do a lot of different things over my career and just that that work on the nuclear weapon side or being an inventor or a sociologist and working across a lot of different functions. But at the heart of it really goes to kind of a curiosity, the idea that I want to try a bunch of new things. And AI fits right in that space of something new, something exciting, being on the cutting edge. And so for me, that was the sweet spot.

Clyde Calhoun: 01:40

When I had the opportunity back in early twenty twenty three to play with one of the first versions of ChatGPT, I just got excited because I could immediately see how transformative the technology would be, plus as well how challenging it is for organizations to change. I knew there would be an opportunity to develop a business in that space.

Adam Larson: 02:02

Well, and it seems like a whole world of time away from the first version of ChatGPT, which was just a couple of years ago, which is kind of blows your mind just how quickly the technology is moving forward. You know, what was it in that moment that you that made you think there's a real problem here that I want that nobody's trying to solve yet?

Clyde Calhoun: 02:20

Yeah. Well, you know, I thought about it from the perspective of, yes, the technology is exciting and new because the things that you could do, even in those early versions of ChatGPT, were pretty impressive. But more importantly, the problem was an old one. And that being, how do you change? Because I recognize that AI was one of those things that was going to require change at a massive scale and at a pace that organizations are just not accustomed to.

Clyde Calhoun: 02:46

And after thirty years in the corporate space, the one thing that I knew is how hard it is, particularly for large organizations, to even make small changes, much less the magnitude of what we're seeing with the rise of AI. And I'll give organizations credit because I think even in the last year, most organizations have gotten on board at this point and are at least experimenting with AI, if not having rolled out some general LLMs across the board. But that creates a whole nother set of challenges for organizations as well. So it's the next level of problems that they need to solve. And of course, the pace at which they need to do that isn't slowing down anytime soon.

Adam Larson: 03:28

You know, when you and I were first talking, you made you made a distinction. You said, you know, there's a difference between governing AI tools and governing AI decisions. So, you know, at the surface, you know, when you first asked me, I was reflecting on it. I was like, wait a second, those sound the same, but I know they're not. So maybe we could talk a little bit about that.

Clyde Calhoun: 03:48

Yeah. And let me just kind of take a step back too, because I do think at this point, most organizations have figured out or are in the process of figuring out how to govern the tools, but very few have figured out how to govern the decisions that those tools are shaping. When I think about AI tools governance, I think about what systems are we going to have in the company? Who has the right to access those, how are we going to protect information and keep it safe. And maybe there's some policies that add on to that in terms of what are we going to do with suppliers or customers and what clauses aren't included in agreements.

Clyde Calhoun: 04:25

It's the rules around how we use the tools. But when I think about AI decision governance, it's how are we managing these decisions that are now being shaped by tools. And I'll give you a great example of that. When you think about pricing as a great example, if AI is making a recommendation that we increase pricing by 3%, do we accept that or do we not? Yes, ultimately a human is making the final call on that.

Clyde Calhoun: 04:53

But as we see in our personal life, people are becoming increasingly reliant on the recommendations from AI. So unless you've got guardrails in place to make sure that you're making the right decision, whether or not you agree with AI or whether or not you're overriding it, But you've got a consistent method of evaluating these decisions so that you're not just falling into the trap of blindly accepting a recommendation or doing something just based on your gut that could impact the company in a very negative way.

Adam Larson: 05:26

So let's let's dig into that a little bit, you know, because when you're thinking about business, a lot of times people are like, oh, what's your gut say? What does your gut feel? And more and more people are becoming more reliant on the AI to help them make those help them make decisions. You know, where is that? How do those guardrails get set up?

Adam Larson: 05:42

Because, you know, there's been studies that are happening that are saying that A. I. Like people aren't really more and more people aren't even checking the output from A. I. And just putting it out there in the world.

Clyde Calhoun: 05:52

Yeah, very true. And that's the challenge. And let me be clear too, because we've seen issues on both sides of the ledger. We've seen cases, high profile cases, where AI had the right call and humans overrode it. I'll just throw there was a fitness equipment manufacturer from, the pandemic days that you saw their commercials all the time.

Clyde Calhoun: 06:14

I'm not going to put the name out on blast, but they actually did a lot of investment in new capacity and inventory and everything like that, even though their machine learning programs or AI effectively were telling them that demand was really cooling and would cool rapidly. But they chose to avoid that and they paid a really hefty price for that the market. And on the flip side, you have other companies. There's a company in the home real estate space that made some huge bets where they trusted AI to make decisions on house valuations. And they got into a real pickle because they lost their shirt with bad recommendations from AI as the models couldn't evolve.

Clyde Calhoun: 07:03

So you see it on both sides. So what's important is that you're thoughtful upfront in terms of how are we going to make these decisions? What criteria do we need to see? How do we interrogate these recommendations? And how do we document that and improve the model over time so that we're getting better decisions as we go forward?

Clyde Calhoun: 07:23

And that's the thing that organizations really need to think about is that it's too easy to get in the trap of either going with your gut or just accepting the AI blindly. You've got to be more thoughtful in terms of how you make better decisions over time.

Adam Larson: 07:39

So let's talk about a little bit about, where AI is getting kind of baked into systems that you already use. You know, companies like SAP, Oracle, you can name hundreds of others that are especially for finance and accounting leaders, you know, that they're saying, oh, we have AI here. There's even ERP is that are saying that they're AI first or they're AI first in their systems. You know, we've had some of those folks come on this podcast and talk about what they're doing. What are your thoughts on, you know, how systems are using and how can you create your guardrails when you're not in charge of how the AI works within a platform you're using?

Clyde Calhoun: 08:11

Yeah, yeah. And that's a really interesting question because there are three layers that you need to think about in terms of AI in your organization. And like let's be clear, AI is running in your organization. Whether you formally put it there or not. You've got those deployed solutions that you've intentionally put out there.

Clyde Calhoun: 08:29

But Adam, to your point, increasingly, it's SAP or Salesforce or Zoho or so many other tools, AI is becoming embedded into the core functionality. And some of those things are controllable in terms of your version control. Some of those things are available and individuals in your company may be able to turn some of those features on. So they are running. And then on top of that, you also have shadow AI or off the books AI with individual employees using personal AI solutions at work.

Clyde Calhoun: 09:04

And the data's pretty stark on that. About six in 10 employees use or acknowledge to uploading sensitive company information into their personal LLM. So that use is pretty prevalent as well. So to get your arms around that, you first have to understand where are all the touch points for AI's influencing decisions in the organization. And that takes a lot of work.

Clyde Calhoun: 09:28

I mean, there's some real work up front, not just to look at the deployed and embedded AI, but also the off the books AI. And that's often where we get pulled in to help our clients as well, is that, again, employees often aren't that eager to share how they're using their off the books AI solutions with their leadership.

Adam Larson: 09:49

Okay, so that statistic is very startling, you know, with the six to 10 out of 10 in 10 employees upload sensitive company data. Now I can understand how that would happen. You especially you're trying it out. You're not you don't you might not necessarily know the effects of uploading your company's personal data in there, you know, but when when you're talking to CFOs and you're talking to senior leaders that hear that, what's their reaction when they hear that?

Clyde Calhoun: 10:12

Yeah. You know, it's funny because, there are multiple sources that will give similar statistics on that. But I think it's hard for a lot of senior leaders to accept that it's happening in their organization because they think that, well, hey, our IT department is really strong. I mean, we've got a policy in place. We've got great people.

Clyde Calhoun: 10:34

Surely not in our organization. And the reality is that it is happening out there. And I think it's one of those things that employees tend to rationalize it because they're being more productive in the work that they're doing. And the analogy that I often make is that you look at people on the highways. Yes, the speed limit is posted.

Clyde Calhoun: 10:56

The rules are very clear, but you know, five miles over the limit, I'm probably okay. 10 miles, I'm still probably okay. And there are a lot of us that will bend the rules a little bit because we think it is helping us to get where we're going a little bit faster. And we're thinking ultimately it's for the benefit of the organization. And I think the same thing is very true with AI.

Clyde Calhoun: 11:19

Employees are using these tools because they see real value in them. And they're thinking that, Hey, this information is not that sensitive. If it got out, maybe it should be all right. I don't think people are intentionally, you know, putting things out there that are compromising the company, but at the same time it is happening.

Adam Larson: 11:38

Yeah, I've I've had some folks on some philosophers on this podcast talking about fraud and things like that. And a lot of times it starts with just small, very small decisions. And well, I've always been I've always followed the rules, so it's okay if I do it just this once and that can kind of snowball over time to something that you weren't even planning to, you know, be that huge. You know, how are companies supposed to kind of put guardrails against that? Cause you're right.

Adam Larson: 12:02

You know, somebody will post something well, well that, you know, that's a stupid rule. I don't need to follow that, you know, but they don't really is the impact that it could have on our organization.

Clyde Calhoun: 12:11

Yeah. Yeah. And I think it's multi tiered because I think number one, they have to go back to understand, first of all, where is it happening and why is Because it that why piece is really important as well. Sometimes it's because, again, the tools that employees have access to at work are not the ones that they like or feel most accustomed to. And I'll use this one as a great example.

Clyde Calhoun: 12:39

Microsoft Copilot, a lot of organizations have rolled that out because, again, it fits in the Office family. It's very integrated. There's some great features with it. But at the same time, just being candid, it's not as user friendly as, say, ChatGPT or Claude. And if you're an employee and you've worked with either one of those tools in your personal life and you're not finding the same functionality or the same type of experience at work, it's natural for you to just pull up your phone or your personal laptop, do what you need to do, and move forward with the analysis there.

Clyde Calhoun: 13:14

So I think that understanding of where and why are critically important. And then the next thing I think along with that is creating a culture of decision control. And what I mean by that is, number one, you have to create the awareness of the risk for the organization and make sure that that's clear for employees. But you also have to give them frameworks that enable them to evaluate those inputs from AI and make the best call there as well. And then you also have to have the right level of accountability where you've got people in the loop to evaluate and then also document what's happening.

Clyde Calhoun: 13:53

So there's a tiered framework that you need to have in place to really get control of that. But it starts with the employees and what are they doing? Why are they doing it? And how can we enroll them in this thing that we're trying to do to keep control of information, but also to make the best decisions possible for the organization.

Adam Larson: 14:13

Well, yeah, and that it's it's a huge impact on the organization and and most internal control policies were written before AI was even a factor, you know, so companies kind of have to scramble and kind of, hey, let's get let's get all these things up to date. Then, but once you write it on a document doesn't mean it can become part of the culture. Then you have to start to ingrain it. And so it's going to take time, but it's like you don't have the time that it really needs to take to get it set in people's ways.

Clyde Calhoun: 14:41

Yeah. And that's the challenge too, because it does take some time to get these things set up. And I think there's been a rush to deploy the technology to the detriment of making sure that the controls are up to date to match. And when you think about it, a lot of organizations are scaling risk just as fast as they're scaling the technology piece. And it's really challenging because your exposure is huge.

Clyde Calhoun: 15:05

One of the things that Ernst and Young shared recently is that for enterprise level organizations, billion dollars up in revenue, about 64% of those organizations have already had at least one AI negative incident that costs them more than $1,000,000. And so that exposure piece is huge. The average AI negative incident is about $4,400,000 in terms of impact you know, for an organization. So you can't wait until after you deploy to govern. You have to govern as you deploy.

Adam Larson: 15:41

So do you think in those in those cases of those losses that you just mentioned, is it the A. Getting it wrong or is it the people who stopped looking at what the AI was telling them? You know, there are there any measures on that?

Clyde Calhoun: 15:53

Yeah, it's a combination. If you look at some of the leading causes for those types of negative impacts, it is that bad data and bad decisions from AI recommendations on either side. There are some cases where people overrode the AI, and there are some cases where they trusted the AI. But the impact was very real from basically bad decisions. And then the related piece is misuse of information as well, or information that wasn't factual that got out and caused a negative customer incident.

Clyde Calhoun: 16:27

There is a big four firm that recently had an issue in Australia. Again, not to name any names, but they had an incident where they were doing some work for the government and quoted some information from sources that didn't exist. And they had to give back a portion of their fees as a result, which was quite a big blow for organizations of that size. And so those are the types of things that can happen. The other major source for those penalties around AI is on the legal side, because we're seeing a rise regulation and then also AI litigation as well.

Clyde Calhoun: 17:12

You look at things like the EU AI Act that applies to any organization that either has employees in Europe or does business in Europe, which covers the vast majority of mid market and enterprise organizations. But the penalties for non compliance there can be up to €35,000,000 or 7% of global revenues, which again are quite significant in terms of teeth with compliance violations. And then you look at litigation doubling in 2025 versus 'twenty four related to AI. So all those things tie into these big negative consequences that can be out there if organizations don't get control of the decisions and the technology itself.

Adam Larson: 17:59

And that 7% can mean even much more greater the smaller the businesses. So let's say you have a smaller business that you're trying to grow and let's say you make that mistake, then that that's as much user, a much larger impact on your revenue. And so small businesses are even having to keep up, which is much more difficult because the costs and trying to keep up with that, you know, what are, what are your recommendations to small businesses who are trying to, trying to wade in this water, trying to keep up so they can, can be successful, but they're like, we don't have the capital to keep doing that.

Clyde Calhoun: 18:34

Yeah. And again, that's where I go back to this governance piece as being so important because I think the natural, reaction for a lot of small businesses is to say that, well, hey, because there's so many risks on AI, we're just not going to do anything in that space. And I think that's risky from two fronts. Number one, just falling behind with the technology because it does have the ability to scale and amplify the work that most organizations are doing in a way that nothing before it, think, has been able to do. So you risk falling behind competition in that sense.

Clyde Calhoun: 19:11

But then the second thing is that your employees are still going to use these tools. Whether or not you're formally deploying it, they're still going to use it on their phone. It was interesting because I worked with a healthcare organization and they actually had a policy against AI use at work. They actually had their Wi Fi configured so that employees couldn't access AI tools from Wi Fi. But when we were doing some assessment work, what we discovered is that employees were just putting their phone in cellular mode while they were doing what they needed to do on AI, and then they'd switch it back on.

Clyde Calhoun: 19:47

So that's the thing. Employees are using these tools, and you just can't put your head in the sand and hope that that it'll it will go away. You absolutely need to embrace putting the right controls framework in place to manage that risk, as best you can.

Adam Larson: 20:06

So maybe you can give us an example of what this governance actually looks like when it's actually working well, because, you know, we've been talking a lot of theory and saying, hey, this is what it can do, but I'm sure you have some examples of, hey, this is what it looks like when it's layered in and working well.

Clyde Calhoun: 20:19

Yeah. So just to give an example, you know, we, we recently worked with a technical services company. They were interested in putting improvement to their revenue forecasting process and better manage their, their deal pipeline. And so they were looking at integrating their SAP data with their CRM data. And then also we layered on some customer sentiment information on top of that from basically Teams calls that the sales reps were having with prospects and clients.

Clyde Calhoun: 20:51

And so all that information was coming together. That was the technology side. But in terms of the governance piece, we looked at, hey, what's the decision inventory that we need to have? So as you think about that whole process, there were several decisions that needed to be made based on these AI recommendations that were coming out. For example, do we move a deal to the next stage in the pipeline?

Clyde Calhoun: 21:15

Or do we pull forward the date on when we expect the deal to close? Or do we expect the magnitude or the expected value of the deal to change up or down over time? Those are just three decisions in there. So we looked at each one of those and we laid out basically a playbook for how we make those calls on that. So what is the objective criteria to move a deal from one phase to the other?

Clyde Calhoun: 21:41

Is it that we've given a proposal, we've had an affirmation that they want to move forward with some sort of working relationship? Is it that we've got now pricing in their hands? What's the criteria to move from one stage to the other? Then you look at what are the other criteria that the model is bringing into play to help shape the decision? Is it that, hey, they've had three calls in the last month?

Clyde Calhoun: 22:07

Is it that, hey, we've had new product orders coming in on other? So what are those other criteria that the model is shaping? And then you also look at what external context is out there. And for example, is there competitor activity that's not on the radar for AI? Or is there something else that's happening in the marketplace?

Clyde Calhoun: 22:30

And you fold all that into, now this is how you make your decision. And so you've got your playbook for how that works. And so it enables you to go through at every decision point and know how you're going to evaluate that. And then on the back end, we have a layered documentation process for that customer as well to help them know when they need to document and when they just need to trust what's in the system and roll with it. The key thing about this is that you don't want everything to be super cumbersome so that it slows you down.

Clyde Calhoun: 23:00

But you've got to have a right approach so that you're clear on what makes a big difference in terms of impact for the organization financially. And that's where you put your focus with the guardrails.

Adam Larson: 23:13

So when you and I were first talking, you had mentioned there's two kinds of AI errors. There's ones where the AI gets wrong, but there's also when the human may override the AI and that's wrong. You know, in the system that you were just describing, how do you create how do you catch both of those types of errors without creating even more red tape and slowing things down?

Clyde Calhoun: 23:32

Yeah. And so number one is just consistency for how you make the decisions. And so there is a decision criteria as part of that playbook that, okay, when we've got humans and AI disagreeing, here's how we make that call. And so for example, it may be that the sales manager makes the call on that. But whenever there is a conflict, we document it so that we know why that's taking place for those decisions that really matter.

Clyde Calhoun: 24:01

And so again, just to be clear, every decision is going to be wrong to some degree. Because it's either going to be wrong on the AI side or wrong on the human side. The intent is to make better decisions as you go forward. And by being consistent in the way that you make decisions and capturing what's important, you're able to improve the model over time and get better decisions over time as well.

Adam Larson: 24:25

So let's say somebody's listening to this conversation and they're thinking back and they're like, oh, wait, there was a decision that kind of went sideways, but it might have been an AI problem that they didn't recognize. What's that first step that they take to say, hey, let's let's go in the right direction now?

Clyde Calhoun: 24:39

Yeah, I think the first step is to try to go back and recreate exactly what happened in terms of the model assumptions and inputs. Now that becomes difficult for organizations. It's the reason why you want to have a defined documentation process upfront so that you don't have to go back and do that forensic work essentially. But that's the first step, is just trying to find out what happened, what were the assumptions, what were the humans thinking in terms of any context that was not considered in the model. So you can basically take that input and improve for the next time.

Clyde Calhoun: 25:14

But then you have to be forward looking as well. So there's not much you can do about the decision that's already happened. But there's a whole lot that you can do about what you do going forward. And I think the first step for any organization is just understanding what your exposure truly is. I mean, you have to know where AI is influencing things across your organization because you can't govern what you can't see.

Clyde Calhoun: 25:36

And so that's the first step for any organization.

Adam Larson: 25:40

That makes a lot of sense. And so if we kind of take a tall, a higher up step for, you know, from the 40,000 foot depth, you know, we're looking at a finance leader. We know what does that look like for the next thirty, ninety, thirty to ninety days as they're saying, hey, I need to get started on this journey. This is really important that I start moving this direction.

Clyde Calhoun: 25:59

Yeah, I think number one is just making the intentional effort to understand how AI is impacting your organization right now. One of the things that we do with clients is what we call an AI decision scan. It's just a thirty day look to go through not just your deployed AI, but your embedded AI and your off the books AI to give you that clear picture for all the different decisions, whether it's pricing or production planning or inventory management or M and A activity or cash forecasting. So many different areas where AI is making impact. But just getting clear about what those decisions are and what your exposure is.

Clyde Calhoun: 26:41

That's step one in the next thirty days. But then step two on that is getting clear about how you're going to manage that going forward, creating your framework for how you're going to govern decisions. Again, that's obviously in our wheelhouse. We help a lot of organizations figure that through. But whether you use external support or not, you have to get clear and be proactive in understanding not just where people should be in the loop, but what criteria they're going to use to interrogate these AI recommendations when they do and how they're going to validate model outputs.

Clyde Calhoun: 27:18

And then as well, how are you going to document your AI decisions as you go forward? That's the third piece that they need to figure out sooner rather than later. And if they do those things, and again, it doesn't have to be a Herculean effort. I mean, you can do those things to some degree and tamp down on your risk. Obviously, if you really want to be sure that you're putting yourself and your company in the best position possible to avoid some of those negative AI, influences, it's always good to seek, some expert counsel in that space as well.

Adam Larson: 27:53

So, love the advice that you just gave. And when we're thinking about the CFO, because they're the ultimate, you know, end all be all when it comes to the finance and accounting team and a lot of CFOs are having to get into IT, but somebody saying, hey, that's an IT problem. I don't have to deal with this. So what's your advice to the CFO who's listening to this?

Clyde Calhoun: 28:11

Yeah, I mean, I think CFOs absolutely have to own decision governance. And honestly, I believe that a lot of CFOs have deferred to the CIO or CTO because AI is viewed as a technology play. But in the end, this is all about business impact, whether you're looking at leveraging AI to grow the company or improve profitability. But more importantly, when you look at the risk of AI to the P and L, that sits squarely in the CFO's office in terms of putting the guardrails in place in that sense. And when you look at it from that point of view, I think this has to be front and center for every CFO.

Clyde Calhoun: 28:51

You can't let someone else in the organization drive what, yeah, is gonna impact ultimately the business financials that has to be owned in in your office.

Adam Larson: 29:01

Well, Clyde, I really appreciate you coming on the podcast, sharing your sage advice. I encourage everybody to check out Root Ideas website, the links, link in the show notes and, connect with Clyde on LinkedIn and just make sure that you, we're keeping this conversation going because it's something that all organizations are going to have to deal with as we continue to move forward. So thank you so much for coming on the podcast.

Clyde Calhoun: 29:20

A pleasure to be here, Adam, and thanks so much.

Announcer: 29:24

This has been Count Me In, IMA's podcast, providing you with the latest perspectives of thought leaders from the accounting and finance profession. If you like what you heard and you'd like to be counted in for more relevant accounting and finance education, visit IMA's website at www.imanet.org.