Welcome back to Count Me In. I'm Adam Larson, and today I'm excited to share our conversation with one of our returning guests, Brian Hock, a true expert in professional exam prep for certifications like the CMA and CSCA. In this episode, I sit down with Brian to talk about how artificial intelligence is reshaping the way candidates prepare for certification exams. We explore everything from the real risk of relying too heavily on AI generated study tools to strategies for using AI in smart and efficient ways. Brian shares his perspective on where candidates can get off track with technology, why foundational knowledge is still key, and how AI can actually enhance accessibility for learners around the globe.

If you're curious about making the most of new learning technologies, or worried about common pitfalls, you won't want to miss Brian's advice on using AI as a tool, not a crutch. So let's jump right in and hear how to approach your exam prep with the right balance of innovation and trusted resources. Brian, welcome back to Count Me In. So always so great to have you.

Adam, it's great to be back. Thank you.

So today, we're gonna be talking about AI. And from your perspective, I think it's a great question to ask you because as a certification provide as a preparation provider, you're seeing a lot of different things. And maybe you can start by talking about the way what the how are professionals preparing in a new way with AI, and how's it reshaping, especially when you're preparing for the CMA, CSCA, everything else?

Well, part of the issue, way it's changing how people prepare is I think there's some people that are using AI in ways that aren't actually helpful to what it is that they're trying to do as they prepare for exam. They think AI can do things that it can't do, and in some cases, is doing more than they need it to. And so we get people that'll write and say, well, this is what I researched and this is what I found out, and it may be correct, but it's not necessarily relevant to the exam. It's not from the exam perspective. And so as we'll talk about, AI is a tremendously good tool.

It's a tremendously powerful tool for both students and the people that are doing the teaching, but it's a tool that if you don't use it properly is really you run the risk of going down a path that you don't need to go down and studying things that you don't need to study and also studying things that may not be correct.

So let's dig into that a little bit. Let's talk about some what are some of those risks, especially if you're relying too heavily on AI generated study? You know, it's funny when, like, it's nowadays people like, oh, I was talking to Chatty or I was talking to and, like, you weren't talking. You're just typing or you're

Right.

Verbally talking to an AI thing. It it seems to be more and more popular that people are just having normal conversations with their with their AI bots.

Yeah. Well, one of the risks is people talk about unlimited questions. I I you know, we we provide unlimited questions, and really all that is is having some AI tool generate questions out of either a textbook or the Internet, and sure, those are unlimited questions. Yes, you can sit down and ask for another question about accounts receivable and your tool will give you question after question after question. But from our standpoint in trying to use that tool as a materials provider, a lot of those questions are either wrong or so simplistic that they're not beneficial, or test things from a way that's not tested on the exam, test an element of a topic that isn't tested on the exam.

And so, when you just generate questions, random questions, and have an AI tool generate those questions, you really don't know whether or not those questions are helping you prepare for the exam. You may think you know the topic really, really well because you got all of those questions correct. But even when somebody who knows what they're doing is trying to get good questions out of AI, it's a challenge. And once you include a number in your question, you really you really can't trust that number. It might be right, but you absolutely have to check it and if you're studying, you don't know how to check it because that answer will sound great and its explanation will sound like the most confident thing you've ever read about why this answer is correct and how they calculated it, but it may not be correct.

It may not be following the standards, it may not be following the formula that it's supposed to be following. And so using AI to generate questions without having a very, very strong review process, you're really not learning anything that's gonna help because it's either too simple, not relevant, or wrong.

It seems like no matter how much we use AI as a tool, you still have to have the foundational knowledge and understand, like, understand all the concepts in order to use it as a tool properly.

Well, you need to know what to ask, and you need to know enough to evaluate whether or not its answer or what it's telling you is correct. And this is I know we're we're talking about exam prep here, but this is the same element connected to work and whether or not, you know, you study CMA, whether or not you're gonna have a job two years from now. Yes. AI has all the all this ability to to take data and manipulate data and do things to it, but somebody has to tell it what to do. You have to make certain that what it does is in line with what it is you're trying to do, and somebody ultimately has to make a decision.

And so that whole kind of way you have to use AI at work, all of those issues still exist just trying to prepare for an exam.

So we've talked about the risks. What are ways that you can use AI to actually help you if you're using it the proper way?

Our standpoint as a materials developer, there's a couple of ways that AI is really, really helpful. And one of them is, even if it can't calculate the answer correctly, it can generate the scenario. It can generate the question information. And one of the hardest parts about making a question is sitting down with, used to be a blank word document and starting ABC company produces a product, and you just have to think of what is the scenario, what are the amounts, what information needs to be included, what's a piece of information that I should include that's not relevant to the question? And then come up with the choices.

And it can't be for one correct answer in three random numbers. Those incorrect answers need to be kind of plausible. Either you add something instead of subtracting it or something reasonable, and so just generating the foundation of the question is difficult. With AI, it can generate that question. You may have to check it, and you may have to do all of the answers and confirm all of the answers, but it eliminates having to sit down and write the story.

Write the background of the question, come up with the numbers. And so it's a huge time saver just in the generation of the base of the questions that you're going to ask. And you can also have it do it, as as we know from anything we do in AI, have it generate And then you can just go through and and delete the 10 that are clearly not good questions or too simple, and you have a few that you can work with to get that those good questions that are going to be relevant and are going to be useful to people preparing the exam. We get all kinds of requests. People want more questions and want more questions.

We want more questions. Well, there's a limit. You don't need an unlimited number of questions because once you can calculate depreciation, once you know how to do the inventory calculations, you practice it a few times, and you're good. To do more questions about inventory, it's just variations on a theme. You change the numbers, but there's a process.

There's a template essentially you're using to answer those questions. And so an unlimited number of bad questions doesn't help. A few good questions that require you to think and require you to understand what it is you're doing in that topic are much, much more useful to a candidate than a bunch of questions that don't really challenge them.

That's interesting. You know, let's take a quick sidebar and talk about that, the amount of questions you need to study for a certification. Because, you know, I've I've heard both sides where, like, a smaller amount of questions but that are well done questions can help prepare you better than, you know, a thousand questions that may not be as good. So is there an in between in there where, like, hey. I need more questions because I feel like I wanna practice that more?

Because if you do the same question over and over again, you might know how to do it. You know?

Right. There is a balance, and it and for each candidate it's actually different. And even within that, for each topic for a candidate, it's different based on your experience, what you like, what you understand, what comes naturally, what doesn't come naturally to you. And so it's a matter of having enough questions that you can see the different variations there are, the different ways something is able to be presented. And if you understand the concept, not just I know that this is see because, or you recognize the three key words, but when you understand the concept, they can give you a slightly different question, and you're okay.

Because you have that foundation. It's when people focus on kind of a shortcut, or you begin to memorize the answers of the questions that you're doing, and not understanding what it is that you're actually doing. And a great example of this is internal controls. A lot of candidates hate internal controls because there's no calculations, there's no number, there's no formula, and it it comes down to what duties should be segregated, what duties can the same person do, what controls should the company have, and so there's a lot of kind of gray area in that. And even with AI, we could not create every possible question about internal controls that can be asked because every part of the business has internal controls, every transaction, every variation of a trans everything, there's almost literally an unlimited number of questions that can be made.

So we can't practice every question on some of those theoretical topics. But if we understand the theory, if we know what it is that we're actually discussing, what the question is actually about, if we know what duties need to be segregated, we're able to look at any of those unlimited questions that we get on the exam and sort it out ourselves, not having to remember, oh, in this case, not a memorization, but an understanding, and then applying that to that question. And so we need to have enough questions that we learn how to apply it. That, you know, one question's not enough because you might get lucky and think you understand it when you don't, or it's a bad question, and it's a difficult question. So enough questions to learn how to apply that concept to different situations, And then you're ready.

And the other thing too in terms of how many questions we need to do, maybe I'm not supposed to say this, but you don't have to get a 100% of every topic. And so if you if you look at a syllabus, you can't get a question about every line item on the syllabus. There's just not enough questions on the exam or there's too many lines on the syllabus. And so you know some things aren't going to be tested. And you know some things are going to be tested at a more basic level and there's going to be a few more challenging questions, but you don't know which topic those more challenging questions are going to come from.

And so if your mindset is, I want to go into the exam a 100% confident in a 100% of the topics, you're never going to take the exam. You're going to study for twenty years and you're still not gonna be 100% confident about 100% of the topics. And so it's a matter of feeling comfortable, I'm gonna get all the basic level questions correct from every topic. I'm gonna get most of the medium level questions correct, and I might get some of the difficult questions correct, but there aren't so many difficult questions that that's a problem. You know, we're not trying to get a 100% on the exam.

We're trying to pass. So that mindset of how many questions is enough, well, it it becomes you get enough questions long before AI has stopped finding questions to generate for you, and and you just just get lost and buried in all of the questions.

Yeah. So, you know, we've talked about, you know, creating the questions and and the amount. I really appreciate that insight because I I know that there's people who struggle with that a lot. But when you talk about, you know, AI with accessibility, because accessibility is a is a big deal because, you know, if you're in a different region and you may not be as familiar with the language or, you know, even sort of, you know, you know, color contrasting, you know, from, you know, from a visual standpoint, all those aspects. How do you see AI kinda contributing to helping with the globalization of our of our of our certification?

Well, it certainly makes it easier for people to get more information about a topic that they don't know or a term they don't know. In terms of the accessibility and computer display, that's more of a technological issue that I'm sure is is being solved, and and I know that we have tools that are available now for reading content and things like that. But one of the things that we've implemented is we have an an AI chatbot that you can access that uses our textbook to answer your questions. It doesn't go out into the whole world of the Internet. It just uses our textbook.

And so if you're going through continuing the segregation of duties, if you're reading that and you think, well, really don't not sure I understand that. You can ask it, please explain to me what the duties are that need to be segregated, and it will do it in a different way. And then you can ask it, please use a different example. But what's that's doing is it's using the benefits of AI and kind of that unlimited nature of asking questions, but asking questions within the context of the textbook. And when you do the context of the textbook as your source document, you know that every answer that it gives you is going to be correct.

And you know it's going to be relevant to the exam. And so we tell people, don't go into Google and type teach me this topic because you're hoping that the most popular Google site actually knows what they're talking about and they don't always. But within a textbook, if you have a good textbook that you trust and that you know is accurate and relevant to the exam, AI can re represent that to you. Present it again. Say it in a more basic language.

Say it in different language. And that is invaluable to students, especially students that don't have necessarily the same work experience or don't have as much work experience. It's a matter of getting it said in a different way. And I know from teaching, just saying it in a different way works for different people. The same words, kind of the same concept, but just looking at it from a different angle or using different language to explain it.

And that's what AI, when you use it within a controlled environment, is able to do for you. Take this topic, give it to me again. Try a different example, try a different way of explaining it, and that works. That that's a huge benefit. Definition.

So you're you're doing a question and you want a definition. Well, go into the textbook chat program and ask for a definition of that term from the CMA textbook. It's going to give you a CMA definition, not a general business definition or something like that, ratios. Want a formula for a ratio? On the CMA exam, there is a a way that the ratios are calculated.

In the real world, companies have different variations on it. So if you go out into the Internet and ask for this ratio, you're gonna get probably that ratio, but maybe not. But if you use AI within that controlled environment of what you know is correct, it's a great tool to answer questions, answer those short questions. This is kind of a little off topic, but I'm always amazed on different online chats and groups. People post something like, what time does a store open?

And I always wonder why do you post into a chat what time is a store open? Because in the amount of time you posted that, you could have searched that store and found out from the store what time they opened, not what time your neighbor thinks that they maybe opened the last time they were there two months ago. And the same with a question about a definition. If there's a basic question about what a definition means or what formula is, you can search that. You can use the AI tool within a textbook to get that answer very quickly that is still the answer that's relevant to the exam.

It's still relevant to what it is that you're studying. I mean, we're still happy to answer the question if you ask us, but sometimes in some of these questions, there is a faster way to get that answer and still know that it's correct.

Yeah. So when you're when you're thinking about as a professional and you're wanting to study, you know, so we've you've talked about, like, making sure that the AI tool is using the right source material. What are some other things that they should, like, think about when in when saying, I wanna utilize AI tools in a better way, more creative way for studying? So, like, you're making sure the source material is good. What are some things I should keep in mind maybe?

Well, one of the things is people want summaries. They want a shortcut. Mhmm. And they say, oh, the book is too long. Can you give me the three most important pages?

They say, well, no, I can't. And so what I think some people use AI for is they say, teach me process costing, but tell me in a paragraph, or make it really short or really simple. And there aren't shortcuts, and it's not something and this is something where I think AI is perhaps overused for exams. Yes, you have areas that you are better at and you're weaker at, but everybody in the world who takes the exam has to study the same content. And so whether you're strong or not, you're gonna have a question about fixed assets, you're gonna have a question about ratios, you're gonna have a question about overhead allocation.

And so there really isn't a personalized study plan. This isn't you for your career, and based on the job that you have right now, and what your promotion is going to be, these are the things that you need to know in your job. This is a standardized content. Everybody who takes it has to study the same things, and so there are no shortcuts. And even if you say, oh, well, I've worked for fixed assets for twenty five years.

I know everything about fixed assets. Great. You probably do. I still want you to go and do those fixed asset questions so you make certain you understand how fixed assets are taught or are covered on the exam. What's the terminology that's used?

Because maybe there's a different term than you've been using all of your career and you don't recognize what they're calling salvage value, or whatever the case may be. And so, when people talk about a personalized study plan for an exam, I understand if you're not good in a topic, you need to keep studying it. Once you understand a topic, you don't need to study it anymore, but you have to study every topic. You have to know every topic. And so the thing that is that I would say is don't try to use AI to make your exam study process shorter or to find the shortcuts or to find what it is that you need to know or what you can skip.

It's it's an exam. There's a standardized syllabus, and you need to know it. The only way to study it is to go through it. You know? The only way to be ready is to go through that whole content.

Now you can use AI as that tool to explain things, to get examples, whatever the case may be, but there isn't a magic way to short circuit or to make a shortcut for how much time it takes to learn those topics, other than if you've already learned them in the past, in which case you've already invested that time. Wasn't a shortcut. It's just you learned those topics in the past, and you don't have to spend that time now.

It's like in this case, AI isn't there to help you save time when studying.

No. It's there to help you learn the topics that you need to learn perhaps a little bit more efficiently.

Mhmm.

And and like I said at the beginning, if you if you start asking AI to give you questions, you're there for days before and it's still giving you questions, and you may think that you're doing something useful because, oh, I did another 500 questions today. Well, did you learn anything new? I mean, it's one of those things where sometimes I think people confuse, I studied with I learned something. Now if you study and you do the same type of question 50 times, you haven't learned anything. You've done that same question and maybe you're a little bit faster doing it, but you haven't learned anything.

And so using AI to help you feel like you've done something without actually learning more of the content for the exam is a risk.

So we've we've talked about these risks. And I guess, are there some pointers you can give folks to maybe help distinguish, like, hey. This is helpful AI generated content, and this this might leave me off track. Are there things they should look for?

Well, I think, really, it it comes down to where did the information come from. What was the source to that? And if the source is the Internet as a whole, it's probably correct. I mean, I know we say, well, it might be wrong if it's off of the Internet. Yes, it might be, but it's probably correct.

But it's probably not covering what it is you need to know for the exam in the way that you need to know it for the exam. Every topic on the exam can become an incredibly complicated process in the real world if we add, well, what if this, this, this, and this happens? And that's kind of what happens with AI. It can teach and present all these different scenarios about analysis or ratios or whatever the case may be, and maybe they're interesting, but if that's not the way ratios are tested on the exam, if that's not what you're going to have to do with the ratios, it isn't helpful to what it is that you're doing. So if the source of where you're getting that information is the exam textbook, it is off of the questions that you've been studying that you know are connected to the exam and what it is that you need to know, then that's fine.

You can have AI create those questions, you can have AI give examples, if you're confident that you know that the source is limited to what you need to know. The risk is you just study stuff that's not relevant to the exam. And like I said, might be interesting. It might be helpful at work, but for your exam time, you're not going to use that information because it's not relevant to the exam. Now, in terms of how should students use AI, one of the ways that students should use AI in the exam prep is passively and indirectly, and without knowing that they're doing it.

I don't want a student to go into whatever AI tool and say, give me 15 questions about process costing. I don't want a student to do that. What's gonna happen is I'm going to have done that, and I'm going to have checked those questions. And out of those 15 questions that it generated, only four of them are going to survive and make it into our database of questions. Because we don't need to calculate one thing 15 times, and we need to make certain that it is actually an accurate calculation and not an AI kind of accurate except for this one thing.

And so what you're looking for from your review materials or your course materials is that that company has used AI on your behalf to help get you the things that you need to study. Better questions, a few a handful of more questions. Not you don't need another 2,000 questions, but 200 good questions on good tab. So the use that students should really have for AI is is, like I said, passive or indirect. You're using it without necessarily knowing that you're using it.

It was used to help make the textbook a little clear in a topic that wasn't as clear as it could have been. The publisher may have used it to generate an example, or to take an example and make it a little bit more challenging, or add a second element to that example. That's not the student doesn't need to do that, but the person working on the material should be using those tools to give you better study tools, to help create the flashcards, to help create podcasts about the topic, or whatever the case may be. The student doesn't need to do that. The person writing the materials should be using AI to make the materials better, make it more efficient and effective for the student so that the student isn't the one going out to AI saying, give me 15 questions.

I appreciate that answer. When I when we first started talking about AI big time, especially, like, in the learning space, you know, there's the Elliot Mazie. He's, you know, he's the godfather of, like, elearning. He's been doing this stuff for a long time. He I remember him saying that, you know, you can't have AI without HI.

So you need the artificial intelligence, but you need that human intelligence, that human element. And I like that idea of, like, knowing where your content's coming knowing what the AI is coming from. Like, you've done all that work. You've done all that checking as a provider so the student can come in and say, I can trust what I'm getting from here because the provider has done that legwork for me. And that's kinda that that human element that is needed for especially when studying for such a certification and stuff.

Well, and give an example, you were talking about accessibility. And for CSCA, we actually created an audio version of our textbook. So there's an audiobook of our textbook, and maybe I'm not supposed to say these things, but it's not a person reading it, And it didn't take I mean, how long however many hours it takes to read to listen to that audio textbook, it didn't take that long for the audio textbook to be created. And so this is something where, you know, the the whole question about is a what what is AI going to do to jobs? Well, in this case, we would not have prepared an audio textbook if we had to pay somebody to read it and edit it, and then every time they coughed in the middle of it, we had to start over, we wouldn't have done that.

But with the tools that are available now to have it created where it sounds perfectly human. You would never guess it wasn't a person, and there aren't mistakes. Everything I mean, there aren't stumbles, and it that allows that tool to be created. No jobs were lost, but instead, there's a study tool that didn't exist. There's a way for somebody studying for CSEA to listen to the textbook.

They don't want to read it, they have trouble reading the screen, whatever the case, they're walking, they're running, they're whatever the case is, if they want to listen to it, instead of read it, that tool is now available. And so I think AI will do things like that where it will create help create tools, it will help create language variants. Now CMA and CSCA are only in English, and so there isn't an element of preparing materials, FMEA is available in Arabic. And so will these tools allow more will AI allow more study tools to be created in Arabic that will help candidates taking that exam in Arabic? Whereas, if it's translated by a person, there's a huge cost.

May not be any more accurate than if AI does it, but it will and and this comes back again to where the the provider is using AI so that the student doesn't have to. The student doesn't need to go out and say, translate all of this into Arabic. It's already been done and controlled and managed and presented rather than having the student go out and do all of these things themselves for the first time.

Yeah. Yeah. You know, last last time we chatted, we were talking about making sure that you make a real world application of your certification. Once you have it, you know, you wanna do it. And I think about, you know, can AI be used as a tool to kind of bridge that gap between certification and real world app like, can can we use AI as one of those tools?

Yeah. I think so. Part of it is AI helps you find examples, stories. I mean, mean, you used to be able to go into Google and search for give me an example of or or find things, but I think with there's more tools now where AI is able to get that information. I mean, one of the things we talk about in terms of experience is the only way you get twenty years of experience is if you work for twenty years.

But you can kinda short circuit that if you listen to other people, if you read other people's stories, and you learn from other people's experiences, you can kind of get some other people's experience prior to doing it all yourself. And I think AI is part of that. It makes some of these stories, some of these examples more available. You're able to learn from more people, and and and part of this, I'm not even sure if this is necessarily AI, but all the podcasts that are available. You know, everybody with a microphone has a podcast and telling their story, and some are better than others, as we know.

But listening to that, taking advantage of all of the information that's available to us now allows us to get that experience, kind of ask those questions about what's next in my career.

Mhmm.

I mean, you're talking the bridge between certification and real world. I mean, this isn't probably what we're trying to do with AI, but how do you make your resume more applicable to that job you're applying for? The AI tools that help you do that, help you rewrite something, help you polish something. Now this is a fine line because we don't wanna use AI to write our job application or our resume, but we can use AI to polish it, to find something that's missing. You know?

Here's here's the job requirements. Here's my CV. Is there something in my the job requirements that's not listed in my CV? Is there something in my CV that's specific to these job requirements that I should be certain to highlight? You know, it's a tool.

Yeah. You and this is I mean, you know this. You you read down LinkedIn. You know which post the person wrote and which post the person gave a prompt, and you read this huge prompt, you know, this huge article that just doesn't sound real. It doesn't sound human.

It it's AI without the HI element to it. And so one of the things that we need to keep in mind is that AI is it's a bridge. It's not a replacement. And it's a it's a tool, and this is the same thing. You know, when when you're working, AI is a tool.

You have to know how to use it. Five years ago, you needed to know how to use Excel. Well, you still need to know how to use Excel, but now you also need to know how to use AI. You need to know how to use AI. AI is not a replacement.

You know, if if you write a report that's entirely AI generated, your boss's first question should be, why do I need you? I can put in that prompt just as easily as you can. And so if you write the report and you ask AI what's missing, what is the question that my boss is gonna ask me when they read this report, and you then use that information to answer that question in your report, great. That's an absolutely wonderful use of AI. But to use AI to write the report, your boss is gonna know that and you're not needed.

You know, they have they have access to the same AI tool you do, and and so you have to add value to it, and that adding value is using AI to polish, to find the hole that's missing, whatever the case may be in your report, not to write the report itself. Awesome.

Well, Brian, it's always wonderful to have you on Count Me In. I love chatting with you about these things. Yeah. Absolutely. Thank you so much for coming on again.

My pleasure. Great to talk with you again.

has been Count Me In, IMA's podcast, providing you with the latest perspectives of thought leaders from the accounting and finance profession. If you like what you heard and you'd like to be counted in for more relevant accounting and finance education, visit IMA's website at www.imanet.org.