Auditors leveraging AI in productivity and fraud detection still need human validation, says Gies Business expert

“Technology is a double-edged sword,” says Andrea Rozario, assistant professor of accountancy at Gies College of Business.

Rozario, who studies artificial intelligence as it relates to auditing, is heavily versed in data analytics. She shares her expertise in the classroom and with her academic peers through research, including a recent study on artificial intelligence regulation in investing.

Gies Business is at the forefront in not only studying, but also adapting emerging technology in business endeavors. Professors like Robert Brunner, associate dean for innovation and chief disruption officer at Gies, are regularly called upon for their expertise in technologies, including AI.

On one hand, AI can aid in productivity when combined with human input and oversight. But it does have its limits and even a dark side.

On the positive side, Rozario points out that companies are using AI to produce a real-time dashboard to generate weekly predictions about sales and inventory, tools that internal auditors can leverage to monitor operational risks.

“That is useful from an operational perspective, which eventually bleeds into the financial statements,” she said. “That also promises to have a dynamic application to monitor and audit clients’ internal operations and financial statements, for example, in the future.”

Learning cautionary AI tales from industry

However, AI, when left unchecked, might just produce results that are more hallucinations than facts. Even larger, reputable companies are not immune. More alarming is the use of AI to commit fraud. Hackers are using vulnerabilities in large-language models like Claude to commit fraud.

“Security researchers have found weaknesses in AI tools like Claude that, in some cases, could let hackers trick the AI into revealing parts of a user’s private chat by using hidden instructions called prompt-injection attacks,” Rozario said.

Fingerprint writer Evelyn Chea notes that 41 percent of fraud attacks on companies are committed using AI, and companies are, in turn, using the technology to detect fraud. But there needs to be caution when it comes to trusting these reports. Rozario says that although AI is useful, it should only be used to narrow the possibility of fraud.

“We have to consider whether AI is producing a prediction that a human can use to justify their decision or not,” Rozario said. “There is a fair amount of research out there suggesting that these models can detect financial fraud compared to simpler methods, and there are a few academic papers that already show that we can use machine learning to detect patterns that would be very difficult to detect with conventional methods. Internal auditors, in particular, can use AI to continuously monitor transactions and flag suspicious patterns across large datasets, something that would be extremely difficult to do manually. However, fraud is extremely difficult to detect. It can be comprised of many factors.”

Although auditors are exploring AI for fraud predictions, they still don’t provide the reliability needed to actually detect fraud.

“I think a lot of it has to do with the lack of transparency of these models,” Rozario said. “Without being able to explain how the model comes up with that decision, the prediction is not that useful. It would ultimately be up to the human user to be able to validate that prediction.”

In that light, Rozario and others are turning to explainable artificial intelligence (XAI) – techniques widely used in computer science – and applying it to accounting, finance, and other business-related disciplines. For instance, auditors can use Shapley values, a technique grounded in cooperative game theory where a player is assigned a value based on how much each player contributes both positively and negatively, to help come up with a fraud prediction (players in this case are the factors that drive the fraud prediction).

“When a machine learning model predicts fraud (for example, giving a company an 85 percent fraud risk score), Shapley values break down how much each factor contributed to that prediction,” she said. “If there are 10 factors that can drive fraud - like unusual revenue patterns, or other accounting irregularities, Shapley values show which factors increased or decreased the fraud risk and by how much. This allows auditors to focus their attention on the most suspicious areas identified by the model. However, even with these fancy tools that allow us to look inside the black box, we still need to understand whether fraud is real or not. These explainability techniques show correlations, not necessarily causation. The human element is still important.”

AI could bypass critical steps in career growth

Rozario is concerned that using Gen AI can be a crutch and diminish critical thinking skills going forward, or even eliminate a critical career growth step. For instance, PWC believes AI will eventually do the work of junior accountants, so they are training them to be managers.

“It is difficult for me to foresee them jumping into the role of the audit manager because auditing is an apprenticeship model,” Rozario said. “So, if a lot of that doing goes away and employees are there to make higher-level decisions as soon as they graduate, I’m not sure how that translates to higher-quality financial statements.”

Whether used for financial forecasting or fraud detection, AI is an important tool that is becoming quicker and more accurate, but it’s not an exact science. It is a temptation for firms to eliminate human oversight, which can be risky.

However, as Gies Assistant Professor of Business Administration Eren Ahsen demonstrated, AI can be a tool that, when combined with human innovation, can have huge dividends. For instance, Ahsen discovered that through a “delegation” strategy, AI could aid human radiologists in cutting mammography costs by as much as 30 percent.

“The bottom line is we have to use these tools to enhance the way we work, but we have to question their output,” Rozario said. “In auditing, whether external financial audits or internal audit functions, there is always this big push for professional skepticism; with AI, that becomes even more important. External auditors add value to clients by understanding their financials, while internal auditors help management improve operations and controls. In both cases, we can’t delegate that work to AI without questioning its output."