Blog · AI strategy

Is your AI paying off? A practical way to measure the return

A new framework from UTS and KPMG explains why most organisations struggle to measure the return on AI, and what a smaller business can do about it.

In September 2026, the Human Technology Institute at the University of Technology Sydney published a practical framework for assessing the return on AI investments, written with KPMG Australia. It is aimed at boards and executives, but most of it applies just as well to a business of twenty people.

The report starts from a simple observation: most organisations are not measuring the return on AI well. The authors argue that this is not for lack of effort or skill. The methods most businesses use were designed for traditional software, which behaves the same way every time, and AI does not.

Feeling faster is not the same as being faster

One study the report cites makes the point clearly. In a 2025 trial, experienced software developers using AI coding assistants believed they were working about 20% faster. When their work was measured, they were 19% slower. If the only measure you use is whether your team feels that AI is helping, you may get a confident answer that turns out to be wrong.

The four steps

The framework has four steps, summarised here:

  • Measure one specific use at a time. A goal such as “improve productivity” is too vague to measure. The report suggests a target like reducing the time a contract review takes from four hours to 90 minutes.
  • Set expectations to suit the stage you are at. A short trial, getting AI working inside real workflows, and rolling it out across the business each produce different returns, and results from a trial often do not hold once the tool is in wider use.
  • Count the indirect benefits as well as the direct ones. Time saved is the obvious one. The report also points to better data, stronger AI skills across the team, better risk management, and improvements for staff and customers.
  • Count all the costs. Licences are the visible part. Training, redesigning workflows, the dip in productivity while people adjust, and governance all belong in the calculation. The report says change management is consistently the biggest factor in whether AI delivers the return that was expected.

Three assumptions worth testing

The report names three factors that decide whether the expected benefits actually arrive:

  • Adoption: how many of the people who could use it actually do. The report notes that support from direct managers matters more than endorsement from the top.
  • Translation efficiency: whether time saved by one person turns into value for the business, or gets lost in handovers, approvals and rework.
  • Accuracy and performance: whether the output is reliable and consistent enough to depend on.

A worked example worth reading

The report includes a hypothetical 150-person marketing firm rolling out Microsoft Copilot to speed up proposal writing. On paper, a 15% time saving is worth $216,000 a year. Once the firm assumes that half its staff will use it in the first year, and that only 70% of each person's time saving will reach the business, the expected benefit falls to about $75,600 against a first-year cost of around $106,000.

The firm goes ahead anyway, because it values the skills and governance it is building, but it does so with realistic numbers and a plan to revisit them. The same arithmetic applies whichever AI platform a business uses.

Where rework fits in

Two of the report's ideas line up closely with the problem an AI Brain is designed to solve. The first is translation efficiency. When AI produces a draft that has to be heavily rewritten before anyone can use it, much of the time saving disappears before it reaches the business. The second is consistency. If each person gets a different answer from AI because it has no shared source to work from, the reliable output the report calls for is hard to achieve.

Both problems usually have the same cause: the AI does not have the business's own rules, processes and standards to work from. In one of our construction case studies, giving the AI the business's own quoting knowledge cut the time to produce a draft quote from up to four hours to under ten minutes.

If you are a smaller business, start here

The report recognises that not every organisation can apply the full framework straight away. For mid-sized firms, and for those earlier in their use of AI, it suggests three disciplines:

  • Decide what specific improvement each use of AI should deliver, and how you will measure it, before you start.
  • Track who is actually using it and how consistently. If use is lower than expected, find out why before spending more.
  • Review the costs and benefits as you go, adjust your assumptions as you learn, and be willing to stop if the benefits are not there.

The report also ends with a useful list of questions for management and boards to ask before and after investing in AI. It is worth reading in full.

Source: Human Technology Institute, University of Technology Sydney, with KPMG Australia, AI ROI: A practical framework for assessing the return on AI investments (September 2026).

Not sure where to start?

Pick one task and measure it properly.

The discovery call is free and takes thirty minutes. Bring one task your team repeats every week, and we will talk through what it costs you now and what could change.