Don’t get caught in the ROI trap…

Return on investment (ROI) is one of the most commonly used frameworks for evaluating technology investments.  While it’s a great metric to use sometimes, other times relying on ROI leads to more problems than it solves.

Just for a quick refresher, ROI is used when organizations must choose among competing uses of capital and it compares the return generated by an investment to its cost.  For example, if a firm spends $1,000,000 developing a system that reduces expenses by $100,000 annually, the ROI would be 10%. If they invest $1,000,000 that generates $100,000 in income, it’s still an ROI of 10%.  I know that’s probably basic for most investment people, but I wanted to make sure we’re all clear on how I define it.

In reality, there are not just two discrete scenarios for ROI.  Most will fall somewhere in between the two ends of the spectrum.  And almost all initiatives will have many parts, some of which are easy to quantify and others that are not.  For example, the cost of a data subscription might be easy to quantify, but the time needed to properly process and engineer that data will be less clear.  And the insight/signal you’re able to extract from that data to help in the decision-making process will also likely be uncertain.   

A big part of this process is identifying what those parts are – every little piece of information, every function that will need to process that information, every step along the way – and figure out which of those parts you can measure well, which you can’t, and then identify the risk and uncertainty that you can use to determine whether to pursue that specific initiative.  If cost is high and uncertainty is also high, there’s a good amount of risk involved.  If cost is low and risk is high, perhaps it’s worth giving it a shot to see how it plays out. 

The more you can define and scope initiatives in detail and measure each part of that for risk/return/cost characteristics, the more informed your decision-making will be.  And the more effective your ROI calculations will be.

So when does ROI work really well? 

ROI is most appropriate for evaluating operational and automation-focused applications. These projects typically involve well-defined tasks with clear costs and benefits. For example, replacing manual data entry or accounting workflows with third-party software allows organizations to compare subscription costs with labor savings.  Similarly, industry-agnostic technologies - such as accounting systems or document processing tools - often have straightforward pricing and predictable efficiency gains. In these cases, ROI provides a useful decision-making framework because both inputs and outputs are relatively stable and measurable.  

Even when these systems involve significant upfront costs, they often follow a familiar pattern: a large initial investment followed by lower ongoing maintenance expenses.  This dynamic resembles real estate development, where development/construction costs are incurred upfront and operating costs are comparatively smaller once the property is stabilized.  So even if there is some uncertainty in these types of projects, that uncertainty is usually within a known range and can be constrained to certain steps in the process. 

So ROI works really well when you have known knowns - when you know the tasks, know the costs, and when there is little uncertainty in those costs. 

When does ROI cause more problems than it solves?

ROI becomes far less reliable when evaluating strategic capabilities or projects with more uncertainty.   These initiatives often involve uncertain timelines, requirements that change as you learn more about the systems and processes, and indirect benefits.  Costs may change as data gaps are discovered, technical challenges arise, and timelines are extended (which always happens).

Returns are even more difficult to quantify, as they may come about through improved decision-making, competitive advantage, or risk reduction rather than direct cost savings.  It’s easy to put a number on an automation process, but how do you put a number on the capability of improved market selection for higher growth over the hold the period?  How much more growth (direction of the market is easier to measure than the magnitude of that direction)?  More growth compared to what (current holdings or other markets/property types)?  Over what hold period (do you even know what the hold period will be)?  Does that growth come with higher risk as well? 

We could come up with thousands of scenarios involving long-term and uncertain decisions real estate executives must make on a daily basis, but our goal is not to build that comprehensive list.  It’s to say that better analysis will hopefully lead to better decisions, but often those “better” decisions are difficult to quantify, making a definitive ROI number difficult to capture.  So should these initiatives not be pursued because ROI is not known?  I don’t think so.

Another challenge in ROI evaluation is the presence of ambiguous or indirect benefits, particularly when external technologies support strategic initiatives. Data subscriptions, infrastructure tools, or analytics platforms often have well-defined costs but unclear individual contributions to overall performance when there are many components involved.  It may be difficult to isolate how much value a specific dataset adds to an investment decision or portfolio outcome.  You know the cost, but maybe not the value contribution. 

Why forcing an ROI is a bad idea

Executives often push teams for an ROI to make their decisions easier.  But this can backfire in a big way and we often hear frustration from teams about the tradeoff they’re faced with.  One scenario is that teams present the uncertainty in their ROI assumptions to executives.  Even a well-scoped project with well-explained uncertainties is often discouraged because it makes go/stop decisions about projects more difficult.  But this isn’t the teams’ fault.  This is one of the decisions executives need to make.  My opinion is that executives don’t like this ambiguity because they don’t know how to properly evaluate those tradeoffs and uncertainties, so they default to a “no.”  Now, I don’t blame them for not liking the ambiguity, but if they want teams to be honest about risk and return potential, they need to be willing to evaluate it rigorously.

The second scenario is that teams come up with an ROI number to make the presentation to executives easier.  Given the number of factors and ambiguity inherent in many of them, they know uncertainties exist and they know it’s difficult to quantify many initiatives because of those uncertainties.  But they also know that if they pursue scenario one above the answer will likely be a “no.”  So they do their best to estimate the risk and returns associated with an initiative with “an ROI.” 

But this approach usually leads to rigid expectations, along with set budgets and timelines, that are not appropriate given the uncertainties involved and don’t accommodate the needed experimentation or learning curves that are inherent in pushing new capabilities.  If timelines get pushed out or budgets grow, teams are often met with negative reactions from the executive team. 

Going forward

A hammer doesn’t solve every problem.  A rigid reliance on ROI can lead to bad decisions and the rejection of really good ideas that could have provided valuable new capabilities. 

ROI is frequently applied too simplistically, particularly in the context of artificial intelligence.  In reality, ROI involves multiple components beyond direct cost savings, including risk, lifetime value, opportunity cost, and strategic/competitive positioning.  Some of these elements are measurable while others are intangible, making ROI analysis more nuanced than it initially appears, especially for initiatives that are more strategic in nature. 

Despite this ambiguity, companies must avoid rejecting investments solely because ROI cannot be precisely calculated.  In many cases, the cost of inaction - failing to develop necessary infrastructure - may outweigh the uncertainty in projected returns.  Many of these components may still be essential for building a broader capability and there should be an acceptance, rather than intolerance, of this ambiguity.  Ultimately, ROI should be treated as one tool among many, applied selectively where appropriate and supplemented with strategic judgment when evaluating complex AI investments.