Energysuccess
How an Electric Utility Used AI to Prevent the Next Blackout
Risk ManagementPilot Discipline
← All Case Studies
Real Estatefailure
When AI Valuations Go Wrong: A Commercial Real Estate Cautionary Tale
National Commercial Real Estate Firm
$340M
overvaluation
14
bad Deals
$89M
loss Realized
11 months
timeline
The Challenge
A national commercial real estate firm managing $12B in assets wanted to accelerate deal flow by using AI to evaluate potential acquisitions. Human analysts could evaluate 3-4 properties per week; the AI system promised to screen 200+ properties daily, identifying undervalued opportunities faster than competitors. The firm licensed a model from a proptech startup that had impressive backtesting results showing 94% valuation accuracy on historical data. The firm's leadership, excited by the potential for competitive advantage, fast-tracked deployment.
The Approach
The AI valuation model was deployed across the firm's acquisition pipeline with minimal customization. Analysts were told to use AI valuations as "starting points" but were evaluated on deal volume—creating incentives to accept AI recommendations rather than challenge them. No systematic validation process compared AI valuations against independent appraisals. The model had been trained primarily on pre-2023 data and had limited exposure to the post-pandemic commercial real estate environment, where office occupancy patterns and remote work had fundamentally changed valuation dynamics. No monitoring system tracked whether the model's predictions matched actual market outcomes over time.
The Results
Over 11 months, the firm acquired 14 properties at prices the AI had flagged as "undervalued opportunities." Post-acquisition analysis revealed that 12 of the 14 were overvalued by an average of 24%—the AI had not adequately accounted for post-pandemic shifts in commercial occupancy, changing remote work patterns, and local market conditions that differed from training data. Total overvaluation exposure reached $340M. The firm realized $89M in actual losses on three properties that had to be sold at distressed prices. Two senior partners resigned. The AI model was pulled from production and the firm returned to human-led valuations with selective AI support.
Seven Pillar Insights
Risk Management
No independent validation of AI valuations against traditional appraisals removed the safety net that would have caught systematic overvaluation.
Continuous Evolution
The model was never updated to reflect post-pandemic commercial real estate dynamics, making it increasingly wrong with each passing month.
Pilot Discipline
Deploying across the entire acquisition pipeline without a controlled pilot meant the firm discovered the model's limitations through real financial losses.
Key Lessons
1
Backtesting accuracy on historical data does not guarantee performance in a changed market environment
2
Incentivizing deal volume while deploying AI valuations created a dangerous alignment failure—humans rubber-stamped AI rather than critically evaluating it
3
AI models trained on pre-disruption data are unreliable when market fundamentals have shifted
4
Without ongoing validation comparing predictions to outcomes, model degradation is invisible until losses accumulate
Related Case Studies
Ready to Avoid These Pitfalls?
Take the AI Leadership Assessment to identify your organization's strengths and vulnerabilities.
Want expert guidance on your AI strategy?
Schedule a consultation with Neil to explore how these lessons apply to your organization.
Schedule a Consultation