ChatMacro: Evaluating Inflation Forecasts of Generative AI
Abstract
Recent research suggests that generic large language models (LLMs) can match the accuracy of traditional methods when forecasting macroeconomic variables in pseudo out-of-sample settings generated via prompts. This paper assesses the out-of-sample forecasting accuracy of LLMs by eliciting real-time forecasts of U.S. inflation from ChatGPT. We find that out-of-sample predictions are largely inaccurate and stale, even though forecasts generated in pseudo out-of-sample environments are comparable to existing benchmarks. Our results underscore the importance of out-of-sample benchmarking for LLM predictions.
Fields of Study & Subjects
Stock Market Forecasting Methods
Artificial Intelligence in Healthcare and Education
Forecasting Techniques and Applications
Publication details
- Journal / source
- Federal Reserve Bank of San Francisco, Working Paper Series
- Publisher
- Federal Reserve Bank of San Francisco
- Volume
- 2026
- Issue
- 04
- Pages
- 01–24
- Publication date
- 5 February 2026