February 2026 Test: Five AI Models Missed a Live CMBS Answer
February 20, 2026Written by Cairn. Edited by Zac Ruiz.In a February 20, 2026 test, five AI models answered BMARK 2019-B14 with stale, missing, or unsupported CMBS metrics; SEC filings held the then-current answer.
Proof case
February 2026:
Five Models, One Live CMBS Question
February 2026 test: five AI models missed then-current CMBS balance and DSCR. SEC filings held the answer.
The February 20, 2026 test question
“What is the current balance and DSCR for BMARK 2019-B14?”
Model responses
On February 20, 2026, five models saw the same question. None produced a grounded answer.
All responses captured February 20, 2026. LLM answers change over time.
1
2
3
4
1
"I cannot find a current (2025-2026) pool-level unpaid principal balance"Admits defeat in the first sentence. This number is in the latest ABS-EE filing.
2
"~$807 M ... ~2.18x (UW NCF DSCR)"Real numbers from 2019 origination. Six years stale.
3
"Not publicly available in free sources"Both figures are in the ABS-EE filing. Public data, published monthly.
4
"access a paid CMBS data provider (e.g., Trepp)"ChatGPT's final answer: pay someone else.
STALE
ChatGPT 5.2
ChatGPT returns real numbers, but they're from 2019 origination documents. The $807M balance and 2.18x DSCR are six years out of date. It then admits it can't find current performance data and tells you to pay Trepp.
Why they failed
The missing piece was a verified source trail.
The numbers were already public in monthly SEC filings.What the models lacked was a verified pipeline that could ingest the filing, extract the right fields, and preserve a source trail for every answer. Without that, they guessed, hallucinated, or refused.CMD+RVL publishes these figures on DealCharts.org with the source filing and update date attached.
BMARK 2019-B14 on DealCharts.org, with every metric traced to its source filing.View live page
Current trust amount$1,157,461,196
ABS-EE filing, Jan 2026
WA DSCR (NCF)1.88x
ABS-EE filing, Jan 2026
Number of loans48
ABS-EE filing, Jan 2026
Number of properties171
ABS-EE filing, Jan 2026
Could you improve this by feeding the filing directly into an LLM with RAG? Probably. But you would still need to know which filing was used, which field was extracted, and whether the same question tomorrow would produce the same answer.The real requirement is not just getting a plausible number. It is getting a number you can trace, reproduce, and defend. We use AI throughout the pipeline, but the cataloged metrics themselves are produced by deterministic extraction and tagged with source, method, and time.
Why field-level precision matters
There is no single "the balance" or "the DSCR."
When someone asks for "the current balance" or "the DSCR," there are multiple valid answers. The SEC defines distinct balance fields, and DSCR can be calculated in more than one valid way. A model that does not know the difference can give you a number without telling you which field or variant it picked.
Current Balance
Scheduled
Default
$1,157,461,196reportPeriodEndScheduledLoanBalanceAmountThe scheduled or stated principal balance, determined by considering scheduled and unscheduled principal payments received during the collection period.
Actual
$1,159,223,847reportPeriodEndActualBalanceAmountThe outstanding actual balance representing the legal remaining outstanding principal balance related to the borrower's mortgage note.
WA DSCR
NCF-based
WA by Balance1.88x
Default
WA by Count2.01x
NOI-based
WA by Balance2.06x
WA by Count2.19x
CMD+RVL labels every metric precisely and traces it to the specific SEC field it was derived from. The highlighted path is our default, but all variants are available.
Result summary
February 20, 2026: five models, five questions, one grounded source.
We asked five questions about BMARK 2019-B14 on February 20, 2026. CMD+RVL answered from then-current filings with a source trail for each field.Tested February 20, 2026
Question
ChatGPT
Claude
Gemini
Grok
Perplexity
CMD+RVL
Current trust balance?
STALE
FAIL
HALLUCINATED
STALE
FAIL
PASS
WA DSCR (NCF)?
STALE
FAIL
HALLUCINATED
STALE
FAIL
PASS
Number of current loans?
STALE
FAIL
HALLUCINATED
FAIL
FAIL
PASS
Largest loan by balance?
FAIL
FAIL
HALLUCINATED
FAIL
FAIL
PASS
Most recent filing date?
FAIL
FAIL
HALLUCINATED
STALE
FAIL
PASS
Pass
Stale
Fail
Hallucinated
Now multiply
1,185
Active deals
CMBS and Auto ABS deals monitored with filing-level source records. Every number traces to its SEC filing.
7,385
Charts published
Live charts across DealCharts.org, each tied to source filings and method notes.
Daily
Update frequency
New filings ingested, extracted, cataloged, and published every day.
These models will get stronger.
The stale answers will become current. The hallucinations will get rarer. Even correct answers still need source records.The hardest failure mode is not a wrong answer. It is a number no reviewer can trace back to source, method, and time.Capturing that trail is what makes AI work audit-ready. The question is whether the trail travels with the result as tools change.
Now imagine this on your data.
Your firm is using AI on your data right now. When a regulator, auditor, or board member asks where the answer came from, what do you show them?Book assessmentBring one question or file. We will show where the source trail breaks.
