VMSI™ Validation Record
Forward Testing, Audit History & Prospective Validation
Record established: September 21, 2026
Baseline study: Markets Become More Predictable at Scale
Baseline published: September 4, 2026
Next scheduled audit: November 6, 2026
VICA Partners maintains a continuing validation record for the VICA Market State Index™.
The objective is straightforward: preserve the original evidence, test subsequent observations prospectively, disclose contradictory results, and determine whether institutional-state information continues to add measurable information as the record expands.
Previously validated observations are not rewritten because subsequent market outcomes become known. Corrections are permitted only for documented factual, mathematical, source, transcription, or methodology-implementation errors.
The purpose of this record is not to protect a prior result.
It is to determine whether the result survives.
Audit 001 — Initial Forward-Information Test
The initial experiment tested a narrow empirical question:
Does measured institutional state contain incremental information about subsequent market outcomes?
The test examined subsequent 20-trading-day S&P 500 price returns across 34 availability-aware out-of-sample forecast origins from October 2, 2025 through July 31, 2026.
A conventional specification incorporating rates, yield-curve structure, volatility, credit, participation and momentum was compared with the same framework after adding VMSI™ as an institutional-state variable.
Core Results
| Measure | Conventional Set | + VMSI™ | Result |
|---|---|---|---|
| MAE | 3.845 | 3.441 | 10.5% lower |
| RMSE | 5.200 | 4.447 | 14.5% lower |
| OOS R² vs. expanding mean | −0.539 | −0.126 | +0.414 |
Adding VMSI™ reduced aggregate forecast error on both primary error measures.
The result was not based on a majority of individual forecast wins. VMSI produced lower absolute error in 47.1% of forecast origins. Its aggregate advantage therefore came from the magnitude and distribution of forecast errors rather than from winning most observations.
That distinction matters.
Robustness Audit
The aggregate result was subjected to additional tests designed to examine overlap, concentration, timing and dependence.
| Validation Test | Result |
|---|---|
| Overlap-adjusted Clark-West test | Favorable direction; one-sided p = 0.088 at HAC lag 3 and 0.098 at lag 4 |
| Forecast-encompassing regression | Coefficient 2.169; t = 4.286; one-sided p = 0.000078; R² = 33.7% |
| Every-fourth-origin sequences | 4 of 4 improved both MAE and RMSE |
| Every-fifth-origin sequences | 5 of 5 improved both MAE and RMSE |
| Leave-one-origin-out | Improvement remained in 34 of 34 exclusions |
| Remove four consecutive origins | Improvement remained in 31 of 31 tests |
| Circular timing test | Actual alignment ranked 1st of 34; empirical p = 0.0294 |
| Block-order timing test | MAE p = 0.0182; RMSE p = 0.0133 |
The robustness tests indicate that the aggregate improvement was not explained by one forecast origin, one contiguous group of observations, or arbitrary temporal alignment.
They do not eliminate model risk.
Temporal Stability
The result was not uniform through time.
During the first half of the evaluation period, adding VMSI™ worsened MAE by approximately 4.7% and RMSE by approximately 2.3%.
During the second half, adding VMSI™ improved MAE by approximately 19.8% and RMSE by approximately 25.3%.
This variation remains part of the record.
Possible explanations include regime dependence, expanding-sample effects, structural change and sampling variation. The present evidence does not establish which explanation controls.
The result will therefore be evaluated prospectively rather than interpreted retrospectively.
What Audit 001 Does Not Establish
Audit 001 does not establish that VMSI™ will improve every forecast, every regime, every asset or every model.
Absolute out-of-sample R² remained negative after VMSI was added.
The primary overlap-adjusted statistical comparison remained favorable but did not reach the conventional 5% significance threshold.
The initial sample is limited and contains overlapping 20-trading-day forecast windows.
Independent prospective confirmation across a larger record remains outstanding.
This validation record reports the verified forecast-level results and associated statistical robustness testing. It should not be interpreted as a complete independent reconstruction of every upstream model-implementation detail.
Audit 001 Conclusion
The evidence supports a bounded conclusion:
Within the tested specification, the VMSI™ institutional-state representation contained incremental forward information and reduced aggregate out-of-sample forecast error.
The result survived multiple reduced-overlap, concentration and temporal-alignment challenges.
It does not establish universal forecasting superiority.
It establishes a result that warrants a harder prospective test.
Next Scheduled Audit — November 6, 2026
Audit 002 — Prospective Holdout Validation
Audit 002 will evaluate the prospective record accumulated after publication of Markets Become More Predictable at Scale.
Unlike the initial study, this audit will focus specifically on observations evaluated after the baseline result was publicly established.
Only forecast origins whose full 20-trading-day outcomes have matured by the audit cutoff will be eligible for evaluation.
The sequence will remain:
Original locked forecast → matured subsequent outcome → conventional-model error → +VMSI™ error → aggregate and robustness evidence
The original VMSI™ readings will remain unchanged.
The target horizon will remain unchanged.
The comparison framework will remain unchanged.
No parameter, benchmark, variable definition or evaluation rule may be altered because subsequent outcomes are known.
VMSI™ Validation Standard
Every future audit in this record will follow the same governing principles.
1. Prior results remain immutable.
Previously published forecasts, results and conclusions remain part of the record. Any correction requires a dated explanation.
2. No look-ahead is permitted.
Only information available at the relevant forecast origin may enter an evaluation.
3. Methodology remains fixed within a test.
Targets, availability rules, benchmark construction, evaluation metrics and testing procedures cannot be changed after outcomes are known.
4. Prospective and historical evidence remain separate.
Reconstructed, original and prospective observations may be compared but will not be blended in a way that obscures their status.
5. Contradictory evidence is retained.
Periods of underperformance, failed significance tests, instability and other adverse results remain part of the published record.
6. The same core scorecard is maintained.
Future audits will report forecast count, evaluation period, MAE, RMSE, OOS R², individual-origin performance, dependence-aware testing, temporal stability and concentration analysis where applicable.
7. Calculations use source precision.
Published results may be rounded for readability, but all calculations are performed using the underlying full-precision record.
8. Source provenance precedes interpretation.
Each audit must be traceable to a defined forecast-level record, aggregate result set and documented methodology.
9. No retrospective optimization is permitted.
Historical observations or methodology-normalized data may not be altered to improve empirical performance.
10. Evidence controls the conclusion.
If subsequent evidence weakens, reverses or contradicts the initial result, the validation record will state that directly.
The Standard Going Forward
VMSI™ was developed to measure institutional market state.
Forward testing asks a separate question:
Does that measured state contain information about what follows?
Audit 001 provides initial evidence that it can.
Audit 002 begins the more demanding test:
Will that information persist after the original result is already known?
That is the purpose of the continuing validation record.
Evidence before narrative.
Measurement before interpretation.
The market remains the final test.