VICA PARTNERS / SCIENTIFIC RESEARCH
VMSI™ Scientific Validation Record
Forward Testing, Audit History & Cumulative Evidence
The VMSI™ Scientific Validation Record is VICA Partners’ permanent, cumulative research publication documenting the empirical evaluation of its market-state framework. It distinguishes retrospective findings from forward testing, preserves the original results and limitations of each completed audit, and records subsequent evidence, methodological disclosures, and dated revisions. Its purpose is to establish a transparent, reproducible record of what the research supports, what remains unproven, and how conclusions evolve as new evidence becomes available.
Institutional reading brief / November 2026
What VMSI™ could add to a capital decision
VMSI™ organizes market-state evidence from liquidity, momentum, hedging and safe-haven behavior. For allocators and underwriters, the practical test is whether this information improves risk-budget, funding or underwriting decisions beyond conventional rates, credit, volatility and momentum inputs. The historical comparison reduced aggregate forecast error; Audit 002 identified significant counterevidence and left prospective superiority unproven.
Where an institutional evaluation could matter
These are potential evaluation uses, not demonstrated investment outcomes or documented economic alpha. Institutional evaluation framework
Cumulative Validation Scorecard
This scorecard consolidates the results of completed VMSI™ validation audits, distinguishing retrospective evidence from forward testing. All reported findings retain their original measurement periods, statistical qualifications, and limitations. Cumulative assessments are updated only when new audit results have been completed and verified.
Illustrates the original historical model comparison only. Errors are measured in return percentage points for subsequent 20-trading-day S&P 500 price returns. Both models had negative out-of-sample R² against the expanding mean; these bars are not new Audit 002 prospective results.
Primary source values and full limitations remain in the unaltered historical record below.
Scroll horizontally to view all columns
| Measure | Audit 001 | Audit 002 | Cumulative Evidence |
|---|---|---|---|
Evaluation Period & Sample | 34 historical forecast origins; Oct. 2, 2025–July 31, 2026 | 5 calendar-aligned outcomes; 4 publication-aligned matured outcomes. No certified new paired forecasts. | 34 original paired observations; prospective replication remains unverified. |
| Primary Empirical Finding | Historical MAE reduced 10.5% (3.845 to 3.441); RMSE reduced 14.5% (5.200 to 4.447). | Historical results reproduced. No new prospective model superiority established; performance concentration identified. | Historical incremental improvement documented; independent prospective confirmation remains outstanding. |
| Statistical Robustness & Limitations | Clark–West one-sided p=0.088–0.098 (HAC). Favorable robustness checks, but 34 overlapping forecast origins limit statistical inference. | Six historical origins contributed 124% of net MAE improvement; removing them reverses the advantage. Simpler benchmarks also outperformed the enhanced model. | Historical findings remain conditional. Concentrated gains, competing benchmarks, and absent certified prospective paired forecasts limit conclusions. |
| Forward-Testing Status | Historical out-of-sample evaluation covering 34 forecast origins. Not an independent post-publication prospective test. | Prospective paired-model test not certified. Original pre-outcome conventional and VMSI-enhanced numeric forecasts unavailable. | Historical evidence reproduced; independent prospective superiority not established. Forward testing remains open. |
Scheduled Validation & Audit Cycle
VICA Partners maintains a structured scientific validation program consisting of 30-day monitoring, 60-day internal research reviews, and formal cumulative validation audits approximately every 90 days.
Each completed formal audit is appended chronologically to the permanent VMSI™ Scientific Validation Record. Original findings, statistical limitations, contradictory evidence, and dated revisions are preserved without retrospective alteration.
An expanded annual assessment evaluates longitudinal robustness, market-regime consistency, and the durability of observed relationships.
Next Formal Audit: Approximately 90 days following the verified completion date of Audit 002, subject to adequate new evidence and data quality.
Audit History & Cumulative Evidence
Audit 001 — September 6, 2026
Audit conducted: September 6, 2026
Validation Record first published: September 21, 2026
Baseline study published: September 4, 2026
Forward Testing, Audit History & Prospective Validation
Record established: September 21, 2026
Baseline study: Markets Become More Predictable at Scale
Baseline published: September 4, 2026
Next scheduled audit: November 6, 2026
VICA Partners maintains a continuing validation record for the VICA Market State Index™.
The objective is straightforward: preserve the original evidence, test subsequent observations prospectively, disclose contradictory results, and determine whether institutional-state information continues to add measurable information as the record expands.
Previously validated observations are not rewritten because subsequent market outcomes become known. Corrections are permitted only for documented factual, mathematical, source, transcription, or methodology-implementation errors.
The purpose of this record is not to protect a prior result.
It is to determine whether the result survives.
Audit 001 — Initial Forward-Information Test
The initial experiment tested a narrow empirical question:
Does measured institutional state contain incremental information about subsequent market outcomes?
The test examined subsequent 20-trading-day S&P 500 price returns across 34 availability-aware out-of-sample forecast origins from October 2, 2025 through July 31, 2026.
A conventional specification incorporating rates, yield-curve structure, volatility, credit, participation and momentum was compared with the same framework after adding VMSI™ as an institutional-state variable.
Core Results
Scroll horizontally to view all columns
| Measure | Conventional Set | + VMSI™ | Result |
|---|---|---|---|
| MAE | 3.845 | 3.441 | 10.5% lower |
| RMSE | 5.200 | 4.447 | 14.5% lower |
| OOS R² vs. expanding mean | −0.539 | −0.126 | +0.414 |
Adding VMSI™ reduced aggregate forecast error on both primary error measures.
The result was not based on a majority of individual forecast wins. VMSI produced lower absolute error in 47.1% of forecast origins. Its aggregate advantage therefore came from the magnitude and distribution of forecast errors rather than from winning most observations.
That distinction matters.
Robustness Audit
The aggregate result was subjected to additional tests designed to examine overlap, concentration, timing and dependence.
Scroll horizontally to view all columns
| Validation Test | Result |
|---|---|
| Overlap-adjusted Clark-West test | Favorable direction; one-sided p = 0.088 at HAC lag 3 and 0.098 at lag 4 |
| Forecast-encompassing regression | Coefficient 2.169; t = 4.286; one-sided p = 0.000078; R² = 33.7% |
| Every-fourth-origin sequences | 4 of 4 improved both MAE and RMSE |
| Every-fifth-origin sequences | 5 of 5 improved both MAE and RMSE |
| Leave-one-origin-out | Improvement remained in 34 of 34 exclusions |
| Remove four consecutive origins | Improvement remained in 31 of 31 tests |
| Circular timing test | Actual alignment ranked 1st of 34; empirical p = 0.0294 |
| Block-order timing test | MAE p = 0.0182; RMSE p = 0.0133 |
The robustness tests indicate that the aggregate improvement was not explained by one forecast origin, one contiguous group of observations, or arbitrary temporal alignment.
They do not eliminate model risk.
Temporal Stability
The result was not uniform through time.
During the first half of the evaluation period, adding VMSI™ worsened MAE by approximately 4.7% and RMSE by approximately 2.3%.
During the second half, adding VMSI™ improved MAE by approximately 19.8% and RMSE by approximately 25.3%.
This variation remains part of the record.
Possible explanations include regime dependence, expanding-sample effects, structural change and sampling variation. The present evidence does not establish which explanation controls.
The result will therefore be evaluated prospectively rather than interpreted retrospectively.
What Audit 001 Does Not Establish
Audit 001 does not establish that VMSI™ will improve every forecast, every regime, every asset or every model.
Absolute out-of-sample R² remained negative after VMSI was added.
The primary overlap-adjusted statistical comparison remained favorable but did not reach the conventional 5% significance threshold.
The initial sample is limited and contains overlapping 20-trading-day forecast windows.
Independent prospective confirmation across a larger record remains outstanding.
This validation record reports the verified forecast-level results and associated statistical robustness testing. It should not be interpreted as a complete independent reconstruction of every upstream model-implementation detail.
Audit 001 Conclusion
The evidence supports a bounded conclusion:
Within the tested specification, the VMSI™ institutional-state representation contained incremental forward information and reduced aggregate out-of-sample forecast error.
The result survived multiple reduced-overlap, concentration and temporal-alignment challenges.
It does not establish universal forecasting superiority.
It establishes a result that warrants a harder prospective test.
Next Scheduled Audit — November 6, 2026
Audit 002 — Prospective Holdout Validation
Audit 002 will evaluate the prospective record accumulated after publication of Markets Become More Predictable at Scale.
Unlike the initial study, this audit will focus specifically on observations evaluated after the baseline result was publicly established.
Only forecast origins whose full 20-trading-day outcomes have matured by the audit cutoff will be eligible for evaluation.
The sequence will remain:
Original locked forecast → matured subsequent outcome → conventional-model error → +VMSI™ error → aggregate and robustness evidence
The original VMSI™ readings will remain unchanged.
The target horizon will remain unchanged.
The comparison framework will remain unchanged.
No parameter, benchmark, variable definition or evaluation rule may be altered because subsequent outcomes are known.
VMSI™ Validation Standard
Every future audit in this record will follow the same governing principles.
1. Prior results remain immutable.
Previously published forecasts, results and conclusions remain part of the record. Any correction requires a dated explanation.
2. No look-ahead is permitted.
Only information available at the relevant forecast origin may enter an evaluation.
3. Methodology remains fixed within a test.
Targets, availability rules, benchmark construction, evaluation metrics and testing procedures cannot be changed after outcomes are known.
4. Prospective and historical evidence remain separate.
Reconstructed, original and prospective observations may be compared but will not be blended in a way that obscures their status.
5. Contradictory evidence is retained.
Periods of underperformance, failed significance tests, instability and other adverse results remain part of the published record.
6. The same core scorecard is maintained.
Future audits will report forecast count, evaluation period, MAE, RMSE, OOS R², individual-origin performance, dependence-aware testing, temporal stability and concentration analysis where applicable.
7. Calculations use source precision.
Published results may be rounded for readability, but all calculations are performed using the underlying full-precision record.
8. Source provenance precedes interpretation.
Each audit must be traceable to a defined forecast-level record, aggregate result set and documented methodology.
9. No retrospective optimization is permitted.
Historical observations or methodology-normalized data may not be altered to improve empirical performance.
10. Evidence controls the conclusion.
If subsequent evidence weakens, reverses or contradicts the initial result, the validation record will state that directly.
The Standard Going Forward
VMSI™ was developed to measure institutional market state.
Forward testing asks a separate question:
Does that measured state contain information about what follows?
Audit 001 provides initial evidence that it can.
Audit 002 begins the more demanding test:
Will that information persist after the original result is already known?
That is the purpose of the continuing validation record.
Evidence before narrative.
Measurement before interpretation.
The market remains the final test.
Audit 002 — November 10, 2026
Audit cutoff: November 6, 2026
Research review: November 10, 2026
Status: Partial evidence review — prospective paired-model forecast test not certified.
Executive Finding
Audit 001 can be reproduced using its original forecast-level records. For Audit 002, the weekly VMSI market states and subsequent S&P 500 closes are observable, but the original conventional and enhanced numeric forecasts locked before each outcome have not been located. Market-state readings alone do not supply the paired forecasts. Therefore, the original experiment cannot yet be repeated, and no new MAE/RMSE superiority or prospective statistical significance is claimed.
A material timing finding strengthens the audit: weekly reports dated Fridays were published publicly on subsequent Sundays. Using their Friday closes as predictive origins without separately timestamped Friday forecast locks would introduce information-timing risk. Five calendar-aligned outcomes are observable; only four conservatively publication-aligned 20-session outcomes had matured by November 6. Neither series constitutes the original two-model experiment.
Original Locked Experiment — Historical Results Verified
Audit 002 reproduced the original Audit 001 experimental results using the unchanged forecast-level record, test3_predictions.csv, crosschecked against test3_results.csv.
The historical experiment comprised 34 forecast origins from October 2, 2025, through July 31, 2026. Its target was the subsequent 20-trading-day S&P 500 price return, measured in percentage points.
The comparison evaluated a conventional model incorporating rates, yield-curve structure, volatility, credit, participation, and momentum against the same model enhanced by the VMSI™ institutional-state variable.
The original historical findings were reproduced without recalibration or retrospective alteration. These results do not constitute a new prospective validation.
Scroll horizontally to view all columns
| Metric | Conventional | +VMSI™ | Historical Difference |
|---|---|---|---|
| MAE | 3.845 | 3.441 | 10.5% reduction |
| RMSE | 5.200 | 4.447 | 14.5% reduction |
| OOS R² vs. expanding mean | −0.539 | −0.126 | +0.414 |
| Individual lower-error origins | — | 16/34 | 47.1% |
Historical Statistical Robustness & Limitations
Audit 001’s original robustness findings were reproduced without recalibration. Clark–West one-sided tests produced p-values of 0.088 (HAC lag 3) and 0.098 (HAC lag 4). The forecast-encompassing regression reported a coefficient of 2.169 (t = 4.286; one-sided p = 0.000078). Leave-one-out testing preserved the historical improvement across all 34 exclusions, while contiguous four-origin deletions preserved it across all 31 tested windows.
These findings provide evidence of historical incremental information, but they do not establish universal forecasting superiority. Overlapping forecast horizons, limited effective sample independence, and sensitivity to particular observation periods remain material limitations. Audit 002 separately identified performance concentration and competitive simpler forecasting benchmarks, which require explicit consideration before drawing stronger conclusions.
New Finding — Historical Performance Concentration
Audit 002 identified a material concentration in the original historical forecasting advantage.
Six forecast origins between May 15 and June 19, 2026, contributed 124.0% of the net absolute-error reduction achieved by the VMSI™-enhanced model. The remaining 28 observations collectively detracted from the overall improvement.
When these six observations were excluded retrospectively, the original MAE advantage reversed: the conventional model recorded an MAE of 2.813, compared with 2.931 for the VMSI-enhanced model.
This six-observation exclusion was a post-hoc diagnostic, not one of Audit 001’s originally specified four-origin robustness tests. It does not invalidate those original results, but demonstrates that the historical incremental advantage was concentrated rather than uniformly distributed.
Additional retrospective comparisons found that zero-return, expanding-mean, and fixed 50% forecast-shrinkage benchmarks produced lower overall MAE and RMSE than the original VMSI-enhanced model. These comparisons were not part of the original prespecified conventional-model experiment.
The findings introduce a critical unresolved question: whether VMSI™ contributes independently measurable institutional information or whether part of its historical benefit reflects forecast calibration and shrinkage effects that simpler models can reproduce.
Publication Timing & Forward-Test Integrity
Audit 002 identified a material distinction between market observation dates, public report dates, and valid prospective forecast origins.
VICA’s weekly institutional reports were dated on Fridays but published publicly on subsequent Sundays. Using Friday closing prices as prospective forecast origins requires independently timestamped evidence that the corresponding model forecasts were finalized and locked before subsequent market outcomes became observable.
Five calendar-aligned 20-trading-day market outcomes were available through the November 6, 2026 audit cutoff. However, only four conservatively publication-aligned outcomes had matured by that date.
The five Friday-aligned observations produced an average subsequent S&P 500 return of −1.36%. The four publication-aligned observations produced an average return of −0.39%. These are descriptive market outcomes, not results from the original paired-model forecasting experiment.
The difference demonstrates why information availability, publication timing, and forecast provenance must be controlled before evaluating predictive performance.
Neither observation set establishes prospective forecasting superiority. The original conventional and VMSI-enhanced numeric forecasts, independently locked before each outcome, remain necessary to perform the prescribed comparison.
Future formal audits will distinguish observed market-state readings, retrospective calculations, and prospectively registered forecasts. No result will be represented as independently forward-tested without adequate evidence of its original information availability and forecast timestamp.
Market-State Evidence & Contradictory Observations
Audit 002 examined whether subsequent market behavior provided additional evidence concerning the VMSI™ institutional-state framework. The observations revealed measurable changes in market conditions, but did not constitute independent prospective validation of the original forecasting experiment.
Market-State Deterioration. VMSI™ declined from 59.7 on September 11, 2026, to 51.7 on November 6, 2026. The original historical sample contained 8 negative 20-day returns across 34 observations, whereas three of the five later calendar-aligned market windows produced negative returns. This represents a descriptive distributional change, not proof of predictive significance.
October 9 Contradictory Observation. VMSI™ increased from 55.9 to 57.6, IC-VMSI™ increased from 64.6 to 64.9, and ICMI™ increased from 50.8 to 51.7. Nevertheless, the S&P 500 subsequently declined 5.56% over the corresponding Friday-aligned 20-trading-day window. This contradicts any simplistic interpretation that improving institutional-state readings necessarily imply positive subsequent equity returns.
Independent Information. VMSI™, IC-VMSI™, and ICMI™ incorporate partially overlapping market evidence. Agreement among their readings cannot be counted as three independent forecasting confirmations. Additional financing, institutional holdings, and execution evidence is required to test the proposed capital-transmission mechanisms independently.
Statistical Limitations. The five observed market windows overlap substantially and do not represent five independent scientific replications. Their limited sample size prevents reliable statistical inference regarding forecasting superiority, causal mechanisms, or persistent market-state relationships.
Research Implication. These observations support continued investigation of the institutional-state framework while reinforcing the distinction between measurable market conditions, inferred transmission mechanisms, and independently validated forecasting performance.
Registered Forward-Testing Protocol & Falsification Criteria
Primary Validation Protocol
The original Audit 001 experiment remains unchanged. Future validation must compare the conventional forecasting model against the same model enhanced by VMSI™, using the original 20-trading-day S&P 500 return target, information-availability requirements, error metrics, and dependence-aware statistical tests.
Before any forecast outcome becomes observable, VICA Partners will archive the conventional forecast, VMSI-enhanced forecast, expanding-mean benchmark, forecast difference, contemporaneous VMSI™/IC-VMSI™/ICMI™ readings, information-availability timestamp, model identification, and market-price anchor.
Additional Benchmark Tests
A separately registered investigation will examine whether VMSI™ identifies structurally overextended conventional forecasts. Its performance will be compared against the original conventional model, a zero-return forecast, an expanding-mean forecast, a fixed 50% forecast-shrinkage benchmark, and a conventional-only calibration fitted exclusively on previously matured observations.
This secondary investigation will not replace or retrospectively modify the original experiment.
Explicit Falsification Conditions
Evidence against incremental VMSI™ forecasting value would include prospective underperformance relative to the unchanged conventional model; equivalent results from simpler conventional-only calibration or shrinkage; excessive false-positive warnings; benefits disappearing when particular observation clusters are excluded; or failure to substantiate proposed capital-transmission mechanisms with independent observational evidence.
Scientific Standard
Every formal audit will distinguish completed tests, inconclusive results, adverse findings, and unavailable evidence. Statistical conclusions will depend on effective sample size, independent observations, point-in-time data integrity, and appropriate controls for overlapping forecast horizons.
Neither favorable historical performance nor subsequent market-state observations will be represented as independently verified prospective superiority without the required supporting evidence.
Audit Disposition — November 10, 2026
Audit 001 — Historical Evidence Verified
The original 34-observation experiment was reproduced, preserving its historical results, statistical tests, methodological qualifications, and limitations. The original findings remain unchanged.
Audit 002 — Partial Evidence Review Completed
As of the November 6, 2026 audit cutoff, subsequent market outcomes and weekly institutional-state observations were reviewed. However, the original conventional and VMSI-enhanced numerical forecasts, independently locked before their respective outcomes, were unavailable.
Consequently, prospective paired-model forecasting superiority is NOT ESTABLISHED. No new MAE, RMSE, out-of-sample R², or statistical significance result is certified.
Audit 002 additionally identified material historical performance concentration, competitive simpler forecasting benchmarks, publication-timing constraints, and contradictory market-state observations. These findings are retained as part of the permanent scientific record.
Requirements for Subsequent Validation
The next formal audit must recover the original forecasting implementation or obtain contemporaneously archived, timestamped conventional and VMSI-enhanced forecasts. Subsequent evaluations must preserve the original experimental specification and distinguish independently verified results from retrospective diagnostics.
Final Disposition
Audit 002 constitutes a completed comparative evidence review, not a successful independent prospective replication. Historical evidence remains documented; incremental prospective forecasting value remains an open empirical question.
Evidence before narrative. Measurement before interpretation. The market remains the final test.
Research Sources & Data Provenance
Original Scientific Record
- Markets Become More Predictable at Scale — original baseline study, published September 4, 2026.
- VMSI™ Validation Record — Audit 001 conducted September 6, 2026; original validation record published September 21, 2026.
- Original forecast-level dataset:
test3_predictions.csv, containing 34 historical observations from October 2, 2025, through July 31, 2026. - Original aggregate results:
test3_results.csv.
Audit 002 Research
VMSI_Audit_002_Comparative_Research_2026-11-06.md- Market-data cutoff: November 6, 2026.
- Comparative research review: November 10, 2026.
- Original historical forecasting results reproduced; prospective paired-model testing not independently certified.
Market Data & Supporting Evidence
- S&P 500 daily closing-price series, Federal Reserve Economic Data (FRED): https://fred.stlouisfed.org/data/SP500
- VICA Partners’ dated weekly institutional reports, September–November 2026.
- Original statistical robustness documentation and historical forecast-level records.
Evidence Classification
Historical results, retrospective diagnostics, subsequent market observations, and prospective forecasts are identified separately throughout this publication. References to archived research or market observations do not establish independent prospective model validation.
Original research findings and subsequent adverse evidence are retained without retrospective alteration. Material corrections will be documented through dated amendments.
Dated Revision History
September 4, 2026 — Baseline Research Published
Markets Become More Predictable at Scale established the original research baseline.
September 6, 2026 — Audit 001 Conducted
Initial scientific validation audit completed.
September 21, 2026 — Original Validation Record Published
Audit 001 findings, statistical robustness results, limitations, and ten permanent validation standards entered the public research record.
November 6, 2026 — Audit 002 Evidence Cutoff
Historical and subsequent market observations evaluated through this date.
November 10, 2026 — Audit 002 Research Review
Historical results reproduced. Additional performance concentration, benchmark comparisons, and timing limitations documented. Prospective paired-model superiority not certified.
November 11, 2026 — Standalone Record Published
The cumulative scientific record was published, preserving Audit 001 and incorporating Audit 002 findings, limitations and contradictory evidence.
November 12, 2026 — Publication-Status Clarification
Updated the November 11 entry to reflect publication status. No scientific findings, scores or methodology changed.
Revision Policy
All future completed audits and material corrections will be appended with explicit dates. Original findings, adverse evidence, methodological limitations, and historical conclusions will remain preserved. Corrections will be documented rather than silently replacing earlier results.