feat(admin): one-call demo bootstrap, and price simulation for the whole market

A fresh deployment came up with a populated portfolio, an empty feeds screen and
no alerts, because everything downstream of the seeder was an admin call whose
order lived only in a runbook. POST /admin/bootstrap/demo now runs the six steps
in dependency order and reports each one.

The price simulator widens from the four commodities with no feed to every
active basket pair, so price_z_score gets its three-year seasonal mean and TW-05
can fire with no DATAGOV_API_KEY present at all. Levels come from a per-crop
table of ordinary mandi rates and the shape from the commodity group, because
the group already encodes perishability - a vegetable moves on a weather event
within days and a cereal does not, and flattening that would erase the one
distinction the z-score exists to read.

A pair holding any non-simulated row is skipped unless forced. That ordering is
the safety property: run this after a real ingestion and it fills the gaps
around the real series rather than over them.

The orchestrator is deliberately not transactional. The seeder holds one
transaction across its whole run, and when the machine slept mid-run nothing
committed until morning while the console confidently served pre-seed state. A
bootstrap generating three years of prices across every pair is long enough to
hit exactly that, so each step commits alone and a failure in one leaves the
others standing.

Still not trainable, and coverage still says so. The seeder derives DPD from the
band and the UPI simulator derives inflow from the same band, so a model fitted
here recovers the seeder rule and reports fine metrics for doing it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
LincPay
2026-08-15 15:56:41 +05:30
co-authored by Claude Opus 5
parent e28ac52803
commit 81e66a9b57
6 changed files with 386 additions and 18 deletions
+26 -18
View File
@@ -221,27 +221,34 @@ The seeder writes the portfolio, the ledger and the forecasts. It does **not** w
loan history, UPI inflows or the no-feed price series: those are admin calls, so a fresh
database comes up without them and several screens look thinner than they should.
Run these once, in order, after the seeder's final log line:
**One call does all of it.** After the seeder's final log line:
```bash
BASE=http://localhost:8082/api/v1/udyam
TOKEN=<a supervisor token from POST $BASE/auth/login>
# 1. DPD history — 24 monthly snapshots per loan, stamped SEEDED_HISTORY
curl -X POST -H "Authorization: Bearer $TOKEN" "$BASE/admin/loan-history/generate?months=24"
# 2. UPI inflows — 26 weeks per enterprise, stamped SIMULATED.
# Stands in for Account Aggregator data until Sahamati FIU registration exists.
curl -X POST -H "Authorization: Bearer $TOKEN" "$BASE/admin/upi/simulate"
# 3. Prices for the commodities with no API anywhere — milk, egg, fodder, sugarcane
curl -X POST -H "Authorization: Bearer $TOKEN" "$BASE/admin/ingestion/simulate/no-feed"
# 4. Derive the features, then score against them
curl -X POST -H "Authorization: Bearer $TOKEN" "$BASE/admin/features/compute"
curl -X POST -H "Authorization: Bearer $TOKEN" "$BASE/admin/scoring/run"
curl -X POST -H "Authorization: Bearer $TOKEN" "$BASE/admin/bootstrap/demo?months=24"
```
That runs six steps in dependency order and reports each one:
| Step | What it writes |
|---|---|
| `loan-history` | 24 monthly DPD snapshots per loan, `SEEDED_HISTORY` |
| `upi-inflows` | 26 weeks per enterprise, `SIMULATED` — the Account Aggregator stand-in |
| `prices-no-feed` | milk, egg, fodder, sugarcane — the four with no API anywhere |
| `prices-all` | three years for **every remaining basket pair**, so `price_z_score` has a seasonal mean and TW-05 can fire without a data.gov.in key |
| `features` | the 15 derivations across the book |
| `scoring` | stress scores, bands, trip-wires, alerts |
It is **idempotent** — re-running is safe, and each step commits on its own, so a
failure in one leaves the others intact and the response names which failed.
The individual endpoints remain if you want to run a stage on its own:
`/admin/loan-history/generate`, `/admin/upi/simulate`,
`/admin/ingestion/simulate/no-feed`, `/admin/ingestion/simulate/all`,
`/admin/features/compute`, `/admin/scoring/run`.
Verify with the coverage endpoints, both of which report synthetic and real separately:
```bash
@@ -264,10 +271,11 @@ Two consequences to state out loud rather than discover later:
derives inflow drift from the same band, so a model trained on this would learn the
seeder's own rule and validate beautifully while knowing nothing. The synthetic data
is here to exercise the pipeline and fill the console, not to train.
- **This sequence does not give you the real mandi price history.** Steps 1–3 are
synthetic; the three-year Agmarknet backfill is real data and needs `DATAGOV_API_KEY`
plus a long one-off run, or a dump of a database that already has it. Without it
`price_z_score` has no seasonal mean and TW-05 stays quiet.
- **No external dependency at all.** The bootstrap needs no `DATAGOV_API_KEY`, no
Account Aggregator consent and no bank connection: every screen fills from generated
data. If a key *is* present, run the real ingestion first and the price simulator will
skip any pair that already holds real rows, filling the gaps around the real series
rather than across it.
## What is deliberately off