v1.0-nabard-submission
6
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 LincPayandClaude Opus 5
|
2cb7d3b14a |
feat(seed): a national portfolio, band-coherent ledger, and a forecast worth trusting
PortfolioPlan makes the demo book explicit — 263 enterprises across six states,
one per NABARD zone, over eight partner banks and sixteen branches — and
PortfolioPurge is the destructive escape hatch for the one boot where that shape
changes. The seeder never rewrites an existing enterprise, so a plan change
without a purge leaves orphans rather than corrupting rows.
udyam.forecast.trust-seeded now defaults to false. It was true only because
ledger coverage reached six enterprises of 142, so without the fallback the
console had almost no forecasts and showed 34 trip-wires of which 5 were real.
The seeder now writes five months of ledger for every enterprise, band-coherent
— a RED enterprise's monthly net genuinely sits below its instalment — so the
computed forecast is the one worth showing and the fallback is no longer
load-bearing.
jwt.secret becomes UDYAM_JWT_SECRET-overridable. The default stays committed so
a fresh clone boots without setup, which is precisely why it is not a secret:
any deployment holding real borrower data must set it.
Two traps this run surfaced, both silent:
- dealBands() now deals round-robin across sectors. Appending HORTICULTURE to
the sector list put it last and the fixed-seed band shuffle was drained by
the five packs ahead of it, so all 19 seeded STANDARD.
- Anything added to the seeder after a database was first populated never
reaches it, because of the count() > 0 guards. backfillPortfolio() now
covers loans and ledger and FeatureCatalog.seed() reconciles tolerances, but
trip-wire rules and sector weights still carry the hazard.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
LincPayandClaude Opus 5
|
7972bfe32d |
feat(simulation): fill what has no feed, marked so it can never pass as real
Nine of fifteen features now reach 263/263 and all six trip-wires fire for the
first time — TW-02 and TW-05 had never fired in the project's life. That
coverage comes partly from simulation, and every simulated row says so:
source = SIMULATED, upserted on a key that includes source, and refused if the
existing row is real even with overwrite=true. Real and simulated coexist; the
simulator can never overwrite a fact.
UpiSimulationService writes 26 weeks of upi_summary for each enterprise and
respects consentActive — a simulator that ignored consent would erase the very
distinction NO_CONSENT exists to hold.
NoFeedPriceSimulationService covers milk, egg, fodder and sugarcane, which have
no usable API anywhere: data.gov.in carries only annual state averages, NECC
publishes HTML, and cane is sold to mills at a set price and never reaches a
mandi. Milk's level is anchored on a real 2018 cooperative procurement figure
carried forward.
Calibrating a generated series took three passes, each plausible as a chart and
wrong as an input to a specific rule. Real Agmarknet series average a
price_z_score of 0.44, which is what exposed all of it:
- Too smooth. Index series at +/-1.5% weekly noise gave a tiny seasonal stdev,
so the newest value was an outlier by construction and TW-05 fired for 197
of 263. Noise must be large relative to trend.
- Monotonic. Even at +/-4% noise, a 5%/yr trend compounding over three years
swamped a +/-9% year shift, so every newest week was a record. Trend cut to
1.5-2.5%/yr and the year shift widened to +/-13%, so year-over-year
variation dominates. Portfolio mean z 1.39 -> 0.91, TW-05 197 -> 72.
- Wrong window. A linear decline over 26 weeks moves an 8-vs-8-week comparison
by only a third of its total, so a 40% drift showed as 13% and never reached
TW-02's -20% threshold. The decline now starts at 40% of the window and
curves. Check a generated series against the rule's own window, not against
its total drift.
Alerts are cumulative, so stale rows from the miscalibrated runs had to be
cleared: risk_alert held 206 TW-05 rows while the corrected run raised one.
The four output indices are categorically weaker than the rest and are labelled
separately in the console. Milk and egg are real prices lacking an API; a
finished sari has no price series to lack.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
LincPayandClaude Opus 5
|
ec057a8124 |
feat(ingestion): per-district commodity baskets, and a probe that verifies them
Commodities 7 -> 21 and a district_commodity table holding a per-district basket of 60 pairs, replacing the district x commodity cross product that would have been 252 mostly-impossible calls a night. District, not zone: Kolar and Belagavi are both SOUTH and trade almost disjoint baskets. Only the four original cereals had verified Agmarknet spellings, so the rest carry sourceVerified = false until POST /admin/ingestion/probe confirms them. The probe separates a wrong spelling from an absent local market by cross-referencing which districts and commodities returned rows anywhere — a distinction that matters because both fail silently and identically. Three district names were costing us every price in their district: Nasik -> Nashik, Ahmadnagar -> Ahilyanagar, Belgaum -> Belagavi. The last is the nastiest: Belgaum still answers with 1000 rows, but they stop in May 2026 while Belagavi is current, so a retired name serving a frozen archive reads as a healthy feed. Only the arrival dates give it away. data.gov.in meters by quota per window, so the limit is reached by the run and not by any one call: a 62-pair run straight after a 62-pair probe returned 429 on all 62 and recorded the whole ingestion FAILED with zero rows. DataGovClient now retries with a 45s backoff, long because the window is minutes not seconds. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
LincPayandClaude Opus 5
|
4a1fe69ff6 |
feat(institution): regional office tier, and one choke point for its scope
Adds a fourth supervisory level between apex and bank — HO -> RO -> Bank -> Branch -> Enterprise — as 25 offices with a state mapping, plus /nabard/ regional-offices and its bank drill. An RO is Tier.APEX carrying a regionalOfficeId, not a tier of its own, so it inherits mayReadBorrowers() == false rather than restating it. The office is resolved branch -> district -> state -> office and never stored on the bank: six of eight partner banks are state-bound RRBs, so a bank-level column would look right today and break the moment an SFB opens a second-state branch. The two new endpoints scoped correctly from the start while the eight pre-existing apex reads ignored the office claim entirely — the supervisory boundary was decorative, though no borrower leaked. Every read now passes through ApexService.scopeDistricts(tenant), one choke point rather than eight guards. All six offices sum exactly to the national 263 units / 8 banks / 342 alerts. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
LincPayandClaude Opus 5
|
8efd9fd6ed |
feat(loan-history): DPD history the model can be trained against, with provenance
The book held one snapshot per loan, so there was no history at all and nothing a model could learn from. Two routes now fill it, and the difference between them is stated rather than implied: generate synthetic monthly snapshots for the prototype, or backfill month-end extracts from the lender's core, which is the real path — every core can answer "what did the book look like at each month end last year", because it is the query behind their regulatory returns. LoanRecord.source is stamped at every write site: SEEDED, SEEDED_HISTORY, UPLOAD, or the connector profile code. Coverage reports three buckets — synthetic, real, and unknown provenance — and never folds unknown into real, because an unlabelled row is not evidence of a lender extract. With no real extract on the book it reports trainable:false and says why in words. Generation never overwrites an existing row, so a real extract survives a regeneration untouched, and purge removes only what was generated. The generator's arithmetic had to be got right and was not at first. Real DPD steps: pay and cure to zero, or miss and roll forward one instalment period. The first version drew the day-of-month slack per month, so consecutive months differed by anything from 1 to 59 days — a shape no loan book produces, and one that would have taught a model a transition matrix nothing real generates. The slack is now fixed per delinquency episode, since a due date does not move between two missed payments: an account 5 days down that misses again is 35 days down, not 12. For the run reaching the anchor it is derived from the anchor's own DPD, so the series joins today's number exactly rather than near it. DpdWalk is shared with the simulated adapters, which previously returned today's DPD for every requested date — a backfill through them would have been a flat line, which is exactly the shape that makes a training label useless and which a chart would have hidden. Verified against a running instance: 3,256 snapshots across 142 loans, 24 monthly extract dates spanning 2024-08 to 2026-07, 100% of enterprises now carrying history, every generated series rolling by exactly 30 — the only non-30 steps being joins to pre-existing rows the generator correctly refused to overwrite. A 6-month Finacle backfill pulled 264 rows and staged them for review. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
LincPayandClaude Opus 5
|
b79b92fe8d |
feat(connector): core banking / LMS seam, one real adapter and four dialects
The loan book is the training label and the one input no public feed
supplies. Integrating with a lender's core is not a technical unknown —
it is an onboarding process per bank, with a schema agreed in a room.
What can be built before that room is the seam, and this is it.
CoreBankingConnector fetches and returns rows; it never writes. Mapped
rows go to LoanBookEntryService.stage — the same validator an uploaded
spreadsheet goes through, landing in the same manual_entry_batch with
the same preview. A parallel validator for the automated path would be
the obvious shape and would eventually enforce the no-PII rule
differently.
FileDropConnector is real: it reads the newest matching file from a
directory a bank delivers to, and leaves it there so a failed run is
repeatable. Finacle, FLEXCUBE, BaNCS and an in-house REST LMS are
simulated, each emitting the same portfolio in its own column dialect —
DPD_DAYS, OVERDUE_DAYS, DAYS_OVERDUE, daysPastDue — which is what makes
the case for a mapping layer rather than asserting it.
The mapping is {canonical: vendor}, keyed by canonical name, so an
unmapped vendor column has no route through and is dropped at the
boundary. CUST_NAME and MOBILE_NO stay out structurally rather than by
anyone remembering. Auto-commit is off on every profile: a sync stages
and stops, keeping the rule that nothing is written until someone has
seen what will be. simulated is stamped per run, not read from the
profile, so switching a profile live cannot make old runs look real.
Fixes a real defect this surfaced: FORBIDDEN_COLUMNS matched on the
exact normalised name, so CUST_NAME — what Finacle actually calls it —
was missed, and a directly uploaded bank file would have kept the
borrower's name in manual_entry_line.raw_row. Matching now splits
separators and camelCase, catching borrowerMobile too, while
loan_account_ref still splits clean.
Verified against a running instance: Finacle 44/44 accepted with 4
columns dropped, the file drop 2 accepted and 1 UNKNOWN_ENTERPRISE
rejected from a real CSV, and both CUST_NAME and borrowerMobile refused
at mapping save.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|