feat(loan-history): DPD history the model can be trained against, with provenance

The book held one snapshot per loan, so there was no history at all and
nothing a model could learn from. Two routes now fill it, and the
difference between them is stated rather than implied: generate
synthetic monthly snapshots for the prototype, or backfill month-end
extracts from the lender's core, which is the real path — every core can
answer "what did the book look like at each month end last year",
because it is the query behind their regulatory returns.

LoanRecord.source is stamped at every write site: SEEDED,
SEEDED_HISTORY, UPLOAD, or the connector profile code. Coverage reports
three buckets — synthetic, real, and unknown provenance — and never
folds unknown into real, because an unlabelled row is not evidence of a
lender extract. With no real extract on the book it reports
trainable:false and says why in words. Generation never overwrites an
existing row, so a real extract survives a regeneration untouched, and
purge removes only what was generated.

The generator's arithmetic had to be got right and was not at first.
Real DPD steps: pay and cure to zero, or miss and roll forward one
instalment period. The first version drew the day-of-month slack per
month, so consecutive months differed by anything from 1 to 59 days — a
shape no loan book produces, and one that would have taught a model a
transition matrix nothing real generates. The slack is now fixed per
delinquency episode, since a due date does not move between two missed
payments: an account 5 days down that misses again is 35 days down, not
12. For the run reaching the anchor it is derived from the anchor's own
DPD, so the series joins today's number exactly rather than near it.

DpdWalk is shared with the simulated adapters, which previously returned
today's DPD for every requested date — a backfill through them would
have been a flat line, which is exactly the shape that makes a training
label useless and which a chart would have hidden.

Verified against a running instance: 3,256 snapshots across 142 loans,
24 monthly extract dates spanning 2024-08 to 2026-07, 100% of
enterprises now carrying history, every generated series rolling by
exactly 30 — the only non-30 steps being joins to pre-existing rows the
generator correctly refused to overwrite. A 6-month Finacle backfill
pulled 264 rows and staged them for review.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
LincPay
2026-08-07 18:57:56 +05:30
co-authored by Claude Opus 5
parent b79b92fe8d
commit 8efd9fd6ed
12 changed files with 755 additions and 5 deletions
+36
View File
@@ -218,6 +218,7 @@ believes.
| Risk engine + rule engine + admin API | ✅ Phase D — see §6.3 |
| Console wire-up: rules screen, market panel, freshness | ✅ Phase E — see §6.4 |
| Core banking / LMS connector seam | ✅ Phase F — see §6.5 |
| DPD history: generated + connector backfill | ✅ Phase F — see §6.6 |
| LightGBM quantile heads + SHAP | ○ R1 milestone |
Phases B, C and D each terminate in a table you can inspect, so each is testable
@@ -379,6 +380,41 @@ That fix belongs to the manual upload path as much as to the connector.
synced by something the bank already runs), scheduled execution (`scheduleCron` is
recorded and read by nothing), and incremental watermarks.
### 6.6 DPD history — the training label, and where it comes from
Package `com.lincfusion.udyamlinc.loanhistory`, plus a fourth tab on
`/officer/integration`.
The book held **one snapshot per loan**, so there was no history at all and
nothing a model could be trained against. Two routes now fill it, and the
difference between them is stated rather than implied.
| Route | What it is |
|---|---|
| `LoanHistoryService.generate` | Synthetic monthly snapshots, every row stamped `SEEDED_HISTORY`. Exercises the feature store, the point-in-time joins and the label plumbing before a bank has delivered anything. |
| `ConnectorSyncService.backfill` | One extract per month across a range, mapped and staged as a single batch. This is the real path: every core can answer "what did the book look like at each month end last year". |
**Provenance is the whole design.** `LoanRecord.source` is stamped at every write
site, and coverage reports **three** buckets — synthetic, real, unknown — never
folding unknown into real. Generation refuses to overwrite an existing row, so a
lender's extract survives a regeneration.
**The generator's arithmetic had to be got right, and was not at first.** Real DPD
steps: pay and cure to zero, or miss and roll forward one instalment period. The
first version added day-of-month slack **per month**, so consecutive months
differed by anything from 1 to 59 days — a shape no loan book produces, and one
that would have taught a model a nonsense transition matrix. The slack is now
fixed per delinquency episode (a due date does not move between two missed
payments), and for the run reaching the anchor it is derived from the anchor's own
DPD so the series joins today's number exactly. Verified against a running
instance: every generated series now rolls by exactly 30, and the only steps that
are not are the joins to pre-existing rows the generator correctly refused to
touch.
**What this does not claim.** With no lender extract on the book, coverage reports
`trainable: false` and says so in words. Training on generated history and
reporting the accuracy would be measuring the generator.
---
## 7. Explicitly not being built yet