feat(loan-history): DPD history the model can be trained against, with provenance
The book held one snapshot per loan, so there was no history at all and nothing a model could learn from. Two routes now fill it, and the difference between them is stated rather than implied: generate synthetic monthly snapshots for the prototype, or backfill month-end extracts from the lender's core, which is the real path — every core can answer "what did the book look like at each month end last year", because it is the query behind their regulatory returns. LoanRecord.source is stamped at every write site: SEEDED, SEEDED_HISTORY, UPLOAD, or the connector profile code. Coverage reports three buckets — synthetic, real, and unknown provenance — and never folds unknown into real, because an unlabelled row is not evidence of a lender extract. With no real extract on the book it reports trainable:false and says why in words. Generation never overwrites an existing row, so a real extract survives a regeneration untouched, and purge removes only what was generated. The generator's arithmetic had to be got right and was not at first. Real DPD steps: pay and cure to zero, or miss and roll forward one instalment period. The first version drew the day-of-month slack per month, so consecutive months differed by anything from 1 to 59 days — a shape no loan book produces, and one that would have taught a model a transition matrix nothing real generates. The slack is now fixed per delinquency episode, since a due date does not move between two missed payments: an account 5 days down that misses again is 35 days down, not 12. For the run reaching the anchor it is derived from the anchor's own DPD, so the series joins today's number exactly rather than near it. DpdWalk is shared with the simulated adapters, which previously returned today's DPD for every requested date — a backfill through them would have been a flat line, which is exactly the shape that makes a training label useless and which a chart would have hidden. Verified against a running instance: 3,256 snapshots across 142 loans, 24 monthly extract dates spanning 2024-08 to 2026-07, 100% of enterprises now carrying history, every generated series rolling by exactly 30 — the only non-30 steps being joins to pre-existing rows the generator correctly refused to overwrite. A 6-month Finacle backfill pulled 264 rows and staged them for review. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -218,6 +218,7 @@ believes.
|
||||
| Risk engine + rule engine + admin API | ✅ Phase D — see §6.3 |
|
||||
| Console wire-up: rules screen, market panel, freshness | ✅ Phase E — see §6.4 |
|
||||
| Core banking / LMS connector seam | ✅ Phase F — see §6.5 |
|
||||
| DPD history: generated + connector backfill | ✅ Phase F — see §6.6 |
|
||||
| LightGBM quantile heads + SHAP | ○ R1 milestone |
|
||||
|
||||
Phases B, C and D each terminate in a table you can inspect, so each is testable
|
||||
@@ -379,6 +380,41 @@ That fix belongs to the manual upload path as much as to the connector.
|
||||
synced by something the bank already runs), scheduled execution (`scheduleCron` is
|
||||
recorded and read by nothing), and incremental watermarks.
|
||||
|
||||
### 6.6 DPD history — the training label, and where it comes from
|
||||
|
||||
Package `com.lincfusion.udyamlinc.loanhistory`, plus a fourth tab on
|
||||
`/officer/integration`.
|
||||
|
||||
The book held **one snapshot per loan**, so there was no history at all and
|
||||
nothing a model could be trained against. Two routes now fill it, and the
|
||||
difference between them is stated rather than implied.
|
||||
|
||||
| Route | What it is |
|
||||
|---|---|
|
||||
| `LoanHistoryService.generate` | Synthetic monthly snapshots, every row stamped `SEEDED_HISTORY`. Exercises the feature store, the point-in-time joins and the label plumbing before a bank has delivered anything. |
|
||||
| `ConnectorSyncService.backfill` | One extract per month across a range, mapped and staged as a single batch. This is the real path: every core can answer "what did the book look like at each month end last year". |
|
||||
|
||||
**Provenance is the whole design.** `LoanRecord.source` is stamped at every write
|
||||
site, and coverage reports **three** buckets — synthetic, real, unknown — never
|
||||
folding unknown into real. Generation refuses to overwrite an existing row, so a
|
||||
lender's extract survives a regeneration.
|
||||
|
||||
**The generator's arithmetic had to be got right, and was not at first.** Real DPD
|
||||
steps: pay and cure to zero, or miss and roll forward one instalment period. The
|
||||
first version added day-of-month slack **per month**, so consecutive months
|
||||
differed by anything from 1 to 59 days — a shape no loan book produces, and one
|
||||
that would have taught a model a nonsense transition matrix. The slack is now
|
||||
fixed per delinquency episode (a due date does not move between two missed
|
||||
payments), and for the run reaching the anchor it is derived from the anchor's own
|
||||
DPD so the series joins today's number exactly. Verified against a running
|
||||
instance: every generated series now rolls by exactly 30, and the only steps that
|
||||
are not are the joins to pre-existing rows the generator correctly refused to
|
||||
touch.
|
||||
|
||||
**What this does not claim.** With no lender extract on the book, coverage reports
|
||||
`trainable: false` and says so in words. Training on generated history and
|
||||
reporting the accuracy would be measuring the generator.
|
||||
|
||||
---
|
||||
|
||||
## 7. Explicitly not being built yet
|
||||
|
||||
Reference in New Issue
Block a user