feat(features): backfill the feature store so a model can be trained on it

feature_value held three as-of dates. A point-in-time training set needs
history, so three paths now generate it:

  POST /admin/features/backfill?from=&to=   loops computeAll(monthEnd) across
                                            a date range — 23 months, ~6 min
  POST /admin/bootstrap/ledger-history      fills the gap before the earliest
                                            entry only, stamped SEEDED_HISTORY
                                            so generated rows stay
                                            distinguishable from recorded ones
  ClimateIngestionService                    honours the lookbackWeeks the
                                            controller was already passing and
                                            had been ignoring, so the
                                            Open-Meteo archive can be pulled

Together these took the store to 21 monthly as-of dates over 263 enterprises,
which is the 5,523-row dataset the published metrics were measured on.

Documentation catches up with the code: API-SPEC lists the new endpoints,
ARCHITECTURE §7 records what was deliberately not built and why.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
LincPay
2026-08-25 13:44:54 +05:30
co-authored by Claude Opus 5
parent 1c42376a92
commit 7d192dfe9a
7 changed files with 277 additions and 5 deletions
+2 -2
View File
@@ -355,7 +355,7 @@ ahead of it is the shape of the seam, and that is what this is.
|---|---|
| `CoreBankingConnector` | Fetches and returns rows. Never writes — an adapter that could write would be an adapter that could bypass the no-PII rule. |
| `FileDropConnector` | **The one that is real.** Reads the newest matching file from a drop directory a bank delivers to. Leaves the file where it found it, so a failed run is repeatable. |
| `SimulatedCoreConnector` × 4 | Finacle, FLEXCUBE, BaNCS and an in-house REST LMS, each emitting the same portfolio in its own column dialect. Rows derived from real enterprises and loan records, drift seeded from the account ref so an extract is reproducible. |
| `SimulatedCoreConnector` | CBSNexora — LincPay's own core banking platform — emitting the real portfolio in its own field names. Rows derived from real enterprises and loan records, drift seeded from the account ref so an extract is reproducible. **No third-party core banking product is named anywhere in this system;** where a lender runs someone else's core, the per-profile field mapping absorbs their column names. |
| `FieldMapper` | `{canonical_column: vendor_column}`. Validated before it is stored, not at the next nightly run. |
| `ConnectorSyncService` | Fetch → map → **`LoanBookEntryService.stage`** → run record. |
| `ConnectorProfile`, `ConnectorSyncRun` | Per-bank configuration and per-attempt audit. |
@@ -371,7 +371,7 @@ profile, so switching a profile to a live adapter cannot retroactively make its
old runs look real.
**One real defect this surfaced.** `FORBIDDEN_COLUMNS` was matched by exact
normalised name, so `CUST_NAME` — what Finacle actually calls it — was not
normalised name, so `CUST_NAME` — a column name real extracts do carry — was not
caught, and a directly uploaded bank file would have kept the borrower's name in
`manual_entry_line.raw_row`. Matching now also splits separators and camelCase.
That fix belongs to the manual upload path as much as to the connector.