The loan book is the training label and the one input no public feed
supplies. Integrating with a lender's core is not a technical unknown —
it is an onboarding process per bank, with a schema agreed in a room.
What can be built before that room is the seam, and this is it.
CoreBankingConnector fetches and returns rows; it never writes. Mapped
rows go to LoanBookEntryService.stage — the same validator an uploaded
spreadsheet goes through, landing in the same manual_entry_batch with
the same preview. A parallel validator for the automated path would be
the obvious shape and would eventually enforce the no-PII rule
differently.
FileDropConnector is real: it reads the newest matching file from a
directory a bank delivers to, and leaves it there so a failed run is
repeatable. Finacle, FLEXCUBE, BaNCS and an in-house REST LMS are
simulated, each emitting the same portfolio in its own column dialect —
DPD_DAYS, OVERDUE_DAYS, DAYS_OVERDUE, daysPastDue — which is what makes
the case for a mapping layer rather than asserting it.
The mapping is {canonical: vendor}, keyed by canonical name, so an
unmapped vendor column has no route through and is dropped at the
boundary. CUST_NAME and MOBILE_NO stay out structurally rather than by
anyone remembering. Auto-commit is off on every profile: a sync stages
and stops, keeping the rule that nothing is written until someone has
seen what will be. simulated is stamped per run, not read from the
profile, so switching a profile live cannot make old runs look real.
Fixes a real defect this surfaced: FORBIDDEN_COLUMNS matched on the
exact normalised name, so CUST_NAME — what Finacle actually calls it —
was missed, and a directly uploaded bank file would have kept the
borrower's name in manual_entry_line.raw_row. Matching now splits
separators and camelCase, catching borrowerMobile too, while
loan_account_ref still splits clean.
Verified against a running instance: Finacle 44/44 accepted with 4
columns dropped, the file drop 2 accepted and 1 UNKNOWN_ENTERPRISE
rejected from a real CSV, and both CUST_NAME and borrowerMobile refused
at mapping save.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Egg, milk and fodder have no usable API for the pilot districts — NECC
publishes HTML, dairy rates exist only as annual state averages, and every
recent Agmarknet fodder row is one market in Ambala. This is how a bank
gets that data in.
Upload stages a PREVIEW batch and writes nothing. It reports what would
land, what was rejected and why, and which rows would overwrite an
existing value. Only commit touches market_signal.
Effective dating: effective_date is the date the fact describes and comes
from the file; observedAt is when we learned it and is stamped at commit.
The uploader cannot reach observedAt — a March rate uploaded in August
must stay invisible to a March scoring run, or the model trains on data
that did not exist yet.
Rejections name themselves: UNKNOWN_DISTRICT, UNKNOWN_COMMODITY,
FUTURE_DATE, BAD_DATE, BAD_NUMBER, UNIT_MISMATCH, PRICE_ORDER,
DUPLICATE_IN_FILE, MISSING_REQUIRED. A price far from last week warns but
is accepted — a genuine price shock is the event this system exists to
catch, and rejecting it would be backwards.
Rows are written with source=MANUAL, which the unique index keeps separate
from a feed's own row for the same week, so manual data never silently
replaces a government feed.
Also distinguishes INSUFFICIENT_HISTORY from NO_SIGNAL in the price
derivations. A newly onboarded manual feed spends its first weeks below
the four-week minimum, and reporting NO_SIGNAL there reads as a broken
integration rather than a series still filling up.
Result: margin_squeeze and output_price_change compute for the first time
— 57 of 142 enterprises, exactly the DAIRY (30) and POULTRY (27) sectors
that have an output commodity.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
