88702f2b4312cdf7992ecc724675e5d13523b0bd
18
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 LincPay
|
88702f2b43 | docs: deployment notes for nginx, Apache httpd and Tomcat | ||
|
LincPay
|
053a1cee05 | fix(config): make CORS origins overridable by environment | ||
|
LincPayandClaude Opus 5
|
2cb7d3b14a |
feat(seed): a national portfolio, band-coherent ledger, and a forecast worth trusting
PortfolioPlan makes the demo book explicit — 263 enterprises across six states,
one per NABARD zone, over eight partner banks and sixteen branches — and
PortfolioPurge is the destructive escape hatch for the one boot where that shape
changes. The seeder never rewrites an existing enterprise, so a plan change
without a purge leaves orphans rather than corrupting rows.
udyam.forecast.trust-seeded now defaults to false. It was true only because
ledger coverage reached six enterprises of 142, so without the fallback the
console had almost no forecasts and showed 34 trip-wires of which 5 were real.
The seeder now writes five months of ledger for every enterprise, band-coherent
— a RED enterprise's monthly net genuinely sits below its instalment — so the
computed forecast is the one worth showing and the fallback is no longer
load-bearing.
jwt.secret becomes UDYAM_JWT_SECRET-overridable. The default stays committed so
a fresh clone boots without setup, which is precisely why it is not a secret:
any deployment holding real borrower data must set it.
Two traps this run surfaced, both silent:
- dealBands() now deals round-robin across sectors. Appending HORTICULTURE to
the sector list put it last and the fixed-seed band shuffle was drained by
the five packs ahead of it, so all 19 seeded STANDARD.
- Anything added to the seeder after a database was first populated never
reaches it, because of the count() > 0 guards. backfillPortfolio() now
covers loans and ledger and FeatureCatalog.seed() reconciles tolerances, but
trip-wire rules and sector weights still carry the hazard.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
v1.0-nabard-submission
|
||
|
LincPayandClaude Opus 5
|
7972bfe32d |
feat(simulation): fill what has no feed, marked so it can never pass as real
Nine of fifteen features now reach 263/263 and all six trip-wires fire for the
first time — TW-02 and TW-05 had never fired in the project's life. That
coverage comes partly from simulation, and every simulated row says so:
source = SIMULATED, upserted on a key that includes source, and refused if the
existing row is real even with overwrite=true. Real and simulated coexist; the
simulator can never overwrite a fact.
UpiSimulationService writes 26 weeks of upi_summary for each enterprise and
respects consentActive — a simulator that ignored consent would erase the very
distinction NO_CONSENT exists to hold.
NoFeedPriceSimulationService covers milk, egg, fodder and sugarcane, which have
no usable API anywhere: data.gov.in carries only annual state averages, NECC
publishes HTML, and cane is sold to mills at a set price and never reaches a
mandi. Milk's level is anchored on a real 2018 cooperative procurement figure
carried forward.
Calibrating a generated series took three passes, each plausible as a chart and
wrong as an input to a specific rule. Real Agmarknet series average a
price_z_score of 0.44, which is what exposed all of it:
- Too smooth. Index series at +/-1.5% weekly noise gave a tiny seasonal stdev,
so the newest value was an outlier by construction and TW-05 fired for 197
of 263. Noise must be large relative to trend.
- Monotonic. Even at +/-4% noise, a 5%/yr trend compounding over three years
swamped a +/-9% year shift, so every newest week was a record. Trend cut to
1.5-2.5%/yr and the year shift widened to +/-13%, so year-over-year
variation dominates. Portfolio mean z 1.39 -> 0.91, TW-05 197 -> 72.
- Wrong window. A linear decline over 26 weeks moves an 8-vs-8-week comparison
by only a third of its total, so a 40% drift showed as 13% and never reached
TW-02's -20% threshold. The decline now starts at 40% of the window and
curves. Check a generated series against the rule's own window, not against
its total drift.
Alerts are cumulative, so stale rows from the miscalibrated runs had to be
cleared: risk_alert held 206 TW-05 rows while the corrected run raised one.
The four output indices are categorically weaker than the rest and are labelled
separately in the console. Milk and egg are real prices lacking an API; a
finished sari has no price series to lack.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
LincPayandClaude Opus 5
|
ec057a8124 |
feat(ingestion): per-district commodity baskets, and a probe that verifies them
Commodities 7 -> 21 and a district_commodity table holding a per-district basket of 60 pairs, replacing the district x commodity cross product that would have been 252 mostly-impossible calls a night. District, not zone: Kolar and Belagavi are both SOUTH and trade almost disjoint baskets. Only the four original cereals had verified Agmarknet spellings, so the rest carry sourceVerified = false until POST /admin/ingestion/probe confirms them. The probe separates a wrong spelling from an absent local market by cross-referencing which districts and commodities returned rows anywhere — a distinction that matters because both fail silently and identically. Three district names were costing us every price in their district: Nasik -> Nashik, Ahmadnagar -> Ahilyanagar, Belgaum -> Belagavi. The last is the nastiest: Belgaum still answers with 1000 rows, but they stop in May 2026 while Belagavi is current, so a retired name serving a frozen archive reads as a healthy feed. Only the arrival dates give it away. data.gov.in meters by quota per window, so the limit is reached by the run and not by any one call: a 62-pair run straight after a 62-pair probe returned 429 on all 62 and recorded the whole ingestion FAILED with zero rows. DataGovClient now retries with a 45s backoff, long because the window is minutes not seconds. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
LincPayandClaude Opus 5
|
4a1fe69ff6 |
feat(institution): regional office tier, and one choke point for its scope
Adds a fourth supervisory level between apex and bank — HO -> RO -> Bank -> Branch -> Enterprise — as 25 offices with a state mapping, plus /nabard/ regional-offices and its bank drill. An RO is Tier.APEX carrying a regionalOfficeId, not a tier of its own, so it inherits mayReadBorrowers() == false rather than restating it. The office is resolved branch -> district -> state -> office and never stored on the bank: six of eight partner banks are state-bound RRBs, so a bank-level column would look right today and break the moment an SFB opens a second-state branch. The two new endpoints scoped correctly from the start while the eight pre-existing apex reads ignored the office claim entirely — the supervisory boundary was decorative, though no borrower leaked. Every read now passes through ApexService.scopeDistricts(tenant), one choke point rather than eight guards. All six offices sum exactly to the national 263 units / 8 banks / 342 alerts. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
LincPayandClaude Opus 5
|
8efd9fd6ed |
feat(loan-history): DPD history the model can be trained against, with provenance
The book held one snapshot per loan, so there was no history at all and nothing a model could learn from. Two routes now fill it, and the difference between them is stated rather than implied: generate synthetic monthly snapshots for the prototype, or backfill month-end extracts from the lender's core, which is the real path — every core can answer "what did the book look like at each month end last year", because it is the query behind their regulatory returns. LoanRecord.source is stamped at every write site: SEEDED, SEEDED_HISTORY, UPLOAD, or the connector profile code. Coverage reports three buckets — synthetic, real, and unknown provenance — and never folds unknown into real, because an unlabelled row is not evidence of a lender extract. With no real extract on the book it reports trainable:false and says why in words. Generation never overwrites an existing row, so a real extract survives a regeneration untouched, and purge removes only what was generated. The generator's arithmetic had to be got right and was not at first. Real DPD steps: pay and cure to zero, or miss and roll forward one instalment period. The first version drew the day-of-month slack per month, so consecutive months differed by anything from 1 to 59 days — a shape no loan book produces, and one that would have taught a model a transition matrix nothing real generates. The slack is now fixed per delinquency episode, since a due date does not move between two missed payments: an account 5 days down that misses again is 35 days down, not 12. For the run reaching the anchor it is derived from the anchor's own DPD, so the series joins today's number exactly rather than near it. DpdWalk is shared with the simulated adapters, which previously returned today's DPD for every requested date — a backfill through them would have been a flat line, which is exactly the shape that makes a training label useless and which a chart would have hidden. Verified against a running instance: 3,256 snapshots across 142 loans, 24 monthly extract dates spanning 2024-08 to 2026-07, 100% of enterprises now carrying history, every generated series rolling by exactly 30 — the only non-30 steps being joins to pre-existing rows the generator correctly refused to overwrite. A 6-month Finacle backfill pulled 264 rows and staged them for review. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
LincPayandClaude Opus 5
|
b79b92fe8d |
feat(connector): core banking / LMS seam, one real adapter and four dialects
The loan book is the training label and the one input no public feed
supplies. Integrating with a lender's core is not a technical unknown —
it is an onboarding process per bank, with a schema agreed in a room.
What can be built before that room is the seam, and this is it.
CoreBankingConnector fetches and returns rows; it never writes. Mapped
rows go to LoanBookEntryService.stage — the same validator an uploaded
spreadsheet goes through, landing in the same manual_entry_batch with
the same preview. A parallel validator for the automated path would be
the obvious shape and would eventually enforce the no-PII rule
differently.
FileDropConnector is real: it reads the newest matching file from a
directory a bank delivers to, and leaves it there so a failed run is
repeatable. Finacle, FLEXCUBE, BaNCS and an in-house REST LMS are
simulated, each emitting the same portfolio in its own column dialect —
DPD_DAYS, OVERDUE_DAYS, DAYS_OVERDUE, daysPastDue — which is what makes
the case for a mapping layer rather than asserting it.
The mapping is {canonical: vendor}, keyed by canonical name, so an
unmapped vendor column has no route through and is dropped at the
boundary. CUST_NAME and MOBILE_NO stay out structurally rather than by
anyone remembering. Auto-commit is off on every profile: a sync stages
and stops, keeping the rule that nothing is written until someone has
seen what will be. simulated is stamped per run, not read from the
profile, so switching a profile live cannot make old runs look real.
Fixes a real defect this surfaced: FORBIDDEN_COLUMNS matched on the
exact normalised name, so CUST_NAME — what Finacle actually calls it —
was missed, and a directly uploaded bank file would have kept the
borrower's name in manual_entry_line.raw_row. Matching now splits
separators and camelCase, catching borrowerMobile too, while
loan_account_ref still splits clean.
Verified against a running instance: Finacle 44/44 accepted with 4
columns dropped, the file drop 2 accepted and 1 UNKNOWN_ENTERPRISE
rejected from a real CSV, and both CUST_NAME and borrowerMobile refused
at mapping save.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
LincPayandClaude Opus 5
|
22870eb480 |
docs: README and API-SPEC, both promised by the original scaffold
A judge who cannot run the code reads the README, and until now the reasoning behind this system lived only in Java comments and commit messages — including the one thing a NABARD panel will check first, which is why the apex tier cannot reach a borrower. README covers the institutional model and why NABARD's restriction is the point rather than a limitation, how to run both projects, the seven console logins with what each sees, the demo path, and the design decisions worth defending. It also states plainly what is real and what is not: live Agmarknet and Open-Meteo data and a working rule engine, a forecast that is deterministic rather than machine-learned, and a seeded loan book. It says that trust-seeded defaults to true and that the honest setting turns 34 trip-wires into 5. A demo that implies more than it has is worth less than one that is straight about the gap. API-SPEC documents every endpoint across the four surfaces, the response envelope, the tenancy rule, small-cell suppression, effective dating and the rejection reasons — plus the one defect found while writing it: SecurityConfig whitelists /health but nothing is mapped there, so it returns 500. The working probe is /auth/health. Every credential and figure quoted was verified against the running service: seven logins, the enterprise PIN, and the 44 / 39 / 59 / 12 portfolio counts. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
LincPayandClaude Opus 5
|
3531e6e79f |
feat(apex): India → zone → state → district, and bank → branch
NABARD reads a national portfolio geographically before it reads it by
institution, so the apex tier needs the hierarchy above the district that
the pilot never had.
District carries a zone. Held as data rather than derived in a service, so
a state moving zone is a data change, not a deploy — and Zones maps state
to zone from a table rather than guessing, because a portfolio rolled into
the wrong zone is worse than one not rolled up at all: confidently wrong,
and nothing downstream can detect it. An unmapped state becomes UNMAPPED
and is reported as such.
/nabard/geography takes level plus optional zone and state. Those come from
the request because they select a view, not an entitlement — an apex user
may read every zone, so naming one grants nothing. Contrast the institution
id, which is never accepted from a request.
/nabard/banks/{code}/branches is the deepest this tier goes. Below a branch
is a borrower, and that belongs to the bank.
Suppression applies at every level and bites hardest at the bottom, which
is the intended behaviour of a drill-down: the further in you go, the
closer you are to a person.
Verified: India→Central→Madhya Pradesh→4 districts resolves, and MPGB
splits into Sehore (12) and Vidisha (32) branches.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
LincPayandClaude Opus 5
|
1e5f446672 |
feat(institution): three-tier model — NABARD, partner banks, enterprises
NABARD does not lend to enterprises. It refinances and supervises the banks that do. A single field-officer console collapsed those two institutions into one and implicitly cast the apex bank as the lender, which is wrong about the sector and wrong about who may see a borrower. Institution and Branch are new; enterprise and udyam_user carry institution_id and branch_id. That pair is the tenancy key. TenantContext is resolved once per request from the signed token and injected into handlers as an ordinary argument. The institution is never read from a path variable, query parameter or body — a tenancy key the caller can type is not a tenancy key. Every borrower-level read now takes its filter from there, and fails closed: a caller with no institution sees nothing rather than everything. Three entitlements, not three ranks of one. APEX reads aggregates across every partner and no borrower row at all; that restriction is what makes portfolio-wide early warning grantable to a supervisor in the first place. BANK reads its own book, narrowed to a branch for branch-scoped users. ENTERPRISE reads itself. Alerts and visits are scoped through the borrowers a caller may see rather than by a column of their own, and dispositioning an alert re-checks tenancy — closing another bank's alert would be invisible to them. A borrower on another bank's book returns the same 'no such enterprise' as one that does not exist. A distinguishable 403 would confirm the id is real and on somebody's book, which is itself the leak. ApexService suppresses any cell below five borrowers and reports the suppression rather than omitting the row, because a missing district reads as 'no activity' — a different and wrong claim. Verified live: MPGB sees 44, DCCB 39, a branch officer 12; NABARD is 403 on /officer/**, banks are 403 on /nabard/**; a real cross-tenant id is byte-identical to a fabricated one; and none of the six apex endpoints emits a single MSME id. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
LincPayandClaude Opus 5
|
cf27258e26 |
fix(features): attribute staleness to the data that built the feature
Staleness was district-wide: the newest market row for the district, whatever commodity it belonged to, stamped onto every price feature. That was a fair approximation while every commodity came from the same daily Agmarknet pull. Manual entry ended it — milk and egg arrive monthly, so a dairy enterprise's output_price_change, built entirely from a month-old milk rate, inherited yesterday's maize freshness and sailed through the staleness gate that exists to catch exactly that. Each price feature now carries the age of its own oldest contributing commodity, and margin_squeeze takes the staler of its two halves — a squeeze is only as current as the older side of it. Measured: output_price_change and margin_squeeze now report 3–24 days where they previously reported 1–8, which is the truth rather than maize's answer to a question about milk. Catalogue tolerances for the price features go to 35 days, the slowest source that legitimately feeds them. Ten days would gate every dairy and poultry price feature out of every rule permanently — uploaded data landing in the table and changing nothing. RuleEvaluator takes the stricter of rule and catalogue, so this is the permissive floor and the trip-wire remains where tightening belongs. FeatureCatalog.seed now reconciles existing rows instead of only inserting absent ones. Insert-if-absent left every database seeded before a tolerance changed running on the old value, silently, in the direction that decides whether a rule may fire at all. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
LincPayandClaude Opus 5
|
f7fd461e73 |
fix(ingestion): name the district and commodity on signal endpoints
The console's mandi and climate tables read districtName and commodityName; the endpoints returned districtId and commodityId, so every row rendered blank against a real backend. It went unnoticed because until today nothing had ever run and the screen only ever showed its demo fallback. Names are added alongside the entity's own fields, so a screen does not need the whole master list to render one row. Climate keys now match the UI's contract exactly (tmaxC, hotDays, rainfallAnomalyPct). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
LincPayandClaude Opus 5
|
a18c2f335f |
feat(manual-entry): loan-book upload, with DPD history the schema could not hold
The CBS/LMS extract is the deepest gap in the system: emi_delay and emi_to_inflow are computed from the loan book, and DPD history is the training label without which the model cannot be trained at all. This is how a real book gets in before a lender builds a nightly feed. Two schema problems had to be fixed first. loan_record had a unique index on accountRef alone, so the table could hold only each loan's current state — every monthly extract overwriting the evidence the model is meant to learn from. The key is now (accountRef, asOfDate): one row per loan per extract. The old constraint must be dropped by hand, as ddl-auto adds indexes but never removes them. emi_delay and emi_to_inflow took the worst DPD across every row at or before the scoring date, which with snapshots present would report a loan sixty days down in March as sixty days down today. Both now read the newest extract per accountRef. The history is there to train on, not to be summarised as the present. The no-PII promise is enforced at the parse boundary. A file whose headers carry a name, phone, Aadhaar, PAN, account number or address is rejected outright with the offending columns named, rather than having the column quietly dropped — a bank must learn the data is unwanted before it builds a nightly job that keeps sending it. ManualEntryLine now stores the row as supplied, so commit reads the file's own columns rather than a re-parse of a file nobody kept, and an auditor sees what the bank sent rather than our reading of it. Verified: a six-month, two-loan book uploads as 12 snapshots; a file with customer_name and mobile_no is refused whole; and emi_delay reports 2 for a loan that peaked at 3 and is now 2. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
LincPayandClaude Opus 5
|
d710176f36 |
feat(forecast): deterministic quantile path so dscr_p10 stops reading seeded rows
dscr_p10 is the feature TW-01 fires on, and so the origin of nearly every alert an officer sees — but nothing had written forecast_point since the seeder. A headline number computed from demo data is worse than no headline number, because it looks identical to a real one. ForecastStrategy is a seam shaped like ScoringStrategy: the LightGBM quantile heads implement it later and the swap is one property. The forecast and the score answer different questions and fail differently, so they get separate seams. The method is fully explainable. Baseline is the median whole-month net position from the ledger, median so one festival month does not become the expectation, and the partial current month is discarded. Drift comes from margin_squeeze, else feed_cost_change, capped so one sharp month does not extrapolate into a cliff. Spread widens with the square root of the horizon and again for every input we did not have. Missing inputs widen the band, never lower the centre. A pessimistic P50 built on ignorance would manufacture breaches. An enterprise with too little ledger gets no path rather than a fabricated one, so dscr_p10 resolves NO_FORECAST visibly. forecast_point gains engine/model_version. Seeded rows carry neither, and udyam.forecast.trust-seeded decides whether dscr_p10 may fall back to them. Left true so the demo portfolio stays populated; with it false the picture is honest — 6 enterprises forecast, 136 NO_FORECAST, and trip-wires firing drops from 34 to 5. The forecast reads margin features and dscr_p10 reads the forecast, so features are computed twice per enterprise rather than letting either half read a previous run's answer. Seeder now writes 150 days of ledger, not 31: the baseline needs whole months and one month leaves nothing to take a median of. Ledger joins loans in the backfill pass, keyed on clientId. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
LincPayandClaude Opus 5
|
5401496a64 |
feat(manual-entry): bulk and single entry for the feeds no API serves
Egg, milk and fodder have no usable API for the pilot districts — NECC publishes HTML, dairy rates exist only as annual state averages, and every recent Agmarknet fodder row is one market in Ambala. This is how a bank gets that data in. Upload stages a PREVIEW batch and writes nothing. It reports what would land, what was rejected and why, and which rows would overwrite an existing value. Only commit touches market_signal. Effective dating: effective_date is the date the fact describes and comes from the file; observedAt is when we learned it and is stamped at commit. The uploader cannot reach observedAt — a March rate uploaded in August must stay invisible to a March scoring run, or the model trains on data that did not exist yet. Rejections name themselves: UNKNOWN_DISTRICT, UNKNOWN_COMMODITY, FUTURE_DATE, BAD_DATE, BAD_NUMBER, UNIT_MISMATCH, PRICE_ORDER, DUPLICATE_IN_FILE, MISSING_REQUIRED. A price far from last week warns but is accepted — a genuine price shock is the event this system exists to catch, and rejecting it would be backwards. Rows are written with source=MANUAL, which the unique index keeps separate from a feed's own row for the same week, so manual data never silently replaces a government feed. Also distinguishes INSUFFICIENT_HISTORY from NO_SIGNAL in the price derivations. A newly onboarded manual feed spends its first weeks below the four-week minimum, and reporting NO_SIGNAL there reads as a broken integration rather than a series still filling up. Result: margin_squeeze and output_price_change compute for the first time — 57 of 142 enterprises, exactly the DAIRY (30) and POULTRY (27) sectors that have an output commodity. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
LincPayandClaude Opus 5
|
20879b5904 |
fix(ingestion): make the pipeline actually run against live feeds
First end-to-end run against Postgres and the real government APIs, which surfaced four defects that static checks could not. - Agmarknet pointed at 9ef84268-…, "Current Daily Price": a today-only snapshot holding one arrival_date, in which none of the four pilot districts appear. Switched to 35985678-…, the historical resource. - Filter fields were lowercase. This resource matches exactly on the capitalised name; the lowercase form is tokenised, so "Madhya Pradesh" also matched Andhra, Himachal and Uttar Pradesh — wrong-state prices written under a pilot district, with freshness looking healthy. - Rows arrive unordered and the lookback filter is client side, so a page was mostly years-old data and every commodity wrote zero weeks. Added sort[Arrival_Date]=desc. - ExternalFeedConfig injected a RestClient.Builder bean Boot does not expose, so the context never built and the app could not start. Also: Open-Meteo meters by request cost, so the 30-year baseline call exhausted the minutely allowance and 429'd two districts out of their climate data. Rate-limited calls now retry with backoff, while genuine failures still fail once and are reported. FODDER moves to MANUAL — Agmarknet carries Green Fodder, but every recent row is a single market in Ambala, Haryana. Result: 16,000 rows fetched, 202 market weeks and 52 climate weeks written, 0 failed calls; 142 scored, 33 trip-wires fired. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
LincPayandClaude Opus 5
|
1b9b02f2c7 |
chore: initial commit — UdyamLinc backend, phases A-E
Spring Boot 4 / Java 17 / Postgres. Masters and serving (A), Agmarknet and climate ingestion (B), feature store with 14 derivations (C), SpEL rule engine and risk scoring (D). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |