commit 1b9b02f2c720649addf4cccc54af234327fdc14f Author: LincPay Date: Thu Aug 6 13:30:14 2026 +0530 chore: initial commit — UdyamLinc backend, phases A-E Spring Boot 4 / Java 17 / Postgres. Masters and serving (A), Agmarknet and climate ingestion (B), feature store with 14 derivations (C), SpEL rule engine and risk scoring (D). Co-Authored-By: Claude Opus 5 (1M context) diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..c4b4782 --- /dev/null +++ b/.gitignore @@ -0,0 +1,29 @@ +# Build output +target/ +*.jar +*.war + +# Secrets — the data.gov.in key and any local credential override live in the +# environment, never in the repo. application-local.properties is the escape +# hatch for developers who want them on disk. +.env +.env.* +application-local.properties +application-local.yml +src/main/resources/application-local.properties + +# IDE +.idea/ +*.iml +.vscode/ +.settings/ +.project +.classpath + +# OS +.DS_Store +Thumbs.db + +# Logs +*.log +logs/ diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md new file mode 100644 index 0000000..5b06c27 --- /dev/null +++ b/ARCHITECTURE.md @@ -0,0 +1,347 @@ +# UdyamLinc — Technical Architecture + +Team LincFusion · LincPay Solutions Pvt. Ltd. · NABARD Hackathon + +This document is the build contract for `UdyamLinc-Backend`. It records what the +system is, what is built, and what is deliberately deferred. + +--- + +## 1. Shape of the system + +``` + Enterprise PWA / Field Officer Console + │ + ▼ + Spring Boot application (one deployable) + │ + ┌──────────────┬──────────────────┼──────────────────┬──────────────┐ + ▼ ▼ ▼ ▼ ▼ +enterprise institutional ingestion rules ai +(ledger) (CBS, DPD, UPI) (mandi, climate) (trip-wires) (LightGBM) + │ │ │ │ │ + └──────────────┴────────┬─────────┴──────────────────┴──────────────┘ + ▼ + feature store (tall = truth, wide = read) + │ + ▼ + risk engine (rule-based now, model later) + │ + ▼ + alerts · dashboard · reports · officer playbook +``` + +### One deployable, not seven services + +The module boundaries above are **Java packages, not microservices**. At pilot +scale — four districts, 142 enterprises — separate deployables would buy +distributed transactions, cross-service auth and five release pipelines in +exchange for nothing. The package boundaries are enforced by convention (no +cross-package entity imports; talk to a neighbouring module through its service), +so `ingestion` or `ai` can be lifted out later when volume actually justifies it. + +For the same reason there is no separate API gateway component: at this scale the +application *is* the gateway. + +--- + +## 2. Three principles that decide whether this survives real data + +### 2.1 External feeds are never called in the request path + +Mandi prices, egg rates and climate series are pulled by a scheduled job at +02:00, validated, and written to `market_signal` / `climate_signal`. Scoring and +login read those tables and nothing else. A government endpoint being down +degrades tomorrow's freshness; it never fails a user request. + +### 2.2 The feature store is the only interface to the model + +Ingestion writes signals. Feature engineering turns signals into indicators. The +AI layer consumes **only** engineered features. This is what allows the model to +be retrained or replaced without touching ingestion, storage or the API. + +The store has two shapes: + +| Table | Shape | Purpose | +|---|---|---| +| `feature_value` | tall — one row per (enterprise, feature, date) | System of record. Auditable, schema-free as features are added. | +| `feature_snapshot` | wide — one row per (enterprise, scoring run) | What the engine and the trainer actually read. | + +Tall alone would mean ~5,700 row reads per scoring pass and far worse for +training. Wide alone would mean a migration every time a feature is added. + +### 2.3 Point-in-time correctness + +Every feature row carries **two** timestamps: + +- `valid_for_date` — the date the fact describes +- `observed_at` — when we learned it + +A mandi price for 3 March scraped on 7 March must be invisible to a 4 March +scoring run. Without this rule, backtests look excellent and production does not +reproduce them. This is what makes the claim of "no forward leakage across folds" +true rather than aspirational. + +--- + +## 3. Data model + +### Master + +| Table | Notes | +|---|---| +| `district` | The join key for every external signal. | +| `market` | A mandi, belonging to a district. | +| `commodity` | Maize, soya, egg, milk, wheat… | +| `sector` | The five risk packs, carrying sub-score weights in basis points. | +| `sector_commodity` | Which commodities a sector consumes vs sells. The **input/output** distinction is what makes `margin_squeeze` computable. | +| `role` | RBAC for the console. | + +### Enterprise & institutional + +| Table | Notes | +|---|---| +| `enterprise` | Opaque id, sector, district, lat/long. No owner name, phone, Aadhaar or account number — the no-PII promise is enforced by schema, not policy. | +| `ledger_entry` | Three-tap entries. `client_id` is generated on the phone so an offline entry keeps identity through sync. | +| `loan_record` | Per loan: sanctioned, outstanding, EMI, due date, **DPD**, bucket. DPD history is the training label. | +| `upi_summary` | Consented transaction aggregates. Source pluggable — Account Aggregator (needs FIU registration under Sahamati), PSP settlement reports, or manual. | + +### Signals & features + +| Table | Notes | +|---|---| +| `market_signal` | `(district, commodity, week)` — modal/min/max price. ~40 rows a day for four districts, not 142 per-enterprise fetches. | +| `climate_signal` | `(district, week)` — rainfall, tmax, anomalies vs 30-year normal. | +| `feature_definition` | Feature catalogue: code, family, unit, staleness tolerance. | +| `feature_value` | Tall store, with `valid_for_date` + `observed_at`. | +| `feature_snapshot` | Wide store, one per enterprise per run. | + +### Scoring & decision + +| Table | Notes | +|---|---| +| `scoring_run` | One pass. Carries `model_version` and `source` (`SERVER` \| `ON_DEVICE`) so a historic alert traces to the exact engine that raised it. | +| `forecast_point` | P10 / P50 / P90 and the obligation due, per month, h = 1…6. | +| `stress_score` | Composite 0–100 plus the five stored sub-scores, `engine`, `confidence`. | +| `reason_code` | SHAP driver as a sentence in both languages. Stored, not regenerated, so the wording an officer acted on is preserved. | +| `tripwire_rule` | Configurable rule: SpEL expression over the feature snapshot. | +| `risk_alert` | Raised by the model or by a rule. References the rule and the snapshot, so it is reproducible. | +| `field_visit` | Playbook intervention and its outcome — the feedback edge of the loop. | + +### Audit + +`api_log` (outbound calls to external feeds), `ingestion_run` (scheduler results, +counts, failures), and `scoring_run` doubling as the model prediction log. + +--- + +## 4. External data — provenance + +| Signal | Source | Access | Cadence | +|---|---|---|---| +| Mandi prices | Agmarknet, via the data.gov.in Open Government Data API | API key, JSON | Daily | +| Egg realisation | NECC (e2necc.com) | No API — scrape the daily city rate table | Daily | +| Rainfall, temperature | NASA POWER or Open-Meteo | Free, no key, long history for anomalies | Daily | +| District advisories | IMD Agromet | District-level advisory text | Tue / Fri | +| Milk procurement rate | State dairy federation | No API anywhere — MoU or admin entry | Monthly | +| Institutional extract | Lender CBS / LMS | SFTP CSV or read-only DB view | Nightly | +| UPI aggregates | Account Aggregator (RBI / Sahamati) | FIU registration + consent artefact | Weekly | + +IMD supplies the advisory; NASA POWER or Open-Meteo supplies the numbers, because +anomalies need decades of history that live endpoints do not serve. + +### Feature derivations + +Raw prices are not features. An absolute maize price of ₹2,140 means nothing. + +| Feature | Derivation | +|---|---| +| `feed_cost_change` | modal price vs 8-week trailing mean | +| `output_price_change` | same, on the sale-side commodity | +| `margin_squeeze` | input change − output change | +| `price_z_score` | (current − 3yr seasonal mean) / stdev | +| `rainfall_anomaly` | season-to-date vs 30yr normal | +| `heat_anomaly` | days above threshold vs normal | +| `cash_flow_change` | ledger net position vs trailing mean | +| `emi_delay` | DPD from `loan_record` | +| `inventory_days` | stock entries vs sales velocity | +| `upi_inflow_change` | 8-week rolling inflow delta | +| `feed_staleness_days` | age of the newest observation — guards all of the above | + +### Degradation rules + +- Carry the last known value forward, stamped with `feed_staleness_days`. +- **Trip-wires refuse to fire on stale data.** A rule firing on a three-week-old + price is worse than no rule. +- Missing shock features fall back to the sector's seasonal prior. The forecast + widens — P10 drops — which is the honest response to knowing less. +- Scoring never blocks on an external feed. The institutional record and the + ledger are always present; market data is an enhancement. + +--- + +## 5. Risk and rules + +The **risk engine** is rule-based first: a sector-weighted combination of the five +sub-score families, with weights held in the `sector` table so tuning needs no +deploy. It sits behind a `ScoringStrategy` interface, so the LightGBM engine +replaces it as a config change. + +The **rule engine** is deliberately independent. `tripwire_rule` rows carry a SpEL +expression evaluated against the feature snapshot: + +``` +emi_delay > 30 and cash_flow_change < -0.40 +``` + +Business teams edit rows; nobody deploys. SpEL rather than Drools — seven rules do +not justify a rules platform. + +A trip-wire alert survives a model retrain. That is the entire point: every alert +stays auditable and explainable independently of whatever the model currently +believes. + +--- + +## 6. Build status + +| Area | State | +|---|---| +| Field officer console — 8 screens | ✅ Built | +| Enterprise PWA — 4 tabs, offline, Hindi-first | ✅ Built | +| Ledger + offline queue + sync | ✅ Built | +| Auth, JWT, two scoped surfaces | ✅ Built | +| Serving entities, repositories, services | ✅ Built | +| Master data, controllers, seeder | ◐ Phase A | +| Ingestion (Agmarknet, climate, scheduler) | ✅ Phase B — see §6.1 | +| Feature store + derivations | ✅ Phase C — see §6.2 | +| Risk engine + rule engine + admin API | ✅ Phase D — see §6.3 | +| Console wire-up: rules screen, market panel, freshness | ✅ Phase E — see §6.4 | +| LightGBM quantile heads + SHAP | ○ R1 milestone | + +Phases B, C and D each terminate in a table you can inspect, so each is testable +without any of the machine learning existing. + +### 6.1 What Phase B delivered + +Package `com.lincfusion.udyamlinc.ingestion`. + +| Piece | Notes | +|---|---| +| `DataGovClient` | Agmarknet via data.gov.in. Case-insensitive field reads, because the resource's own casing changes between versions. Returns `null` on a failed call and an empty list on a filter that matched nothing — an outage and a typo must not look alike. | +| `OpenMeteoClient` | Reanalysis archive for the district centroid. No key. | +| `MandiIngestionService` | Folds daily arrivals into one `market_signal` per (district, commodity, ISO week). Upserts, so re-running corrects a partial week. | +| `ClimateIngestionService` | Weekly `climate_signal` with the 30-year normal alongside, and `climate_normal` computed once per district from a single baseline request. | +| `IngestionScheduler` | The 02:00 cron. Off by default; one pass at a time; every outcome — including a skip — lands in `ingestion_run`. | +| `IngestionController` | `/admin/ingestion/{runs,calls,signals/market,signals/climate,freshness}`, plus `POST /run` and `/run/{source}`. Reads open to any officer, triggers supervisor-only under the existing `/admin/**` rule. | + +Both signal tables carry `valid_for_date` + `observed_at`, and each repository +exposes a `findTrailing(...)` that filters on both — the point-in-time window +Phase C's derivations will read. `/admin/ingestion/freshness` is the staleness +number Phase D's trip-wires must consult before firing. + +**Configuration.** `udyam.ingestion.enabled` is `false` by default so a demo boot +never calls out. `udyam.ingestion.datagov.api-key` (env `DATAGOV_API_KEY`) is +required for Agmarknet; without it the run records `SKIPPED` rather than failing. +Climate needs no key. + +**Still deferred from §4.** NECC egg rates (no API — needs an HTML scrape), IMD +Agromet advisories, and the dairy federation rate (no API anywhere). Their +commodities are seeded and their `source` values reserved; nothing pulls them yet. + +### 6.2 What Phase C delivered + +Package `com.lincfusion.udyamlinc.features`, plus the two institutional tables of +§3 the derivations read from (`loan_record`, `upi_summary`). + +| Piece | Notes | +|---|---| +| `feature_definition` | The catalogue: family, unit, source kind, direction, and `staleness_tolerance_days` — the column a trip-wire consults before it is allowed to fire. Seeded idempotently at boot. | +| `feature_value` | The tall store. A feature that cannot be computed is written as **null with a `missing_reason`**, never skipped — `NO_SIGNAL`, `NO_LEDGER`, `NO_CBS`, `NO_CONSENT`, `NO_FORECAST`, `INSUFFICIENT_HISTORY`. "We did not know" and "we never looked" are different facts. | +| `feature_snapshot` | The wide store, one per (enterprise, as-of date), carrying `feature_count` / `missing_count` / `max_staleness_days` so a snapshot mostly made of holes cannot read as confident. `asFeatureMap()` exposes the same values under their snake_case codes — the shape a SpEL rule is written against. | +| `FeatureEngineeringService` | All fourteen derivations. Every read is point-in-time on both `valid_for_date` and `observed_at`, so recomputing last month uses only what was knowable last month. | +| `FeatureController` | `/admin/features/{definitions,snapshot/{id},values/{id},coverage}` and `POST /compute`, `POST /compute/{id}`. `coverage` reports the populated share per feature — a feature at 4 % coverage is a column, not a feature. | + +**Catalogue.** §4's eleven, plus three the seeded trip-wires already named: +`dscr_p10` (worst month of `min(p10 / obligation)` over the latest run), +`days_since_last_entry`, and `emi_to_inflow`. A rule referring to a feature +nobody computes is a rule that silently never fires. These three also give the +OBLIGATION_LOAD family the inputs it otherwise lacked. + +**Demo inputs.** The seeder now writes one `loan_record` per enterprise with DPD +tracking the band, so `emi_delay` and `emi_to_inflow` are real without a CBS +feed. `upi_summary` has an entity and a point-in-time query but no source until +Account Aggregator registration — `upi_inflow_change` therefore resolves to null +with `NO_CONSENT`, which is the honest answer and exercises the degradation path. +Price and climate features stay null until Phase B has actually run. + +### 6.3 What Phase D delivered + +Packages `rules` and `risk`. + +| Piece | Notes | +|---|---| +| `RuleEvaluator` | SpEL over the feature rows, in a `SimpleEvaluationContext` restricted to property access — these expressions are authored by business users, so no method calls or bean lookups. Compiled expressions cached by rule code **and version**, so an edit invalidates its own entry; the admin write endpoints also clear the cache. | +| `RuleOutcome` | Every rule returns FIRED / NOT_FIRED / SKIPPED_STALE / SKIPPED_MISSING / SKIPPED_SECTOR / ERROR. "No alert because the enterprise is fine" and "no alert because the feed has been down a fortnight" must never look the same. | +| `ScoringStrategy` | The seam the model arrives through. `udyam.risk.engine` binds the implementation; LightGBM later is that one line. Implementations read features and nothing else — no reaching back into the signal tables. | +| `RuleBasedScoringStrategy` | Five families onto 0–100, combined with the sector's basis-point weights. Each family takes its **worst** signal, not the mean — averaging lets a calm dimension mask an acute one, and the acute one is where the officer needs to go. | +| `RiskEngineService` | Compute features → score → evaluate trip-wires against the **same feature rows, never the score** → persist score, top-3 reason codes, alerts. | +| `RiskEngineController` | `POST /admin/scoring/run`, `/run/{id}`, `GET /admin/scoring/runs`, and `GET /admin/scoring/explain/{id}` — every rule's outcome with no writes. | +| `ScoringScheduler` | 03:00, an hour behind ingestion so it scores what that job wrote. Off by default. | + +**The staleness gate.** Before any expression runs, each of a rule's +`requiredFeatures` must have a value and be within the **stricter** of the rule's +`maxStalenessDays` and the catalogue's per-feature tolerance — a supervisor +tightening a rule must not be overridden by a generous default. This is the +promise of §4's degradation rules, enforced rather than documented. + +**Band versus score.** A fired trip-wire escalates the band; it never rewrites +the score. They answer different questions — "how stressed is this enterprise" +and "has a specific agreed line been crossed" — and an audit needs both. +A family with no usable feature falls back to `udyam.risk.family-prior` rather +than scoring zero, because zero reads as "safe here" when the truth is "unknown", +and each family on a prior costs confidence. + +**What the seeded rules can actually do today.** TW-01 (`dscr_p10`) and TW-03 +(`days_since_last_entry`) evaluate from seeded data. TW-04 (`emi_to_inflow`) +evaluates once loan records exist, which the seeder now writes. TW-02 +(`upi_inflow_change`) and TW-05 (`price_z_score`, `heat_anomaly`) will report +SKIPPED_MISSING until Account Aggregator consent and Phase B ingestion +respectively — visibly skipped, which is the intended behaviour rather than a +silent no-op. + +**Not built here.** The quantile forecast is still the seeded path; this engine +scores and alerts but does not produce `forecast_point` rows. `dscr_p10` reads +whatever the latest run stored. Generating forecasts is part of the R1 model +milestone. + +### 6.4 What Phase E delivered + +In `UdyamLinc-UI`, two console screens and one piece of chrome — all three about +provenance, because the failure mode that matters is silent. A score computed +from a three-week-old mandi price looks exactly like one computed this morning. + +| Piece | Notes | +|---|---| +| `features/officer/DataFeedsPage.tsx` | `/officer/feeds` — freshness per district first, then the mandi panel, the rainfall/heat panel with the 30-year normal beside each observation, and the ingestion run log. A skipped run appears as a row, so "nothing ran last night" never reads as "it ran and found nothing". | +| `features/officer/RulesPage.tsx` | `/officer/rules` — every trip-wire with its expression and tolerance, its outcome for a chosen enterprise, and the feature snapshot underneath with each null's reason. A banner counts rules that could not run. | +| `FreshnessChip` in `OfficerShell` | Data age in the header on every screen, amber past ten days, linking to the feeds page. | +| `lib/demo/feeds.ts` | Demo fallback that applies the **same two guards in the same order** as `RuleEvaluator` — missing, then stale, then the expression — so the offline demo cannot claim a rule fired that the real engine would have skipped. | + +Both screens follow the house pattern: `useQuery` against the real endpoint, +in-file demo fallback, and a "Demo data" pill when the backend is unreachable. +`npm run build` and `oxlint` are clean. + +**Deliberately not added:** rule *editing* from the console. The backend accepts +`POST`/`PUT /admin/rules` and bumps the version on every edit, but a supervisor +changing a live threshold from a demo build with no audit trail in front of it is +a worse idea than the round trip through an API client. + +--- + +## 7. Explicitly not being built yet + +Kafka, Neo4j, Elasticsearch, Keycloak, MinIO, and separate deployables. All appear +on the submission's technology slide; none earn their keep at four districts. +Postgres plus a scheduled job carries the pilot comfortably. Kafka arrives when +ingestion volume or a second consumer demands it — not before. diff --git a/pom.xml b/pom.xml new file mode 100644 index 0000000..c726a1c --- /dev/null +++ b/pom.xml @@ -0,0 +1,137 @@ + + + 4.0.0 + + + org.springframework.boot + spring-boot-starter-parent + 4.0.1 + + + + com.lincfusion + udyamlinc-backend + 0.0.1-SNAPSHOT + jar + + udyamlinc-backend + UdyamLinc — predictive cash-flow intelligence and early warning for rural microenterprises (NABARD Hackathon) + + + 17 + + + + + + + org.springframework.boot + spring-boot-starter-web + + + + + org.springframework.boot + spring-boot-starter-security + + + + + org.springframework.boot + spring-boot-starter-data-jpa + + + + + org.springframework.boot + spring-boot-starter-validation + + + + + org.postgresql + postgresql + runtime + + + + + io.jsonwebtoken + jjwt-api + 0.11.5 + + + io.jsonwebtoken + jjwt-impl + 0.11.5 + runtime + + + io.jsonwebtoken + jjwt-jackson + 0.11.5 + runtime + + + + + com.fasterxml.jackson.core + jackson-databind + + + com.fasterxml.jackson.datatype + jackson-datatype-jsr310 + + + + + org.projectlombok + lombok + 1.18.36 + provided + + + + + org.springframework.boot + spring-boot-starter-test + test + + + + + + udyamlinc-backend + + + + + org.springframework.boot + spring-boot-maven-plugin + + com.lincfusion.udyamlinc.UdyamLincApplication + + + + + org.apache.maven.plugins + maven-compiler-plugin + 3.11.0 + + 17 + + + org.projectlombok + lombok + 1.18.36 + + + + + + + + + diff --git a/src/main/java/com/lincfusion/udyamlinc/UdyamLincApplication.java b/src/main/java/com/lincfusion/udyamlinc/UdyamLincApplication.java new file mode 100644 index 0000000..14c2f92 --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/UdyamLincApplication.java @@ -0,0 +1,21 @@ +package com.lincfusion.udyamlinc; + +import org.springframework.boot.SpringApplication; +import org.springframework.boot.autoconfigure.SpringBootApplication; + +/** + * UdyamLinc — predictive cash-flow intelligence and early warning for rural + * microenterprises. NABARD Hackathon submission by Team LincFusion. + * + *

This service is the delivery layer of the architecture on the deck: it + * serves the field officer console and the enterprise PWA from stored forecast + * runs and scores. The AI/ML layer (LightGBM quantile heads, SHAP) and the + * ingestion pipeline are separate components and are not implemented here. + */ +@SpringBootApplication +public class UdyamLincApplication { + + public static void main(String[] args) { + SpringApplication.run(UdyamLincApplication.class, args); + } +} diff --git a/src/main/java/com/lincfusion/udyamlinc/admin/AdminController.java b/src/main/java/com/lincfusion/udyamlinc/admin/AdminController.java new file mode 100644 index 0000000..9708a64 --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/admin/AdminController.java @@ -0,0 +1,128 @@ +package com.lincfusion.udyamlinc.admin; + +import com.lincfusion.udyamlinc.common.ApiResponse; +import com.lincfusion.udyamlinc.common.ResourceNotFoundException; +import com.lincfusion.udyamlinc.master.*; +import com.lincfusion.udyamlinc.rules.RuleEvaluator; +import com.lincfusion.udyamlinc.rules.TripWireRule; +import com.lincfusion.udyamlinc.rules.TripWireRuleRepository; +import jakarta.validation.Valid; +import lombok.RequiredArgsConstructor; +import org.springframework.security.core.annotation.AuthenticationPrincipal; +import org.springframework.web.bind.annotation.*; + +import java.time.LocalDateTime; +import java.util.List; + +/** + * Configuration surface for supervisors: sector weights and trip-wire rules. + * + *

These are the two things the deck promises business teams can change + * without a deploy. Reads are open to any officer; writes require a supervisor — + * enforced by path and method in {@code SecurityConfig}. + */ +@RestController +@RequestMapping("/admin") +@RequiredArgsConstructor +public class AdminController { + + private final SectorRepository sectorRepository; + private final DistrictRepository districtRepository; + private final CommodityRepository commodityRepository; + private final TripWireRuleRepository ruleRepository; + private final RuleEvaluator ruleEvaluator; + + // ── masters ───────────────────────────────────────────── + + @GetMapping("/sectors") + public ApiResponse> sectors() { + return ApiResponse.ok(sectorRepository.findAll()); + } + + @GetMapping("/districts") + public ApiResponse> districts() { + return ApiResponse.ok(districtRepository.findByActiveTrue()); + } + + @GetMapping("/commodities") + public ApiResponse> commodities() { + return ApiResponse.ok(commodityRepository.findByActiveTrue()); + } + + /** + * Retune a sector pack. Weights are basis points and must sum to 10000 — + * rejected otherwise, because a pack that does not sum silently rescales + * every score in that sector. + */ + @PutMapping("/sectors/{code}/weights") + public ApiResponse updateWeights(@PathVariable String code, @RequestBody Sector body) { + Sector sector = sectorRepository.findByCode(code) + .orElseThrow(() -> new ResourceNotFoundException("No sector " + code)); + sector.setWLiquidity(body.getWLiquidity()); + sector.setWBehavioural(body.getWBehavioural()); + sector.setWTrendVolatility(body.getWTrendVolatility()); + sector.setWObligationLoad(body.getWObligationLoad()); + sector.setWExternalShock(body.getWExternalShock()); + if (!sector.weightsBalanced()) { + throw new IllegalArgumentException("Sector weights must sum to 10000 basis points"); + } + return ApiResponse.ok("Weights updated", sectorRepository.save(sector)); + } + + // ── trip-wire rules ───────────────────────────────────── + + @GetMapping("/rules") + public ApiResponse> rules() { + return ApiResponse.ok(ruleRepository.findAll()); + } + + @PostMapping("/rules") + public ApiResponse create( + @Valid @RequestBody TripWireRule rule, + @AuthenticationPrincipal String actor) { + rule.setId(null); + rule.setVersion(1); + rule.setUpdatedBy(actor); + rule.setUpdatedAt(LocalDateTime.now()); + ruleEvaluator.invalidate(); + return ApiResponse.ok("Rule created", ruleRepository.save(rule)); + } + + /** Editing bumps the version, so an old alert can still name the rule that fired. */ + @PutMapping("/rules/{code}") + public ApiResponse update( + @PathVariable String code, + @RequestBody TripWireRule body, + @AuthenticationPrincipal String actor) { + TripWireRule rule = ruleRepository.findByCode(code) + .orElseThrow(() -> new ResourceNotFoundException("No rule " + code)); + rule.setLabel(body.getLabel()); + rule.setLabelHi(body.getLabelHi()); + rule.setExpression(body.getExpression()); + rule.setThresholdText(body.getThresholdText()); + rule.setSeverity(body.getSeverity()); + rule.setSectorCode(body.getSectorCode()); + rule.setRequiredFeatures(body.getRequiredFeatures()); + rule.setMaxStalenessDays(body.getMaxStalenessDays()); + rule.setMessageEn(body.getMessageEn()); + rule.setMessageHi(body.getMessageHi()); + rule.setActive(body.getActive()); + rule.setVersion(rule.getVersion() == null ? 1 : rule.getVersion() + 1); + rule.setUpdatedBy(actor); + rule.setUpdatedAt(LocalDateTime.now()); + ruleEvaluator.invalidate(); + return ApiResponse.ok("Rule updated", ruleRepository.save(rule)); + } + + @DeleteMapping("/rules/{code}") + public ApiResponse deactivate(@PathVariable String code, @AuthenticationPrincipal String actor) { + TripWireRule rule = ruleRepository.findByCode(code) + .orElseThrow(() -> new ResourceNotFoundException("No rule " + code)); + rule.setActive(false); + rule.setUpdatedBy(actor); + rule.setUpdatedAt(LocalDateTime.now()); + ruleRepository.save(rule); + ruleEvaluator.invalidate(); + return ApiResponse.ok("Rule deactivated", code); + } +} diff --git a/src/main/java/com/lincfusion/udyamlinc/app/EnterpriseAppController.java b/src/main/java/com/lincfusion/udyamlinc/app/EnterpriseAppController.java new file mode 100644 index 0000000..afaef4c --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/app/EnterpriseAppController.java @@ -0,0 +1,63 @@ +package com.lincfusion.udyamlinc.app; + +import com.lincfusion.udyamlinc.common.ApiResponse; +import com.lincfusion.udyamlinc.dto.UdyamRequests.ConsentRequest; +import com.lincfusion.udyamlinc.dto.UdyamRequests.LedgerEntryRequest; +import com.lincfusion.udyamlinc.dto.UdyamResponses.AppHomeResponse; +import com.lincfusion.udyamlinc.dto.UdyamResponses.SyncResultResponse; +import com.lincfusion.udyamlinc.enterprise.Enterprise; +import com.lincfusion.udyamlinc.enterprise.LedgerEntry; +import com.lincfusion.udyamlinc.risk.RiskAlert; +import jakarta.validation.Valid; +import lombok.RequiredArgsConstructor; +import org.springframework.security.core.annotation.AuthenticationPrincipal; +import org.springframework.web.bind.annotation.*; + +import java.util.List; + +/** + * The enterprise PWA API. + * + *

The enterprise id always comes from the token's subject, never from the + * request — an owner cannot read another enterprise's record by changing a path + * parameter, because there is no path parameter to change. + */ +@RestController +@RequestMapping("/app") +@RequiredArgsConstructor +public class EnterpriseAppController { + + private final EnterpriseAppService appService; + + @GetMapping("/home") + public ApiResponse home(@AuthenticationPrincipal String enterpriseId) { + return ApiResponse.ok(appService.home(enterpriseId)); + } + + @GetMapping("/ledger") + public ApiResponse> ledger(@AuthenticationPrincipal String enterpriseId) { + return ApiResponse.ok(appService.ledger(enterpriseId)); + } + + /** Drain the phone's offline queue. Safe to call twice with the same batch. */ + @PostMapping("/ledger/sync") + public ApiResponse sync( + @AuthenticationPrincipal String enterpriseId, + @Valid @RequestBody List batch) { + return ApiResponse.ok("Synced", appService.sync(enterpriseId, batch)); + } + + @GetMapping("/alerts") + public ApiResponse> alerts(@AuthenticationPrincipal String enterpriseId) { + return ApiResponse.ok(appService.alerts(enterpriseId)); + } + + @PutMapping("/consent") + public ApiResponse consent( + @AuthenticationPrincipal String enterpriseId, + @RequestBody ConsentRequest request) { + return ApiResponse.ok( + request.active() ? "Consent recorded" : "Consent withdrawn", + appService.setConsent(enterpriseId, request.active())); + } +} diff --git a/src/main/java/com/lincfusion/udyamlinc/app/EnterpriseAppService.java b/src/main/java/com/lincfusion/udyamlinc/app/EnterpriseAppService.java new file mode 100644 index 0000000..ed30b53 --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/app/EnterpriseAppService.java @@ -0,0 +1,156 @@ +package com.lincfusion.udyamlinc.app; + +import com.lincfusion.udyamlinc.common.ResourceNotFoundException; +import com.lincfusion.udyamlinc.dto.UdyamRequests.LedgerEntryRequest; +import com.lincfusion.udyamlinc.dto.UdyamResponses.AppHomeResponse; +import com.lincfusion.udyamlinc.dto.UdyamResponses.SyncResultResponse; +import com.lincfusion.udyamlinc.enterprise.Enterprise; +import com.lincfusion.udyamlinc.enterprise.EnterpriseRepository; +import com.lincfusion.udyamlinc.enterprise.LedgerEntry; +import com.lincfusion.udyamlinc.enterprise.LedgerEntryRepository; +import com.lincfusion.udyamlinc.master.Sector; +import com.lincfusion.udyamlinc.master.SectorRepository; +import com.lincfusion.udyamlinc.risk.ForecastPointRepository; +import com.lincfusion.udyamlinc.risk.RiskAlert; +import com.lincfusion.udyamlinc.risk.RiskAlertRepository; +import lombok.RequiredArgsConstructor; +import org.springframework.stereotype.Service; +import org.springframework.transaction.annotation.Transactional; + +import java.math.BigDecimal; +import java.time.LocalDate; +import java.time.LocalDateTime; +import java.util.List; +import java.util.Map; + +/** + * The enterprise PWA's server side. + * + *

Note what this deliberately does not do: it never fails a ledger + * write because of connectivity. The phone owns the entry, generates its id and + * queues it; this service only accepts the queue and de-duplicates it. + */ +@Service +@RequiredArgsConstructor +public class EnterpriseAppService { + + private final EnterpriseRepository enterpriseRepository; + private final LedgerEntryRepository ledgerRepository; + private final ForecastPointRepository pointRepository; + private final RiskAlertRepository alertRepository; + private final SectorRepository sectorRepository; + + public AppHomeResponse home(String enterpriseId) { + Enterprise enterprise = requireEnterprise(enterpriseId); + RiskAlert alert = alertRepository.findByEnterpriseIdOrderByRaisedAtDesc(enterpriseId).stream() + .filter(a -> !"CLOSED".equals(a.getState())) + .findFirst() + .orElse(null); + + List today = ledgerRepository.findByEnterpriseIdAndEntryDate(enterpriseId, LocalDate.now()); + Map advice = adviceFor(sectorCode(enterprise)); + + return new AppHomeResponse( + enterprise, + alert, + pointRepository.findByEnterpriseIdOrderByHorizonAsc(enterpriseId), + sum(today, "IN"), + sum(today, "OUT"), + advice.get("en"), + advice.get("hi"), + lastSyncedAt(enterpriseId)); + } + + public List ledger(String enterpriseId) { + return ledgerRepository.findByEnterpriseIdOrderByEntryDateDesc(enterpriseId); + } + + public List alerts(String enterpriseId) { + return alertRepository.findByEnterpriseIdOrderByRaisedAtDesc(enterpriseId); + } + + /** + * Drain the phone's offline queue. Idempotent on {@code clientId}, so a queue + * submitted twice after a flaky reconnect cannot double-count. + */ + @Transactional + public SyncResultResponse sync(String enterpriseId, List batch) { + requireEnterprise(enterpriseId); + int accepted = 0; + int duplicates = 0; + + for (LedgerEntryRequest r : batch) { + if (ledgerRepository.findByClientId(r.clientId()).isPresent()) { + duplicates++; + continue; + } + ledgerRepository.save(LedgerEntry.builder() + .clientId(r.clientId()) + .enterpriseId(enterpriseId) + .kind(r.kind().toUpperCase()) + .amount(r.amount()) + .category(r.category()) + .note(r.note()) + .entryDate(r.entryDate() == null ? LocalDate.now() : LocalDate.parse(r.entryDate())) + .syncedAt(LocalDateTime.now()) + .build()); + accepted++; + } + return new SyncResultResponse(accepted, duplicates, LocalDateTime.now().toString()); + } + + /** Consent is revocable and purpose-bound; withdrawing it stops scoring. */ + @Transactional + public Enterprise setConsent(String enterpriseId, boolean active) { + Enterprise enterprise = requireEnterprise(enterpriseId); + enterprise.setConsentActive(active); + enterprise.setConsentUpdatedAt(LocalDateTime.now()); + return enterpriseRepository.save(enterprise); + } + + private String lastSyncedAt(String enterpriseId) { + return ledgerRepository.findByEnterpriseIdOrderByEntryDateDesc(enterpriseId).stream() + .map(LedgerEntry::getSyncedAt) + .filter(java.util.Objects::nonNull) + .max(LocalDateTime::compareTo) + .map(LocalDateTime::toString) + .orElse(null); + } + + private BigDecimal sum(List entries, String kind) { + return entries.stream() + .filter(e -> kind.equals(e.getKind())) + .map(LedgerEntry::getAmount) + .reduce(BigDecimal.ZERO, BigDecimal::add); + } + + private String sectorCode(Enterprise e) { + return sectorRepository.findById(e.getSectorId()).map(Sector::getCode).orElse("RURAL_RETAIL"); + } + + private Enterprise requireEnterprise(String enterpriseId) { + return enterpriseRepository.findByEnterpriseId(enterpriseId) + .orElseThrow(() -> new ResourceNotFoundException("No enterprise " + enterpriseId)); + } + + /** Sector-specific suggestion shown on the Home tab, in both languages. */ + private Map adviceFor(String sector) { + return switch (sector) { + case "POULTRY" -> Map.of( + "en", "Buy feed now — the rate is up 18% and may rise further.", + "hi", "अभी दाना ख़रीद लें — रेट 18% बढ़ा है और आगे और बढ़ सकता है।"); + case "DAIRY" -> Map.of( + "en", "Stock fodder now and press the society for faster payment.", + "hi", "चारा अभी भर लें और समिति से भुगतान जल्दी माँगें।"); + case "FOOD_PROC" -> Map.of( + "en", "Buy raw material in bulk and hold buyers to 15-day settlement.", + "hi", "कच्चा माल थोक में लें और खरीदार से 15 दिन में भुगतान तय करें।"); + case "HANDICRAFTS" -> Map.of( + "en", "Add a second buyer so sales do not depend on one account.", + "hi", "तैयार माल की बिक्री के लिए दूसरा खरीदार भी जोड़ें।"); + default -> Map.of( + "en", "Reduce credit sales and collect daily over UPI.", + "hi", "उधार बिक्री कम करें और रोज़ UPI से वसूली करें।"); + }; + } +} diff --git a/src/main/java/com/lincfusion/udyamlinc/auth/AuthController.java b/src/main/java/com/lincfusion/udyamlinc/auth/AuthController.java new file mode 100644 index 0000000..219a398 --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/auth/AuthController.java @@ -0,0 +1,37 @@ +package com.lincfusion.udyamlinc.auth; + +import com.lincfusion.udyamlinc.common.ApiResponse; +import com.lincfusion.udyamlinc.dto.UdyamRequests.AppLoginRequest; +import com.lincfusion.udyamlinc.dto.UdyamRequests.OfficerLoginRequest; +import com.lincfusion.udyamlinc.dto.UdyamResponses.EnterpriseSessionResponse; +import com.lincfusion.udyamlinc.dto.UdyamResponses.OfficerSessionResponse; +import jakarta.validation.Valid; +import lombok.RequiredArgsConstructor; +import org.springframework.web.bind.annotation.*; + +import java.util.Map; + +/** The only unauthenticated surface. */ +@RestController +@RequestMapping("/auth") +@RequiredArgsConstructor +public class AuthController { + + private final AuthService authService; + + @PostMapping("/officer-login") + public ApiResponse officerLogin(@Valid @RequestBody OfficerLoginRequest request) { + return ApiResponse.ok("Signed in", authService.officerLogin(request)); + } + + @PostMapping("/app-login") + public ApiResponse appLogin(@Valid @RequestBody AppLoginRequest request) { + return ApiResponse.ok("Signed in", authService.appLogin(request)); + } + + /** Liveness probe, also used by the UI to decide whether to fall back to demo data. */ + @GetMapping("/health") + public ApiResponse> health() { + return ApiResponse.ok(Map.of("service", "udyamlinc-backend", "status", "UP")); + } +} diff --git a/src/main/java/com/lincfusion/udyamlinc/auth/AuthService.java b/src/main/java/com/lincfusion/udyamlinc/auth/AuthService.java new file mode 100644 index 0000000..af91972 --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/auth/AuthService.java @@ -0,0 +1,94 @@ +package com.lincfusion.udyamlinc.auth; + +import com.lincfusion.udyamlinc.common.ResourceNotFoundException; +import com.lincfusion.udyamlinc.dto.UdyamRequests.AppLoginRequest; +import com.lincfusion.udyamlinc.dto.UdyamRequests.OfficerLoginRequest; +import com.lincfusion.udyamlinc.dto.UdyamResponses.EnterpriseSessionResponse; +import com.lincfusion.udyamlinc.dto.UdyamResponses.OfficerSessionResponse; +import com.lincfusion.udyamlinc.enterprise.Enterprise; +import com.lincfusion.udyamlinc.enterprise.EnterpriseRepository; +import com.lincfusion.udyamlinc.master.Sector; +import com.lincfusion.udyamlinc.master.SectorRepository; +import com.lincfusion.udyamlinc.security.JwtUtil; +import lombok.RequiredArgsConstructor; +import org.springframework.security.authentication.BadCredentialsException; +import org.springframework.security.crypto.password.PasswordEncoder; +import org.springframework.stereotype.Service; +import org.springframework.transaction.annotation.Transactional; + +import java.time.LocalDateTime; +import java.util.Arrays; +import java.util.List; + +/** + * Sign-in for both surfaces: resolve the user, verify the secret, issue a token + * scoped to exactly what that user may read. + */ +@Service +@RequiredArgsConstructor +public class AuthService { + + private final UdyamUserRepository userRepository; + private final EnterpriseRepository enterpriseRepository; + private final SectorRepository sectorRepository; + private final PasswordEncoder passwordEncoder; + private final JwtUtil jwtUtil; + + @Transactional + public OfficerSessionResponse officerLogin(OfficerLoginRequest request) { + UdyamUser user = userRepository.findByUsernameIgnoreCase(request.username().trim()) + .filter(u -> "OFFICER".equalsIgnoreCase(u.getScope())) + .orElseThrow(() -> new BadCredentialsException("No field officer is registered with that username")); + + verify(user, request.password()); + touch(user); + + String token = jwtUtil.generateOfficerToken(user.getUsername(), user.getRole(), user.getBlocks()); + List blocks = user.getBlocks() == null || user.getBlocks().isBlank() + ? List.of() + : Arrays.stream(user.getBlocks().split(",")).map(String::trim).toList(); + + return new OfficerSessionResponse( + token, "Bearer", user.getUsername(), user.getFullName(), user.getRole(), blocks, user.getBranch()); + } + + @Transactional + public EnterpriseSessionResponse appLogin(AppLoginRequest request) { + String enterpriseId = request.enterpriseId().trim().toUpperCase(); + + Enterprise enterprise = enterpriseRepository.findByEnterpriseId(enterpriseId) + .orElseThrow(() -> new ResourceNotFoundException("No enterprise is enrolled as " + enterpriseId)); + if (!Boolean.TRUE.equals(enterprise.getConsentActive())) { + throw new BadCredentialsException("Consent has been withdrawn for this enterprise"); + } + + UdyamUser user = userRepository.findByEnterpriseIdAndScopeIgnoreCase(enterpriseId, "ENTERPRISE") + .orElseThrow(() -> new BadCredentialsException("No app login exists for this enterprise")); + + verify(user, request.pin()); + touch(user); + + String sector = sectorRepository.findById(enterprise.getSectorId()) + .map(Sector::getCode) + .orElse("RURAL_RETAIL"); + + String token = jwtUtil.generateEnterpriseToken(enterprise.getEnterpriseId(), sector); + return new EnterpriseSessionResponse( + token, "Bearer", enterprise.getEnterpriseId(), enterprise.getName(), enterprise.getNameHi(), + sector, enterprise.getBlock(), "hi"); + } + + private void verify(UdyamUser user, String secret) { + if (!Boolean.TRUE.equals(user.getActive())) { + throw new BadCredentialsException("This login is deactivated"); + } + if (user.getPassword() == null || !passwordEncoder.matches(secret, user.getPassword())) { + throw new BadCredentialsException("Invalid credentials"); + } + } + + private void touch(UdyamUser user) { + user.setLastLoginAt(LocalDateTime.now()); + userRepository.save(user); + } +} diff --git a/src/main/java/com/lincfusion/udyamlinc/auth/UdyamUser.java b/src/main/java/com/lincfusion/udyamlinc/auth/UdyamUser.java new file mode 100644 index 0000000..91a92c1 --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/auth/UdyamUser.java @@ -0,0 +1,72 @@ +package com.lincfusion.udyamlinc.auth; + +import jakarta.persistence.*; +import lombok.*; + +import java.time.LocalDateTime; + +/** + * A console or app login. Deliberately thin: no phone, address or document + * number. The only identity here is a work handle for an officer, or an + * enterprise id for an owner. + */ +@Entity +@Table(name = "udyam_user", indexes = { + @Index(name = "ix_user_username", columnList = "username", unique = true), + @Index(name = "ix_user_enterprise", columnList = "enterpriseId"), +}) +@Getter +@Setter +@NoArgsConstructor +@AllArgsConstructor +@Builder +public class UdyamUser { + + @Id + @GeneratedValue(strategy = GenerationType.IDENTITY) + private Long id; + + @Column(nullable = false, unique = true, length = 64) + private String username; + + /** Display handle for the console header. Not a customer name. */ + @Column(length = 96) + private String fullName; + + @Column(nullable = false) + private String password; + + /** Denormalised role code. {@code roleId} is the authority once RBAC lands. */ + @Column(nullable = false, length = 32) + private String role; + + private Long roleId; + + /** OFFICER | ENTERPRISE — decides which API surface the token opens. */ + @Column(nullable = false, length = 16) + private String scope; + + /** Set only for scope=ENTERPRISE logins. */ + @Column(length = 24) + private String enterpriseId; + + /** CSV of blocks an officer covers. */ + @Column(length = 255) + private String blocks; + + @Column(length = 96) + private String branch; + + @Column(nullable = false) + private Boolean active; + + private LocalDateTime lastLoginAt; + + private LocalDateTime createdAt; + + @PrePersist + void onCreate() { + if (createdAt == null) createdAt = LocalDateTime.now(); + if (active == null) active = Boolean.TRUE; + } +} diff --git a/src/main/java/com/lincfusion/udyamlinc/auth/UdyamUserRepository.java b/src/main/java/com/lincfusion/udyamlinc/auth/UdyamUserRepository.java new file mode 100644 index 0000000..4e32b51 --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/auth/UdyamUserRepository.java @@ -0,0 +1,12 @@ +package com.lincfusion.udyamlinc.auth; + +import org.springframework.data.jpa.repository.JpaRepository; + +import java.util.Optional; + +public interface UdyamUserRepository extends JpaRepository { + + Optional findByUsernameIgnoreCase(String username); + + Optional findByEnterpriseIdAndScopeIgnoreCase(String enterpriseId, String scope); +} diff --git a/src/main/java/com/lincfusion/udyamlinc/common/ApiResponse.java b/src/main/java/com/lincfusion/udyamlinc/common/ApiResponse.java new file mode 100644 index 0000000..1b73604 --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/common/ApiResponse.java @@ -0,0 +1,29 @@ +package com.lincfusion.udyamlinc.common; + +import lombok.AllArgsConstructor; +import lombok.Getter; + +/** + * The single response envelope every endpoint returns: {@code {success, message, data}}. + * The UI's api.ts unwraps {@code .data} — keep this shape stable. + */ +@Getter +@AllArgsConstructor +public class ApiResponse { + + private final boolean success; + private final String message; + private final T data; + + public static ApiResponse ok(T data) { + return new ApiResponse<>(true, "OK", data); + } + + public static ApiResponse ok(String message, T data) { + return new ApiResponse<>(true, message, data); + } + + public static ApiResponse fail(String message) { + return new ApiResponse<>(false, message, null); + } +} diff --git a/src/main/java/com/lincfusion/udyamlinc/common/GlobalExceptionHandler.java b/src/main/java/com/lincfusion/udyamlinc/common/GlobalExceptionHandler.java new file mode 100644 index 0000000..0fd455b --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/common/GlobalExceptionHandler.java @@ -0,0 +1,44 @@ +package com.lincfusion.udyamlinc.common; + +import org.springframework.http.HttpStatus; +import org.springframework.http.ResponseEntity; +import org.springframework.security.authentication.BadCredentialsException; +import org.springframework.web.bind.MethodArgumentNotValidException; +import org.springframework.web.bind.annotation.ExceptionHandler; +import org.springframework.web.bind.annotation.RestControllerAdvice; + +import java.util.stream.Collectors; + +/** Every failure leaves the service in the same envelope the UI already parses. */ +@RestControllerAdvice +public class GlobalExceptionHandler { + + @ExceptionHandler(ResourceNotFoundException.class) + public ResponseEntity> notFound(ResourceNotFoundException e) { + return ResponseEntity.status(HttpStatus.NOT_FOUND).body(ApiResponse.fail(e.getMessage())); + } + + @ExceptionHandler(BadCredentialsException.class) + public ResponseEntity> badCredentials(BadCredentialsException e) { + return ResponseEntity.status(HttpStatus.UNAUTHORIZED).body(ApiResponse.fail(e.getMessage())); + } + + @ExceptionHandler(MethodArgumentNotValidException.class) + public ResponseEntity> invalid(MethodArgumentNotValidException e) { + String detail = e.getBindingResult().getFieldErrors().stream() + .map(f -> f.getField() + " " + f.getDefaultMessage()) + .collect(Collectors.joining("; ")); + return ResponseEntity.badRequest().body(ApiResponse.fail(detail)); + } + + @ExceptionHandler(IllegalArgumentException.class) + public ResponseEntity> illegal(IllegalArgumentException e) { + return ResponseEntity.badRequest().body(ApiResponse.fail(e.getMessage())); + } + + @ExceptionHandler(Exception.class) + public ResponseEntity> unexpected(Exception e) { + return ResponseEntity.status(HttpStatus.INTERNAL_SERVER_ERROR) + .body(ApiResponse.fail("Unexpected error: " + e.getMessage())); + } +} diff --git a/src/main/java/com/lincfusion/udyamlinc/common/ResourceNotFoundException.java b/src/main/java/com/lincfusion/udyamlinc/common/ResourceNotFoundException.java new file mode 100644 index 0000000..772800a --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/common/ResourceNotFoundException.java @@ -0,0 +1,8 @@ +package com.lincfusion.udyamlinc.common; + +public class ResourceNotFoundException extends RuntimeException { + + public ResourceNotFoundException(String message) { + super(message); + } +} diff --git a/src/main/java/com/lincfusion/udyamlinc/config/ExternalFeedConfig.java b/src/main/java/com/lincfusion/udyamlinc/config/ExternalFeedConfig.java new file mode 100644 index 0000000..6699229 --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/config/ExternalFeedConfig.java @@ -0,0 +1,34 @@ +package com.lincfusion.udyamlinc.config; + +import org.springframework.beans.factory.annotation.Value; +import org.springframework.context.annotation.Bean; +import org.springframework.context.annotation.Configuration; +import org.springframework.http.client.SimpleClientHttpRequestFactory; +import org.springframework.scheduling.annotation.EnableScheduling; +import org.springframework.web.client.RestClient; + +/** + * The HTTP client used for external feeds, and the scheduler that drives them. + * + *

Short, explicit timeouts are the point: a government endpoint that hangs + * must fail the nightly job quickly, not hold a thread until someone notices. + * Nothing in the request path uses this client — see ARCHITECTURE §2.1. + */ +@Configuration +@EnableScheduling +public class ExternalFeedConfig { + + @Value("${udyam.ingestion.http.connect-timeout-ms:10000}") + private int connectTimeoutMs; + + @Value("${udyam.ingestion.http.read-timeout-ms:30000}") + private int readTimeoutMs; + + @Bean("externalFeedClient") + public RestClient externalFeedClient(RestClient.Builder builder) { + SimpleClientHttpRequestFactory factory = new SimpleClientHttpRequestFactory(); + factory.setConnectTimeout(connectTimeoutMs); + factory.setReadTimeout(readTimeoutMs); + return builder.requestFactory(factory).build(); + } +} diff --git a/src/main/java/com/lincfusion/udyamlinc/config/SecurityConfig.java b/src/main/java/com/lincfusion/udyamlinc/config/SecurityConfig.java new file mode 100644 index 0000000..013e196 --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/config/SecurityConfig.java @@ -0,0 +1,69 @@ +package com.lincfusion.udyamlinc.config; + +import com.lincfusion.udyamlinc.security.JwtAuthFilter; +import lombok.RequiredArgsConstructor; +import org.springframework.beans.factory.annotation.Value; +import org.springframework.context.annotation.Bean; +import org.springframework.context.annotation.Configuration; +import org.springframework.http.HttpMethod; +import org.springframework.security.config.annotation.web.builders.HttpSecurity; +import org.springframework.security.config.annotation.web.configuration.EnableWebSecurity; +import org.springframework.security.config.http.SessionCreationPolicy; +import org.springframework.security.crypto.bcrypt.BCryptPasswordEncoder; +import org.springframework.security.crypto.password.PasswordEncoder; +import org.springframework.security.web.SecurityFilterChain; +import org.springframework.security.web.authentication.UsernamePasswordAuthenticationFilter; +import org.springframework.web.cors.CorsConfiguration; +import org.springframework.web.cors.CorsConfigurationSource; +import org.springframework.web.cors.UrlBasedCorsConfigurationSource; + +import java.util.Arrays; +import java.util.List; + +/** Stateless JWT security. /auth/** is open; everything else needs a token. */ +@Configuration +@EnableWebSecurity +@RequiredArgsConstructor +public class SecurityConfig { + + private final JwtAuthFilter jwtAuthFilter; + + @Value("${udyam.cors.allowed-origins}") + private String allowedOrigins; + + @Bean + public SecurityFilterChain securityFilterChain(HttpSecurity http) throws Exception { + http + .cors(cors -> cors.configurationSource(corsConfigurationSource())) + .csrf(csrf -> csrf.disable()) + .sessionManagement(s -> s.sessionCreationPolicy(SessionCreationPolicy.STATELESS)) + .authorizeHttpRequests(auth -> auth + .requestMatchers("/auth/**", "/health").permitAll() + // The officer console and the enterprise app are separate surfaces. + .requestMatchers("/officer/**", "/forecast/**").hasAuthority("SCOPE_OFFICER") + .requestMatchers("/app/**").hasAuthority("SCOPE_ENTERPRISE") + // Any officer may read the configuration; only a supervisor may change it. + .requestMatchers(HttpMethod.GET, "/admin/**").hasAuthority("SCOPE_OFFICER") + .requestMatchers("/admin/**").hasAuthority("ROLE_SUPERVISOR") + .anyRequest().authenticated()) + .addFilterBefore(jwtAuthFilter, UsernamePasswordAuthenticationFilter.class); + return http.build(); + } + + @Bean + public CorsConfigurationSource corsConfigurationSource() { + CorsConfiguration cfg = new CorsConfiguration(); + cfg.setAllowedOrigins(Arrays.asList(allowedOrigins.split(","))); + cfg.setAllowedMethods(List.of("GET", "POST", "PUT", "DELETE", "OPTIONS")); + cfg.setAllowedHeaders(List.of("*")); + cfg.setAllowCredentials(true); + UrlBasedCorsConfigurationSource source = new UrlBasedCorsConfigurationSource(); + source.registerCorsConfiguration("/**", cfg); + return source; + } + + @Bean + public PasswordEncoder passwordEncoder() { + return new BCryptPasswordEncoder(); + } +} diff --git a/src/main/java/com/lincfusion/udyamlinc/dto/UdyamRequests.java b/src/main/java/com/lincfusion/udyamlinc/dto/UdyamRequests.java new file mode 100644 index 0000000..aab881c --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/dto/UdyamRequests.java @@ -0,0 +1,48 @@ +package com.lincfusion.udyamlinc.dto; + +import jakarta.validation.constraints.NotBlank; +import jakarta.validation.constraints.Positive; + +import java.math.BigDecimal; + +/** Request bodies. Records, so the JSON shape is the type. */ +public final class UdyamRequests { + + private UdyamRequests() { + } + + public record OfficerLoginRequest(@NotBlank String username, @NotBlank String password) { + } + + public record AppLoginRequest(@NotBlank String enterpriseId, @NotBlank String pin) { + } + + /** + * A ledger entry pushed from the phone. {@code clientId} was generated + * offline; the server treats a repeat as an idempotent no-op. + */ + public record LedgerEntryRequest( + @NotBlank String clientId, + @NotBlank String kind, + @Positive BigDecimal amount, + @NotBlank String category, + String note, + String entryDate) { + } + + /** Officer disposition on an alert. */ + public record AlertActionRequest(@NotBlank String state, String note) { + } + + public record ScheduleVisitRequest( + @NotBlank String enterpriseId, + @NotBlank String actionCode, + String dueDate) { + } + + public record VisitOutcomeRequest(@NotBlank String outcome, Boolean officerOverride) { + } + + public record ConsentRequest(boolean active) { + } +} diff --git a/src/main/java/com/lincfusion/udyamlinc/dto/UdyamResponses.java b/src/main/java/com/lincfusion/udyamlinc/dto/UdyamResponses.java new file mode 100644 index 0000000..7a2102e --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/dto/UdyamResponses.java @@ -0,0 +1,123 @@ +package com.lincfusion.udyamlinc.dto; + +import com.lincfusion.udyamlinc.enterprise.Enterprise; +import com.lincfusion.udyamlinc.enterprise.LedgerEntry; +import com.lincfusion.udyamlinc.officer.FieldVisit; +import com.lincfusion.udyamlinc.risk.ForecastPoint; +import com.lincfusion.udyamlinc.risk.ReasonCode; +import com.lincfusion.udyamlinc.risk.RiskAlert; +import com.lincfusion.udyamlinc.risk.StressScore; + +import java.math.BigDecimal; +import java.util.List; + +/** Response shapes. These mirror the interfaces in UdyamLinc-UI/src/lib/api.ts. */ +public final class UdyamResponses { + + private UdyamResponses() { + } + + public record OfficerSessionResponse( + String token, + String tokenType, + String username, + String fullName, + String role, + List blocks, + String branch) { + } + + public record EnterpriseSessionResponse( + String token, + String tokenType, + String enterpriseId, + String name, + String nameHi, + String sector, + String block, + String lang) { + } + + /** An enterprise flattened with its sector code, as the console's table expects. */ + public record EnterpriseRow( + Long id, + String enterpriseId, + String name, + String nameHi, + String sector, + String block, + String district, + Integer ownerCount, + Integer vintageMonths, + BigDecimal loanOutstanding, + BigDecimal monthlyInstalment, + Integer score, + String band, + List trend, + String lastUpdated) { + } + + /** Dashboard KPI tiles plus the enterprises-by-risk register. */ + public record PortfolioResponse( + String block, + String month, + long total, + long totalDelta, + long red, + long redDelta, + long watch, + long watchDelta, + long standard, + long standardDelta, + int updatedMinutesAgo, + List rows) { + } + + /** Everything the risk detail panel needs for one enterprise. */ + public record EnterpriseDetailResponse( + Enterprise enterprise, + String sector, + String district, + StressScore score, + List reasons, + List forecast, + List alerts, + List visits, + List ledger) { + } + + public record ForecastResponse( + String enterpriseId, + String modelVersion, + String engine, + int horizonMonths, + String breachMonth, + BigDecimal shortfall, + List points) { + } + + /** The enterprise app's Home tab in one call. */ + public record AppHomeResponse( + Enterprise enterprise, + RiskAlert alert, + List forecast, + BigDecimal todayIn, + BigDecimal todayOut, + String adviceEn, + String adviceHi, + String lastSyncedAt) { + } + + public record SyncResultResponse(int accepted, int duplicates, String syncedAt) { + } + + /** One playbook action, offered for a band. */ + public record PlaybookActionResponse( + String code, String label, String labelHi, String appliesTo, String detail) { + } + + /** Sector pack rollup for the Forecasts screen. */ + public record SectorRollupResponse( + String sector, String label, long count, long red, long watch, long standard, int avgScore) { + } +} diff --git a/src/main/java/com/lincfusion/udyamlinc/enterprise/Enterprise.java b/src/main/java/com/lincfusion/udyamlinc/enterprise/Enterprise.java new file mode 100644 index 0000000..4c0669d --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/enterprise/Enterprise.java @@ -0,0 +1,103 @@ +package com.lincfusion.udyamlinc.enterprise; + +import jakarta.persistence.*; +import lombok.*; + +import java.math.BigDecimal; +import java.time.LocalDateTime; + +/** + * A monitored microenterprise. + * + *

No owner name, phone, Aadhaar or account number is held anywhere on this + * record. The deck's promise that no personally identifiable data is stored is + * enforced here by schema design rather than by policy control — there is no + * column to put it in. + * + *

{@code score} and {@code band} are a denormalised cache written by the risk + * engine after each scoring run, so list queries need one table. The system of + * record is {@code stress_score}. + */ +@Entity +@Table(name = "enterprise", indexes = { + @Index(name = "ix_ent_enterprise_id", columnList = "enterpriseId", unique = true), + @Index(name = "ix_ent_district", columnList = "districtId"), + @Index(name = "ix_ent_sector", columnList = "sectorId"), + @Index(name = "ix_ent_band", columnList = "band"), +}) +@Getter +@Setter +@NoArgsConstructor +@AllArgsConstructor +@Builder +public class Enterprise { + + @Id + @GeneratedValue(strategy = GenerationType.IDENTITY) + private Long id; + + /** Opaque public identifier, e.g. MSME-4417. */ + @Column(nullable = false, unique = true, length = 24) + private String enterpriseId; + + @Column(nullable = false, length = 128) + private String name; + + @Column(length = 128) + private String nameHi; + + @Column(nullable = false) + private Long sectorId; + + @Column(nullable = false) + private Long districtId; + + /** Sub-district administrative unit; the console's default scope filter. */ + @Column(nullable = false, length = 64) + private String block; + + /** + * Enterprise location. Held at premises granularity because climate features + * are gridded — this is a place, not a person. + */ + private Double latitude; + + private Double longitude; + + /** The mandi whose prices apply to this enterprise. */ + private Long marketId; + + private Integer ownerCount; + + /** Months on the institution's book. */ + private Integer vintageMonths; + + /** Denormalised from loan_record for list queries. */ + @Column(precision = 14, scale = 2) + private BigDecimal loanOutstanding; + + @Column(precision = 14, scale = 2) + private BigDecimal monthlyInstalment; + + /** Latest Composite Stress Score — cache, see class note. */ + private Integer score; + + /** RED | WATCH | STANDARD — cache, see class note. */ + @Column(length = 16) + private String band; + + /** Consent is revocable and purpose-bound; scoring stops when it is withdrawn. */ + private Boolean consentActive; + + private LocalDateTime consentUpdatedAt; + + private LocalDateTime lastScoredAt; + + private LocalDateTime createdAt; + + @PrePersist + void onCreate() { + if (createdAt == null) createdAt = LocalDateTime.now(); + if (consentActive == null) consentActive = Boolean.TRUE; + } +} diff --git a/src/main/java/com/lincfusion/udyamlinc/enterprise/EnterpriseRepository.java b/src/main/java/com/lincfusion/udyamlinc/enterprise/EnterpriseRepository.java new file mode 100644 index 0000000..1b8bfc8 --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/enterprise/EnterpriseRepository.java @@ -0,0 +1,23 @@ +package com.lincfusion.udyamlinc.enterprise; + +import org.springframework.data.jpa.repository.JpaRepository; + +import java.util.List; +import java.util.Optional; + +public interface EnterpriseRepository extends JpaRepository { + + Optional findByEnterpriseId(String enterpriseId); + + List findByBlockOrderByScoreDesc(String block); + + List findByDistrictIdOrderByScoreDesc(Long districtId); + + List findBySectorIdOrderByScoreDesc(Long sectorId); + + List findAllByOrderByScoreDesc(); + + long countByBand(String band); + + long countByBlockAndBand(String block, String band); +} diff --git a/src/main/java/com/lincfusion/udyamlinc/enterprise/LedgerEntry.java b/src/main/java/com/lincfusion/udyamlinc/enterprise/LedgerEntry.java new file mode 100644 index 0000000..572f8a3 --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/enterprise/LedgerEntry.java @@ -0,0 +1,67 @@ +package com.lincfusion.udyamlinc.enterprise; + +import jakarta.persistence.*; +import lombok.*; + +import java.math.BigDecimal; +import java.time.LocalDate; +import java.time.LocalDateTime; + +/** + * One three-tap ledger entry. + * + *

{@code clientId} is generated on the phone, so an entry made with no + * connectivity has identity before the server ever sees it. Sync is idempotent + * on that id, which is what makes draining a queue twice after a flaky + * reconnect harmless. + */ +@Entity +@Table(name = "ledger_entry", indexes = { + @Index(name = "ix_ledger_enterprise", columnList = "enterpriseId"), + @Index(name = "ix_ledger_client_id", columnList = "clientId", unique = true), + @Index(name = "ix_ledger_date", columnList = "enterpriseId,entryDate"), +}) +@Getter +@Setter +@NoArgsConstructor +@AllArgsConstructor +@Builder +public class LedgerEntry { + + @Id + @GeneratedValue(strategy = GenerationType.IDENTITY) + private Long id; + + @Column(nullable = false, unique = true, length = 64) + private String clientId; + + @Column(nullable = false, length = 24) + private String enterpriseId; + + /** IN | OUT | STOCK */ + @Column(nullable = false, length = 8) + private String kind; + + @Column(nullable = false, precision = 14, scale = 2) + private BigDecimal amount; + + @Column(nullable = false, length = 32) + private String category; + + @Column(length = 255) + private String note; + + /** The day the money moved, which may predate the sync by weeks. */ + @Column(nullable = false) + private LocalDate entryDate; + + /** When the entry reached the server. Null means it is still only on the phone. */ + private LocalDateTime syncedAt; + + private LocalDateTime createdAt; + + @PrePersist + void onCreate() { + if (createdAt == null) createdAt = LocalDateTime.now(); + } +} diff --git a/src/main/java/com/lincfusion/udyamlinc/enterprise/LedgerEntryRepository.java b/src/main/java/com/lincfusion/udyamlinc/enterprise/LedgerEntryRepository.java new file mode 100644 index 0000000..790a1f8 --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/enterprise/LedgerEntryRepository.java @@ -0,0 +1,21 @@ +package com.lincfusion.udyamlinc.enterprise; + +import org.springframework.data.jpa.repository.JpaRepository; + +import java.time.LocalDate; +import java.util.List; +import java.util.Optional; + +public interface LedgerEntryRepository extends JpaRepository { + + List findByEnterpriseIdOrderByEntryDateDesc(String enterpriseId); + + List findByEnterpriseIdAndEntryDate(String enterpriseId, LocalDate entryDate); + + List findByEnterpriseIdAndEntryDateGreaterThanEqualOrderByEntryDateDesc( + String enterpriseId, LocalDate from); + + Optional findByClientId(String clientId); + + long countByEnterpriseId(String enterpriseId); +} diff --git a/src/main/java/com/lincfusion/udyamlinc/enterprise/LoanRecord.java b/src/main/java/com/lincfusion/udyamlinc/enterprise/LoanRecord.java new file mode 100644 index 0000000..ff4fec7 --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/enterprise/LoanRecord.java @@ -0,0 +1,77 @@ +package com.lincfusion.udyamlinc.enterprise; + +import jakarta.persistence.*; +import lombok.*; + +import java.math.BigDecimal; +import java.time.LocalDate; +import java.time.LocalDateTime; + +/** + * One loan on the institution's book, as the nightly CBS extract reports it. + * + *

{@code dpd} is the single most important column in the schema: today it + * drives the {@code emi_delay} feature and the obligation-load sub-score, and its + * history is the training label when the model arrives. No account number and no + * borrower identity — the loan is attached to the opaque enterprise id. + */ +@Entity +@Table(name = "loan_record", indexes = { + @Index(name = "ix_loan_enterprise", columnList = "enterpriseId"), + @Index(name = "ix_loan_account_ref", columnList = "accountRef", unique = true), +}) +@Getter +@Setter +@NoArgsConstructor +@AllArgsConstructor +@Builder +public class LoanRecord { + + @Id + @GeneratedValue(strategy = GenerationType.IDENTITY) + private Long id; + + /** Opaque loan reference from the lender's extract — not an account number. */ + @Column(nullable = false, unique = true, length = 64) + private String accountRef; + + @Column(nullable = false, length = 24) + private String enterpriseId; + + @Column(length = 32) + private String product; + + @Column(precision = 14, scale = 2) + private BigDecimal sanctionedAmount; + + @Column(precision = 14, scale = 2) + private BigDecimal outstandingAmount; + + @Column(precision = 14, scale = 2) + private BigDecimal instalmentAmount; + + private LocalDate nextDueDate; + + private LocalDate lastPaidDate; + + /** Days past due at the extract date. */ + private Integer dpd; + + /** 0 | 1-30 | 31-60 | 61-90 | 90+ */ + @Column(length = 16) + private String bucket; + + private Boolean closed; + + /** The date the extract describes — features must not read past this. */ + @Column(nullable = false) + private LocalDate asOfDate; + + private LocalDateTime observedAt; + + @PrePersist + void onCreate() { + if (observedAt == null) observedAt = LocalDateTime.now(); + if (closed == null) closed = Boolean.FALSE; + } +} diff --git a/src/main/java/com/lincfusion/udyamlinc/enterprise/LoanRecordRepository.java b/src/main/java/com/lincfusion/udyamlinc/enterprise/LoanRecordRepository.java new file mode 100644 index 0000000..44d7cf8 --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/enterprise/LoanRecordRepository.java @@ -0,0 +1,19 @@ +package com.lincfusion.udyamlinc.enterprise; + +import org.springframework.data.jpa.repository.JpaRepository; + +import java.time.LocalDate; +import java.util.List; +import java.util.Optional; + +public interface LoanRecordRepository extends JpaRepository { + + List findByEnterpriseIdAndClosedFalse(String enterpriseId); + + /** Point-in-time: only extracts describing a date at or before the scoring date. */ + List findByEnterpriseIdAndAsOfDateLessThanEqual(String enterpriseId, LocalDate asOfDate); + + Optional findByAccountRef(String accountRef); + + long countByEnterpriseId(String enterpriseId); +} diff --git a/src/main/java/com/lincfusion/udyamlinc/enterprise/UpiSummary.java b/src/main/java/com/lincfusion/udyamlinc/enterprise/UpiSummary.java new file mode 100644 index 0000000..e9a6ec5 --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/enterprise/UpiSummary.java @@ -0,0 +1,67 @@ +package com.lincfusion.udyamlinc.enterprise; + +import jakarta.persistence.*; +import lombok.*; + +import java.math.BigDecimal; +import java.time.LocalDate; +import java.time.LocalDateTime; + +/** + * Consented weekly transaction aggregates — counts and totals, never individual + * transactions, which is what keeps this inside the consent artefact's purpose. + * + *

The source is pluggable: an Account Aggregator flow once FIU registration is + * through, a PSP settlement report, or manual entry during the pilot. + * {@code consentRef} is stored so a revocation can find and delete its rows. + */ +@Entity +@Table(name = "upi_summary", indexes = { + @Index(name = "ix_upi_key", columnList = "enterpriseId,weekStart", unique = true), +}) +@Getter +@Setter +@NoArgsConstructor +@AllArgsConstructor +@Builder +public class UpiSummary { + + @Id + @GeneratedValue(strategy = GenerationType.IDENTITY) + private Long id; + + @Column(nullable = false, length = 24) + private String enterpriseId; + + @Column(nullable = false) + private LocalDate weekStart; + + @Column(precision = 14, scale = 2) + private BigDecimal inflowAmount; + + @Column(precision = 14, scale = 2) + private BigDecimal outflowAmount; + + private Integer inflowCount; + + private Integer outflowCount; + + /** ACCOUNT_AGGREGATOR | PSP_REPORT | MANUAL */ + @Column(length = 32) + private String source; + + /** The consent artefact these rows were collected under. */ + @Column(length = 64) + private String consentRef; + + @Column(nullable = false) + private LocalDate validForDate; + + @Column(nullable = false) + private LocalDateTime observedAt; + + @PrePersist + void onCreate() { + if (observedAt == null) observedAt = LocalDateTime.now(); + } +} diff --git a/src/main/java/com/lincfusion/udyamlinc/enterprise/UpiSummaryRepository.java b/src/main/java/com/lincfusion/udyamlinc/enterprise/UpiSummaryRepository.java new file mode 100644 index 0000000..8f88dc0 --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/enterprise/UpiSummaryRepository.java @@ -0,0 +1,27 @@ +package com.lincfusion.udyamlinc.enterprise; + +import org.springframework.data.jpa.repository.JpaRepository; +import org.springframework.data.jpa.repository.Query; +import org.springframework.data.repository.query.Param; + +import java.time.LocalDate; +import java.time.LocalDateTime; +import java.util.List; +import java.util.Optional; + +public interface UpiSummaryRepository extends JpaRepository { + + Optional findByEnterpriseIdAndWeekStart(String enterpriseId, LocalDate weekStart); + + @Query(""" + select u from UpiSummary u + where u.enterpriseId = :enterpriseId + and u.validForDate <= :asOfDate and u.observedAt <= :asOf + order by u.weekStart desc + """) + List findTrailing(@Param("enterpriseId") String enterpriseId, + @Param("asOfDate") LocalDate asOfDate, + @Param("asOf") LocalDateTime asOf); + + void deleteByEnterpriseId(String enterpriseId); +} diff --git a/src/main/java/com/lincfusion/udyamlinc/features/FeatureCatalog.java b/src/main/java/com/lincfusion/udyamlinc/features/FeatureCatalog.java new file mode 100644 index 0000000..61de54c --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/features/FeatureCatalog.java @@ -0,0 +1,123 @@ +package com.lincfusion.udyamlinc.features; + +import lombok.RequiredArgsConstructor; +import lombok.extern.slf4j.Slf4j; +import org.springframework.stereotype.Component; + +import java.util.List; + +/** + * The engineered indicators of ARCHITECTURE §4, as constants and as rows. + * + *

Raw prices are not features. An absolute maize price of ₹2,140 means + * nothing; "feed cost 18 % above its eight-week mean while the egg rate is flat" + * means something. Every derivation here turns a level into a change, a ratio or + * an anomaly for exactly that reason. + * + *

Codes are snake_case because a trip-wire expression is written against them + * directly: {@code emi_delay > 30 and cash_flow_change < -0.40}. + * + *

Three codes beyond §4's list are here because the seeded trip-wires already + * name them — {@code dscr_p10}, {@code days_since_last_entry} and + * {@code emi_to_inflow}. A rule referring to a feature nobody computes is a rule + * that silently never fires, which is worse than no rule at all. They also give + * the OBLIGATION_LOAD family the inputs it otherwise lacked. + */ +@Component +@RequiredArgsConstructor +@Slf4j +public class FeatureCatalog { + + // ── codes ─────────────────────────────────────────────── + + public static final String FEED_COST_CHANGE = "feed_cost_change"; + public static final String OUTPUT_PRICE_CHANGE = "output_price_change"; + public static final String MARGIN_SQUEEZE = "margin_squeeze"; + public static final String PRICE_Z_SCORE = "price_z_score"; + public static final String RAINFALL_ANOMALY = "rainfall_anomaly"; + public static final String HEAT_ANOMALY = "heat_anomaly"; + public static final String CASH_FLOW_CHANGE = "cash_flow_change"; + public static final String EMI_DELAY = "emi_delay"; + public static final String INVENTORY_DAYS = "inventory_days"; + public static final String UPI_INFLOW_CHANGE = "upi_inflow_change"; + public static final String FEED_STALENESS_DAYS = "feed_staleness_days"; + public static final String DSCR_P10 = "dscr_p10"; + public static final String DAYS_SINCE_LAST_ENTRY = "days_since_last_entry"; + public static final String EMI_TO_INFLOW = "emi_to_inflow"; + + public static final List ALL = List.of( + FEED_COST_CHANGE, OUTPUT_PRICE_CHANGE, MARGIN_SQUEEZE, PRICE_Z_SCORE, + RAINFALL_ANOMALY, HEAT_ANOMALY, CASH_FLOW_CHANGE, EMI_DELAY, + INVENTORY_DAYS, UPI_INFLOW_CHANGE, FEED_STALENESS_DAYS, + DSCR_P10, DAYS_SINCE_LAST_ENTRY, EMI_TO_INFLOW); + + // ── families ──────────────────────────────────────────── + + public static final String LIQUIDITY = "LIQUIDITY"; + public static final String BEHAVIOURAL = "BEHAVIOURAL"; + public static final String TREND_VOLATILITY = "TREND_VOLATILITY"; + public static final String OBLIGATION_LOAD = "OBLIGATION_LOAD"; + public static final String EXTERNAL_SHOCK = "EXTERNAL_SHOCK"; + public static final String QUALITY = "QUALITY"; + + // ── missing reasons ───────────────────────────────────── + + public static final String NO_SIGNAL = "NO_SIGNAL"; + public static final String NO_LEDGER = "NO_LEDGER"; + public static final String NO_CBS = "NO_CBS"; + public static final String NO_CONSENT = "NO_CONSENT"; + public static final String NO_FORECAST = "NO_FORECAST"; + public static final String INSUFFICIENT_HISTORY = "INSUFFICIENT_HISTORY"; + + private final FeatureDefinitionRepository repository; + + /** Idempotent: writes any catalogue row that is not already present. */ + public void seed() { + List catalogue = List.of( + def(FEED_COST_CHANGE, "Feed cost change", EXTERNAL_SHOCK, "RATIO", "AGMARKNET", 10, + "HIGHER_IS_WORSE", "Input-side modal price vs its 8-week trailing mean, weighted across the sector's INPUT commodities"), + def(OUTPUT_PRICE_CHANGE, "Output price change", EXTERNAL_SHOCK, "RATIO", "AGMARKNET", 10, + "LOWER_IS_WORSE", "Same derivation on the sector's OUTPUT commodities"), + def(MARGIN_SQUEEZE, "Margin squeeze", EXTERNAL_SHOCK, "RATIO", "AGMARKNET", 10, + "HIGHER_IS_WORSE", "feed_cost_change − output_price_change; a feed rise matched by an output rise is not a squeeze"), + def(PRICE_Z_SCORE, "Seasonal price z-score", TREND_VOLATILITY, "INDEX", "AGMARKNET", 10, + "HIGHER_IS_WORSE", "(current − seasonal mean for this ISO week ±2) / stdev, over all held history"), + def(RAINFALL_ANOMALY, "Rainfall anomaly", EXTERNAL_SHOCK, "PCT", "CLIMATE", 14, + "LOWER_IS_WORSE", "Season-to-date rainfall over the trailing window vs the 30-year normal for the same weeks"), + def(HEAT_ANOMALY, "Heat anomaly", EXTERNAL_SHOCK, "INDEX", "CLIMATE", 14, + "HIGHER_IS_WORSE", "Mean weekly max temperature minus the 30-year normal, over the trailing window"), + def(CASH_FLOW_CHANGE, "Cash flow change", LIQUIDITY, "RATIO", "LEDGER", 7, + "LOWER_IS_WORSE", "Ledger net position over the recent 4 weeks vs the prior 8-week mean"), + def(EMI_DELAY, "EMI delay", BEHAVIOURAL, "DAYS", "CBS", 35, + "HIGHER_IS_WORSE", "Worst DPD across open loans at the extract date"), + def(INVENTORY_DAYS, "Inventory days", LIQUIDITY, "DAYS", "LEDGER", 14, + "HIGHER_IS_WORSE", "Stock on hand divided by daily sales velocity over the recent window"), + def(UPI_INFLOW_CHANGE, "UPI inflow change", LIQUIDITY, "RATIO", "UPI", 21, + "LOWER_IS_WORSE", "Trailing 8-week consented inflow vs the 8 weeks before it"), + def(FEED_STALENESS_DAYS, "Feed staleness", QUALITY, "DAYS", "AGMARKNET", null, + "HIGHER_IS_WORSE", "Age of the newest market observation for the district — guards every price-derived feature above"), + def(DSCR_P10, "DSCR at P10", OBLIGATION_LOAD, "RATIO", "FORECAST", 7, + "LOWER_IS_WORSE", "Worst month of the horizon: min(p10 / obligation) over the latest scoring run — the downside case failing to cover what is due"), + def(DAYS_SINCE_LAST_ENTRY, "Days since last ledger entry", BEHAVIOURAL, "DAYS", "LEDGER", 3, + "HIGHER_IS_WORSE", "Days between the newest ledger entry date and the scoring date; silence is itself a signal"), + def(EMI_TO_INFLOW, "Instalment share of inflow", OBLIGATION_LOAD, "RATIO", "CBS", 30, + "HIGHER_IS_WORSE", "Monthly instalment across open loans divided by the monthly ledger inflow rate")); + + int written = 0; + for (FeatureDefinition candidate : catalogue) { + if (repository.findByCode(candidate.getCode()).isEmpty()) { + repository.save(candidate); + written++; + } + } + if (written > 0) log.info("Seeded {} feature definitions", written); + } + + private FeatureDefinition def(String code, String label, String family, String unit, String sourceKind, + Integer staleness, String direction, String derivation) { + return FeatureDefinition.builder() + .code(code).label(label).family(family).unit(unit).sourceKind(sourceKind) + .stalenessToleranceDays(staleness).direction(direction).derivation(derivation) + .active(true).build(); + } +} diff --git a/src/main/java/com/lincfusion/udyamlinc/features/FeatureController.java b/src/main/java/com/lincfusion/udyamlinc/features/FeatureController.java new file mode 100644 index 0000000..52d7dd8 --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/features/FeatureController.java @@ -0,0 +1,121 @@ +package com.lincfusion.udyamlinc.features; + +import com.lincfusion.udyamlinc.common.ApiResponse; +import com.lincfusion.udyamlinc.common.ResourceNotFoundException; +import com.lincfusion.udyamlinc.enterprise.Enterprise; +import com.lincfusion.udyamlinc.enterprise.EnterpriseRepository; +import lombok.RequiredArgsConstructor; +import org.springframework.format.annotation.DateTimeFormat; +import org.springframework.web.bind.annotation.*; + +import java.time.LocalDate; +import java.util.*; + +/** + * Phase C terminates in a table you can inspect; this is how you inspect it. + * + *

Under {@code /admin}, so reads are open to any officer and the recompute + * trigger is supervisor-only under the existing rule. + */ +@RestController +@RequestMapping("/admin/features") +@RequiredArgsConstructor +public class FeatureController { + + private final FeatureDefinitionRepository definitionRepository; + private final FeatureValueRepository valueRepository; + private final FeatureSnapshotRepository snapshotRepository; + private final FeatureEngineeringService engineeringService; + private final EnterpriseRepository enterpriseRepository; + + @GetMapping("/definitions") + public ApiResponse> definitions() { + return ApiResponse.ok(definitionRepository.findByActiveTrue()); + } + + /** The wide row the engine reads — latest, or as it stood on a given date. */ + @GetMapping("/snapshot/{enterpriseId}") + public ApiResponse snapshot( + @PathVariable String enterpriseId, + @RequestParam(required = false) @DateTimeFormat(iso = DateTimeFormat.ISO.DATE) LocalDate date) { + + Optional snapshot = date == null + ? snapshotRepository.findFirstByEnterpriseIdOrderByAsOfDateDesc(enterpriseId) + : snapshotRepository.findByEnterpriseIdAndAsOfDate(enterpriseId, date); + + return ApiResponse.ok(snapshot.orElseThrow(() -> + new ResourceNotFoundException("No feature snapshot for " + enterpriseId + + (date == null ? "" : " at " + date)))); + } + + /** The tall rows behind it, including the nulls and why they are null. */ + @GetMapping("/values/{enterpriseId}") + public ApiResponse> values( + @PathVariable String enterpriseId, + @RequestParam(required = false) @DateTimeFormat(iso = DateTimeFormat.ISO.DATE) LocalDate date) { + + LocalDate asOf = date != null ? date + : snapshotRepository.findFirstByEnterpriseIdOrderByAsOfDateDesc(enterpriseId) + .map(FeatureSnapshot::getAsOfDate) + .orElseThrow(() -> new ResourceNotFoundException("No features computed for " + enterpriseId)); + + return ApiResponse.ok(valueRepository.findByEnterpriseIdAndValidForDateOrderByFeatureCode(enterpriseId, asOf)); + } + + /** + * How much of the catalogue is actually populated across the book. A feature + * at 4 % coverage is not a feature — it is a column, and the engine should be + * told to stop weighting it. + */ + @GetMapping("/coverage") + public ApiResponse> coverage( + @RequestParam(required = false) @DateTimeFormat(iso = DateTimeFormat.ISO.DATE) LocalDate date) { + + LocalDate asOf = date == null ? LocalDate.now() : date; + List snapshots = snapshotRepository.findByAsOfDate(asOf); + + Map present = new LinkedHashMap<>(); + FeatureCatalog.ALL.forEach(code -> present.put(code, 0)); + for (FeatureSnapshot snapshot : snapshots) { + snapshot.asFeatureMap().forEach((code, value) -> { + if (value != null) present.merge(code, 1, Integer::sum); + }); + } + + Map body = new LinkedHashMap<>(); + body.put("asOfDate", asOf); + body.put("snapshots", snapshots.size()); + Map byFeature = new LinkedHashMap<>(); + present.forEach((code, count) -> { + Map row = new LinkedHashMap<>(); + row.put("populated", count); + row.put("coveragePct", snapshots.isEmpty() ? 0 + : Math.round(count * 1000.0 / snapshots.size()) / 10.0); + byFeature.put(code, row); + }); + body.put("features", byFeature); + return ApiResponse.ok(body); + } + + // ── trigger ───────────────────────────────────────────── + + /** Recompute the whole book. Supervisor-only under the {@code /admin/**} write rule. */ + @PostMapping("/compute") + public ApiResponse compute( + @RequestParam(required = false) @DateTimeFormat(iso = DateTimeFormat.ISO.DATE) LocalDate date) { + return ApiResponse.ok("Features computed", + engineeringService.computeAll(date == null ? LocalDate.now() : date, null)); + } + + /** One enterprise, for when a single reason code needs explaining. */ + @PostMapping("/compute/{enterpriseId}") + public ApiResponse computeOne( + @PathVariable String enterpriseId, + @RequestParam(required = false) @DateTimeFormat(iso = DateTimeFormat.ISO.DATE) LocalDate date) { + + Enterprise enterprise = enterpriseRepository.findByEnterpriseId(enterpriseId) + .orElseThrow(() -> new ResourceNotFoundException("No enterprise " + enterpriseId)); + return ApiResponse.ok("Features computed", engineeringService.computeFor( + enterprise, date == null ? LocalDate.now() : date, java.time.LocalDateTime.now(), null)); + } +} diff --git a/src/main/java/com/lincfusion/udyamlinc/features/FeatureDefinition.java b/src/main/java/com/lincfusion/udyamlinc/features/FeatureDefinition.java new file mode 100644 index 0000000..f05e013 --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/features/FeatureDefinition.java @@ -0,0 +1,70 @@ +package com.lincfusion.udyamlinc.features; + +import jakarta.persistence.*; +import lombok.*; + +import java.time.LocalDateTime; + +/** + * The feature catalogue. A row here is the contract for one engineered + * indicator: what it means, what unit it carries, and how old it is allowed to + * be before anything downstream may act on it. + * + *

{@code stalenessToleranceDays} is the column trip-wires read. A rule firing + * on a three-week-old price is worse than no rule, and the tolerance is per + * feature because a mandi price goes stale in days while a CBS extract is + * monthly by design. + */ +@Entity +@Table(name = "feature_definition", indexes = @Index(name = "ix_feature_def_code", columnList = "code", unique = true)) +@Getter +@Setter +@NoArgsConstructor +@AllArgsConstructor +@Builder +public class FeatureDefinition { + + @Id + @GeneratedValue(strategy = GenerationType.IDENTITY) + private Long id; + + /** snake_case, matching the name a trip-wire expression uses. */ + @Column(nullable = false, unique = true, length = 48) + private String code; + + @Column(nullable = false, length = 96) + private String label; + + /** LIQUIDITY | BEHAVIOURAL | TREND_VOLATILITY | OBLIGATION_LOAD | EXTERNAL_SHOCK | QUALITY */ + @Column(nullable = false, length = 32) + private String family; + + /** RATIO | PCT | DAYS | INDEX */ + @Column(length = 16) + private String unit; + + /** Which upstream table it derives from — AGMARKNET, CLIMATE, LEDGER, CBS, UPI. */ + @Column(length = 32) + private String sourceKind; + + /** Beyond this age the value must not be acted on. Null means it never goes stale. */ + private Integer stalenessToleranceDays; + + /** HIGHER_IS_WORSE | LOWER_IS_WORSE */ + @Column(length = 24) + private String direction; + + @Column(length = 512) + private String derivation; + + private Boolean active; + + private LocalDateTime updatedAt; + + @PrePersist + @PreUpdate + void stamp() { + if (active == null) active = Boolean.TRUE; + updatedAt = LocalDateTime.now(); + } +} diff --git a/src/main/java/com/lincfusion/udyamlinc/features/FeatureDefinitionRepository.java b/src/main/java/com/lincfusion/udyamlinc/features/FeatureDefinitionRepository.java new file mode 100644 index 0000000..aedc2b1 --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/features/FeatureDefinitionRepository.java @@ -0,0 +1,15 @@ +package com.lincfusion.udyamlinc.features; + +import org.springframework.data.jpa.repository.JpaRepository; + +import java.util.List; +import java.util.Optional; + +public interface FeatureDefinitionRepository extends JpaRepository { + + Optional findByCode(String code); + + List findByActiveTrue(); + + List findByFamilyAndActiveTrue(String family); +} diff --git a/src/main/java/com/lincfusion/udyamlinc/features/FeatureEngineeringService.java b/src/main/java/com/lincfusion/udyamlinc/features/FeatureEngineeringService.java new file mode 100644 index 0000000..4fad867 --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/features/FeatureEngineeringService.java @@ -0,0 +1,535 @@ +package com.lincfusion.udyamlinc.features; + +import com.lincfusion.udyamlinc.enterprise.*; +import com.lincfusion.udyamlinc.ingestion.*; +import com.lincfusion.udyamlinc.master.SectorCommodity; +import com.lincfusion.udyamlinc.master.SectorCommodityRepository; +import com.lincfusion.udyamlinc.risk.ForecastPoint; +import com.lincfusion.udyamlinc.risk.ForecastPointRepository; +import lombok.RequiredArgsConstructor; +import lombok.extern.slf4j.Slf4j; +import org.springframework.beans.factory.annotation.Value; +import org.springframework.stereotype.Service; + +import java.math.BigDecimal; +import java.math.RoundingMode; +import java.time.LocalDate; +import java.time.LocalDateTime; +import java.time.temporal.ChronoUnit; +import java.util.*; + +/** + * Turns signals into indicators. The only writer of the feature store, and the + * only path by which anything reaches the model — ingestion writes signals, this + * writes features, the AI layer consumes features and nothing else. That is what + * lets the model be retrained or replaced without touching ingestion or the API. + * + *

Every read here is point-in-time: each repository query filters on both + * {@code validForDate <= asOfDate} and {@code observedAt <= observedAsOf}, so + * recomputing last month's snapshot uses only what was actually knowable last + * month. Without that, a backtest looks excellent and production does not + * reproduce it. + * + *

A feature that cannot be computed is written as null with a reason, not + * skipped. "We did not know" and "we never looked" are different facts, and the + * forecast widens honestly only if the first one is recorded. + */ +@Service +@RequiredArgsConstructor +@Slf4j +public class FeatureEngineeringService { + + private final EnterpriseRepository enterpriseRepository; + private final SectorCommodityRepository sectorCommodityRepository; + private final MarketSignalRepository marketSignalRepository; + private final ClimateSignalRepository climateSignalRepository; + private final LedgerEntryRepository ledgerRepository; + private final LoanRecordRepository loanRepository; + private final UpiSummaryRepository upiRepository; + private final ForecastPointRepository forecastPointRepository; + private final FeatureValueRepository valueRepository; + private final FeatureSnapshotRepository snapshotRepository; + private final FeatureCatalog catalog; + + /** Weeks of price history averaged for a "vs trailing mean" change. */ + @Value("${udyam.features.price-window-weeks:8}") + private int priceWindowWeeks; + + /** Recent window for ledger and climate aggregates. */ + @Value("${udyam.features.recent-window-weeks:4}") + private int recentWindowWeeks; + + @Value("${udyam.features.climate-window-weeks:8}") + private int climateWindowWeeks; + + /** Minimum observations before a z-score is meaningful rather than noise. */ + @Value("${udyam.features.min-zscore-samples:6}") + private int minZScoreSamples; + + // ── entry points ──────────────────────────────────────── + + /** One pass over the whole book. Consent-revoked enterprises are skipped, not computed. */ + public Summary computeAll(LocalDate asOfDate, Long scoringRunId) { + catalog.seed(); + LocalDateTime observedAsOf = LocalDateTime.now(); + int computed = 0, skipped = 0, missing = 0; + + for (Enterprise enterprise : enterpriseRepository.findAll()) { + if (Boolean.FALSE.equals(enterprise.getConsentActive())) { + skipped++; + continue; + } + FeatureSnapshot snapshot = computeFor(enterprise, asOfDate, observedAsOf, scoringRunId); + computed++; + missing += snapshot.getMissingCount() == null ? 0 : snapshot.getMissingCount(); + } + + String message = "%d snapshots at %s · %d skipped for consent · %d null feature values" + .formatted(computed, asOfDate, skipped, missing); + log.info("Feature engineering — {}", message); + return new Summary(computed, skipped, missing, message); + } + + /** The eleven catalogue features for one enterprise, tall rows and a wide snapshot. */ + public FeatureSnapshot computeFor(Enterprise enterprise, LocalDate asOfDate, + LocalDateTime observedAsOf, Long scoringRunId) { + + Map features = new LinkedHashMap<>(); + + List inputs = sectorCommodityRepository.findBySectorIdAndRole(enterprise.getSectorId(), "INPUT"); + List outputs = sectorCommodityRepository.findBySectorIdAndRole(enterprise.getSectorId(), "OUTPUT"); + + Integer marketStaleness = marketStaleness(enterprise.getDistrictId(), asOfDate); + + Computed feedCost = weightedPriceChange(enterprise.getDistrictId(), inputs, asOfDate, observedAsOf, marketStaleness); + Computed outputPrice = weightedPriceChange(enterprise.getDistrictId(), outputs, asOfDate, observedAsOf, marketStaleness); + + features.put(FeatureCatalog.FEED_COST_CHANGE, feedCost); + features.put(FeatureCatalog.OUTPUT_PRICE_CHANGE, outputPrice); + features.put(FeatureCatalog.MARGIN_SQUEEZE, marginSqueeze(feedCost, outputPrice, marketStaleness)); + features.put(FeatureCatalog.PRICE_Z_SCORE, + priceZScore(enterprise.getDistrictId(), outputs, asOfDate, observedAsOf, marketStaleness)); + features.put(FeatureCatalog.RAINFALL_ANOMALY, rainfallAnomaly(enterprise.getDistrictId(), asOfDate, observedAsOf)); + features.put(FeatureCatalog.HEAT_ANOMALY, heatAnomaly(enterprise.getDistrictId(), asOfDate, observedAsOf)); + features.put(FeatureCatalog.CASH_FLOW_CHANGE, cashFlowChange(enterprise.getEnterpriseId(), asOfDate)); + features.put(FeatureCatalog.EMI_DELAY, emiDelay(enterprise.getEnterpriseId(), asOfDate)); + features.put(FeatureCatalog.INVENTORY_DAYS, inventoryDays(enterprise.getEnterpriseId(), asOfDate)); + features.put(FeatureCatalog.UPI_INFLOW_CHANGE, upiInflowChange(enterprise.getEnterpriseId(), asOfDate, observedAsOf)); + features.put(FeatureCatalog.FEED_STALENESS_DAYS, marketStaleness == null + ? Computed.missing(FeatureCatalog.NO_SIGNAL) + : Computed.of(BigDecimal.valueOf(marketStaleness), marketStaleness)); + features.put(FeatureCatalog.DAYS_SINCE_LAST_ENTRY, daysSinceLastEntry(enterprise.getEnterpriseId(), asOfDate)); + features.put(FeatureCatalog.EMI_TO_INFLOW, emiToInflow(enterprise.getEnterpriseId(), asOfDate)); + features.put(FeatureCatalog.DSCR_P10, dscrP10(enterprise.getEnterpriseId(), asOfDate)); + + return persist(enterprise.getEnterpriseId(), asOfDate, observedAsOf, scoringRunId, features); + } + + // ── price derivations ─────────────────────────────────── + + /** + * A level is not a feature. This is the modal price against its trailing mean, + * averaged across the sector's commodities for that role and weighted by how + * much of the trade each one actually is. + */ + private Computed weightedPriceChange(Long districtId, List commodities, + LocalDate asOfDate, LocalDateTime observedAsOf, Integer staleness) { + if (commodities.isEmpty()) return Computed.missing(FeatureCatalog.NO_SIGNAL); + + BigDecimal weightedSum = BigDecimal.ZERO; + int totalWeight = 0; + boolean anyHistory = false; + + for (SectorCommodity link : commodities) { + List series = marketSignalRepository.findTrailing( + districtId, link.getCommodityId(), asOfDate, observedAsOf); + BigDecimal change = changeVsTrailingMean(series); + if (change == null) continue; + anyHistory = true; + int weight = link.getWeight() == null || link.getWeight() <= 0 ? 1 : link.getWeight(); + weightedSum = weightedSum.add(change.multiply(BigDecimal.valueOf(weight))); + totalWeight += weight; + } + + if (!anyHistory || totalWeight == 0) { + return Computed.missing(FeatureCatalog.NO_SIGNAL); + } + return Computed.of(weightedSum.divide(BigDecimal.valueOf(totalWeight), 4, RoundingMode.HALF_UP), staleness); + } + + /** (latest − mean of the prior window) / mean. Null until there is enough history to mean anything. */ + private BigDecimal changeVsTrailingMean(List descendingByWeek) { + List modal = descendingByWeek.stream() + .map(MarketSignal::getModalPrice) + .filter(Objects::nonNull) + .toList(); + if (modal.size() < 4) return null; + + BigDecimal latest = modal.get(0); + List prior = modal.subList(1, Math.min(modal.size(), priceWindowWeeks + 1)); + BigDecimal mean = mean(prior); + if (mean == null || mean.signum() == 0) return null; + return latest.subtract(mean).divide(mean, 4, RoundingMode.HALF_UP); + } + + /** A feed rise matched by an output rise is not a squeeze; the difference is. */ + private Computed marginSqueeze(Computed feedCost, Computed outputPrice, Integer staleness) { + if (feedCost.value() == null || outputPrice.value() == null) { + return Computed.missing(FeatureCatalog.NO_SIGNAL); + } + return Computed.of(feedCost.value().subtract(outputPrice.value()), staleness); + } + + /** + * Seasonal z-score on the sale-side commodity: this week against the same week + * of the year across all held history, ±2 weeks for sample size. Prices are + * seasonal, so comparing March with the annual mean would call every harvest a + * shock. + */ + private Computed priceZScore(Long districtId, List outputs, + LocalDate asOfDate, LocalDateTime observedAsOf, Integer staleness) { + if (outputs.isEmpty()) return Computed.missing(FeatureCatalog.NO_SIGNAL); + + SectorCommodity primary = outputs.stream() + .max(Comparator.comparing(sc -> sc.getWeight() == null ? 0 : sc.getWeight())) + .orElse(outputs.get(0)); + + List series = marketSignalRepository.findTrailing( + districtId, primary.getCommodityId(), asOfDate, observedAsOf); + if (series.isEmpty()) return Computed.missing(FeatureCatalog.NO_SIGNAL); + + MarketSignal latest = series.get(0); + if (latest.getModalPrice() == null) return Computed.missing(FeatureCatalog.NO_SIGNAL); + int week = Weeks.isoWeek(latest.getWeekStart()); + + List seasonal = series.stream() + .skip(1) + .filter(s -> s.getModalPrice() != null) + .filter(s -> weekDistance(Weeks.isoWeek(s.getWeekStart()), week) <= 2) + .map(MarketSignal::getModalPrice) + .toList(); + + if (seasonal.size() < minZScoreSamples) return Computed.missing(FeatureCatalog.INSUFFICIENT_HISTORY); + + BigDecimal mean = mean(seasonal); + BigDecimal stdev = stdev(seasonal, mean); + if (stdev == null || stdev.signum() == 0) return Computed.missing(FeatureCatalog.INSUFFICIENT_HISTORY); + + return Computed.of(latest.getModalPrice().subtract(mean).divide(stdev, 4, RoundingMode.HALF_UP), staleness); + } + + /** ISO weeks wrap, so week 52 and week 1 are two apart, not fifty-one. */ + private int weekDistance(int a, int b) { + int raw = Math.abs(a - b); + return Math.min(raw, 53 - raw); + } + + // ── climate derivations ───────────────────────────────── + + private Computed rainfallAnomaly(Long districtId, LocalDate asOfDate, LocalDateTime observedAsOf) { + List series = climateSignalRepository.findTrailing(districtId, asOfDate, observedAsOf); + if (series.isEmpty()) return Computed.missing(FeatureCatalog.NO_SIGNAL); + + List window = series.subList(0, Math.min(series.size(), climateWindowWeeks)); + Integer staleness = staleness(window.get(0).getValidForDate(), asOfDate); + + BigDecimal observed = BigDecimal.ZERO, normal = BigDecimal.ZERO; + boolean anyNormal = false; + for (ClimateSignal week : window) { + if (week.getRainfallMm() != null) observed = observed.add(week.getRainfallMm()); + if (week.getRainfallNormalMm() != null) { + normal = normal.add(week.getRainfallNormalMm()); + anyNormal = true; + } + } + if (!anyNormal || normal.signum() == 0) return Computed.missing(FeatureCatalog.INSUFFICIENT_HISTORY); + + BigDecimal pct = observed.subtract(normal) + .multiply(BigDecimal.valueOf(100)) + .divide(normal, 4, RoundingMode.HALF_UP); + return Computed.of(pct, staleness); + } + + private Computed heatAnomaly(Long districtId, LocalDate asOfDate, LocalDateTime observedAsOf) { + List series = climateSignalRepository.findTrailing(districtId, asOfDate, observedAsOf); + if (series.isEmpty()) return Computed.missing(FeatureCatalog.NO_SIGNAL); + + List window = series.subList(0, Math.min(series.size(), climateWindowWeeks)); + Integer staleness = staleness(window.get(0).getValidForDate(), asOfDate); + + List anomalies = window.stream() + .map(ClimateSignal::getHeatAnomalyC) + .filter(Objects::nonNull) + .toList(); + if (anomalies.isEmpty()) return Computed.missing(FeatureCatalog.INSUFFICIENT_HISTORY); + + return Computed.of(mean(anomalies), staleness); + } + + // ── ledger derivations ────────────────────────────────── + + /** + * Net position over the recent weeks against the mean of the weeks before it. + * Entries are filtered on {@code entryDate}, not sync time — an entry made + * offline three weeks ago describes three weeks ago. + */ + private Computed cashFlowChange(String enterpriseId, LocalDate asOfDate) { + List entries = ledgerRepository.findByEnterpriseIdOrderByEntryDateDesc(enterpriseId).stream() + .filter(e -> !e.getEntryDate().isAfter(asOfDate)) + .toList(); + if (entries.isEmpty()) return Computed.missing(FeatureCatalog.NO_LEDGER); + + Integer staleness = staleness(entries.get(0).getEntryDate(), asOfDate); + LocalDate recentFrom = asOfDate.minusWeeks(recentWindowWeeks); + LocalDate priorFrom = recentFrom.minusWeeks(priceWindowWeeks); + + BigDecimal recent = net(entries, recentFrom, asOfDate); + BigDecimal prior = net(entries, priorFrom, recentFrom); + if (prior.signum() == 0) return Computed.missing(FeatureCatalog.INSUFFICIENT_HISTORY); + + // Both windows normalised to a weekly rate before comparison. + BigDecimal recentRate = recent.divide(BigDecimal.valueOf(recentWindowWeeks), 4, RoundingMode.HALF_UP); + BigDecimal priorRate = prior.divide(BigDecimal.valueOf(priceWindowWeeks), 4, RoundingMode.HALF_UP); + if (priorRate.signum() == 0) return Computed.missing(FeatureCatalog.INSUFFICIENT_HISTORY); + + BigDecimal change = recentRate.subtract(priorRate).divide(priorRate.abs(), 4, RoundingMode.HALF_UP); + return Computed.of(change, staleness); + } + + private BigDecimal net(List entries, LocalDate from, LocalDate to) { + BigDecimal sum = BigDecimal.ZERO; + for (LedgerEntry e : entries) { + if (e.getEntryDate().isBefore(from) || !e.getEntryDate().isBefore(to)) continue; + if ("IN".equals(e.getKind())) sum = sum.add(e.getAmount()); + else if ("OUT".equals(e.getKind())) sum = sum.subtract(e.getAmount()); + } + return sum; + } + + /** Stock on hand against sales velocity. Rising inventory days is cash turning into unsold goods. */ + private Computed inventoryDays(String enterpriseId, LocalDate asOfDate) { + LocalDate from = asOfDate.minusWeeks(recentWindowWeeks); + List entries = ledgerRepository.findByEnterpriseIdOrderByEntryDateDesc(enterpriseId).stream() + .filter(e -> !e.getEntryDate().isAfter(asOfDate) && !e.getEntryDate().isBefore(from)) + .toList(); + if (entries.isEmpty()) return Computed.missing(FeatureCatalog.NO_LEDGER); + + Integer staleness = staleness(entries.get(0).getEntryDate(), asOfDate); + BigDecimal stock = BigDecimal.ZERO, sales = BigDecimal.ZERO; + for (LedgerEntry e : entries) { + if ("STOCK".equals(e.getKind())) stock = stock.add(e.getAmount()); + else if ("IN".equals(e.getKind())) sales = sales.add(e.getAmount()); + } + if (sales.signum() == 0) return Computed.missing(FeatureCatalog.INSUFFICIENT_HISTORY); + + long days = ChronoUnit.DAYS.between(from, asOfDate); + BigDecimal dailySales = sales.divide(BigDecimal.valueOf(Math.max(days, 1)), 4, RoundingMode.HALF_UP); + if (dailySales.signum() == 0) return Computed.missing(FeatureCatalog.INSUFFICIENT_HISTORY); + + return Computed.of(stock.divide(dailySales, 4, RoundingMode.HALF_UP), staleness); + } + + /** Silence is a signal. An enterprise that stops recording has usually stopped trading normally. */ + private Computed daysSinceLastEntry(String enterpriseId, LocalDate asOfDate) { + Optional newest = ledgerRepository.findByEnterpriseIdOrderByEntryDateDesc(enterpriseId).stream() + .map(LedgerEntry::getEntryDate) + .filter(date -> !date.isAfter(asOfDate)) + .max(LocalDate::compareTo); + return newest + .map(date -> { + int days = staleness(date, asOfDate); + return Computed.of(BigDecimal.valueOf(days), days); + }) + .orElseGet(() -> Computed.missing(FeatureCatalog.NO_LEDGER)); + } + + // ── institutional derivations ─────────────────────────── + + private Computed emiDelay(String enterpriseId, LocalDate asOfDate) { + List loans = loanRepository + .findByEnterpriseIdAndAsOfDateLessThanEqual(enterpriseId, asOfDate).stream() + .filter(l -> !Boolean.TRUE.equals(l.getClosed())) + .toList(); + if (loans.isEmpty()) return Computed.missing(FeatureCatalog.NO_CBS); + + LocalDate newest = loans.stream().map(LoanRecord::getAsOfDate).max(LocalDate::compareTo).orElse(asOfDate); + int worstDpd = loans.stream().map(LoanRecord::getDpd).filter(Objects::nonNull) + .mapToInt(Integer::intValue).max().orElse(0); + return Computed.of(BigDecimal.valueOf(worstDpd), staleness(newest, asOfDate)); + } + + private Computed upiInflowChange(String enterpriseId, LocalDate asOfDate, LocalDateTime observedAsOf) { + List series = upiRepository.findTrailing(enterpriseId, asOfDate, observedAsOf); + if (series.isEmpty()) return Computed.missing(FeatureCatalog.NO_CONSENT); + + Integer staleness = staleness(series.get(0).getValidForDate(), asOfDate); + int window = priceWindowWeeks; + if (series.size() < window * 2) return Computed.missing(FeatureCatalog.INSUFFICIENT_HISTORY); + + BigDecimal recent = sumInflow(series.subList(0, window)); + BigDecimal prior = sumInflow(series.subList(window, Math.min(series.size(), window * 2))); + if (prior.signum() == 0) return Computed.missing(FeatureCatalog.INSUFFICIENT_HISTORY); + + return Computed.of(recent.subtract(prior).divide(prior, 4, RoundingMode.HALF_UP), staleness); + } + + /** + * What the instalment takes out of what comes in. Inflow is the ledger's own + * money-in over the recent window, normalised to a month — the enterprise's + * record of its trade, not an institutional estimate of it. + */ + private Computed emiToInflow(String enterpriseId, LocalDate asOfDate) { + List loans = loanRepository + .findByEnterpriseIdAndAsOfDateLessThanEqual(enterpriseId, asOfDate).stream() + .filter(l -> !Boolean.TRUE.equals(l.getClosed())) + .toList(); + if (loans.isEmpty()) return Computed.missing(FeatureCatalog.NO_CBS); + + BigDecimal instalment = loans.stream() + .map(LoanRecord::getInstalmentAmount) + .filter(Objects::nonNull) + .reduce(BigDecimal.ZERO, BigDecimal::add); + if (instalment.signum() == 0) return Computed.missing(FeatureCatalog.NO_CBS); + + LocalDate from = asOfDate.minusWeeks(recentWindowWeeks); + BigDecimal inflow = ledgerRepository.findByEnterpriseIdOrderByEntryDateDesc(enterpriseId).stream() + .filter(e -> "IN".equals(e.getKind())) + .filter(e -> !e.getEntryDate().isAfter(asOfDate) && !e.getEntryDate().isBefore(from)) + .map(LedgerEntry::getAmount) + .reduce(BigDecimal.ZERO, BigDecimal::add); + if (inflow.signum() == 0) return Computed.missing(FeatureCatalog.NO_LEDGER); + + long days = Math.max(ChronoUnit.DAYS.between(from, asOfDate), 1); + BigDecimal monthlyInflow = inflow.divide(BigDecimal.valueOf(days), 4, RoundingMode.HALF_UP) + .multiply(BigDecimal.valueOf(30)); + if (monthlyInflow.signum() == 0) return Computed.missing(FeatureCatalog.NO_LEDGER); + + LocalDate newest = loans.stream().map(LoanRecord::getAsOfDate).max(LocalDate::compareTo).orElse(asOfDate); + return Computed.of(instalment.divide(monthlyInflow, 4, RoundingMode.HALF_UP), staleness(newest, asOfDate)); + } + + /** + * The product's entire trigger, as a number: the worst month of the horizon + * where the downside case has to cover what is due. Read from the latest + * scoring run's stored quantile path, so it reflects the engine that actually + * produced the forecast an officer saw. + */ + private Computed dscrP10(String enterpriseId, LocalDate asOfDate) { + List points = forecastPointRepository.findByEnterpriseIdOrderByHorizonAsc(enterpriseId); + if (points.isEmpty()) return Computed.missing(FeatureCatalog.NO_FORECAST); + + Long latestRun = points.stream().map(ForecastPoint::getRunId) + .filter(Objects::nonNull).max(Long::compareTo).orElse(null); + if (latestRun == null) return Computed.missing(FeatureCatalog.NO_FORECAST); + + BigDecimal worst = null; + for (ForecastPoint point : points) { + if (!latestRun.equals(point.getRunId())) continue; + if (point.getP10() == null || point.getObligation() == null || point.getObligation().signum() == 0) continue; + BigDecimal ratio = point.getP10().divide(point.getObligation(), 4, RoundingMode.HALF_UP); + if (worst == null || ratio.compareTo(worst) < 0) worst = ratio; + } + return worst == null ? Computed.missing(FeatureCatalog.NO_FORECAST) : Computed.of(worst, 0); + } + + private BigDecimal sumInflow(List weeks) { + return weeks.stream() + .map(UpiSummary::getInflowAmount) + .filter(Objects::nonNull) + .reduce(BigDecimal.ZERO, BigDecimal::add); + } + + // ── persistence ───────────────────────────────────────── + + private FeatureSnapshot persist(String enterpriseId, LocalDate asOfDate, LocalDateTime observedAsOf, + Long scoringRunId, Map features) { + + FeatureSnapshot snapshot = snapshotRepository.findByEnterpriseIdAndAsOfDate(enterpriseId, asOfDate) + .orElseGet(() -> FeatureSnapshot.builder() + .enterpriseId(enterpriseId) + .asOfDate(asOfDate) + .build()); + + int present = 0, missing = 0, maxStaleness = 0; + + for (Map.Entry entry : features.entrySet()) { + String code = entry.getKey(); + Computed computed = entry.getValue(); + + FeatureValue row = valueRepository + .findByEnterpriseIdAndFeatureCodeAndValidForDate(enterpriseId, code, asOfDate) + .orElseGet(() -> FeatureValue.builder() + .enterpriseId(enterpriseId) + .featureCode(code) + .validForDate(asOfDate) + .build()); + row.setValue(computed.value()); + row.setStalenessDays(computed.stalenessDays()); + row.setMissingReason(computed.missingReason()); + row.setObservedAt(observedAsOf); + valueRepository.save(row); + + snapshot.put(code, computed.value()); + if (computed.value() == null) missing++; + else present++; + if (computed.stalenessDays() != null) maxStaleness = Math.max(maxStaleness, computed.stalenessDays()); + } + + snapshot.setScoringRunId(scoringRunId); + snapshot.setFeatureCount(present); + snapshot.setMissingCount(missing); + snapshot.setMaxStalenessDays(maxStaleness); + snapshot.setComputedAt(observedAsOf); + return snapshotRepository.save(snapshot); + } + + // ── helpers ───────────────────────────────────────────── + + private Integer marketStaleness(Long districtId, LocalDate asOfDate) { + return marketSignalRepository.findFirstByDistrictIdOrderByValidForDateDesc(districtId) + .map(signal -> staleness(signal.getValidForDate(), asOfDate)) + .orElse(null); + } + + private Integer staleness(LocalDate validFor, LocalDate asOfDate) { + if (validFor == null) return null; + return (int) Math.max(0, ChronoUnit.DAYS.between(validFor, asOfDate)); + } + + private BigDecimal mean(List values) { + if (values == null || values.isEmpty()) return null; + return values.stream().reduce(BigDecimal.ZERO, BigDecimal::add) + .divide(BigDecimal.valueOf(values.size()), 4, RoundingMode.HALF_UP); + } + + /** Sample standard deviation — n−1, because these are samples of a season, not the population. */ + private BigDecimal stdev(List values, BigDecimal mean) { + if (values.size() < 2 || mean == null) return null; + BigDecimal sumSquares = BigDecimal.ZERO; + for (BigDecimal v : values) { + BigDecimal diff = v.subtract(mean); + sumSquares = sumSquares.add(diff.multiply(diff)); + } + double variance = sumSquares.divide(BigDecimal.valueOf(values.size() - 1L), 6, RoundingMode.HALF_UP) + .doubleValue(); + return BigDecimal.valueOf(Math.sqrt(variance)).setScale(4, RoundingMode.HALF_UP); + } + + /** + * One feature's outcome. A null value always carries a reason — that is the + * difference between a hole we can explain and one we cannot. + */ + public record Computed(BigDecimal value, Integer stalenessDays, String missingReason) { + + static Computed of(BigDecimal value, Integer stalenessDays) { + return new Computed(value, stalenessDays, null); + } + + static Computed missing(String reason) { + return new Computed(null, null, reason); + } + } + + /** What one pass over the book did. */ + public record Summary(int snapshots, int skippedForConsent, int nullValues, String message) {} +} diff --git a/src/main/java/com/lincfusion/udyamlinc/features/FeatureSnapshot.java b/src/main/java/com/lincfusion/udyamlinc/features/FeatureSnapshot.java new file mode 100644 index 0000000..305b51b --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/features/FeatureSnapshot.java @@ -0,0 +1,164 @@ +package com.lincfusion.udyamlinc.features; + +import jakarta.persistence.*; +import lombok.*; + +import java.math.BigDecimal; +import java.time.LocalDate; +import java.time.LocalDateTime; +import java.util.LinkedHashMap; +import java.util.Map; + +/** + * The wide store — one row per (enterprise, scoring run). What the risk engine + * and, later, the trainer actually read. + * + *

The tall table alone would mean thousands of row reads per scoring pass and + * far worse for training; this alone would mean a migration every time a feature + * is added. Both exist on purpose, and {@link FeatureValue} remains the system of + * record — this is a materialised read. + * + *

{@link #asFeatureMap()} exposes the same values under their snake_case + * catalogue codes, which is what a trip-wire expression is written against. + */ +@Entity +@Table(name = "feature_snapshot", indexes = { + @Index(name = "ix_feature_snap_key", columnList = "enterpriseId,asOfDate", unique = true), + @Index(name = "ix_feature_snap_run", columnList = "scoringRunId"), +}) +@Getter +@Setter +@NoArgsConstructor +@AllArgsConstructor +@Builder +public class FeatureSnapshot { + + @Id + @GeneratedValue(strategy = GenerationType.IDENTITY) + private Long id; + + @Column(nullable = false, length = 24) + private String enterpriseId; + + @Column(nullable = false) + private LocalDate asOfDate; + + /** Set when the snapshot was built for a specific scoring pass. */ + private Long scoringRunId; + + // ── external shock ────────────────────────────────────── + + @Column(precision = 16, scale = 4) + private BigDecimal feedCostChange; + + @Column(precision = 16, scale = 4) + private BigDecimal outputPriceChange; + + @Column(precision = 16, scale = 4) + private BigDecimal marginSqueeze; + + @Column(precision = 16, scale = 4) + private BigDecimal priceZScore; + + @Column(precision = 16, scale = 4) + private BigDecimal rainfallAnomaly; + + @Column(precision = 16, scale = 4) + private BigDecimal heatAnomaly; + + // ── liquidity & behaviour ─────────────────────────────── + + @Column(precision = 16, scale = 4) + private BigDecimal cashFlowChange; + + @Column(precision = 16, scale = 4) + private BigDecimal emiDelay; + + @Column(precision = 16, scale = 4) + private BigDecimal inventoryDays; + + @Column(precision = 16, scale = 4) + private BigDecimal upiInflowChange; + + /** Silence is a signal: an enterprise that stops recording has usually stopped trading normally. */ + @Column(precision = 16, scale = 4) + private BigDecimal daysSinceLastEntry; + + // ── obligation load ───────────────────────────────────── + + /** Worst month of the horizon — min(p10 / obligation). Below 1.0 the downside case does not cover what is due. */ + @Column(precision = 16, scale = 4) + private BigDecimal dscrP10; + + @Column(precision = 16, scale = 4) + private BigDecimal emiToInflow; + + // ── the guard ─────────────────────────────────────────── + + /** Age of the newest market observation behind this row. Guards all of the above. */ + @Column(precision = 16, scale = 4) + private BigDecimal feedStalenessDays; + + // ── provenance ────────────────────────────────────────── + + /** How many catalogue features carry a value. */ + private Integer featureCount; + + /** How many came back null — a snapshot mostly made of holes must not look confident. */ + private Integer missingCount; + + /** The oldest input age across all features, in days. */ + private Integer maxStalenessDays; + + @Column(nullable = false) + private LocalDateTime computedAt; + + @PrePersist + void onCreate() { + if (computedAt == null) computedAt = LocalDateTime.now(); + } + + /** + * The catalogue-code view, for rule evaluation and for the model. Insertion + * ordered so a debug dump reads the way the catalogue does. + */ + public Map asFeatureMap() { + Map map = new LinkedHashMap<>(); + map.put(FeatureCatalog.FEED_COST_CHANGE, feedCostChange); + map.put(FeatureCatalog.OUTPUT_PRICE_CHANGE, outputPriceChange); + map.put(FeatureCatalog.MARGIN_SQUEEZE, marginSqueeze); + map.put(FeatureCatalog.PRICE_Z_SCORE, priceZScore); + map.put(FeatureCatalog.RAINFALL_ANOMALY, rainfallAnomaly); + map.put(FeatureCatalog.HEAT_ANOMALY, heatAnomaly); + map.put(FeatureCatalog.CASH_FLOW_CHANGE, cashFlowChange); + map.put(FeatureCatalog.EMI_DELAY, emiDelay); + map.put(FeatureCatalog.INVENTORY_DAYS, inventoryDays); + map.put(FeatureCatalog.UPI_INFLOW_CHANGE, upiInflowChange); + map.put(FeatureCatalog.FEED_STALENESS_DAYS, feedStalenessDays); + map.put(FeatureCatalog.DSCR_P10, dscrP10); + map.put(FeatureCatalog.DAYS_SINCE_LAST_ENTRY, daysSinceLastEntry); + map.put(FeatureCatalog.EMI_TO_INFLOW, emiToInflow); + return map; + } + + /** Setter counterpart, so the engineering service writes columns by code. */ + public void put(String code, BigDecimal value) { + switch (code) { + case FeatureCatalog.FEED_COST_CHANGE -> feedCostChange = value; + case FeatureCatalog.OUTPUT_PRICE_CHANGE -> outputPriceChange = value; + case FeatureCatalog.MARGIN_SQUEEZE -> marginSqueeze = value; + case FeatureCatalog.PRICE_Z_SCORE -> priceZScore = value; + case FeatureCatalog.RAINFALL_ANOMALY -> rainfallAnomaly = value; + case FeatureCatalog.HEAT_ANOMALY -> heatAnomaly = value; + case FeatureCatalog.CASH_FLOW_CHANGE -> cashFlowChange = value; + case FeatureCatalog.EMI_DELAY -> emiDelay = value; + case FeatureCatalog.INVENTORY_DAYS -> inventoryDays = value; + case FeatureCatalog.UPI_INFLOW_CHANGE -> upiInflowChange = value; + case FeatureCatalog.FEED_STALENESS_DAYS -> feedStalenessDays = value; + case FeatureCatalog.DSCR_P10 -> dscrP10 = value; + case FeatureCatalog.DAYS_SINCE_LAST_ENTRY -> daysSinceLastEntry = value; + case FeatureCatalog.EMI_TO_INFLOW -> emiToInflow = value; + default -> throw new IllegalArgumentException("Unknown feature code: " + code); + } + } +} diff --git a/src/main/java/com/lincfusion/udyamlinc/features/FeatureSnapshotRepository.java b/src/main/java/com/lincfusion/udyamlinc/features/FeatureSnapshotRepository.java new file mode 100644 index 0000000..b6b00aa --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/features/FeatureSnapshotRepository.java @@ -0,0 +1,20 @@ +package com.lincfusion.udyamlinc.features; + +import org.springframework.data.jpa.repository.JpaRepository; + +import java.time.LocalDate; +import java.util.List; +import java.util.Optional; + +public interface FeatureSnapshotRepository extends JpaRepository { + + Optional findByEnterpriseIdAndAsOfDate(String enterpriseId, LocalDate asOfDate); + + Optional findFirstByEnterpriseIdOrderByAsOfDateDesc(String enterpriseId); + + List findByAsOfDate(LocalDate asOfDate); + + List findByScoringRunId(Long scoringRunId); + + long countByAsOfDate(LocalDate asOfDate); +} diff --git a/src/main/java/com/lincfusion/udyamlinc/features/FeatureValue.java b/src/main/java/com/lincfusion/udyamlinc/features/FeatureValue.java new file mode 100644 index 0000000..6b7184d --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/features/FeatureValue.java @@ -0,0 +1,64 @@ +package com.lincfusion.udyamlinc.features; + +import jakarta.persistence.*; +import lombok.*; + +import java.math.BigDecimal; +import java.time.LocalDate; +import java.time.LocalDateTime; + +/** + * The tall store — one row per (enterprise, feature, date). This is the system of + * record: auditable, and schema-free as features are added. + * + *

Both timestamps are carried for the reason in ARCHITECTURE §2.3. + * {@code stalenessDays} is computed at write time from the age of the newest + * input the value was derived from, so a consumer never has to re-derive + * freshness from the source tables. + */ +@Entity +@Table(name = "feature_value", indexes = { + @Index(name = "ix_feature_value_key", columnList = "enterpriseId,featureCode,validForDate", unique = true), + @Index(name = "ix_feature_value_code", columnList = "featureCode,validForDate"), +}) +@Getter +@Setter +@NoArgsConstructor +@AllArgsConstructor +@Builder +public class FeatureValue { + + @Id + @GeneratedValue(strategy = GenerationType.IDENTITY) + private Long id; + + @Column(nullable = false, length = 24) + private String enterpriseId; + + @Column(nullable = false, length = 48) + private String featureCode; + + /** Null is a value: it means the inputs were absent, not that the row is missing. */ + @Column(precision = 16, scale = 4) + private BigDecimal value; + + /** The date the fact describes. */ + @Column(nullable = false) + private LocalDate validForDate; + + /** When we computed it. */ + @Column(nullable = false) + private LocalDateTime observedAt; + + /** Age of the newest input behind this value, in days. */ + private Integer stalenessDays; + + /** Why the value is null — NO_SIGNAL | NO_LEDGER | NO_CBS | NO_CONSENT | INSUFFICIENT_HISTORY */ + @Column(length = 32) + private String missingReason; + + @PrePersist + void onCreate() { + if (observedAt == null) observedAt = LocalDateTime.now(); + } +} diff --git a/src/main/java/com/lincfusion/udyamlinc/features/FeatureValueRepository.java b/src/main/java/com/lincfusion/udyamlinc/features/FeatureValueRepository.java new file mode 100644 index 0000000..4eec6ec --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/features/FeatureValueRepository.java @@ -0,0 +1,34 @@ +package com.lincfusion.udyamlinc.features; + +import org.springframework.data.jpa.repository.JpaRepository; +import org.springframework.data.jpa.repository.Query; +import org.springframework.data.repository.query.Param; + +import java.time.LocalDate; +import java.time.LocalDateTime; +import java.util.List; +import java.util.Optional; + +public interface FeatureValueRepository extends JpaRepository { + + Optional findByEnterpriseIdAndFeatureCodeAndValidForDate( + String enterpriseId, String featureCode, LocalDate validForDate); + + List findByEnterpriseIdAndValidForDateOrderByFeatureCode(String enterpriseId, LocalDate validForDate); + + List findByEnterpriseIdAndFeatureCodeOrderByValidForDateDesc(String enterpriseId, String featureCode); + + /** The training read: one feature's history for one enterprise, point-in-time. */ + @Query(""" + select f from FeatureValue f + where f.enterpriseId = :enterpriseId and f.featureCode = :featureCode + and f.validForDate <= :asOfDate and f.observedAt <= :asOf + order by f.validForDate desc + """) + List findTrailing(@Param("enterpriseId") String enterpriseId, + @Param("featureCode") String featureCode, + @Param("asOfDate") LocalDate asOfDate, + @Param("asOf") LocalDateTime asOf); + + long countByValidForDate(LocalDate validForDate); +} diff --git a/src/main/java/com/lincfusion/udyamlinc/ingestion/ApiLog.java b/src/main/java/com/lincfusion/udyamlinc/ingestion/ApiLog.java new file mode 100644 index 0000000..413e8ac --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/ingestion/ApiLog.java @@ -0,0 +1,58 @@ +package com.lincfusion.udyamlinc.ingestion; + +import jakarta.persistence.*; +import lombok.*; + +import java.time.LocalDateTime; + +/** + * One outbound call to an external feed. Kept because "the mandi price looked + * wrong on Tuesday" is only answerable if we recorded what we asked and what + * came back. The API key is never stored — the URL is redacted before write. + */ +@Entity +@Table(name = "api_log", indexes = { + @Index(name = "ix_api_log_source_time", columnList = "source,calledAt"), +}) +@Getter +@Setter +@NoArgsConstructor +@AllArgsConstructor +@Builder +public class ApiLog { + + @Id + @GeneratedValue(strategy = GenerationType.IDENTITY) + private Long id; + + /** AGMARKNET | OPEN_METEO | NECC | IMD */ + @Column(nullable = false, length = 32) + private String source; + + @Column(length = 8) + private String method; + + /** Redacted — the api-key value is replaced before this is written. */ + @Column(length = 1024) + private String url; + + private Integer httpStatus; + + private Long durationMs; + + /** Records the response carried, before any filtering. */ + private Integer recordCount; + + private Boolean success; + + @Column(length = 1024) + private String error; + + @Column(nullable = false) + private LocalDateTime calledAt; + + @PrePersist + void onCreate() { + if (calledAt == null) calledAt = LocalDateTime.now(); + } +} diff --git a/src/main/java/com/lincfusion/udyamlinc/ingestion/ApiLogRepository.java b/src/main/java/com/lincfusion/udyamlinc/ingestion/ApiLogRepository.java new file mode 100644 index 0000000..38c0f85 --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/ingestion/ApiLogRepository.java @@ -0,0 +1,13 @@ +package com.lincfusion.udyamlinc.ingestion; + +import org.springframework.data.domain.Pageable; +import org.springframework.data.jpa.repository.JpaRepository; + +import java.util.List; + +public interface ApiLogRepository extends JpaRepository { + + List findBySourceOrderByCalledAtDesc(String source, Pageable pageable); + + List findAllByOrderByCalledAtDesc(Pageable pageable); +} diff --git a/src/main/java/com/lincfusion/udyamlinc/ingestion/ClimateIngestionService.java b/src/main/java/com/lincfusion/udyamlinc/ingestion/ClimateIngestionService.java new file mode 100644 index 0000000..4febc2a --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/ingestion/ClimateIngestionService.java @@ -0,0 +1,203 @@ +package com.lincfusion.udyamlinc.ingestion; + +import com.lincfusion.udyamlinc.ingestion.client.DailyWeather; +import com.lincfusion.udyamlinc.ingestion.client.OpenMeteoClient; +import com.lincfusion.udyamlinc.master.District; +import com.lincfusion.udyamlinc.master.DistrictRepository; +import lombok.RequiredArgsConstructor; +import lombok.extern.slf4j.Slf4j; +import org.springframework.beans.factory.annotation.Value; +import org.springframework.stereotype.Service; + +import java.math.BigDecimal; +import java.math.RoundingMode; +import java.time.LocalDate; +import java.time.LocalDateTime; +import java.util.*; +import java.util.stream.Collectors; + +/** + * Pulls the district weather series and folds it into weekly + * {@link ClimateSignal} rows, each carrying its 30-year normal. + * + *

The normal is computed once per district from a single baseline request and + * cached in {@link ClimateNormal}; nightly runs only fetch the recent window. An + * absolute rainfall figure is not a feature — "40 % below normal for this week + * of the year" is, and that needs the baseline sitting next to the observation. + */ +@Service +@RequiredArgsConstructor +@Slf4j +public class ClimateIngestionService { + + private final OpenMeteoClient client; + private final DistrictRepository districtRepository; + private final ClimateSignalRepository signalRepository; + private final ClimateNormalRepository normalRepository; + + @Value("${udyam.ingestion.climate.lookback-weeks:12}") + private int lookbackWeeks; + + /** The archive trails real time; asking for yesterday returns nothing. */ + @Value("${udyam.ingestion.climate.lag-days:5}") + private int lagDays; + + @Value("${udyam.ingestion.climate.heat-threshold-c:40}") + private double heatThresholdC; + + @Value("${udyam.ingestion.climate.baseline-from:1995-01-01}") + private String baselineFrom; + + @Value("${udyam.ingestion.climate.baseline-to:2024-12-31}") + private String baselineTo; + + /** Not one transaction, for the same reason as {@link MandiIngestionService#ingest()}. */ + public IngestionResult ingest() { + List districts = districtRepository.findByActiveTrue(); + if (districts.isEmpty()) { + return new IngestionResult(0, 0, 0, 0, "No active districts"); + } + + LocalDate to = LocalDate.now().minusDays(lagDays); + LocalDate from = Weeks.startOf(to).minusWeeks(lookbackWeeks); + LocalDateTime observedAt = LocalDateTime.now(); + int fetched = 0, written = 0, skipped = 0, failed = 0; + + for (District district : districts) { + if (district.getLatitude() == null || district.getLongitude() == null) { + skipped++; + log.warn("District {} has no centroid — cannot pick a climate grid cell", district.getCode()); + continue; + } + + ensureNormals(district); + + List days = client.fetchDaily( + district.getLatitude(), district.getLongitude(), from, to); + if (days == null) { + failed++; + continue; + } + fetched += days.size(); + + Map> byWeek = days.stream() + .collect(Collectors.groupingBy(d -> Weeks.startOf(d.date()))); + + for (Map.Entry> week : byWeek.entrySet()) { + upsert(district, week.getKey(), week.getValue(), observedAt); + written++; + } + } + + String message = "%d districts · %d days fetched · %d weeks written · %d calls failed" + .formatted(districts.size(), fetched, written, failed); + log.info("Climate ingestion — {}", message); + return new IngestionResult(fetched, written, skipped, failed, message); + } + + // ── weekly observation ────────────────────────────────── + + private void upsert(District district, LocalDate weekStart, List days, LocalDateTime observedAt) { + int isoWeek = Weeks.isoWeek(weekStart); + + BigDecimal rain = round(days.stream().map(DailyWeather::rainfallMm) + .filter(Objects::nonNull).mapToDouble(Double::doubleValue).sum(), 2); + Double tmaxMean = days.stream().map(DailyWeather::tmaxC).filter(Objects::nonNull) + .mapToDouble(Double::doubleValue).average().orElse(Double.NaN); + Double tminMean = days.stream().map(DailyWeather::tminC).filter(Objects::nonNull) + .mapToDouble(Double::doubleValue).average().orElse(Double.NaN); + int hotDays = (int) days.stream().map(DailyWeather::tmaxC) + .filter(Objects::nonNull).filter(t -> t >= heatThresholdC).count(); + + ClimateSignal signal = signalRepository + .findByDistrictIdAndWeekStartAndSource(district.getId(), weekStart, OpenMeteoClient.SOURCE) + .orElseGet(() -> ClimateSignal.builder() + .districtId(district.getId()) + .weekStart(weekStart) + .source(OpenMeteoClient.SOURCE) + .build()); + + signal.setIsoWeek(isoWeek); + signal.setRainfallMm(rain); + signal.setTmaxC(Double.isNaN(tmaxMean) ? null : round(tmaxMean, 2)); + signal.setTminC(Double.isNaN(tminMean) ? null : round(tminMean, 2)); + signal.setHotDays(hotDays); + signal.setDayCount(days.size()); + signal.setValidForDate(days.stream().map(DailyWeather::date).max(LocalDate::compareTo).orElse(weekStart)); + signal.setObservedAt(observedAt); + + normalRepository.findByDistrictIdAndIsoWeek(district.getId(), isoWeek).ifPresent(normal -> { + signal.setRainfallNormalMm(normal.getNormalRainfallMm()); + signal.setTmaxNormalC(normal.getNormalTmaxC()); + if (normal.getNormalRainfallMm() != null && normal.getNormalRainfallMm().signum() > 0 && rain != null) { + signal.setRainfallAnomalyPct(rain.subtract(normal.getNormalRainfallMm()) + .multiply(BigDecimal.valueOf(100)) + .divide(normal.getNormalRainfallMm(), 2, RoundingMode.HALF_UP)); + } + if (normal.getNormalTmaxC() != null && signal.getTmaxC() != null) { + signal.setHeatAnomalyC(signal.getTmaxC().subtract(normal.getNormalTmaxC())); + } + }); + + signalRepository.save(signal); + } + + // ── the 30-year baseline ──────────────────────────────── + + /** One request per district, ever. Skipped entirely once normals are stored. */ + private void ensureNormals(District district) { + if (normalRepository.countByDistrictId(district.getId()) > 0) return; + + LocalDate from = LocalDate.parse(baselineFrom); + LocalDate to = LocalDate.parse(baselineTo); + log.info("Computing climate normals for {} from {}..{} — one-time", district.getCode(), from, to); + + List days = client.fetchDaily(district.getLatitude(), district.getLongitude(), from, to); + if (days == null || days.isEmpty()) { + log.warn("No baseline series for {} — anomalies stay null until this succeeds", district.getCode()); + return; + } + + // year-week → that year's weekly totals, then mean across years for each ISO week + Map> byYearWeek = days.stream() + .collect(Collectors.groupingBy(d -> Weeks.isoYear(d.date()) + "-" + Weeks.isoWeek(d.date()))); + + Map> rainByWeek = new HashMap<>(); + Map> tmaxByWeek = new HashMap<>(); + Map> yearsByWeek = new HashMap<>(); + + byYearWeek.forEach((key, group) -> { + int year = Integer.parseInt(key.substring(0, key.indexOf('-'))); + int week = Integer.parseInt(key.substring(key.indexOf('-') + 1)); + double weeklyRain = group.stream().map(DailyWeather::rainfallMm) + .filter(Objects::nonNull).mapToDouble(Double::doubleValue).sum(); + OptionalDouble weeklyTmax = group.stream().map(DailyWeather::tmaxC) + .filter(Objects::nonNull).mapToDouble(Double::doubleValue).average(); + rainByWeek.computeIfAbsent(week, k -> new ArrayList<>()).add(weeklyRain); + weeklyTmax.ifPresent(v -> tmaxByWeek.computeIfAbsent(week, k -> new ArrayList<>()).add(v)); + yearsByWeek.computeIfAbsent(week, k -> new HashSet<>()).add(year); + }); + + List normals = new ArrayList<>(); + for (Map.Entry> e : rainByWeek.entrySet()) { + int week = e.getKey(); + normals.add(ClimateNormal.builder() + .districtId(district.getId()) + .isoWeek(week) + .normalRainfallMm(round(e.getValue().stream().mapToDouble(Double::doubleValue).average().orElse(0), 2)) + .normalTmaxC(tmaxByWeek.containsKey(week) + ? round(tmaxByWeek.get(week).stream().mapToDouble(Double::doubleValue).average().orElse(0), 2) + : null) + .years(yearsByWeek.getOrDefault(week, Set.of()).size()) + .baselineFrom(from) + .baselineTo(to) + .build()); + } + normalRepository.saveAll(normals); + log.info("Stored {} weekly normals for {}", normals.size(), district.getCode()); + } + + private BigDecimal round(double value, int scale) { + return BigDecimal.valueOf(value).setScale(scale, RoundingMode.HALF_UP); + } +} diff --git a/src/main/java/com/lincfusion/udyamlinc/ingestion/ClimateNormal.java b/src/main/java/com/lincfusion/udyamlinc/ingestion/ClimateNormal.java new file mode 100644 index 0000000..64b0423 --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/ingestion/ClimateNormal.java @@ -0,0 +1,58 @@ +package com.lincfusion.udyamlinc.ingestion; + +import jakarta.persistence.*; +import lombok.*; + +import java.math.BigDecimal; +import java.time.LocalDate; +import java.time.LocalDateTime; + +/** + * The 30-year baseline for one district and ISO week. Computed once per district + * from a single archive request over the baseline period, then reused — an + * anomaly is meaningless without it, and re-fetching decades of history nightly + * would be absurd. + */ +@Entity +@Table(name = "climate_normal", indexes = { + @Index(name = "ix_climate_normal_key", columnList = "districtId,isoWeek", unique = true), +}) +@Getter +@Setter +@NoArgsConstructor +@AllArgsConstructor +@Builder +public class ClimateNormal { + + @Id + @GeneratedValue(strategy = GenerationType.IDENTITY) + private Long id; + + @Column(nullable = false) + private Long districtId; + + /** 1–53. */ + @Column(nullable = false) + private Integer isoWeek; + + /** Mean weekly rainfall across the baseline years. */ + @Column(precision = 8, scale = 2) + private BigDecimal normalRainfallMm; + + @Column(precision = 6, scale = 2) + private BigDecimal normalTmaxC; + + /** How many baseline years contributed — a thin normal is a weak normal. */ + private Integer years; + + private LocalDate baselineFrom; + + private LocalDate baselineTo; + + private LocalDateTime computedAt; + + @PrePersist + void onCreate() { + if (computedAt == null) computedAt = LocalDateTime.now(); + } +} diff --git a/src/main/java/com/lincfusion/udyamlinc/ingestion/ClimateNormalRepository.java b/src/main/java/com/lincfusion/udyamlinc/ingestion/ClimateNormalRepository.java new file mode 100644 index 0000000..f72b5c7 --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/ingestion/ClimateNormalRepository.java @@ -0,0 +1,15 @@ +package com.lincfusion.udyamlinc.ingestion; + +import org.springframework.data.jpa.repository.JpaRepository; + +import java.util.List; +import java.util.Optional; + +public interface ClimateNormalRepository extends JpaRepository { + + Optional findByDistrictIdAndIsoWeek(Long districtId, Integer isoWeek); + + List findByDistrictId(Long districtId); + + long countByDistrictId(Long districtId); +} diff --git a/src/main/java/com/lincfusion/udyamlinc/ingestion/ClimateSignal.java b/src/main/java/com/lincfusion/udyamlinc/ingestion/ClimateSignal.java new file mode 100644 index 0000000..6987eef --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/ingestion/ClimateSignal.java @@ -0,0 +1,94 @@ +package com.lincfusion.udyamlinc.ingestion; + +import jakarta.persistence.*; +import lombok.*; + +import java.math.BigDecimal; +import java.time.LocalDate; +import java.time.LocalDateTime; + +/** + * A week of weather for one district, with the 30-year normal alongside the + * observation. IMD supplies the advisory text; the numbers come from a + * reanalysis archive, because anomalies need decades of history that live + * endpoints do not serve. + */ +@Entity +@Table(name = "climate_signal", indexes = { + @Index(name = "ix_climate_signal_key", columnList = "districtId,weekStart,source", unique = true), + @Index(name = "ix_climate_signal_week", columnList = "weekStart"), +}) +@Getter +@Setter +@NoArgsConstructor +@AllArgsConstructor +@Builder +public class ClimateSignal { + + @Id + @GeneratedValue(strategy = GenerationType.IDENTITY) + private Long id; + + @Column(nullable = false) + private Long districtId; + + /** Monday of the ISO week. */ + @Column(nullable = false) + private LocalDate weekStart; + + @Column(nullable = false) + private Integer isoWeek; + + @Column(precision = 8, scale = 2) + private BigDecimal rainfallMm; + + @Column(precision = 6, scale = 2) + private BigDecimal tmaxC; + + @Column(precision = 6, scale = 2) + private BigDecimal tminC; + + /** Days in the week above the heat threshold. */ + private Integer hotDays; + + // ── the 30-year normal for this district and ISO week, copied in at write time ── + + @Column(precision = 8, scale = 2) + private BigDecimal rainfallNormalMm; + + @Column(precision = 6, scale = 2) + private BigDecimal tmaxNormalC; + + /** (observed − normal) / normal, as a percentage. Null when no normal is held. */ + @Column(precision = 8, scale = 2) + private BigDecimal rainfallAnomalyPct; + + /** Observed hot days minus the normal's hot days. */ + @Column(precision = 6, scale = 2) + private BigDecimal heatAnomalyC; + + /** How many daily observations backed this week — a short week is a partial week. */ + private Integer dayCount; + + @Column(nullable = false, length = 32) + private String source; + + @Column(nullable = false) + private LocalDate validForDate; + + @Column(nullable = false) + private LocalDateTime observedAt; + + private LocalDateTime updatedAt; + + @PrePersist + void onCreate() { + if (observedAt == null) observedAt = LocalDateTime.now(); + updatedAt = LocalDateTime.now(); + } + + @PreUpdate + void onUpdate() { + updatedAt = LocalDateTime.now(); + } +} diff --git a/src/main/java/com/lincfusion/udyamlinc/ingestion/ClimateSignalRepository.java b/src/main/java/com/lincfusion/udyamlinc/ingestion/ClimateSignalRepository.java new file mode 100644 index 0000000..e69d69c --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/ingestion/ClimateSignalRepository.java @@ -0,0 +1,32 @@ +package com.lincfusion.udyamlinc.ingestion; + +import org.springframework.data.jpa.repository.JpaRepository; +import org.springframework.data.jpa.repository.Query; +import org.springframework.data.repository.query.Param; + +import java.time.LocalDate; +import java.time.LocalDateTime; +import java.util.List; +import java.util.Optional; + +public interface ClimateSignalRepository extends JpaRepository { + + Optional findByDistrictIdAndWeekStartAndSource(Long districtId, LocalDate weekStart, String source); + + List findByDistrictIdOrderByWeekStartDesc(Long districtId); + + List findByWeekStartGreaterThanEqualOrderByWeekStartDesc(LocalDate from); + + /** Point-in-time trailing window, same rule as {@link MarketSignalRepository#findTrailing}. */ + @Query(""" + select s from ClimateSignal s + where s.districtId = :districtId + and s.validForDate <= :asOfDate and s.observedAt <= :asOf + order by s.weekStart desc + """) + List findTrailing(@Param("districtId") Long districtId, + @Param("asOfDate") LocalDate asOfDate, + @Param("asOf") LocalDateTime asOf); + + Optional findFirstByDistrictIdOrderByValidForDateDesc(Long districtId); +} diff --git a/src/main/java/com/lincfusion/udyamlinc/ingestion/IngestionController.java b/src/main/java/com/lincfusion/udyamlinc/ingestion/IngestionController.java new file mode 100644 index 0000000..30729a5 --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/ingestion/IngestionController.java @@ -0,0 +1,109 @@ +package com.lincfusion.udyamlinc.ingestion; + +import com.lincfusion.udyamlinc.common.ApiResponse; +import com.lincfusion.udyamlinc.master.District; +import com.lincfusion.udyamlinc.master.DistrictRepository; +import lombok.RequiredArgsConstructor; +import org.springframework.data.domain.PageRequest; +import org.springframework.web.bind.annotation.*; + +import java.time.LocalDate; +import java.time.temporal.ChronoUnit; +import java.util.ArrayList; +import java.util.List; +import java.util.Map; + +/** + * The operations surface for ingestion: what ran, what it fetched, and how stale + * each district's data now is. + * + *

Mounted under {@code /admin} so the existing rule applies — any officer may + * read, only a supervisor may trigger a run. + */ +@RestController +@RequestMapping("/admin/ingestion") +@RequiredArgsConstructor +public class IngestionController { + + private final IngestionScheduler scheduler; + private final IngestionRunRepository runRepository; + private final MarketSignalRepository marketSignalRepository; + private final ClimateSignalRepository climateSignalRepository; + private final ApiLogRepository apiLogRepository; + private final DistrictRepository districtRepository; + + // ── what ran ──────────────────────────────────────────── + + @GetMapping("/runs") + public ApiResponse> runs(@RequestParam(required = false) String source, + @RequestParam(defaultValue = "30") int limit) { + PageRequest page = PageRequest.of(0, Math.min(limit, 200)); + return ApiResponse.ok(source == null + ? runRepository.findAllByOrderByStartedAtDesc(page) + : runRepository.findBySourceOrderByStartedAtDesc(source, page)); + } + + @GetMapping("/calls") + public ApiResponse> calls(@RequestParam(required = false) String source, + @RequestParam(defaultValue = "50") int limit) { + PageRequest page = PageRequest.of(0, Math.min(limit, 200)); + return ApiResponse.ok(source == null + ? apiLogRepository.findAllByOrderByCalledAtDesc(page) + : apiLogRepository.findBySourceOrderByCalledAtDesc(source, page)); + } + + // ── what it wrote ─────────────────────────────────────── + + @GetMapping("/signals/market") + public ApiResponse> marketSignals(@RequestParam(defaultValue = "12") int weeks) { + return ApiResponse.ok(marketSignalRepository + .findByWeekStartGreaterThanEqualOrderByWeekStartDesc(Weeks.startOf(LocalDate.now()).minusWeeks(weeks))); + } + + @GetMapping("/signals/climate") + public ApiResponse> climateSignals(@RequestParam(defaultValue = "12") int weeks) { + return ApiResponse.ok(climateSignalRepository + .findByWeekStartGreaterThanEqualOrderByWeekStartDesc(Weeks.startOf(LocalDate.now()).minusWeeks(weeks))); + } + + /** + * Per-district freshness — the number a trip-wire consults before it is + * allowed to fire. A rule firing on a three-week-old price is worse than no + * rule. + */ + @GetMapping("/freshness") + public ApiResponse>> freshness() { + LocalDate today = LocalDate.now(); + List> out = new ArrayList<>(); + for (District district : districtRepository.findByActiveTrue()) { + LocalDate market = marketSignalRepository + .findFirstByDistrictIdOrderByValidForDateDesc(district.getId()) + .map(MarketSignal::getValidForDate).orElse(null); + LocalDate climate = climateSignalRepository + .findFirstByDistrictIdOrderByValidForDateDesc(district.getId()) + .map(ClimateSignal::getValidForDate).orElse(null); + Map row = new java.util.LinkedHashMap<>(); + row.put("districtCode", district.getCode()); + row.put("districtName", district.getName()); + row.put("marketValidFor", market); + row.put("marketStalenessDays", market == null ? null : ChronoUnit.DAYS.between(market, today)); + row.put("climateValidFor", climate); + row.put("climateStalenessDays", climate == null ? null : ChronoUnit.DAYS.between(climate, today)); + out.add(row); + } + return ApiResponse.ok(out); + } + + // ── trigger ───────────────────────────────────────────── + + /** Supervisor only, by the {@code /admin/**} write rule in SecurityConfig. */ + @PostMapping("/run") + public ApiResponse> runAll() { + return ApiResponse.ok("Ingestion triggered", scheduler.runAll("MANUAL")); + } + + @PostMapping("/run/{source}") + public ApiResponse runOne(@PathVariable String source) { + return ApiResponse.ok("Ingestion triggered", scheduler.run(source.toUpperCase(), "MANUAL")); + } +} diff --git a/src/main/java/com/lincfusion/udyamlinc/ingestion/IngestionResult.java b/src/main/java/com/lincfusion/udyamlinc/ingestion/IngestionResult.java new file mode 100644 index 0000000..f497dc3 --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/ingestion/IngestionResult.java @@ -0,0 +1,12 @@ +package com.lincfusion.udyamlinc.ingestion; + +/** What one ingestion pass did, on its way into an {@link IngestionRun} row. */ +public record IngestionResult(int fetched, int written, int skipped, int callsFailed, String message) { + + /** A pass that wrote nothing and lost no calls did not really run. */ + public String status() { + if (callsFailed > 0 && written == 0) return IngestionRun.FAILED; + if (callsFailed > 0) return IngestionRun.PARTIAL; + return IngestionRun.SUCCESS; + } +} diff --git a/src/main/java/com/lincfusion/udyamlinc/ingestion/IngestionRun.java b/src/main/java/com/lincfusion/udyamlinc/ingestion/IngestionRun.java new file mode 100644 index 0000000..4641467 --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/ingestion/IngestionRun.java @@ -0,0 +1,65 @@ +package com.lincfusion.udyamlinc.ingestion; + +import jakarta.persistence.*; +import lombok.*; + +import java.time.LocalDate; +import java.time.LocalDateTime; + +/** + * One pass of the nightly job for one source. This is the table the console's + * freshness indicator reads: a scoring run that quietly used three-week-old + * prices is the failure mode this exists to make visible. + */ +@Entity +@Table(name = "ingestion_run", indexes = { + @Index(name = "ix_ingestion_run_source_date", columnList = "source,runDate"), +}) +@Getter +@Setter +@NoArgsConstructor +@AllArgsConstructor +@Builder +public class IngestionRun { + + public static final String SUCCESS = "SUCCESS"; + public static final String PARTIAL = "PARTIAL"; + public static final String FAILED = "FAILED"; + public static final String SKIPPED = "SKIPPED"; + + @Id + @GeneratedValue(strategy = GenerationType.IDENTITY) + private Long id; + + /** AGMARKNET | OPEN_METEO */ + @Column(nullable = false, length = 32) + private String source; + + @Column(nullable = false) + private LocalDate runDate; + + /** SCHEDULED | MANUAL */ + @Column(length = 16) + private String trigger; + + @Column(nullable = false, length = 16) + private String status; + + private Integer rowsFetched; + + private Integer rowsWritten; + + private Integer rowsSkipped; + + /** Upstream calls that failed — a partial run still writes what it got. */ + private Integer callsFailed; + + private LocalDateTime startedAt; + + private LocalDateTime finishedAt; + + private Long durationMs; + + @Column(length = 1024) + private String message; +} diff --git a/src/main/java/com/lincfusion/udyamlinc/ingestion/IngestionRunRepository.java b/src/main/java/com/lincfusion/udyamlinc/ingestion/IngestionRunRepository.java new file mode 100644 index 0000000..2e6182e --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/ingestion/IngestionRunRepository.java @@ -0,0 +1,18 @@ +package com.lincfusion.udyamlinc.ingestion; + +import org.springframework.data.domain.Pageable; +import org.springframework.data.jpa.repository.JpaRepository; + +import java.util.List; +import java.util.Optional; + +public interface IngestionRunRepository extends JpaRepository { + + List findAllByOrderByStartedAtDesc(Pageable pageable); + + List findBySourceOrderByStartedAtDesc(String source, Pageable pageable); + + Optional findFirstBySourceAndStatusOrderByStartedAtDesc(String source, String status); + + Optional findFirstBySourceOrderByStartedAtDesc(String source); +} diff --git a/src/main/java/com/lincfusion/udyamlinc/ingestion/IngestionScheduler.java b/src/main/java/com/lincfusion/udyamlinc/ingestion/IngestionScheduler.java new file mode 100644 index 0000000..0cd0c84 --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/ingestion/IngestionScheduler.java @@ -0,0 +1,113 @@ +package com.lincfusion.udyamlinc.ingestion; + +import com.lincfusion.udyamlinc.ingestion.client.DataGovClient; +import com.lincfusion.udyamlinc.ingestion.client.OpenMeteoClient; +import lombok.RequiredArgsConstructor; +import lombok.extern.slf4j.Slf4j; +import org.springframework.beans.factory.annotation.Value; +import org.springframework.scheduling.annotation.Scheduled; +import org.springframework.stereotype.Component; + +import java.time.LocalDate; +import java.time.LocalDateTime; +import java.util.List; +import java.util.concurrent.atomic.AtomicBoolean; + +/** + * The 02:00 job. This is the only thing that calls an external feed — scoring and + * login read the signal tables and nothing else, so a government endpoint being + * down degrades tomorrow's freshness rather than failing a user request. + * + *

Off by default ({@code udyam.ingestion.enabled=false}) so a demo or a test + * boot never calls out. The manual trigger on {@code /admin/ingestion/run} works + * regardless — that is how you prove the wiring without waiting for 02:00. + */ +@Component +@RequiredArgsConstructor +@Slf4j +public class IngestionScheduler { + + private final MandiIngestionService mandiService; + private final ClimateIngestionService climateService; + private final DataGovClient dataGovClient; + private final IngestionRunRepository runRepository; + + /** One pass at a time. A second trigger while a run is live is refused, not queued. */ + private final AtomicBoolean running = new AtomicBoolean(false); + + @Value("${udyam.ingestion.enabled:false}") + private boolean enabled; + + @Scheduled(cron = "${udyam.ingestion.cron:0 0 2 * * *}", zone = "${udyam.ingestion.zone:Asia/Kolkata}") + public void nightly() { + if (!enabled) { + log.debug("Nightly ingestion skipped — udyam.ingestion.enabled=false"); + return; + } + runAll("SCHEDULED"); + } + + /** Both sources, in order. Used by the scheduler and by the admin trigger. */ + public List runAll(String trigger) { + return List.of( + run(DataGovClient.SOURCE, trigger), + run(OpenMeteoClient.SOURCE, trigger)); + } + + /** + * One source, recorded as an {@link IngestionRun} whatever the outcome — + * including the skip, because "nothing ran last night" and "it ran and found + * nothing" must not look the same in the console. + */ + public IngestionRun run(String source, String trigger) { + if (!running.compareAndSet(false, true)) { + return record(source, trigger, IngestionRun.SKIPPED, + new IngestionResult(0, 0, 0, 0, "An ingestion pass is already running"), + LocalDateTime.now()); + } + + LocalDateTime started = LocalDateTime.now(); + try { + if (DataGovClient.SOURCE.equals(source) && !dataGovClient.configured()) { + return record(source, trigger, IngestionRun.SKIPPED, + new IngestionResult(0, 0, 0, 0, + "No data.gov.in API key — set udyam.ingestion.datagov.api-key or DATAGOV_API_KEY"), + started); + } + + IngestionResult result = switch (source) { + case DataGovClient.SOURCE -> mandiService.ingest(); + case OpenMeteoClient.SOURCE -> climateService.ingest(); + default -> throw new IllegalArgumentException("Unknown ingestion source: " + source); + }; + return record(source, trigger, result.status(), result, started); + + } catch (Exception e) { + log.error("Ingestion pass failed for {}", source, e); + return record(source, trigger, IngestionRun.FAILED, + new IngestionResult(0, 0, 0, 1, e.toString()), started); + } finally { + running.set(false); + } + } + + private IngestionRun record(String source, String trigger, String status, + IngestionResult result, LocalDateTime started) { + LocalDateTime finished = LocalDateTime.now(); + String message = result.message(); + return runRepository.save(IngestionRun.builder() + .source(source) + .runDate(LocalDate.now()) + .trigger(trigger) + .status(status) + .rowsFetched(result.fetched()) + .rowsWritten(result.written()) + .rowsSkipped(result.skipped()) + .callsFailed(result.callsFailed()) + .startedAt(started) + .finishedAt(finished) + .durationMs(java.time.Duration.between(started, finished).toMillis()) + .message(message == null || message.length() <= 1000 ? message : message.substring(0, 1000)) + .build()); + } +} diff --git a/src/main/java/com/lincfusion/udyamlinc/ingestion/MandiIngestionService.java b/src/main/java/com/lincfusion/udyamlinc/ingestion/MandiIngestionService.java new file mode 100644 index 0000000..9b29733 --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/ingestion/MandiIngestionService.java @@ -0,0 +1,140 @@ +package com.lincfusion.udyamlinc.ingestion; + +import com.lincfusion.udyamlinc.ingestion.client.DataGovClient; +import com.lincfusion.udyamlinc.ingestion.client.MandiRecord; +import com.lincfusion.udyamlinc.master.*; +import lombok.RequiredArgsConstructor; +import lombok.extern.slf4j.Slf4j; +import org.springframework.beans.factory.annotation.Value; +import org.springframework.stereotype.Service; + +import java.math.BigDecimal; +import java.math.RoundingMode; +import java.time.LocalDate; +import java.time.LocalDateTime; +import java.util.*; +import java.util.stream.Collectors; + +/** + * Pulls Agmarknet daily arrivals and folds them into one weekly + * {@link MarketSignal} per (district, commodity). + * + *

Weekly, not daily, for two reasons: a mandi does not report every day, so a + * daily series is mostly holes; and every feature that consumes this — feed cost + * change, margin squeeze, price z-score — is defined against a trailing weekly + * mean anyway. + * + *

Upserts on (district, commodity, week, source), so re-running the job on the + * same day corrects a partial week instead of duplicating it. {@code observedAt} + * is stamped fresh on every write, which is what lets a later scoring run tell + * the difference between a price we held on Tuesday and one that arrived today. + */ +@Service +@RequiredArgsConstructor +@Slf4j +public class MandiIngestionService { + + private final DataGovClient client; + private final DistrictRepository districtRepository; + private final CommodityRepository commodityRepository; + private final MarketRepository marketRepository; + private final MarketSignalRepository signalRepository; + + @Value("${udyam.ingestion.mandi.lookback-weeks:12}") + private int lookbackWeeks; + + /** + * One pass over every active district × Agmarknet commodity. Failures are + * counted, not thrown: a partial run still writes what it got. + * + *

Deliberately not one transaction. The loop is dominated by outbound HTTP, + * and holding a database connection open across a slow government endpoint is + * how a nightly job takes the pool down. Each upsert commits on its own. + */ + public IngestionResult ingest() { + List districts = districtRepository.findByActiveTrue(); + List commodities = commodityRepository.findBySourceAndActiveTrue(DataGovClient.SOURCE); + + if (districts.isEmpty() || commodities.isEmpty()) { + return new IngestionResult(0, 0, 0, 0, "Nothing to pull — " + districts.size() + + " active districts, " + commodities.size() + " Agmarknet commodities"); + } + + LocalDate horizon = Weeks.startOf(LocalDate.now()).minusWeeks(lookbackWeeks); + LocalDateTime observedAt = LocalDateTime.now(); + int fetched = 0, written = 0, skipped = 0, failed = 0; + + for (District district : districts) { + Long marketId = marketRepository.findFirstByDistrictIdAndPrimaryMarketTrue(district.getId()) + .map(Market::getId).orElse(null); + + for (Commodity commodity : commodities) { + String sourceCode = commodity.getSourceCode() == null ? commodity.getName() : commodity.getSourceCode(); + List records = client.fetchPrices(district.getState(), district.getName(), sourceCode); + + if (records == null) { + failed++; + continue; + } + fetched += records.size(); + + Map> byWeek = records.stream() + .filter(r -> !Weeks.startOf(r.arrivalDate()).isBefore(horizon)) + .collect(Collectors.groupingBy(r -> Weeks.startOf(r.arrivalDate()))); + + if (byWeek.isEmpty()) { + skipped += records.size(); + continue; + } + + for (Map.Entry> week : byWeek.entrySet()) { + upsert(district, commodity, marketId, week.getKey(), week.getValue(), observedAt); + written++; + } + } + } + + String message = "%d districts × %d commodities · %d rows fetched · %d weeks written · %d calls failed" + .formatted(districts.size(), commodities.size(), fetched, written, failed); + log.info("Agmarknet ingestion — {}", message); + return new IngestionResult(fetched, written, skipped, failed, message); + } + + private void upsert(District district, Commodity commodity, Long marketId, + LocalDate weekStart, List rows, LocalDateTime observedAt) { + + BigDecimal modal = mean(rows.stream().map(MandiRecord::modalPrice).filter(Objects::nonNull).toList()); + BigDecimal min = rows.stream().map(MandiRecord::minPrice).filter(Objects::nonNull) + .min(BigDecimal::compareTo).orElse(null); + BigDecimal max = rows.stream().map(MandiRecord::maxPrice).filter(Objects::nonNull) + .max(BigDecimal::compareTo).orElse(null); + LocalDate latestArrival = rows.stream().map(MandiRecord::arrivalDate).filter(Objects::nonNull) + .max(LocalDate::compareTo).orElse(weekStart); + + MarketSignal signal = signalRepository + .findByDistrictIdAndCommodityIdAndWeekStartAndSource( + district.getId(), commodity.getId(), weekStart, DataGovClient.SOURCE) + .orElseGet(() -> MarketSignal.builder() + .districtId(district.getId()) + .commodityId(commodity.getId()) + .weekStart(weekStart) + .source(DataGovClient.SOURCE) + .build()); + + signal.setMarketId(marketId); + signal.setModalPrice(modal); + signal.setMinPrice(min); + signal.setMaxPrice(max); + signal.setArrivalCount(rows.size()); + signal.setUnit(commodity.getUnit()); + signal.setValidForDate(latestArrival); + signal.setObservedAt(observedAt); + signalRepository.save(signal); + } + + private BigDecimal mean(List values) { + if (values.isEmpty()) return null; + BigDecimal sum = values.stream().reduce(BigDecimal.ZERO, BigDecimal::add); + return sum.divide(BigDecimal.valueOf(values.size()), 2, RoundingMode.HALF_UP); + } +} diff --git a/src/main/java/com/lincfusion/udyamlinc/ingestion/MarketSignal.java b/src/main/java/com/lincfusion/udyamlinc/ingestion/MarketSignal.java new file mode 100644 index 0000000..f36d77b --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/ingestion/MarketSignal.java @@ -0,0 +1,90 @@ +package com.lincfusion.udyamlinc.ingestion; + +import jakarta.persistence.*; +import lombok.*; + +import java.math.BigDecimal; +import java.time.LocalDate; +import java.time.LocalDateTime; + +/** + * A week of mandi prices for one (district, commodity). Pulled per district — + * never per enterprise — which is what keeps ingestion to roughly forty rows a + * day for the pilot's four districts. + * + *

Carries both timestamps demanded by point-in-time correctness: + * {@code validForDate} is the latest arrival the week's prices describe, + * {@code observedAt} is when we learned it. A price for 3 March fetched on + * 7 March must stay invisible to a 4 March scoring run. + */ +@Entity +@Table(name = "market_signal", indexes = { + @Index(name = "ix_market_signal_key", columnList = "districtId,commodityId,weekStart,source", unique = true), + @Index(name = "ix_market_signal_week", columnList = "weekStart"), + @Index(name = "ix_market_signal_observed", columnList = "observedAt"), +}) +@Getter +@Setter +@NoArgsConstructor +@AllArgsConstructor +@Builder +public class MarketSignal { + + @Id + @GeneratedValue(strategy = GenerationType.IDENTITY) + private Long id; + + @Column(nullable = false) + private Long districtId; + + @Column(nullable = false) + private Long commodityId; + + /** The reporting mandi, when the feed named one we recognise. */ + private Long marketId; + + /** Monday of the ISO week these prices belong to. */ + @Column(nullable = false) + private LocalDate weekStart; + + @Column(precision = 12, scale = 2) + private BigDecimal modalPrice; + + @Column(precision = 12, scale = 2) + private BigDecimal minPrice; + + @Column(precision = 12, scale = 2) + private BigDecimal maxPrice; + + /** How many daily arrivals were aggregated into this row — thin weeks are less trustworthy. */ + private Integer arrivalCount; + + /** QUINTAL | KG | HUNDRED | LITRE — copied from the commodity at write time. */ + @Column(length = 16) + private String unit; + + /** AGMARKNET | NECC | DAIRY_FEDERATION | MANUAL */ + @Column(nullable = false, length = 32) + private String source; + + /** The date the fact describes — the latest arrival date inside the week. */ + @Column(nullable = false) + private LocalDate validForDate; + + /** When we learned it. */ + @Column(nullable = false) + private LocalDateTime observedAt; + + private LocalDateTime updatedAt; + + @PrePersist + void onCreate() { + if (observedAt == null) observedAt = LocalDateTime.now(); + updatedAt = LocalDateTime.now(); + } + + @PreUpdate + void onUpdate() { + updatedAt = LocalDateTime.now(); + } +} diff --git a/src/main/java/com/lincfusion/udyamlinc/ingestion/MarketSignalRepository.java b/src/main/java/com/lincfusion/udyamlinc/ingestion/MarketSignalRepository.java new file mode 100644 index 0000000..2a7aea7 --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/ingestion/MarketSignalRepository.java @@ -0,0 +1,38 @@ +package com.lincfusion.udyamlinc.ingestion; + +import org.springframework.data.jpa.repository.JpaRepository; +import org.springframework.data.jpa.repository.Query; +import org.springframework.data.repository.query.Param; + +import java.time.LocalDate; +import java.time.LocalDateTime; +import java.util.List; +import java.util.Optional; + +public interface MarketSignalRepository extends JpaRepository { + + Optional findByDistrictIdAndCommodityIdAndWeekStartAndSource( + Long districtId, Long commodityId, LocalDate weekStart, String source); + + List findByDistrictIdAndCommodityIdOrderByWeekStartDesc(Long districtId, Long commodityId); + + List findByWeekStartGreaterThanEqualOrderByWeekStartDesc(LocalDate from); + + /** + * The trailing window a feature derivation reads, filtered point-in-time: + * only rows we had actually observed by {@code asOf} are visible. + */ + @Query(""" + select s from MarketSignal s + where s.districtId = :districtId and s.commodityId = :commodityId + and s.validForDate <= :asOfDate and s.observedAt <= :asOf + order by s.weekStart desc + """) + List findTrailing(@Param("districtId") Long districtId, + @Param("commodityId") Long commodityId, + @Param("asOfDate") LocalDate asOfDate, + @Param("asOf") LocalDateTime asOf); + + /** Freshness: the newest observation we hold for a district, whatever the commodity. */ + Optional findFirstByDistrictIdOrderByValidForDateDesc(Long districtId); +} diff --git a/src/main/java/com/lincfusion/udyamlinc/ingestion/Weeks.java b/src/main/java/com/lincfusion/udyamlinc/ingestion/Weeks.java new file mode 100644 index 0000000..50bf0ea --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/ingestion/Weeks.java @@ -0,0 +1,30 @@ +package com.lincfusion.udyamlinc.ingestion; + +import java.time.DayOfWeek; +import java.time.LocalDate; +import java.time.temporal.WeekFields; + +/** + * Everything upstream reports daily; everything downstream reads weekly. One + * definition of "week" for both signal tables, so a mandi price and a rainfall + * figure for the same period actually line up. + */ +public final class Weeks { + + private Weeks() {} + + /** Monday of the ISO week the date falls in. */ + public static LocalDate startOf(LocalDate date) { + return date.with(DayOfWeek.MONDAY); + } + + /** ISO week number, 1–53. */ + public static int isoWeek(LocalDate date) { + return date.get(WeekFields.ISO.weekOfWeekBasedYear()); + } + + /** ISO week-based year — not always the calendar year at a year boundary. */ + public static int isoYear(LocalDate date) { + return date.get(WeekFields.ISO.weekBasedYear()); + } +} diff --git a/src/main/java/com/lincfusion/udyamlinc/ingestion/client/DailyWeather.java b/src/main/java/com/lincfusion/udyamlinc/ingestion/client/DailyWeather.java new file mode 100644 index 0000000..78fbaf4 --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/ingestion/client/DailyWeather.java @@ -0,0 +1,11 @@ +package com.lincfusion.udyamlinc.ingestion.client; + +import java.time.LocalDate; + +/** One day of the reanalysis series for a district centroid. */ +public record DailyWeather(LocalDate date, Double rainfallMm, Double tmaxC, Double tminC) { + + public boolean usable() { + return date != null && (rainfallMm != null || tmaxC != null); + } +} diff --git a/src/main/java/com/lincfusion/udyamlinc/ingestion/client/DataGovClient.java b/src/main/java/com/lincfusion/udyamlinc/ingestion/client/DataGovClient.java new file mode 100644 index 0000000..28dcffa --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/ingestion/client/DataGovClient.java @@ -0,0 +1,202 @@ +package com.lincfusion.udyamlinc.ingestion.client; + +import com.lincfusion.udyamlinc.ingestion.ApiLog; +import com.lincfusion.udyamlinc.ingestion.ApiLogRepository; +import lombok.extern.slf4j.Slf4j; +import org.springframework.beans.factory.annotation.Qualifier; +import org.springframework.beans.factory.annotation.Value; +import org.springframework.stereotype.Component; +import org.springframework.web.client.RestClient; +import org.springframework.web.util.UriComponentsBuilder; + +import java.math.BigDecimal; +import java.net.URI; +import java.time.LocalDate; +import java.time.format.DateTimeFormatter; +import java.util.*; + +/** + * Agmarknet mandi prices via the data.gov.in Open Government Data API. + * + *

Two things about this feed drive the design. Its field names change casing + * between resource versions ({@code arrival_date} vs {@code Arrival_Date}), so + * every read goes through a case-insensitive lookup rather than a bound DTO. And + * it answers with HTTP 200 and an empty {@code records} array when a filter + * matches nothing, so "no data" and "wrong filter" look identical — which is why + * the caller records the record count per call. + * + *

Never called from a request thread. + */ +@Component +@Slf4j +public class DataGovClient { + + public static final String SOURCE = "AGMARKNET"; + + /** Agmarknet publishes dd/MM/yyyy; some mirrors publish ISO. Both are accepted. */ + private static final DateTimeFormatter DMY = DateTimeFormatter.ofPattern("dd/MM/yyyy"); + + private final RestClient http; + private final ApiLogRepository apiLogRepository; + + @Value("${udyam.ingestion.datagov.base-url:https://api.data.gov.in/resource}") + private String baseUrl; + + @Value("${udyam.ingestion.datagov.resource-id:9ef84268-d588-465a-a308-a864a43d0070}") + private String resourceId; + + @Value("${udyam.ingestion.datagov.api-key:}") + private String apiKey; + + @Value("${udyam.ingestion.datagov.page-limit:1000}") + private int pageLimit; + + // Filter field names, because they must match the resource's own field names. + @Value("${udyam.ingestion.datagov.field.state:state}") + private String fieldState; + + @Value("${udyam.ingestion.datagov.field.district:district}") + private String fieldDistrict; + + @Value("${udyam.ingestion.datagov.field.commodity:commodity}") + private String fieldCommodity; + + public DataGovClient(@Qualifier("externalFeedClient") RestClient http, ApiLogRepository apiLogRepository) { + this.http = http; + this.apiLogRepository = apiLogRepository; + } + + /** False when no key is configured — ingestion logs and skips rather than calling and failing. */ + public boolean configured() { + return apiKey != null && !apiKey.isBlank(); + } + + /** + * Daily arrivals for one district and commodity. + * + *

Returns {@code null} when the call itself failed and an empty list when + * the feed answered with no rows. The distinction matters: the first is an + * outage to report, the second is a filter that matched nothing. Either way + * the run continues — a feed being down degrades tomorrow's freshness, it + * does not abort ingestion. + */ + @SuppressWarnings("unchecked") + public List fetchPrices(String state, String district, String commoditySourceCode) { + if (!configured()) return null; + + URI uri = UriComponentsBuilder.fromUriString(baseUrl + "/" + resourceId) + .queryParam("api-key", apiKey) + .queryParam("format", "json") + .queryParam("limit", pageLimit) + .queryParam("filters[" + fieldState + "]", state) + .queryParam("filters[" + fieldDistrict + "]", district) + .queryParam("filters[" + fieldCommodity + "]", commoditySourceCode) + .build() + .encode() + .toUri(); + + long started = System.currentTimeMillis(); + try { + Map body = http.get().uri(uri).retrieve().body(Map.class); + List> records = body == null + ? List.of() + : (List>) body.getOrDefault("records", List.of()); + + List parsed = new ArrayList<>(records.size()); + for (Map row : records) { + MandiRecord rec = parse(row); + if (rec.usable()) parsed.add(rec); + } + + logCall(uri, 200, started, records.size(), true, null); + if (records.isEmpty()) { + log.warn("data.gov.in returned no records for {}/{} — check the filter field names", district, commoditySourceCode); + } + return parsed; + + } catch (Exception e) { + logCall(uri, null, started, null, false, e.getMessage()); + log.warn("data.gov.in call failed for {}/{}: {}", district, commoditySourceCode, e.toString()); + return null; + } + } + + // ── parsing ───────────────────────────────────────────── + + private MandiRecord parse(Map row) { + return new MandiRecord( + str(row, "state"), + str(row, "district"), + str(row, "market"), + str(row, "commodity"), + str(row, "variety"), + date(str(row, "arrival_date", "arrivaldate", "date")), + num(str(row, "min_price", "minprice")), + num(str(row, "max_price", "maxprice")), + num(str(row, "modal_price", "modalprice"))); + } + + /** Case- and underscore-insensitive lookup, because the feed's casing is not stable. */ + private String str(Map row, String... keys) { + for (String key : keys) { + for (Map.Entry e : row.entrySet()) { + if (normalise(e.getKey()).equals(normalise(key))) { + Object v = e.getValue(); + if (v == null) return null; + String s = String.valueOf(v).trim(); + return s.isEmpty() || "NA".equalsIgnoreCase(s) ? null : s; + } + } + } + return null; + } + + private String normalise(String s) { + return s.replace("_", "").replace(" ", "").toLowerCase(Locale.ROOT); + } + + private LocalDate date(String raw) { + if (raw == null) return null; + try { + return LocalDate.parse(raw, DMY); + } catch (Exception ignored) { + try { + return LocalDate.parse(raw); + } catch (Exception e) { + return null; + } + } + } + + private BigDecimal num(String raw) { + if (raw == null) return null; + try { + return new BigDecimal(raw.replace(",", "")); + } catch (NumberFormatException e) { + return null; + } + } + + private void logCall(URI uri, Integer status, long started, Integer count, boolean ok, String error) { + try { + apiLogRepository.save(ApiLog.builder() + .source(SOURCE) + .method("GET") + .url(redact(uri.toString())) + .httpStatus(status) + .durationMs(System.currentTimeMillis() - started) + .recordCount(count) + .success(ok) + .error(error == null ? null : error.substring(0, Math.min(1000, error.length()))) + .build()); + } catch (Exception e) { + log.warn("Could not write api_log: {}", e.toString()); + } + } + + /** The key never reaches the audit table. */ + private String redact(String url) { + String redacted = url.replaceAll("api-key=[^&]*", "api-key=***"); + return redacted.length() > 1000 ? redacted.substring(0, 1000) : redacted; + } +} diff --git a/src/main/java/com/lincfusion/udyamlinc/ingestion/client/MandiRecord.java b/src/main/java/com/lincfusion/udyamlinc/ingestion/client/MandiRecord.java new file mode 100644 index 0000000..9e265fb --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/ingestion/client/MandiRecord.java @@ -0,0 +1,25 @@ +package com.lincfusion.udyamlinc.ingestion.client; + +import java.math.BigDecimal; +import java.time.LocalDate; + +/** + * One daily arrival row as Agmarknet publishes it, already normalised out of the + * feed's inconsistent field casing. + */ +public record MandiRecord( + String state, + String district, + String market, + String commodity, + String variety, + LocalDate arrivalDate, + BigDecimal minPrice, + BigDecimal maxPrice, + BigDecimal modalPrice) { + + /** A row with no modal price and no date tells us nothing; the parser drops it. */ + public boolean usable() { + return arrivalDate != null && modalPrice != null && modalPrice.signum() > 0; + } +} diff --git a/src/main/java/com/lincfusion/udyamlinc/ingestion/client/OpenMeteoClient.java b/src/main/java/com/lincfusion/udyamlinc/ingestion/client/OpenMeteoClient.java new file mode 100644 index 0000000..b8b7207 --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/ingestion/client/OpenMeteoClient.java @@ -0,0 +1,118 @@ +package com.lincfusion.udyamlinc.ingestion.client; + +import com.lincfusion.udyamlinc.ingestion.ApiLog; +import com.lincfusion.udyamlinc.ingestion.ApiLogRepository; +import lombok.extern.slf4j.Slf4j; +import org.springframework.beans.factory.annotation.Qualifier; +import org.springframework.beans.factory.annotation.Value; +import org.springframework.stereotype.Component; +import org.springframework.web.client.RestClient; +import org.springframework.web.util.UriComponentsBuilder; + +import java.net.URI; +import java.time.LocalDate; +import java.util.ArrayList; +import java.util.List; +import java.util.Map; + +/** + * Rainfall and temperature for a district centroid, from the Open-Meteo + * reanalysis archive. No key, and decades of history in one request — which is + * the only reason a 30-year normal is computable at all. IMD supplies the + * district advisory text; it does not serve the series anomalies need. + * + *

The archive trails real time by a few days. {@code lag-days} keeps the + * requested window inside what the archive actually holds, so a run does not + * report failure for data that simply is not published yet. + */ +@Component +@Slf4j +public class OpenMeteoClient { + + public static final String SOURCE = "OPEN_METEO"; + + private final RestClient http; + private final ApiLogRepository apiLogRepository; + + @Value("${udyam.ingestion.climate.base-url:https://archive-api.open-meteo.com/v1/archive}") + private String baseUrl; + + @Value("${udyam.ingestion.climate.timezone:Asia/Kolkata}") + private String timezone; + + public OpenMeteoClient(@Qualifier("externalFeedClient") RestClient http, ApiLogRepository apiLogRepository) { + this.http = http; + this.apiLogRepository = apiLogRepository; + } + + /** + * The daily series for one grid cell. {@code null} when the call failed, + * empty when the archive held nothing for the window. + */ + @SuppressWarnings("unchecked") + public List fetchDaily(double latitude, double longitude, LocalDate from, LocalDate to) { + URI uri = UriComponentsBuilder.fromUriString(baseUrl) + .queryParam("latitude", latitude) + .queryParam("longitude", longitude) + .queryParam("start_date", from) + .queryParam("end_date", to) + .queryParam("daily", "precipitation_sum,temperature_2m_max,temperature_2m_min") + .queryParam("timezone", timezone) + .build() + .encode() + .toUri(); + + long started = System.currentTimeMillis(); + try { + Map body = http.get().uri(uri).retrieve().body(Map.class); + Map daily = body == null ? null : (Map) body.get("daily"); + if (daily == null) { + logCall(uri, 200, started, 0, false, "no daily block in response"); + return null; + } + + List dates = (List) daily.getOrDefault("time", List.of()); + List rain = (List) daily.getOrDefault("precipitation_sum", List.of()); + List tmax = (List) daily.getOrDefault("temperature_2m_max", List.of()); + List tmin = (List) daily.getOrDefault("temperature_2m_min", List.of()); + + List out = new ArrayList<>(dates.size()); + for (int i = 0; i < dates.size(); i++) { + DailyWeather day = new DailyWeather( + LocalDate.parse(dates.get(i)), at(rain, i), at(tmax, i), at(tmin, i)); + if (day.usable()) out.add(day); + } + + logCall(uri, 200, started, out.size(), true, null); + return out; + + } catch (Exception e) { + logCall(uri, null, started, null, false, e.getMessage()); + log.warn("Open-Meteo call failed for {},{} {}..{}: {}", latitude, longitude, from, to, e.toString()); + return null; + } + } + + private Double at(List series, int i) { + if (series == null || i >= series.size() || series.get(i) == null) return null; + return series.get(i).doubleValue(); + } + + private void logCall(URI uri, Integer status, long started, Integer count, boolean ok, String error) { + try { + String url = uri.toString(); + apiLogRepository.save(ApiLog.builder() + .source(SOURCE) + .method("GET") + .url(url.length() > 1000 ? url.substring(0, 1000) : url) + .httpStatus(status) + .durationMs(System.currentTimeMillis() - started) + .recordCount(count) + .success(ok) + .error(error == null ? null : error.substring(0, Math.min(1000, error.length()))) + .build()); + } catch (Exception e) { + log.warn("Could not write api_log: {}", e.toString()); + } + } +} diff --git a/src/main/java/com/lincfusion/udyamlinc/master/Commodity.java b/src/main/java/com/lincfusion/udyamlinc/master/Commodity.java new file mode 100644 index 0000000..b67add4 --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/master/Commodity.java @@ -0,0 +1,51 @@ +package com.lincfusion.udyamlinc.master; + +import jakarta.persistence.*; +import lombok.*; + +/** + * A tradeable commodity tracked in a price feed. {@code sourceCode} is the name + * the upstream feed uses, which rarely matches ours — Agmarknet says "Maize", + * NECC says "Egg" with no commodity field at all. + */ +@Entity +@Table(name = "commodity", indexes = @Index(name = "ix_commodity_code", columnList = "code", unique = true)) +@Getter +@Setter +@NoArgsConstructor +@AllArgsConstructor +@Builder +public class Commodity { + + @Id + @GeneratedValue(strategy = GenerationType.IDENTITY) + private Long id; + + @Column(nullable = false, unique = true, length = 32) + private String code; + + @Column(nullable = false, length = 64) + private String name; + + @Column(length = 64) + private String nameHi; + + /** AGMARKNET | NECC | DAIRY_FEDERATION | MANUAL */ + @Column(nullable = false, length = 32) + private String source; + + /** The identifier this commodity carries in the upstream feed. */ + @Column(length = 64) + private String sourceCode; + + /** QUINTAL | KG | HUNDRED | LITRE */ + @Column(length = 16) + private String unit; + + private Boolean active; + + @PrePersist + void onCreate() { + if (active == null) active = Boolean.TRUE; + } +} diff --git a/src/main/java/com/lincfusion/udyamlinc/master/CommodityRepository.java b/src/main/java/com/lincfusion/udyamlinc/master/CommodityRepository.java new file mode 100644 index 0000000..50c1e6d --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/master/CommodityRepository.java @@ -0,0 +1,15 @@ +package com.lincfusion.udyamlinc.master; + +import org.springframework.data.jpa.repository.JpaRepository; + +import java.util.List; +import java.util.Optional; + +public interface CommodityRepository extends JpaRepository { + + Optional findByCode(String code); + + List findBySourceAndActiveTrue(String source); + + List findByActiveTrue(); +} diff --git a/src/main/java/com/lincfusion/udyamlinc/master/District.java b/src/main/java/com/lincfusion/udyamlinc/master/District.java new file mode 100644 index 0000000..cc3cd29 --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/master/District.java @@ -0,0 +1,48 @@ +package com.lincfusion.udyamlinc.master; + +import jakarta.persistence.*; +import lombok.*; + +/** + * The join key for every external signal. Mandi prices and climate series are + * pulled per district, never per enterprise — that is what keeps ingestion to + * roughly forty rows a day instead of one call per enterprise. + */ +@Entity +@Table(name = "district", indexes = @Index(name = "ix_district_code", columnList = "code", unique = true)) +@Getter +@Setter +@NoArgsConstructor +@AllArgsConstructor +@Builder +public class District { + + @Id + @GeneratedValue(strategy = GenerationType.IDENTITY) + private Long id; + + /** SEHORE, VIDISHA… */ + @Column(nullable = false, unique = true, length = 32) + private String code; + + @Column(nullable = false, length = 64) + private String name; + + @Column(length = 64) + private String nameHi; + + @Column(nullable = false, length = 64) + private String state; + + /** District centroid — used to pick the climate grid cell. */ + private Double latitude; + + private Double longitude; + + private Boolean active; + + @PrePersist + void onCreate() { + if (active == null) active = Boolean.TRUE; + } +} diff --git a/src/main/java/com/lincfusion/udyamlinc/master/DistrictRepository.java b/src/main/java/com/lincfusion/udyamlinc/master/DistrictRepository.java new file mode 100644 index 0000000..0fc0002 --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/master/DistrictRepository.java @@ -0,0 +1,15 @@ +package com.lincfusion.udyamlinc.master; + +import org.springframework.data.jpa.repository.JpaRepository; + +import java.util.List; +import java.util.Optional; + +public interface DistrictRepository extends JpaRepository { + + Optional findByCode(String code); + + Optional findByNameIgnoreCase(String name); + + List findByActiveTrue(); +} diff --git a/src/main/java/com/lincfusion/udyamlinc/master/Market.java b/src/main/java/com/lincfusion/udyamlinc/master/Market.java new file mode 100644 index 0000000..07a9609 --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/master/Market.java @@ -0,0 +1,46 @@ +package com.lincfusion.udyamlinc.master; + +import jakarta.persistence.*; +import lombok.*; + +/** A mandi. Enterprises are attributed to the reporting market in their district. */ +@Entity +@Table(name = "market", indexes = { + @Index(name = "ix_market_code", columnList = "code", unique = true), + @Index(name = "ix_market_district", columnList = "districtId"), +}) +@Getter +@Setter +@NoArgsConstructor +@AllArgsConstructor +@Builder +public class Market { + + @Id + @GeneratedValue(strategy = GenerationType.IDENTITY) + private Long id; + + @Column(nullable = false, unique = true, length = 48) + private String code; + + @Column(nullable = false, length = 96) + private String name; + + @Column(nullable = false) + private Long districtId; + + /** The market name as Agmarknet reports it, for the ingestion filter. */ + @Column(length = 96) + private String sourceName; + + /** Preferred reporting market for its district when several report. */ + private Boolean primaryMarket; + + private Boolean active; + + @PrePersist + void onCreate() { + if (active == null) active = Boolean.TRUE; + if (primaryMarket == null) primaryMarket = Boolean.FALSE; + } +} diff --git a/src/main/java/com/lincfusion/udyamlinc/master/MarketRepository.java b/src/main/java/com/lincfusion/udyamlinc/master/MarketRepository.java new file mode 100644 index 0000000..241552a --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/master/MarketRepository.java @@ -0,0 +1,15 @@ +package com.lincfusion.udyamlinc.master; + +import org.springframework.data.jpa.repository.JpaRepository; + +import java.util.List; +import java.util.Optional; + +public interface MarketRepository extends JpaRepository { + + Optional findByCode(String code); + + List findByDistrictId(Long districtId); + + Optional findFirstByDistrictIdAndPrimaryMarketTrue(Long districtId); +} diff --git a/src/main/java/com/lincfusion/udyamlinc/master/Role.java b/src/main/java/com/lincfusion/udyamlinc/master/Role.java new file mode 100644 index 0000000..ba706b3 --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/master/Role.java @@ -0,0 +1,41 @@ +package com.lincfusion.udyamlinc.master; + +import jakarta.persistence.*; +import lombok.*; + +/** Console RBAC. Permissions are a comma-separated set of MODULE:ACTION keys. */ +@Entity +@Table(name = "role", indexes = @Index(name = "ix_role_code", columnList = "code", unique = true)) +@Getter +@Setter +@NoArgsConstructor +@AllArgsConstructor +@Builder +public class Role { + + @Id + @GeneratedValue(strategy = GenerationType.IDENTITY) + private Long id; + + /** FIELD_OFFICER | SUPERVISOR | OWNER */ + @Column(nullable = false, unique = true, length = 32) + private String code; + + @Column(nullable = false, length = 64) + private String name; + + /** e.g. "PORTFOLIO:VIEW,ALERT:ACT,VISIT:LOG" */ + @Column(length = 600) + private String permissions; + + /** System roles cannot be deleted from the console. */ + private Boolean systemRole; + + private Boolean active; + + @PrePersist + void onCreate() { + if (active == null) active = Boolean.TRUE; + if (systemRole == null) systemRole = Boolean.FALSE; + } +} diff --git a/src/main/java/com/lincfusion/udyamlinc/master/RoleRepository.java b/src/main/java/com/lincfusion/udyamlinc/master/RoleRepository.java new file mode 100644 index 0000000..e217ba9 --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/master/RoleRepository.java @@ -0,0 +1,10 @@ +package com.lincfusion.udyamlinc.master; + +import org.springframework.data.jpa.repository.JpaRepository; + +import java.util.Optional; + +public interface RoleRepository extends JpaRepository { + + Optional findByCode(String code); +} diff --git a/src/main/java/com/lincfusion/udyamlinc/master/Sector.java b/src/main/java/com/lincfusion/udyamlinc/master/Sector.java new file mode 100644 index 0000000..d28c5ab --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/master/Sector.java @@ -0,0 +1,67 @@ +package com.lincfusion.udyamlinc.master; + +import jakarta.persistence.*; +import lombok.*; + +/** + * A sector risk pack — the weights that make the model trade-aware rather than + * generic. Feed indices matter for poultry; fodder and yield for dairy; buyer + * concentration for handicrafts. + * + *

Weights live here rather than in code so tuning a pack needs no deploy. + * They are basis points and must sum to 10000. + */ +@Entity +@Table(name = "sector", indexes = @Index(name = "ix_sector_code", columnList = "code", unique = true)) +@Getter +@Setter +@NoArgsConstructor +@AllArgsConstructor +@Builder +public class Sector { + + @Id + @GeneratedValue(strategy = GenerationType.IDENTITY) + private Long id; + + /** DAIRY | POULTRY | FOOD_PROC | HANDICRAFTS | RURAL_RETAIL */ + @Column(nullable = false, unique = true, length = 32) + private String code; + + @Column(nullable = false, length = 64) + private String label; + + @Column(length = 64) + private String labelHi; + + /** Short form for table columns, e.g. "Food proc." */ + @Column(length = 32) + private String shortLabel; + + private Integer wLiquidity; + + private Integer wBehavioural; + + private Integer wTrendVolatility; + + private Integer wObligationLoad; + + private Integer wExternalShock; + + private Boolean active; + + /** True when the five weights sum to 10000 basis points. */ + public boolean weightsBalanced() { + return nz(wLiquidity) + nz(wBehavioural) + nz(wTrendVolatility) + nz(wObligationLoad) + nz(wExternalShock) + == 10_000; + } + + private int nz(Integer v) { + return v == null ? 0 : v; + } + + @PrePersist + void onCreate() { + if (active == null) active = Boolean.TRUE; + } +} diff --git a/src/main/java/com/lincfusion/udyamlinc/master/SectorCommodity.java b/src/main/java/com/lincfusion/udyamlinc/master/SectorCommodity.java new file mode 100644 index 0000000..b93fcf1 --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/master/SectorCommodity.java @@ -0,0 +1,42 @@ +package com.lincfusion.udyamlinc.master; + +import jakarta.persistence.*; +import lombok.*; + +/** + * Which commodities a sector buys and which it sells. + * + *

The INPUT / OUTPUT distinction is not cosmetic — it is what makes + * {@code margin_squeeze} computable. Poultry buys maize and soya and sells eggs; + * a feed rise with a flat egg rate is a squeeze, and a feed rise matched by an + * egg rise is not. + */ +@Entity +@Table(name = "sector_commodity", indexes = { + @Index(name = "ix_sc_sector", columnList = "sectorId"), + @Index(name = "ix_sc_pair", columnList = "sectorId,commodityId", unique = true), +}) +@Getter +@Setter +@NoArgsConstructor +@AllArgsConstructor +@Builder +public class SectorCommodity { + + @Id + @GeneratedValue(strategy = GenerationType.IDENTITY) + private Long id; + + @Column(nullable = false) + private Long sectorId; + + @Column(nullable = false) + private Long commodityId; + + /** INPUT (a cost) | OUTPUT (a revenue). */ + @Column(nullable = false, length = 8) + private String role; + + /** Relative importance within its role, basis points. */ + private Integer weight; +} diff --git a/src/main/java/com/lincfusion/udyamlinc/master/SectorCommodityRepository.java b/src/main/java/com/lincfusion/udyamlinc/master/SectorCommodityRepository.java new file mode 100644 index 0000000..f17c68f --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/master/SectorCommodityRepository.java @@ -0,0 +1,12 @@ +package com.lincfusion.udyamlinc.master; + +import org.springframework.data.jpa.repository.JpaRepository; + +import java.util.List; + +public interface SectorCommodityRepository extends JpaRepository { + + List findBySectorId(Long sectorId); + + List findBySectorIdAndRole(Long sectorId, String role); +} diff --git a/src/main/java/com/lincfusion/udyamlinc/master/SectorRepository.java b/src/main/java/com/lincfusion/udyamlinc/master/SectorRepository.java new file mode 100644 index 0000000..0933d67 --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/master/SectorRepository.java @@ -0,0 +1,13 @@ +package com.lincfusion.udyamlinc.master; + +import org.springframework.data.jpa.repository.JpaRepository; + +import java.util.List; +import java.util.Optional; + +public interface SectorRepository extends JpaRepository { + + Optional findByCode(String code); + + List findByActiveTrue(); +} diff --git a/src/main/java/com/lincfusion/udyamlinc/officer/FieldVisit.java b/src/main/java/com/lincfusion/udyamlinc/officer/FieldVisit.java new file mode 100644 index 0000000..9fbed8c --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/officer/FieldVisit.java @@ -0,0 +1,86 @@ +package com.lincfusion.udyamlinc.officer; + +import jakarta.persistence.*; +import lombok.*; + +import java.time.LocalDate; +import java.time.LocalDateTime; + +/** + * A playbook intervention and its outcome. + * + *

The outcome is the feedback edge of the monitoring loop — it is what the + * next retraining cycle learns from. {@code officerOverride} records the cases + * where the officer disagreed with the model, which is the most valuable + * training signal in the table. + */ +@Entity +@Table(name = "field_visit", indexes = { + @Index(name = "ix_visit_visit_id", columnList = "visitId", unique = true), + @Index(name = "ix_visit_enterprise", columnList = "enterpriseId"), + @Index(name = "ix_visit_block", columnList = "block"), +}) +@Getter +@Setter +@NoArgsConstructor +@AllArgsConstructor +@Builder +public class FieldVisit { + + @Id + @GeneratedValue(strategy = GenerationType.IDENTITY) + private Long id; + + @Column(nullable = false, unique = true, length = 24) + private String visitId; + + @Column(nullable = false, length = 24) + private String enterpriseId; + + /** Denormalised for the console's block filter. */ + @Column(length = 128) + private String enterpriseName; + + @Column(length = 64) + private String block; + + /** Playbook code, e.g. PB-01. */ + @Column(length = 16) + private String actionCode; + + @Column(length = 200) + private String action; + + @Column(length = 200) + private String actionHi; + + /** The alert that prompted this visit, if any. */ + private Long alertId; + + /** 1 = act first. */ + private Integer priority; + + private LocalDate dueDate; + + /** PLANNED | DONE | OVERDUE */ + @Column(nullable = false, length = 16) + private String state; + + @Column(length = 600) + private String outcome; + + private Boolean officerOverride; + + @Column(length = 64) + private String officer; + + private LocalDateTime loggedAt; + + private LocalDateTime createdAt; + + @PrePersist + void onCreate() { + if (createdAt == null) createdAt = LocalDateTime.now(); + if (state == null) state = "PLANNED"; + } +} diff --git a/src/main/java/com/lincfusion/udyamlinc/officer/FieldVisitRepository.java b/src/main/java/com/lincfusion/udyamlinc/officer/FieldVisitRepository.java new file mode 100644 index 0000000..df582fc --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/officer/FieldVisitRepository.java @@ -0,0 +1,19 @@ +package com.lincfusion.udyamlinc.officer; + +import org.springframework.data.jpa.repository.JpaRepository; + +import java.util.List; +import java.util.Optional; + +public interface FieldVisitRepository extends JpaRepository { + + Optional findByVisitId(String visitId); + + List findByEnterpriseIdOrderByDueDateAsc(String enterpriseId); + + List findByBlockOrderByPriorityAscDueDateAsc(String block); + + List findAllByOrderByPriorityAscDueDateAsc(); + + long countByStateAndPriority(String state, Integer priority); +} diff --git a/src/main/java/com/lincfusion/udyamlinc/officer/OfficerController.java b/src/main/java/com/lincfusion/udyamlinc/officer/OfficerController.java new file mode 100644 index 0000000..1cf1da7 --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/officer/OfficerController.java @@ -0,0 +1,91 @@ +package com.lincfusion.udyamlinc.officer; + +import com.lincfusion.udyamlinc.common.ApiResponse; +import com.lincfusion.udyamlinc.dto.UdyamRequests.AlertActionRequest; +import com.lincfusion.udyamlinc.dto.UdyamRequests.ScheduleVisitRequest; +import com.lincfusion.udyamlinc.dto.UdyamRequests.VisitOutcomeRequest; +import com.lincfusion.udyamlinc.dto.UdyamResponses.*; +import com.lincfusion.udyamlinc.risk.RiskAlert; +import jakarta.validation.Valid; +import lombok.RequiredArgsConstructor; +import org.springframework.security.core.annotation.AuthenticationPrincipal; +import org.springframework.web.bind.annotation.*; + +import java.util.List; + +/** + * The field officer console API. Every route here is gated on SCOPE_OFFICER by + * {@code SecurityConfig}, so an enterprise token can never reach it. + */ +@RestController +@RequestMapping("/officer") +@RequiredArgsConstructor +public class OfficerController { + + private final OfficerService officerService; + + @GetMapping("/portfolio") + public ApiResponse portfolio( + @RequestParam(defaultValue = OfficerService.ALL_BLOCKS) String block, + @RequestParam(required = false) String month) { + return ApiResponse.ok(officerService.portfolio(block, month)); + } + + @GetMapping("/enterprises") + public ApiResponse> enterprises( + @RequestParam(defaultValue = OfficerService.ALL_BLOCKS) String block) { + return ApiResponse.ok(officerService.enterprises(block)); + } + + @GetMapping("/enterprises/{enterpriseId}") + public ApiResponse detail(@PathVariable String enterpriseId) { + return ApiResponse.ok(officerService.detail(enterpriseId)); + } + + @GetMapping("/alerts") + public ApiResponse> alerts( + @RequestParam(defaultValue = OfficerService.ALL_BLOCKS) String block, + @RequestParam(required = false) String state) { + return ApiResponse.ok(officerService.alerts(block, state)); + } + + @PutMapping("/alerts/{alertId}") + public ApiResponse disposition( + @PathVariable String alertId, + @Valid @RequestBody AlertActionRequest request, + @AuthenticationPrincipal String actor) { + return ApiResponse.ok("Alert updated", officerService.disposition(alertId, request.state(), request.note(), actor)); + } + + @GetMapping("/visits") + public ApiResponse> visits( + @RequestParam(defaultValue = OfficerService.ALL_BLOCKS) String block) { + return ApiResponse.ok(officerService.visits(block)); + } + + @PostMapping("/visits") + public ApiResponse schedule( + @Valid @RequestBody ScheduleVisitRequest request, + @AuthenticationPrincipal String officer) { + return ApiResponse.ok("Visit scheduled", officerService.schedule(request, officer)); + } + + @PutMapping("/visits/{visitId}/outcome") + public ApiResponse logOutcome( + @PathVariable String visitId, + @Valid @RequestBody VisitOutcomeRequest request) { + return ApiResponse.ok("Outcome logged", + officerService.logOutcome(visitId, request.outcome(), request.officerOverride())); + } + + @GetMapping("/playbook") + public ApiResponse> playbook() { + return ApiResponse.ok(PlaybookCatalog.asResponses()); + } + + @GetMapping("/sectors") + public ApiResponse> sectors( + @RequestParam(defaultValue = OfficerService.ALL_BLOCKS) String block) { + return ApiResponse.ok(officerService.sectorRollup(block)); + } +} diff --git a/src/main/java/com/lincfusion/udyamlinc/officer/OfficerService.java b/src/main/java/com/lincfusion/udyamlinc/officer/OfficerService.java new file mode 100644 index 0000000..dfd9400 --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/officer/OfficerService.java @@ -0,0 +1,253 @@ +package com.lincfusion.udyamlinc.officer; + +import com.lincfusion.udyamlinc.common.ResourceNotFoundException; +import com.lincfusion.udyamlinc.dto.UdyamRequests.ScheduleVisitRequest; +import com.lincfusion.udyamlinc.dto.UdyamResponses.EnterpriseDetailResponse; +import com.lincfusion.udyamlinc.dto.UdyamResponses.EnterpriseRow; +import com.lincfusion.udyamlinc.dto.UdyamResponses.PortfolioResponse; +import com.lincfusion.udyamlinc.dto.UdyamResponses.SectorRollupResponse; +import com.lincfusion.udyamlinc.enterprise.Enterprise; +import com.lincfusion.udyamlinc.enterprise.EnterpriseRepository; +import com.lincfusion.udyamlinc.enterprise.LedgerEntryRepository; +import com.lincfusion.udyamlinc.master.District; +import com.lincfusion.udyamlinc.master.DistrictRepository; +import com.lincfusion.udyamlinc.master.Sector; +import com.lincfusion.udyamlinc.master.SectorRepository; +import com.lincfusion.udyamlinc.risk.*; +import lombok.RequiredArgsConstructor; +import org.springframework.stereotype.Service; +import org.springframework.transaction.annotation.Transactional; + +import java.time.LocalDate; +import java.time.LocalDateTime; +import java.util.List; +import java.util.Map; +import java.util.function.Function; +import java.util.stream.Collectors; + +/** + * Read side of the field officer console, plus the two writes an officer makes: + * dispositioning an alert and scheduling or closing a visit. + * + *

Everything served here is read from stored scoring runs — this service does + * not score. Scoring belongs to the risk engine, which writes the rows this + * service reads back. + */ +@Service +@RequiredArgsConstructor +public class OfficerService { + + /** Sentinel used by the console when no block filter is applied. */ + public static final String ALL_BLOCKS = "ALL"; + + private final EnterpriseRepository enterpriseRepository; + private final LedgerEntryRepository ledgerRepository; + private final SectorRepository sectorRepository; + private final DistrictRepository districtRepository; + private final StressScoreRepository scoreRepository; + private final ReasonCodeRepository reasonRepository; + private final ForecastPointRepository pointRepository; + private final RiskAlertRepository alertRepository; + private final FieldVisitRepository visitRepository; + + public PortfolioResponse portfolio(String block, String month) { + List rows = enterprisesIn(block); + long red = countBand(rows, "RED"); + long watch = countBand(rows, "WATCH"); + long standard = rows.size() - red - watch; + + return new PortfolioResponse( + block, month, rows.size(), 0, red, 0, watch, 0, standard, 0, + minutesSinceLastScore(rows), toRows(rows)); + } + + public List enterprises(String block) { + return toRows(enterprisesIn(block)); + } + + public EnterpriseDetailResponse detail(String enterpriseId) { + Enterprise enterprise = requireEnterprise(enterpriseId); + StressScore score = scoreRepository.findFirstByEnterpriseIdOrderByComputedAtDesc(enterpriseId).orElse(null); + + return new EnterpriseDetailResponse( + enterprise, + sectorCode(enterprise), + districtName(enterprise), + score, + reasonRepository.findByEnterpriseIdOrderByRankAsc(enterpriseId), + pointRepository.findByEnterpriseIdOrderByHorizonAsc(enterpriseId), + alertRepository.findByEnterpriseIdOrderByRaisedAtDesc(enterpriseId), + visitRepository.findByEnterpriseIdOrderByDueDateAsc(enterpriseId), + ledgerRepository.findByEnterpriseIdOrderByEntryDateDesc(enterpriseId).stream().limit(60).toList()); + } + + public List alerts(String block, String state) { + List all = state == null || state.isBlank() || "ALL".equalsIgnoreCase(state) + ? alertRepository.findAllByOrderByScoreDesc() + : alertRepository.findByStateOrderByScoreDesc(state.toUpperCase()); + if (ALL_BLOCKS.equalsIgnoreCase(block)) return all; + + // One lookup for the whole page rather than a query per alert. + Map blockOf = enterpriseRepository.findAll().stream() + .collect(Collectors.toMap(Enterprise::getEnterpriseId, Enterprise::getBlock, (a, b) -> a)); + return all.stream().filter(a -> block.equalsIgnoreCase(blockOf.get(a.getEnterpriseId()))).toList(); + } + + /** Acknowledge / escalate / snooze / close. Every change records who and when. */ + @Transactional + public RiskAlert disposition(String alertId, String state, String note, String actor) { + RiskAlert alert = alertRepository.findByAlertId(alertId) + .orElseThrow(() -> new ResourceNotFoundException("No alert " + alertId)); + String next = state.toUpperCase(); + if (!List.of("OPEN", "ACKNOWLEDGED", "ESCALATED", "SNOOZED", "CLOSED").contains(next)) { + throw new IllegalArgumentException("Unknown alert state: " + state); + } + alert.setState(next); + alert.setDispositionNote(note); + alert.setDispositionedBy(actor); + alert.setDispositionedAt(LocalDateTime.now()); + return alertRepository.save(alert); + } + + public List visits(String block) { + return ALL_BLOCKS.equalsIgnoreCase(block) + ? visitRepository.findAllByOrderByPriorityAscDueDateAsc() + : visitRepository.findByBlockOrderByPriorityAscDueDateAsc(block); + } + + @Transactional + public FieldVisit schedule(ScheduleVisitRequest request, String officer) { + Enterprise enterprise = requireEnterprise(request.enterpriseId()); + PlaybookCatalog.Action action = PlaybookCatalog.byCode(request.actionCode()); + + FieldVisit visit = FieldVisit.builder() + .visitId("FV-" + (9000 + visitRepository.count())) + .enterpriseId(enterprise.getEnterpriseId()) + .enterpriseName(enterprise.getName()) + .block(enterprise.getBlock()) + .actionCode(action.code()) + .action(action.label()) + .actionHi(action.labelHi()) + .priority("RED".equals(enterprise.getBand()) ? 1 : 2) + .dueDate(request.dueDate() == null ? LocalDate.now().plusDays(3) : LocalDate.parse(request.dueDate())) + .state("PLANNED") + .officer(officer) + .build(); + return visitRepository.save(visit); + } + + /** Closing a visit is the feedback edge of the monitoring loop. */ + @Transactional + public FieldVisit logOutcome(String visitId, String outcome, Boolean override) { + FieldVisit visit = visitRepository.findByVisitId(visitId) + .orElseThrow(() -> new ResourceNotFoundException("No visit " + visitId)); + visit.setState("DONE"); + visit.setOutcome(outcome); + visit.setOfficerOverride(Boolean.TRUE.equals(override)); + visit.setLoggedAt(LocalDateTime.now()); + return visitRepository.save(visit); + } + + /** Band mix and average score for each sector pack. */ + public List sectorRollup(String block) { + List rows = enterprisesIn(block); + return sectorRepository.findByActiveTrue().stream() + .map(sector -> { + List mine = rows.stream() + .filter(e -> sector.getId().equals(e.getSectorId())) + .toList(); + long red = countBand(mine, "RED"); + long watch = countBand(mine, "WATCH"); + int avg = mine.isEmpty() ? 0 + : (int) Math.round(mine.stream().mapToInt(e -> nz(e.getScore())).average().orElse(0)); + return new SectorRollupResponse( + sector.getCode(), sector.getLabel(), mine.size(), red, watch, + mine.size() - red - watch, avg); + }) + .toList(); + } + + // ── helpers ───────────────────────────────────────────── + + private List enterprisesIn(String block) { + return block == null || block.isBlank() || ALL_BLOCKS.equalsIgnoreCase(block) + ? enterpriseRepository.findAllByOrderByScoreDesc() + : enterpriseRepository.findByBlockOrderByScoreDesc(block); + } + + /** + * Flatten enterprises for the console table, resolving sector and district + * codes once for the whole page rather than per row. + */ + private List toRows(List rows) { + Map sectors = sectorRepository.findAll().stream() + .collect(Collectors.toMap(Sector::getId, Function.identity())); + Map districts = districtRepository.findAll().stream() + .collect(Collectors.toMap(District::getId, Function.identity())); + + // One query for every score, then grouped — not one query per enterprise. + Map> history = scoreRepository.findAll().stream() + .collect(Collectors.groupingBy(StressScore::getEnterpriseId)); + + return rows.stream() + .map(e -> new EnterpriseRow( + e.getId(), + e.getEnterpriseId(), + e.getName(), + e.getNameHi(), + sectors.containsKey(e.getSectorId()) ? sectors.get(e.getSectorId()).getCode() : null, + e.getBlock(), + districts.containsKey(e.getDistrictId()) ? districts.get(e.getDistrictId()).getName() : null, + e.getOwnerCount(), + e.getVintageMonths(), + e.getLoanOutstanding(), + e.getMonthlyInstalment(), + e.getScore(), + e.getBand(), + trendOf(history.get(e.getEnterpriseId()), e.getScore()), + e.getLastScoredAt() == null ? null : e.getLastScoredAt().toString())) + .toList(); + } + + /** Last six scores, oldest first, padded with the current score when short. */ + private List trendOf(List scores, Integer current) { + if (scores == null || scores.isEmpty()) { + return current == null ? List.of() : List.of(current); + } + List tail = scores.stream() + .sorted((a, b) -> a.getComputedAt().compareTo(b.getComputedAt())) + .map(StressScore::getScore) + .toList(); + return tail.size() <= 6 ? tail : tail.subList(tail.size() - 6, tail.size()); + } + + private int minutesSinceLastScore(List rows) { + return rows.stream() + .map(Enterprise::getLastScoredAt) + .filter(java.util.Objects::nonNull) + .max(LocalDateTime::compareTo) + .map(t -> (int) java.time.Duration.between(t, LocalDateTime.now()).toMinutes()) + .orElse(0); + } + + private long countBand(List rows, String band) { + return rows.stream().filter(e -> band.equals(e.getBand())).count(); + } + + private String sectorCode(Enterprise e) { + return sectorRepository.findById(e.getSectorId()).map(Sector::getCode).orElse(null); + } + + private String districtName(Enterprise e) { + return districtRepository.findById(e.getDistrictId()).map(District::getName).orElse(null); + } + + private Enterprise requireEnterprise(String enterpriseId) { + return enterpriseRepository.findByEnterpriseId(enterpriseId) + .orElseThrow(() -> new ResourceNotFoundException("No enterprise " + enterpriseId)); + } + + private int nz(Integer v) { + return v == null ? 0 : v; + } +} diff --git a/src/main/java/com/lincfusion/udyamlinc/officer/PlaybookCatalog.java b/src/main/java/com/lincfusion/udyamlinc/officer/PlaybookCatalog.java new file mode 100644 index 0000000..81f3c60 --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/officer/PlaybookCatalog.java @@ -0,0 +1,59 @@ +package com.lincfusion.udyamlinc.officer; + +import com.lincfusion.udyamlinc.dto.UdyamResponses.PlaybookActionResponse; + +import java.util.List; + +/** + * The ranked intervention playbook. Fixed rather than table-driven: the set is + * small, every entry maps to a band, and the officer picks from what the model + * recommends instead of typing free text. + */ +public final class PlaybookCatalog { + + private PlaybookCatalog() { + } + + public record Action(String code, String label, String labelHi, String appliesTo, String detail) { + } + + public static final List ACTIONS = List.of( + new Action("PB-01", "Restructure instalment to the earning season", + "किस्त को कमाई के मौसम से मिलाएँ", "RED", + "Move the due date behind the sector cash peak; no principal change."), + new Action("PB-02", "Sanction a short working-capital top-up", + "थोड़ी कार्यशील पूँजी और दें", "RED", + "Bridge the projected P10 gap for two months against stock."), + new Action("PB-03", "Field visit — verify stock and receivables", + "मौके पर जाकर स्टॉक और उधारी जाँचें", "RED", + "Confirm the ledger against physical stock before any action."), + new Action("PB-04", "Input-cost counselling with the sector pack", + "लागत घटाने की सलाह दें", "WATCH", + "Bulk feed or raw purchase timing against the mandi price curve."), + new Action("PB-05", "Enrol for daily ledger reminders", + "रोज़ हिसाब दर्ज करने की याद दिलाएँ", "WATCH", + "Ledger silence is the strongest early behavioural signal."), + new Action("PB-06", "Diversify buyer concentration", + "एक ही खरीदार पर निर्भरता घटाएँ", "WATCH", + "Introduce to the block procurement group."), + new Action("PB-07", "Routine monitoring — no action", + "सिर्फ़ निगरानी, कोई कार्रवाई नहीं", "STANDARD", + "Re-score next cycle.")); + + public static Action byCode(String code) { + return ACTIONS.stream() + .filter(a -> a.code().equalsIgnoreCase(code)) + .findFirst() + .orElseThrow(() -> new IllegalArgumentException("Unknown playbook action: " + code)); + } + + public static List forBand(String band) { + return ACTIONS.stream().filter(a -> a.appliesTo().equalsIgnoreCase(band)).toList(); + } + + public static List asResponses() { + return ACTIONS.stream() + .map(a -> new PlaybookActionResponse(a.code(), a.label(), a.labelHi(), a.appliesTo(), a.detail())) + .toList(); + } +} diff --git a/src/main/java/com/lincfusion/udyamlinc/risk/ForecastController.java b/src/main/java/com/lincfusion/udyamlinc/risk/ForecastController.java new file mode 100644 index 0000000..1ce0a87 --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/risk/ForecastController.java @@ -0,0 +1,28 @@ +package com.lincfusion.udyamlinc.risk; + +import com.lincfusion.udyamlinc.common.ApiResponse; +import com.lincfusion.udyamlinc.dto.UdyamResponses.ForecastResponse; +import lombok.RequiredArgsConstructor; +import org.springframework.web.bind.annotation.*; + +import java.util.List; + +@RestController +@RequestMapping("/forecast") +@RequiredArgsConstructor +public class ForecastController { + + private final ForecastService forecastService; + private final ScoringRunRepository runRepository; + + @GetMapping("/{enterpriseId}") + public ApiResponse forecast(@PathVariable String enterpriseId) { + return ApiResponse.ok(forecastService.forecast(enterpriseId)); + } + + /** Recent scoring runs — the model prediction log, for supervisory review. */ + @GetMapping("/runs") + public ApiResponse> runs() { + return ApiResponse.ok(runRepository.findTop20ByOrderByRunAtDesc()); + } +} diff --git a/src/main/java/com/lincfusion/udyamlinc/risk/ForecastPoint.java b/src/main/java/com/lincfusion/udyamlinc/risk/ForecastPoint.java new file mode 100644 index 0000000..ce158bb --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/risk/ForecastPoint.java @@ -0,0 +1,58 @@ +package com.lincfusion.udyamlinc.risk; + +import jakarta.persistence.*; +import lombok.*; + +import java.math.BigDecimal; + +/** + * One month of the quantile forecast. Where {@code p10 < obligation}, the + * downside case fails to cover what is due and the enterprise escalates — that + * single comparison is the product's entire trigger. + */ +@Entity +@Table(name = "forecast_point", indexes = { + @Index(name = "ix_point_run", columnList = "runId"), + @Index(name = "ix_point_enterprise", columnList = "enterpriseId"), +}) +@Getter +@Setter +@NoArgsConstructor +@AllArgsConstructor +@Builder +public class ForecastPoint { + + @Id + @GeneratedValue(strategy = GenerationType.IDENTITY) + private Long id; + + @Column(nullable = false) + private Long runId; + + @Column(nullable = false, length = 24) + private String enterpriseId; + + /** 1…6 — months ahead of the run's as-of date. */ + @Column(nullable = false) + private Integer horizon; + + /** '2026-10' */ + @Column(nullable = false, length = 7) + private String month; + + @Column(precision = 14, scale = 2) + private BigDecimal p10; + + @Column(precision = 14, scale = 2) + private BigDecimal p50; + + @Column(precision = 14, scale = 2) + private BigDecimal p90; + + /** Instalment plus fixed costs falling due that month. */ + @Column(precision = 14, scale = 2) + private BigDecimal obligation; + + /** True when p10 falls short of the obligation. Stored so queries stay simple. */ + private Boolean breach; +} diff --git a/src/main/java/com/lincfusion/udyamlinc/risk/ForecastPointRepository.java b/src/main/java/com/lincfusion/udyamlinc/risk/ForecastPointRepository.java new file mode 100644 index 0000000..ef14be5 --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/risk/ForecastPointRepository.java @@ -0,0 +1,12 @@ +package com.lincfusion.udyamlinc.risk; + +import org.springframework.data.jpa.repository.JpaRepository; + +import java.util.List; + +public interface ForecastPointRepository extends JpaRepository { + + List findByRunIdOrderByHorizonAsc(Long runId); + + List findByEnterpriseIdOrderByHorizonAsc(String enterpriseId); +} diff --git a/src/main/java/com/lincfusion/udyamlinc/risk/ForecastService.java b/src/main/java/com/lincfusion/udyamlinc/risk/ForecastService.java new file mode 100644 index 0000000..e58e4a1 --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/risk/ForecastService.java @@ -0,0 +1,55 @@ +package com.lincfusion.udyamlinc.risk; + +import com.lincfusion.udyamlinc.common.ResourceNotFoundException; +import com.lincfusion.udyamlinc.dto.UdyamResponses.ForecastResponse; +import lombok.RequiredArgsConstructor; +import org.springframework.beans.factory.annotation.Value; +import org.springframework.stereotype.Service; + +import java.math.BigDecimal; +import java.util.List; + +/** + * Reads back the stored quantile path and derives the two facts the UI needs: + * the first month the downside case fails to cover obligations, and the size of + * that gap. + * + *

The path itself is produced by the risk engine or the model; this service + * neither fits nor predicts anything. + */ +@Service +@RequiredArgsConstructor +public class ForecastService { + + private final ScoringRunRepository runRepository; + private final ForecastPointRepository pointRepository; + + @Value("${udyam.forecast.horizon-months:6}") + private int horizonMonths; + + public ForecastResponse forecast(String enterpriseId) { + ScoringRun run = runRepository.findFirstByEnterpriseIdOrderByRunAtDesc(enterpriseId) + .orElseThrow(() -> new ResourceNotFoundException("No scoring run for " + enterpriseId)); + + List points = pointRepository.findByRunIdOrderByHorizonAsc(run.getId()); + ForecastPoint breach = firstBreach(points); + + return new ForecastResponse( + enterpriseId, + run.getModelVersion(), + run.getEngine(), + run.getHorizonMonths() == null ? horizonMonths : run.getHorizonMonths(), + breach == null ? null : breach.getMonth(), + breach == null ? BigDecimal.ZERO : breach.getObligation().subtract(breach.getP10()), + points); + } + + /** First month where P10 dips below what is due — the escalation trigger. */ + public ForecastPoint firstBreach(List points) { + return points.stream() + .filter(p -> p.getP10() != null && p.getObligation() != null) + .filter(p -> p.getP10().compareTo(p.getObligation()) < 0) + .findFirst() + .orElse(null); + } +} diff --git a/src/main/java/com/lincfusion/udyamlinc/risk/ReasonCode.java b/src/main/java/com/lincfusion/udyamlinc/risk/ReasonCode.java new file mode 100644 index 0000000..323fb89 --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/risk/ReasonCode.java @@ -0,0 +1,53 @@ +package com.lincfusion.udyamlinc.risk; + +import jakarta.persistence.*; +import lombok.*; + +/** + * A SHAP driver rendered as a sentence in both languages. Stored rather than + * generated at read time, so the exact wording an officer acted on is preserved + * even after the sentence templates change. + */ +@Entity +@Table(name = "reason_code", indexes = { + @Index(name = "ix_reason_run", columnList = "runId"), + @Index(name = "ix_reason_enterprise", columnList = "enterpriseId"), +}) +@Getter +@Setter +@NoArgsConstructor +@AllArgsConstructor +@Builder +public class ReasonCode { + + @Id + @GeneratedValue(strategy = GenerationType.IDENTITY) + private Long id; + + @Column(nullable = false) + private Long runId; + + @Column(nullable = false, length = 24) + private String enterpriseId; + + /** liquidity | behavioural | trendVolatility | obligationLoad | externalShock */ + @Column(nullable = false, length = 32) + private String driver; + + @Column(length = 64) + private String driverHi; + + /** SHAP contribution, 0–100. */ + @Column(nullable = false) + private Integer weight; + + /** 1 = strongest driver. Only the top three are surfaced to a user. */ + @Column(name = "driver_rank") + private Integer rank; + + @Column(length = 400) + private String sentenceEn; + + @Column(length = 400) + private String sentenceHi; +} diff --git a/src/main/java/com/lincfusion/udyamlinc/risk/ReasonCodeRepository.java b/src/main/java/com/lincfusion/udyamlinc/risk/ReasonCodeRepository.java new file mode 100644 index 0000000..95018ad --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/risk/ReasonCodeRepository.java @@ -0,0 +1,12 @@ +package com.lincfusion.udyamlinc.risk; + +import org.springframework.data.jpa.repository.JpaRepository; + +import java.util.List; + +public interface ReasonCodeRepository extends JpaRepository { + + List findByRunIdOrderByRankAsc(Long runId); + + List findByEnterpriseIdOrderByRankAsc(String enterpriseId); +} diff --git a/src/main/java/com/lincfusion/udyamlinc/risk/ReasonSentences.java b/src/main/java/com/lincfusion/udyamlinc/risk/ReasonSentences.java new file mode 100644 index 0000000..17eda7b --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/risk/ReasonSentences.java @@ -0,0 +1,120 @@ +package com.lincfusion.udyamlinc.risk; + +import com.lincfusion.udyamlinc.features.FeatureCatalog; + +import java.util.Locale; +import java.util.Map; + +/** + * Turns a driver into a sentence a person can act on, in both languages. + * + *

"externalShock: 26" is not a reason code. "Feed costs are 18 % above their + * eight-week average while the egg rate is flat" is. The officer console shows + * the English; the enterprise app shows the Hindi, and the Hindi is written for + * an owner rather than translated from analyst English. + * + *

Sentences are rendered once and stored on {@link ReasonCode}, so the wording + * an officer acted on survives a later change to these templates. + */ +public final class ReasonSentences { + + private ReasonSentences() {} + + private static final Map FAMILY_HI = Map.of( + RuleBasedScoringStrategy.LIQUIDITY, "नकदी", + RuleBasedScoringStrategy.BEHAVIOURAL, "भुगतान व्यवहार", + RuleBasedScoringStrategy.TREND_VOLATILITY, "बाज़ार में उतार-चढ़ाव", + RuleBasedScoringStrategy.OBLIGATION_LOAD, "किस्त का बोझ", + RuleBasedScoringStrategy.EXTERNAL_SHOCK, "बाहरी झटका"); + + private static final Map FAMILY_EN = Map.of( + RuleBasedScoringStrategy.LIQUIDITY, "Liquidity", + RuleBasedScoringStrategy.BEHAVIOURAL, "Repayment behaviour", + RuleBasedScoringStrategy.TREND_VOLATILITY, "Market volatility", + RuleBasedScoringStrategy.OBLIGATION_LOAD, "Obligation load", + RuleBasedScoringStrategy.EXTERNAL_SHOCK, "External shock"); + + public static String driverHi(String family) { + return FAMILY_HI.getOrDefault(family, family); + } + + public static String driverEn(String family) { + return FAMILY_EN.getOrDefault(family, family); + } + + /** English sentence for the officer console. */ + public static String en(ScoreOutcome.Driver driver) { + if (driver.onPrior() || driver.featureCode() == null || driver.value() == null) { + return FAMILY_EN.getOrDefault(driver.family(), driver.family()) + + " could not be measured — no recent data, so the sector's seasonal prior was used"; + } + double v = driver.value(); + return switch (driver.featureCode()) { + case FeatureCatalog.CASH_FLOW_CHANGE -> "Net cash flow is " + pct(v) + " against its trailing weekly rate"; + case FeatureCatalog.INVENTORY_DAYS -> "Stock on hand covers " + days(v) + " of sales — cash is sitting in goods"; + case FeatureCatalog.EMI_DELAY -> "Instalment is " + days(v) + " past due"; + case FeatureCatalog.DAYS_SINCE_LAST_ENTRY -> "No ledger entry for " + days(v); + case FeatureCatalog.PRICE_Z_SCORE -> "Sale price is " + fixed(Math.abs(v)) + " standard deviations " + + (v < 0 ? "below" : "above") + " its seasonal norm"; + case FeatureCatalog.DSCR_P10 -> "In the worst forecast month the downside case covers " + + fixed(v) + "× what is due" + (v < 1.0 ? " — short of the obligation" : ""); + case FeatureCatalog.EMI_TO_INFLOW -> "The instalment takes " + pct(v) + " of monthly inflow"; + case FeatureCatalog.MARGIN_SQUEEZE -> "Input costs have risen " + pct(v) + " more than output prices"; + case FeatureCatalog.RAINFALL_ANOMALY -> "Season rainfall is " + pct(v / 100) + " against the 30-year normal"; + case FeatureCatalog.HEAT_ANOMALY -> "Weekly maximum temperature is " + fixed(v) + " °C above normal"; + default -> FAMILY_EN.getOrDefault(driver.family(), driver.family()) + " is elevated"; + }; + } + + /** Hindi sentence for the enterprise app — written for an owner, not translated. */ + public static String hi(ScoreOutcome.Driver driver) { + if (driver.onPrior() || driver.featureCode() == null || driver.value() == null) { + return FAMILY_HI.getOrDefault(driver.family(), driver.family()) + + " का पता नहीं चल पाया — नया आंकड़ा नहीं मिला"; + } + double v = driver.value(); + return switch (driver.featureCode()) { + case FeatureCatalog.CASH_FLOW_CHANGE -> v < 0 + ? "पिछले हफ़्तों के मुक़ाबले आमदनी " + pctHi(Math.abs(v)) + " कम है" + : "पिछले हफ़्तों के मुक़ाबले आमदनी " + pctHi(v) + " ज़्यादा है"; + case FeatureCatalog.INVENTORY_DAYS -> "इतना स्टॉक पड़ा है कि " + daysHi(v) + " चल जाए — पैसा माल में फँसा है"; + case FeatureCatalog.EMI_DELAY -> "किस्त " + daysHi(v) + " से बकाया है"; + case FeatureCatalog.DAYS_SINCE_LAST_ENTRY -> daysHi(v) + " से कोई हिसाब दर्ज नहीं हुआ"; + case FeatureCatalog.PRICE_Z_SCORE -> v < 0 + ? "आपके माल का भाव इस मौसम के औसत से नीचे है" + : "आपके माल का भाव इस मौसम के औसत से ऊपर है"; + case FeatureCatalog.DSCR_P10 -> v < 1.0 + ? "आने वाले महीनों में किस्त चुकाने लायक पैसा कम पड़ सकता है" + : "फ़िलहाल किस्त चुकाने लायक पैसा रहने का अनुमान है"; + case FeatureCatalog.EMI_TO_INFLOW -> "महीने की आमदनी का " + pctHi(v) + " किस्त में चला जाता है"; + case FeatureCatalog.MARGIN_SQUEEZE -> "लागत बढ़ी है पर बिक्री का भाव उतना नहीं बढ़ा"; + case FeatureCatalog.RAINFALL_ANOMALY -> v < 0 + ? "इस मौसम में बारिश सामान्य से कम रही है" + : "इस मौसम में बारिश सामान्य से ज़्यादा रही है"; + case FeatureCatalog.HEAT_ANOMALY -> "गर्मी सामान्य से " + fixed(v) + " डिग्री ज़्यादा चल रही है"; + default -> FAMILY_HI.getOrDefault(driver.family(), driver.family()) + " का असर दिख रहा है"; + }; + } + + // ── formatting ────────────────────────────────────────── + + private static String pct(double ratio) { + return String.format(Locale.ENGLISH, "%.0f%%", ratio * 100); + } + + private static String pctHi(double ratio) { + return String.format(Locale.ENGLISH, "%.0f%%", ratio * 100); + } + + private static String days(double value) { + return String.format(Locale.ENGLISH, "%.0f days", value); + } + + private static String daysHi(double value) { + return String.format(Locale.ENGLISH, "%.0f दिन", value); + } + + private static String fixed(double value) { + return String.format(Locale.ENGLISH, "%.2f", value); + } +} diff --git a/src/main/java/com/lincfusion/udyamlinc/risk/RiskAlert.java b/src/main/java/com/lincfusion/udyamlinc/risk/RiskAlert.java new file mode 100644 index 0000000..a6a6900 --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/risk/RiskAlert.java @@ -0,0 +1,95 @@ +package com.lincfusion.udyamlinc.risk; + +import jakarta.persistence.*; +import lombok.*; + +import java.math.BigDecimal; +import java.time.LocalDateTime; + +/** + * An escalation, raised either by the model or by a deterministic trip-wire. + * + *

Both {@code ruleId} and {@code featureSnapshotId} are recorded so any alert + * can be reproduced exactly: the rule that fired, and the inputs it fired on. A + * trip-wire alert therefore stands on its own evidence even after the model has + * been retrained. + */ +@Entity +@Table(name = "risk_alert", indexes = { + @Index(name = "ix_alert_alert_id", columnList = "alertId", unique = true), + @Index(name = "ix_alert_enterprise", columnList = "enterpriseId"), + @Index(name = "ix_alert_state", columnList = "state"), +}) +@Getter +@Setter +@NoArgsConstructor +@AllArgsConstructor +@Builder +public class RiskAlert { + + @Id + @GeneratedValue(strategy = GenerationType.IDENTITY) + private Long id; + + @Column(nullable = false, unique = true, length = 24) + private String alertId; + + @Column(nullable = false, length = 24) + private String enterpriseId; + + private Long runId; + + @Column(nullable = false, length = 16) + private String band; + + private Integer score; + + /** MODEL | TRIPWIRE */ + @Column(nullable = false, length = 16) + private String origin; + + /** Set when origin=TRIPWIRE — the rule that fired. */ + private Long ruleId; + + /** The wide feature row the decision was made on. */ + private Long featureSnapshotId; + + @Column(length = 200) + private String title; + + @Column(length = 200) + private String titleHi; + + @Column(length = 400) + private String reasonEn; + + @Column(length = 400) + private String reasonHi; + + /** Projected gap at P10 in the breach month. */ + @Column(precision = 14, scale = 2) + private BigDecimal shortfall; + + @Column(length = 7) + private String breachMonth; + + /** OPEN | ACKNOWLEDGED | ESCALATED | SNOOZED | CLOSED */ + @Column(nullable = false, length = 16) + private String state; + + @Column(length = 64) + private String dispositionedBy; + + @Column(length = 400) + private String dispositionNote; + + private LocalDateTime dispositionedAt; + + private LocalDateTime raisedAt; + + @PrePersist + void onCreate() { + if (raisedAt == null) raisedAt = LocalDateTime.now(); + if (state == null) state = "OPEN"; + } +} diff --git a/src/main/java/com/lincfusion/udyamlinc/risk/RiskAlertRepository.java b/src/main/java/com/lincfusion/udyamlinc/risk/RiskAlertRepository.java new file mode 100644 index 0000000..0a8ae05 --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/risk/RiskAlertRepository.java @@ -0,0 +1,19 @@ +package com.lincfusion.udyamlinc.risk; + +import org.springframework.data.jpa.repository.JpaRepository; + +import java.util.List; +import java.util.Optional; + +public interface RiskAlertRepository extends JpaRepository { + + Optional findByAlertId(String alertId); + + List findByEnterpriseIdOrderByRaisedAtDesc(String enterpriseId); + + List findAllByOrderByScoreDesc(); + + List findByStateOrderByScoreDesc(String state); + + long countByState(String state); +} diff --git a/src/main/java/com/lincfusion/udyamlinc/risk/RiskEngineController.java b/src/main/java/com/lincfusion/udyamlinc/risk/RiskEngineController.java new file mode 100644 index 0000000..e863ad7 --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/risk/RiskEngineController.java @@ -0,0 +1,63 @@ +package com.lincfusion.udyamlinc.risk; + +import com.lincfusion.udyamlinc.common.ApiResponse; +import com.lincfusion.udyamlinc.rules.RuleOutcome; +import lombok.RequiredArgsConstructor; +import org.springframework.data.domain.PageRequest; +import org.springframework.format.annotation.DateTimeFormat; +import org.springframework.web.bind.annotation.*; + +import java.time.LocalDate; +import java.util.List; + +/** + * Operating the engine: run it, and see why a rule did or did not fire. + * + *

Under {@code /admin}, so reads are open to any officer and triggering a run + * is supervisor-only under the existing rule. + */ +@RestController +@RequestMapping("/admin/scoring") +@RequiredArgsConstructor +public class RiskEngineController { + + private final RiskEngineService riskEngine; + private final ScoringRunRepository runRepository; + + @GetMapping("/runs") + public ApiResponse> runs(@RequestParam(defaultValue = "30") int limit) { + return ApiResponse.ok(runRepository.findAll( + PageRequest.of(0, Math.min(limit, 200), + org.springframework.data.domain.Sort.by("runAt").descending())) + .getContent()); + } + + /** + * Every rule's outcome for one enterprise, without writing anything. A rule + * that is not firing is usually starved of fresh data rather than badly + * thresholded, and this is what shows which. + */ + @GetMapping("/explain/{enterpriseId}") + public ApiResponse> explain( + @PathVariable String enterpriseId, + @RequestParam(required = false) @DateTimeFormat(iso = DateTimeFormat.ISO.DATE) LocalDate date) { + return ApiResponse.ok(riskEngine.explain(enterpriseId, date == null ? LocalDate.now() : date)); + } + + // ── triggers ──────────────────────────────────────────── + + @PostMapping("/run") + public ApiResponse runAll( + @RequestParam(required = false) @DateTimeFormat(iso = DateTimeFormat.ISO.DATE) LocalDate date) { + return ApiResponse.ok("Scoring run complete", + riskEngine.scoreAll(date == null ? LocalDate.now() : date)); + } + + @PostMapping("/run/{enterpriseId}") + public ApiResponse runOne( + @PathVariable String enterpriseId, + @RequestParam(required = false) @DateTimeFormat(iso = DateTimeFormat.ISO.DATE) LocalDate date) { + return ApiResponse.ok("Rescored", + riskEngine.scoreOne(enterpriseId, date == null ? LocalDate.now() : date)); + } +} diff --git a/src/main/java/com/lincfusion/udyamlinc/risk/RiskEngineService.java b/src/main/java/com/lincfusion/udyamlinc/risk/RiskEngineService.java new file mode 100644 index 0000000..f5b4333 --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/risk/RiskEngineService.java @@ -0,0 +1,311 @@ +package com.lincfusion.udyamlinc.risk; + +import com.lincfusion.udyamlinc.common.ResourceNotFoundException; +import com.lincfusion.udyamlinc.enterprise.Enterprise; +import com.lincfusion.udyamlinc.enterprise.EnterpriseRepository; +import com.lincfusion.udyamlinc.features.*; +import com.lincfusion.udyamlinc.master.Sector; +import com.lincfusion.udyamlinc.master.SectorRepository; +import com.lincfusion.udyamlinc.rules.RuleEvaluator; +import com.lincfusion.udyamlinc.rules.RuleOutcome; +import lombok.extern.slf4j.Slf4j; +import org.springframework.beans.factory.annotation.Value; +import org.springframework.stereotype.Service; + +import java.time.LocalDate; +import java.time.LocalDateTime; +import java.time.format.DateTimeFormatter; +import java.util.*; + +/** + * One scoring pass: features in, score and alerts out. + * + *

The order matters. Features are computed first so the run scores what it + * just derived rather than yesterday's snapshot. The engine then produces a + * score, and the trip-wires run separately against the same feature rows — + * never against the score. That independence is the point: a trip-wire alert + * stays valid and explainable after the model has been retrained, because it + * never depended on the model in the first place. + * + *

A fired trip-wire may escalate the band but never rewrites the score. The + * two answer different questions — "how stressed is this enterprise" and "has a + * specific, agreed line been crossed" — and collapsing them would lose the + * second. + */ +@Service +@Slf4j +public class RiskEngineService { + + private static final DateTimeFormatter ALERT_STAMP = DateTimeFormatter.ofPattern("MMdd"); + + private final EnterpriseRepository enterpriseRepository; + private final SectorRepository sectorRepository; + private final FeatureEngineeringService featureService; + private final FeatureValueRepository featureValueRepository; + private final RuleEvaluator ruleEvaluator; + private final ScoringRunRepository runRepository; + private final StressScoreRepository scoreRepository; + private final ReasonCodeRepository reasonRepository; + private final RiskAlertRepository alertRepository; + private final ScoringStrategy strategy; + + @Value("${udyam.forecast.horizon-months:6}") + private int horizon; + + public RiskEngineService(EnterpriseRepository enterpriseRepository, + SectorRepository sectorRepository, + FeatureEngineeringService featureService, + FeatureValueRepository featureValueRepository, + RuleEvaluator ruleEvaluator, + ScoringRunRepository runRepository, + StressScoreRepository scoreRepository, + ReasonCodeRepository reasonRepository, + RiskAlertRepository alertRepository, + List strategies, + @Value("${udyam.risk.engine:RULE}") String engineName) { + + this.enterpriseRepository = enterpriseRepository; + this.sectorRepository = sectorRepository; + this.featureService = featureService; + this.featureValueRepository = featureValueRepository; + this.ruleEvaluator = ruleEvaluator; + this.runRepository = runRepository; + this.scoreRepository = scoreRepository; + this.reasonRepository = reasonRepository; + this.alertRepository = alertRepository; + this.strategy = strategies.stream() + .filter(s -> s.engine().equalsIgnoreCase(engineName)) + .findFirst() + .orElseThrow(() -> new IllegalStateException( + "No ScoringStrategy for udyam.risk.engine=" + engineName + + "; available: " + strategies.stream().map(ScoringStrategy::engine).toList())); + log.info("Risk engine bound to {} ({})", this.strategy.engine(), this.strategy.modelVersion()); + } + + // ── batch ─────────────────────────────────────────────── + + /** The whole book. Consent-revoked enterprises are not scored at all. */ + public Result scoreAll(LocalDate asOfDate) { + long started = System.currentTimeMillis(); + ScoringRun run = runRepository.save(ScoringRun.builder() + .engine(strategy.engine()) + .modelVersion(strategy.modelVersion()) + .source("SERVER") + .asOfDate(asOfDate) + .horizonMonths(horizon) + .status("RUNNING") + .build()); + + int scored = 0, skipped = 0, alerts = 0, tripwires = 0; + for (Enterprise enterprise : enterpriseRepository.findAll()) { + if (Boolean.FALSE.equals(enterprise.getConsentActive())) { + skipped++; + continue; + } + Outcome outcome = scoreOne(enterprise, run, asOfDate); + scored++; + alerts += outcome.alertsRaised(); + tripwires += outcome.firedRules().size(); + } + + run.setEnterpriseCount(scored); + run.setStatus("COMPLETED"); + run.setDurationMs(System.currentTimeMillis() - started); + run.setNote("%d scored · %d skipped for consent · %d trip-wires fired · %d alerts raised" + .formatted(scored, skipped, tripwires, alerts)); + runRepository.save(run); + + log.info("Scoring run {} — {}", run.getId(), run.getNote()); + return new Result(run.getId(), scored, skipped, tripwires, alerts, run.getNote()); + } + + /** One enterprise, on its own run, for a rescore after a visit or a data fix. */ + public Outcome scoreOne(String enterpriseId, LocalDate asOfDate) { + Enterprise enterprise = enterpriseRepository.findByEnterpriseId(enterpriseId) + .orElseThrow(() -> new ResourceNotFoundException("No enterprise " + enterpriseId)); + + ScoringRun run = runRepository.save(ScoringRun.builder() + .enterpriseId(enterpriseId) + .engine(strategy.engine()) + .modelVersion(strategy.modelVersion()) + .source("SERVER") + .asOfDate(asOfDate) + .horizonMonths(horizon) + .enterpriseCount(1) + .status("COMPLETED") + .build()); + + return scoreOne(enterprise, run, asOfDate); + } + + // ── the pass ──────────────────────────────────────────── + + private Outcome scoreOne(Enterprise enterprise, ScoringRun run, LocalDate asOfDate) { + LocalDateTime observedAsOf = LocalDateTime.now(); + + FeatureSnapshot snapshot = featureService.computeFor(enterprise, asOfDate, observedAsOf, run.getId()); + Map features = new HashMap<>(); + featureValueRepository.findByEnterpriseIdAndValidForDateOrderByFeatureCode( + enterprise.getEnterpriseId(), asOfDate) + .forEach(value -> features.put(value.getFeatureCode(), value)); + + Sector sector = sectorRepository.findById(enterprise.getSectorId()).orElse(null); + String sectorCode = sector == null ? null : sector.getCode(); + + ScoreOutcome scored = strategy.score(enterprise, sector, features); + + // Trip-wires read the same features, never the score. + List outcomes = ruleEvaluator.evaluateAll(sectorCode, features); + List fired = outcomes.stream().filter(RuleOutcome::fired).toList(); + String band = escalate(scored.band(), fired); + + persistScore(enterprise, run, snapshot, scored, band); + persistReasons(enterprise, run, scored); + int raised = persistAlerts(enterprise, run, snapshot, scored, band, fired, asOfDate); + + enterprise.setScore(scored.score()); + enterprise.setBand(band); + enterprise.setLastScoredAt(observedAsOf); + enterpriseRepository.save(enterprise); + + return new Outcome(enterprise.getEnterpriseId(), scored.score(), band, scored.confidence(), + snapshot.getId(), scored, outcomes, fired, raised); + } + + /** + * A RED trip-wire escalates whatever the score said; a WATCH one only lifts a + * STANDARD. The score is left untouched — the band is a decision, the score is + * a measurement, and an audit needs to see both. + */ + private String escalate(String band, List fired) { + boolean red = fired.stream().anyMatch(o -> "RED".equalsIgnoreCase(o.severity())); + if (red) return "RED"; + boolean watch = fired.stream().anyMatch(o -> "WATCH".equalsIgnoreCase(o.severity())); + if (watch && "STANDARD".equals(band)) return "WATCH"; + return band; + } + + private void persistScore(Enterprise enterprise, ScoringRun run, FeatureSnapshot snapshot, + ScoreOutcome scored, String band) { + scoreRepository.save(StressScore.builder() + .runId(run.getId()) + .enterpriseId(enterprise.getEnterpriseId()) + .score(scored.score()) + .band(band) + .engine(strategy.engine()) + .confidence(scored.confidence()) + .liquidity(scored.subScores().get(RuleBasedScoringStrategy.LIQUIDITY)) + .behavioural(scored.subScores().get(RuleBasedScoringStrategy.BEHAVIOURAL)) + .trendVolatility(scored.subScores().get(RuleBasedScoringStrategy.TREND_VOLATILITY)) + .obligationLoad(scored.subScores().get(RuleBasedScoringStrategy.OBLIGATION_LOAD)) + .externalShock(scored.subScores().get(RuleBasedScoringStrategy.EXTERNAL_SHOCK)) + .featureSnapshotId(snapshot.getId()) + .build()); + } + + /** Top three drivers, rendered once and stored, so the wording an officer acted on is preserved. */ + private void persistReasons(Enterprise enterprise, ScoringRun run, ScoreOutcome scored) { + int rank = 1; + List reasons = new ArrayList<>(); + for (ScoreOutcome.Driver driver : scored.drivers()) { + if (rank > 3) break; + reasons.add(ReasonCode.builder() + .runId(run.getId()) + .enterpriseId(enterprise.getEnterpriseId()) + .driver(driver.family()) + .driverHi(ReasonSentences.driverHi(driver.family())) + .weight(driver.weight()) + .rank(rank++) + .sentenceEn(ReasonSentences.en(driver)) + .sentenceHi(ReasonSentences.hi(driver)) + .build()); + } + reasonRepository.saveAll(reasons); + } + + /** + * One alert per fired rule per enterprise per day, plus a model-origin alert + * when the score alone reaches RED without any rule firing — a high score must + * never sit silent. + * + *

Both carry the rule id and the snapshot id, so any alert can be + * reproduced from the inputs it was raised on. + */ + private int persistAlerts(Enterprise enterprise, ScoringRun run, FeatureSnapshot snapshot, + ScoreOutcome scored, String band, List fired, LocalDate asOfDate) { + int raised = 0; + String stamp = asOfDate.format(ALERT_STAMP); + String shortId = enterprise.getEnterpriseId().replace("MSME-", ""); + + for (RuleOutcome outcome : fired) { + String alertId = "AL-" + shortId + "-" + outcome.ruleCode().replace("-", "") + "-" + stamp; + if (alertRepository.findByAlertId(alertId).isPresent()) continue; + + alertRepository.save(RiskAlert.builder() + .alertId(alertId) + .enterpriseId(enterprise.getEnterpriseId()) + .runId(run.getId()) + .band(outcome.severity()) + .score(scored.score()) + .origin("TRIPWIRE") + .ruleId(outcome.ruleId()) + .featureSnapshotId(snapshot.getId()) + .title(outcome.ruleCode() + " · " + outcome.detail()) + .reasonEn(scored.drivers().isEmpty() ? null : ReasonSentences.en(scored.drivers().get(0))) + .reasonHi(scored.drivers().isEmpty() ? null : ReasonSentences.hi(scored.drivers().get(0))) + .state("OPEN") + .build()); + raised++; + } + + if ("RED".equals(band) && fired.isEmpty()) { + String alertId = "AL-" + shortId + "-SCORE-" + stamp; + if (alertRepository.findByAlertId(alertId).isEmpty()) { + alertRepository.save(RiskAlert.builder() + .alertId(alertId) + .enterpriseId(enterprise.getEnterpriseId()) + .runId(run.getId()) + .band(band) + .score(scored.score()) + .origin("MODEL") + .featureSnapshotId(snapshot.getId()) + .title("Composite stress score " + scored.score()) + .reasonEn(scored.drivers().isEmpty() ? null : ReasonSentences.en(scored.drivers().get(0))) + .reasonHi(scored.drivers().isEmpty() ? null : ReasonSentences.hi(scored.drivers().get(0))) + .state("OPEN") + .build()); + raised++; + } + } + return raised; + } + + // ── dry run ───────────────────────────────────────────── + + /** + * Evaluate the rules without writing anything. This is how a supervisor sees + * why a rule is not firing — usually a stale or missing feature rather than a + * threshold that is wrong. + */ + public List explain(String enterpriseId, LocalDate asOfDate) { + Enterprise enterprise = enterpriseRepository.findByEnterpriseId(enterpriseId) + .orElseThrow(() -> new ResourceNotFoundException("No enterprise " + enterpriseId)); + + Map features = new HashMap<>(); + featureValueRepository.findByEnterpriseIdAndValidForDateOrderByFeatureCode(enterpriseId, asOfDate) + .forEach(value -> features.put(value.getFeatureCode(), value)); + + String sectorCode = sectorRepository.findById(enterprise.getSectorId()) + .map(Sector::getCode).orElse(null); + return ruleEvaluator.evaluateAll(sectorCode, features); + } + + // ── results ───────────────────────────────────────────── + + public record Outcome(String enterpriseId, int score, String band, int confidence, + Long featureSnapshotId, ScoreOutcome scored, + List ruleOutcomes, List firedRules, int alertsRaised) {} + + public record Result(Long runId, int scored, int skippedForConsent, + int tripWiresFired, int alertsRaised, String message) {} +} diff --git a/src/main/java/com/lincfusion/udyamlinc/risk/RuleBasedScoringStrategy.java b/src/main/java/com/lincfusion/udyamlinc/risk/RuleBasedScoringStrategy.java new file mode 100644 index 0000000..d92d284 --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/risk/RuleBasedScoringStrategy.java @@ -0,0 +1,214 @@ +package com.lincfusion.udyamlinc.risk; + +import com.lincfusion.udyamlinc.enterprise.Enterprise; +import com.lincfusion.udyamlinc.features.FeatureCatalog; +import com.lincfusion.udyamlinc.features.FeatureValue; +import com.lincfusion.udyamlinc.master.Sector; +import lombok.extern.slf4j.Slf4j; +import org.springframework.beans.factory.annotation.Value; +import org.springframework.stereotype.Component; + +import java.util.*; + +/** + * The engine that ships first: five family sub-scores, combined with the sector's + * own weights. + * + *

Each family maps its features onto a 0–100 stress scale where higher is + * worse, then the families are combined in basis points from the {@code sector} + * row. Tuning a pack is a database edit. The mapping constants below are the + * honest part to argue with — they are calibrated to the pilot's ranges, not + * learned, and that is exactly what the LightGBM heads replace. + * + *

A family with no usable feature falls back to the sector prior rather than + * scoring zero. Zero would read as "this enterprise is safe on this dimension", + * when the truth is that we do not know — and the confidence drop is what makes + * the difference visible. + */ +@Component +@Slf4j +public class RuleBasedScoringStrategy implements ScoringStrategy { + + public static final String LIQUIDITY = "liquidity"; + public static final String BEHAVIOURAL = "behavioural"; + public static final String TREND_VOLATILITY = "trendVolatility"; + public static final String OBLIGATION_LOAD = "obligationLoad"; + public static final String EXTERNAL_SHOCK = "externalShock"; + + @Value("${udyam.risk.model-version:rule-v1.0.0}") + private String modelVersion; + + @Value("${udyam.risk.red-threshold:65}") + private int redThreshold; + + @Value("${udyam.risk.watch-threshold:35}") + private int watchThreshold; + + /** Where a family lands when it has nothing to go on. Neutral, not innocent. */ + @Value("${udyam.risk.family-prior:40}") + private int familyPrior; + + /** A rule engine reports low confidence by design; this is its ceiling. */ + @Value("${udyam.risk.rule-confidence-ceiling:70}") + private int confidenceCeiling; + + @Override + public String engine() { + return "RULE"; + } + + @Override + public String modelVersion() { + return modelVersion; + } + + @Override + public ScoreOutcome score(Enterprise enterprise, Sector sector, Map features) { + Map families = new LinkedHashMap<>(); + families.put(LIQUIDITY, liquidity(features)); + families.put(BEHAVIOURAL, behavioural(features)); + families.put(TREND_VOLATILITY, trendVolatility(features)); + families.put(OBLIGATION_LOAD, obligationLoad(features)); + families.put(EXTERNAL_SHOCK, externalShock(features)); + + Map weights = weights(sector); + int totalWeight = weights.values().stream().mapToInt(Integer::intValue).sum(); + if (totalWeight == 0) totalWeight = 10_000; + + long weighted = 0; + int onPrior = 0; + List drivers = new ArrayList<>(); + + for (Map.Entry entry : families.entrySet()) { + Family family = entry.getValue(); + int weight = weights.getOrDefault(entry.getKey(), 0); + weighted += (long) family.value() * weight; + if (family.onPrior()) onPrior++; + // Contribution as a 0–100 share of the composite, which is what a + // driver bar in the console is showing. + int contribution = (int) Math.round(family.value() * weight / (double) totalWeight); + drivers.add(new ScoreOutcome.Driver(entry.getKey(), contribution, + family.featureCode(), family.featureValue(), family.onPrior())); + } + + int score = clamp((int) Math.round(weighted / (double) totalWeight), 0, 100); + drivers.sort(Comparator.comparingInt(ScoreOutcome.Driver::weight).reversed()); + + Map subScores = new LinkedHashMap<>(); + families.forEach((name, family) -> subScores.put(name, family.value())); + + return new ScoreOutcome(score, band(score), confidence(onPrior), subScores, drivers, onPrior); + } + + // ── families ──────────────────────────────────────────── + + /** Money coming in against money going out, and cash sitting in unsold stock. */ + private Family liquidity(Map features) { + // A 40 % fall against the trailing rate is full stress on this axis. + Signal cash = signal(features, FeatureCatalog.CASH_FLOW_CHANGE, v -> -v * 250); + // Beyond about 55 days of stock, working capital is stuck in goods. + Signal inventory = signal(features, FeatureCatalog.INVENTORY_DAYS, v -> (v - 15) * 2.5); + return worst(cash, inventory); + } + + /** Repayment conduct, and the silence that usually precedes it. */ + private Family behavioural(Map features) { + // 70 days past due is full stress. + Signal dpd = signal(features, FeatureCatalog.EMI_DELAY, v -> v * 1.43); + // A ledger quiet for a month is full stress; a week is nothing. + Signal silence = signal(features, FeatureCatalog.DAYS_SINCE_LAST_ENTRY, v -> (v - 7) * 4); + return worst(dpd, silence); + } + + /** How far out of its own seasonal norm the enterprise's market has moved. */ + private Family trendVolatility(Map features) { + // Three standard deviations either way is full stress. + return worst(signal(features, FeatureCatalog.PRICE_Z_SCORE, v -> Math.abs(v) * 33)); + } + + /** Whether what is due can be covered — the product's central question. */ + private Family obligationLoad(Map features) { + // DSCR of 1.5 is comfortable, 0.5 is full stress. + Signal dscr = signal(features, FeatureCatalog.DSCR_P10, v -> (1.5 - v) * 100); + // An instalment past 55 % of monthly inflow is full stress. + Signal share = signal(features, FeatureCatalog.EMI_TO_INFLOW, v -> (v - 0.15) * 250); + return worst(dscr, share); + } + + /** What the weather and the mandi did, none of which the enterprise controls. */ + private Family externalShock(Map features) { + // A third of margin eaten by input costs is full stress. + Signal squeeze = signal(features, FeatureCatalog.MARGIN_SQUEEZE, v -> v * 300); + // A 60 % rainfall deficit is full stress; surplus does not score. + Signal rain = signal(features, FeatureCatalog.RAINFALL_ANOMALY, v -> -v * 1.67); + // Three degrees above the norm is full stress. + Signal heat = signal(features, FeatureCatalog.HEAT_ANOMALY, v -> v * 33); + return worst(squeeze, rain, heat); + } + + // ── mechanics ─────────────────────────────────────────── + + /** + * The family takes its worst signal rather than their mean. Averaging lets a + * calm dimension mask an acute one, and it is the acute one an officer needs + * to be sent to. + */ + private Family worst(Signal... signals) { + Signal chosen = null; + for (Signal signal : signals) { + if (signal == null) continue; + if (chosen == null || signal.stress() > chosen.stress()) chosen = signal; + } + return chosen == null + ? new Family(clamp(familyPrior, 0, 100), null, null, true) + : new Family(chosen.stress(), chosen.code(), chosen.raw(), false); + } + + private Signal signal(Map features, String code, java.util.function.DoubleUnaryOperator map) { + FeatureValue value = features.get(code); + if (value == null || value.getValue() == null) return null; + double raw = value.getValue().doubleValue(); + return new Signal(code, raw, clamp((int) Math.round(map.applyAsDouble(raw)), 0, 100)); + } + + private Map weights(Sector sector) { + Map weights = new LinkedHashMap<>(); + if (sector == null) { + // No pack configured: equal weight beats silently zero-weighting a family. + FAMILIES.forEach(name -> weights.put(name, 2000)); + return weights; + } + weights.put(LIQUIDITY, nz(sector.getWLiquidity())); + weights.put(BEHAVIOURAL, nz(sector.getWBehavioural())); + weights.put(TREND_VOLATILITY, nz(sector.getWTrendVolatility())); + weights.put(OBLIGATION_LOAD, nz(sector.getWObligationLoad())); + weights.put(EXTERNAL_SHOCK, nz(sector.getWExternalShock())); + return weights; + } + + /** Each family on a prior costs confidence — five of five means we know nothing. */ + private int confidence(int familiesOnPrior) { + return clamp(confidenceCeiling - familiesOnPrior * 12, 10, confidenceCeiling); + } + + private String band(int score) { + if (score >= redThreshold) return "RED"; + if (score >= watchThreshold) return "WATCH"; + return "STANDARD"; + } + + private static final List FAMILIES = + List.of(LIQUIDITY, BEHAVIOURAL, TREND_VOLATILITY, OBLIGATION_LOAD, EXTERNAL_SHOCK); + + private int nz(Integer value) { + return value == null ? 0 : value; + } + + private int clamp(int value, int min, int max) { + return Math.max(min, Math.min(max, value)); + } + + private record Signal(String code, Double raw, int stress) {} + + private record Family(int value, String featureCode, Double featureValue, boolean onPrior) {} +} diff --git a/src/main/java/com/lincfusion/udyamlinc/risk/ScoreOutcome.java b/src/main/java/com/lincfusion/udyamlinc/risk/ScoreOutcome.java new file mode 100644 index 0000000..1117707 --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/risk/ScoreOutcome.java @@ -0,0 +1,28 @@ +package com.lincfusion.udyamlinc.risk; + +import java.util.List; +import java.util.Map; + +/** + * What an engine produced for one enterprise, before any of it is persisted. + * + *

Sub-scores and drivers travel with the composite so the number can always be + * explained by the parts that made it, rather than reverse-engineered later. + */ +public record ScoreOutcome( + int score, + String band, + int confidence, + /** liquidity | behavioural | trendVolatility | obligationLoad | externalShock → 0–100 */ + Map subScores, + List drivers, + /** Families that had no usable feature and fell back to the sector prior. */ + int familiesOnPrior) { + + /** + * One family's contribution to the composite. In the rule engine this is the + * family's weighted share; when the model lands it becomes the SHAP value, + * and the rest of the pipeline does not change. + */ + public record Driver(String family, int weight, String featureCode, Double value, boolean onPrior) {} +} diff --git a/src/main/java/com/lincfusion/udyamlinc/risk/ScoringRun.java b/src/main/java/com/lincfusion/udyamlinc/risk/ScoringRun.java new file mode 100644 index 0000000..d04283c --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/risk/ScoringRun.java @@ -0,0 +1,76 @@ +package com.lincfusion.udyamlinc.risk; + +import jakarta.persistence.*; +import lombok.*; + +import java.time.LocalDate; +import java.time.LocalDateTime; + +/** + * One scoring pass. Every score, forecast point, reason code and alert hangs off + * a run, so an alert raised months ago always traces back to the exact engine + * and inputs that produced it. This doubles as the model prediction log. + * + *

Generalised from the earlier {@code forecast_run}: a run may be produced by + * the rule engine or by the model, and either on the server or on the phone. + */ +@Entity +@Table(name = "scoring_run", indexes = { + @Index(name = "ix_run_enterprise", columnList = "enterpriseId"), + @Index(name = "ix_run_as_of", columnList = "asOfDate"), +}) +@Getter +@Setter +@NoArgsConstructor +@AllArgsConstructor +@Builder +public class ScoringRun { + + @Id + @GeneratedValue(strategy = GenerationType.IDENTITY) + private Long id; + + /** Null for a portfolio-wide batch; set for a single-enterprise rescore. */ + @Column(length = 24) + private String enterpriseId; + + /** RULE | MODEL | BLEND */ + @Column(nullable = false, length = 16) + private String engine; + + @Column(nullable = false, length = 32) + private String modelVersion; + + /** SERVER | ON_DEVICE — edge inference reconciles against the server on sync. */ + @Column(nullable = false, length = 16) + private String source; + + /** + * The as-of date for point-in-time correctness: no feature observed after + * this date may be read by this run. + */ + @Column(nullable = false) + private LocalDate asOfDate; + + private Integer horizonMonths; + + private Integer enterpriseCount; + + /** RUNNING | COMPLETED | FAILED */ + @Column(nullable = false, length = 16) + private String status; + + private Long durationMs; + + @Column(length = 400) + private String note; + + private LocalDateTime runAt; + + @PrePersist + void onCreate() { + if (runAt == null) runAt = LocalDateTime.now(); + if (asOfDate == null) asOfDate = LocalDate.now(); + if (status == null) status = "COMPLETED"; + } +} diff --git a/src/main/java/com/lincfusion/udyamlinc/risk/ScoringRunRepository.java b/src/main/java/com/lincfusion/udyamlinc/risk/ScoringRunRepository.java new file mode 100644 index 0000000..9f3c38f --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/risk/ScoringRunRepository.java @@ -0,0 +1,13 @@ +package com.lincfusion.udyamlinc.risk; + +import org.springframework.data.jpa.repository.JpaRepository; + +import java.util.List; +import java.util.Optional; + +public interface ScoringRunRepository extends JpaRepository { + + Optional findFirstByEnterpriseIdOrderByRunAtDesc(String enterpriseId); + + List findTop20ByOrderByRunAtDesc(); +} diff --git a/src/main/java/com/lincfusion/udyamlinc/risk/ScoringScheduler.java b/src/main/java/com/lincfusion/udyamlinc/risk/ScoringScheduler.java new file mode 100644 index 0000000..fc69137 --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/risk/ScoringScheduler.java @@ -0,0 +1,48 @@ +package com.lincfusion.udyamlinc.risk; + +import lombok.RequiredArgsConstructor; +import lombok.extern.slf4j.Slf4j; +import org.springframework.beans.factory.annotation.Value; +import org.springframework.scheduling.annotation.Scheduled; +import org.springframework.stereotype.Component; + +import java.time.LocalDate; +import java.util.concurrent.atomic.AtomicBoolean; + +/** + * The nightly scoring pass, deliberately an hour behind ingestion so it scores + * the data that job just wrote rather than racing it. + * + *

Off by default for the same reason ingestion is: a demo boot should score + * when asked, not on a timer. + */ +@Component +@RequiredArgsConstructor +@Slf4j +public class ScoringScheduler { + + private final RiskEngineService riskEngine; + private final AtomicBoolean running = new AtomicBoolean(false); + + @Value("${udyam.risk.scheduled:false}") + private boolean enabled; + + @Scheduled(cron = "${udyam.risk.cron:0 0 3 * * *}", zone = "${udyam.ingestion.zone:Asia/Kolkata}") + public void nightly() { + if (!enabled) { + log.debug("Nightly scoring skipped — udyam.risk.scheduled=false"); + return; + } + if (!running.compareAndSet(false, true)) { + log.warn("Scoring pass already running — this trigger is dropped, not queued"); + return; + } + try { + riskEngine.scoreAll(LocalDate.now()); + } catch (Exception e) { + log.error("Nightly scoring failed", e); + } finally { + running.set(false); + } + } +} diff --git a/src/main/java/com/lincfusion/udyamlinc/risk/ScoringStrategy.java b/src/main/java/com/lincfusion/udyamlinc/risk/ScoringStrategy.java new file mode 100644 index 0000000..bcf9222 --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/risk/ScoringStrategy.java @@ -0,0 +1,36 @@ +package com.lincfusion.udyamlinc.risk; + +import com.lincfusion.udyamlinc.enterprise.Enterprise; +import com.lincfusion.udyamlinc.features.FeatureValue; +import com.lincfusion.udyamlinc.master.Sector; + +import java.util.Map; + +/** + * The seam the model arrives through. + * + *

The engine is rule-based first — a sector-weighted combination of the five + * families, with the weights in the {@code sector} table so tuning needs no + * deploy. When the LightGBM heads exist they implement this interface and + * replacing the engine is a configuration change, not a rewrite. Everything + * downstream of here — scores, reason codes, alerts, the console — is written + * against this contract rather than against either engine. + * + *

Implementations read features and nothing else. No implementation may reach + * back into ingestion or the signal tables; that is what makes the model + * retrainable in isolation. + */ +public interface ScoringStrategy { + + /** RULE | MODEL | BLEND — recorded on every run, score and alert. */ + String engine(); + + /** Recorded on the run so a historic alert names the exact engine that raised it. */ + String modelVersion(); + + /** + * @param features the enterprise's feature rows by catalogue code, including + * the nulls — an engine must be able to see what it did not get + */ + ScoreOutcome score(Enterprise enterprise, Sector sector, Map features); +} diff --git a/src/main/java/com/lincfusion/udyamlinc/risk/StressScore.java b/src/main/java/com/lincfusion/udyamlinc/risk/StressScore.java new file mode 100644 index 0000000..468a72e --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/risk/StressScore.java @@ -0,0 +1,71 @@ +package com.lincfusion.udyamlinc.risk; + +import jakarta.persistence.*; +import lombok.*; + +import java.time.LocalDateTime; + +/** + * The Composite Stress Score and the five independent risk families behind it. + * + *

Sub-scores are stored rather than recomputed, so a historic score can always + * be explained with the numbers that actually produced it — even after the + * sector weights have been retuned. + */ +@Entity +@Table(name = "stress_score", indexes = { + @Index(name = "ix_score_enterprise", columnList = "enterpriseId"), + @Index(name = "ix_score_run", columnList = "runId"), +}) +@Getter +@Setter +@NoArgsConstructor +@AllArgsConstructor +@Builder +public class StressScore { + + @Id + @GeneratedValue(strategy = GenerationType.IDENTITY) + private Long id; + + @Column(nullable = false) + private Long runId; + + @Column(nullable = false, length = 24) + private String enterpriseId; + + /** 0–100, sector-weighted combination of the five families below. */ + @Column(nullable = false) + private Integer score; + + @Column(nullable = false, length = 16) + private String band; + + /** RULE | MODEL | BLEND — which engine produced this score. */ + @Column(nullable = false, length = 16) + private String engine; + + /** 0–100. A rule engine reports low confidence by design. */ + private Integer confidence; + + private Integer liquidity; + + private Integer behavioural; + + private Integer trendVolatility; + + private Integer obligationLoad; + + private Integer externalShock; + + /** The wide feature row this score was computed from, for reproducibility. */ + private Long featureSnapshotId; + + private LocalDateTime computedAt; + + @PrePersist + void onCreate() { + if (computedAt == null) computedAt = LocalDateTime.now(); + if (engine == null) engine = "RULE"; + } +} diff --git a/src/main/java/com/lincfusion/udyamlinc/risk/StressScoreRepository.java b/src/main/java/com/lincfusion/udyamlinc/risk/StressScoreRepository.java new file mode 100644 index 0000000..ab304d7 --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/risk/StressScoreRepository.java @@ -0,0 +1,13 @@ +package com.lincfusion.udyamlinc.risk; + +import org.springframework.data.jpa.repository.JpaRepository; + +import java.util.List; +import java.util.Optional; + +public interface StressScoreRepository extends JpaRepository { + + Optional findFirstByEnterpriseIdOrderByComputedAtDesc(String enterpriseId); + + List findByEnterpriseIdOrderByComputedAtDesc(String enterpriseId); +} diff --git a/src/main/java/com/lincfusion/udyamlinc/rules/RuleEvaluator.java b/src/main/java/com/lincfusion/udyamlinc/rules/RuleEvaluator.java new file mode 100644 index 0000000..e9bf40e --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/rules/RuleEvaluator.java @@ -0,0 +1,143 @@ +package com.lincfusion.udyamlinc.rules; + +import com.lincfusion.udyamlinc.features.FeatureDefinition; +import com.lincfusion.udyamlinc.features.FeatureDefinitionRepository; +import com.lincfusion.udyamlinc.features.FeatureValue; +import lombok.RequiredArgsConstructor; +import lombok.extern.slf4j.Slf4j; +import org.springframework.context.expression.MapAccessor; +import org.springframework.expression.Expression; +import org.springframework.expression.spel.standard.SpelExpressionParser; +import org.springframework.expression.spel.support.SimpleEvaluationContext; +import org.springframework.stereotype.Component; + +import java.util.*; +import java.util.concurrent.ConcurrentHashMap; + +/** + * Runs the deterministic trip-wires against an enterprise's features. + * + *

Independent of the model on purpose. A trip-wire alert survives a retrain — + * that is the entire point — so nothing here consults a score, only the feature + * values and their freshness. + * + *

Two guards stand in front of every expression: + * + *

+ * + *

Values are handed to SpEL as doubles rather than {@code BigDecimal}. The + * rules are written as plain arithmetic comparisons and this keeps their + * semantics obvious; nothing here needs decimal exactness. + * + *

Expressions run in a {@code SimpleEvaluationContext} restricted to property + * access. These strings are authored by business users through the admin API, so + * method calls, type references and bean lookups are not on the table. + */ +@Component +@RequiredArgsConstructor +@Slf4j +public class RuleEvaluator { + + private final TripWireRuleRepository ruleRepository; + private final FeatureDefinitionRepository definitionRepository; + + private final SpelExpressionParser parser = new SpelExpressionParser(); + + /** Keyed by rule code and version, so an edit invalidates its own entry. */ + private final Map compiled = new ConcurrentHashMap<>(); + + /** + * Every active rule against one enterprise's feature rows, in catalogue order. + * Returns an outcome for each — including the skips, which are the whole + * reason this returns outcomes rather than a list of alerts. + */ + public List evaluateAll(String sectorCode, Map features) { + Map catalogue = new HashMap<>(); + definitionRepository.findByActiveTrue().forEach(d -> catalogue.put(d.getCode(), d)); + + List outcomes = new ArrayList<>(); + for (TripWireRule rule : ruleRepository.findByActiveTrueOrderByCodeAsc()) { + outcomes.add(evaluate(rule, sectorCode, features, catalogue)); + } + return outcomes; + } + + /** One rule. Package-visible so the dry-run endpoint can explain a single code. */ + public RuleOutcome evaluate(TripWireRule rule, String sectorCode, + Map features, + Map catalogue) { + + if (rule.getSectorCode() != null && !rule.getSectorCode().equals(sectorCode)) { + return RuleOutcome.of(rule, RuleOutcome.Status.SKIPPED_SECTOR, rule.getSectorCode()); + } + + // ── guards ── + for (String code : requiredFeatures(rule)) { + FeatureValue value = features.get(code); + if (value == null || value.getValue() == null) { + String reason = value == null ? "not computed" : value.getMissingReason(); + return RuleOutcome.of(rule, RuleOutcome.Status.SKIPPED_MISSING, code + " (" + reason + ")"); + } + Integer tolerance = tolerance(rule, catalogue.get(code)); + if (tolerance != null && value.getStalenessDays() != null && value.getStalenessDays() > tolerance) { + return RuleOutcome.of(rule, RuleOutcome.Status.SKIPPED_STALE, + code + " is " + value.getStalenessDays() + "d old, tolerance " + tolerance + "d"); + } + } + + // ── evaluation ── + Map root = new HashMap<>(); + features.forEach((code, value) -> { + if (value.getValue() != null) root.put(code, value.getValue().doubleValue()); + }); + + try { + Expression expression = compiled.computeIfAbsent( + rule.getCode() + ":" + rule.getVersion(), key -> parser.parseExpression(rule.getExpression())); + Boolean result = expression.getValue( + SimpleEvaluationContext.forPropertyAccessors(new MapAccessor()).build(), + root, Boolean.class); + return RuleOutcome.of(rule, + Boolean.TRUE.equals(result) ? RuleOutcome.Status.FIRED : RuleOutcome.Status.NOT_FIRED, + rule.getThresholdText()); + + } catch (Exception e) { + log.warn("Trip-wire {} failed to evaluate: {}", rule.getCode(), e.toString()); + return RuleOutcome.of(rule, RuleOutcome.Status.ERROR, e.getMessage()); + } + } + + /** + * The stricter of the two limits. The rule may be more cautious than the + * catalogue, never less — a supervisor tightening a rule must not be + * overridden by a generous default. + */ + private Integer tolerance(TripWireRule rule, FeatureDefinition definition) { + Integer ruleLimit = rule.getMaxStalenessDays(); + Integer featureLimit = definition == null ? null : definition.getStalenessToleranceDays(); + if (ruleLimit == null) return featureLimit; + if (featureLimit == null) return ruleLimit; + return Math.min(ruleLimit, featureLimit); + } + + private List requiredFeatures(TripWireRule rule) { + if (rule.getRequiredFeatures() == null || rule.getRequiredFeatures().isBlank()) return List.of(); + return Arrays.stream(rule.getRequiredFeatures().split(",")) + .map(String::trim) + .filter(s -> !s.isEmpty()) + .toList(); + } + + /** Clears the compiled cache — called after a rule edit. */ + public void invalidate() { + compiled.clear(); + } +} diff --git a/src/main/java/com/lincfusion/udyamlinc/rules/RuleOutcome.java b/src/main/java/com/lincfusion/udyamlinc/rules/RuleOutcome.java new file mode 100644 index 0000000..88d67b7 --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/rules/RuleOutcome.java @@ -0,0 +1,47 @@ +package com.lincfusion.udyamlinc.rules; + +/** + * What one rule did against one snapshot. + * + *

A rule that did not fire and a rule that was not allowed to run are + * different events, and the console must be able to tell them apart — "no alert" + * because the enterprise is fine reads very differently from "no alert" because + * the mandi feed has been down for a fortnight. + */ +public record RuleOutcome( + String ruleCode, + Long ruleId, + Integer ruleVersion, + String severity, + Status status, + /** The feature that caused a skip, or the expression's evaluated result. */ + String detail) { + + public enum Status { + /** The expression evaluated true. */ + FIRED, + /** It ran and came back false. */ + NOT_FIRED, + /** A required feature was older than the rule tolerates. */ + SKIPPED_STALE, + /** A required feature had no value at all. */ + SKIPPED_MISSING, + /** Sector-scoped rule, different sector. */ + SKIPPED_SECTOR, + /** The expression itself is broken — a configuration fault, not a risk signal. */ + ERROR + } + + public boolean fired() { + return status == Status.FIRED; + } + + public boolean evaluated() { + return status == Status.FIRED || status == Status.NOT_FIRED; + } + + static RuleOutcome of(TripWireRule rule, Status status, String detail) { + return new RuleOutcome(rule.getCode(), rule.getId(), rule.getVersion(), + rule.getSeverity(), status, detail); + } +} diff --git a/src/main/java/com/lincfusion/udyamlinc/rules/TripWireRule.java b/src/main/java/com/lincfusion/udyamlinc/rules/TripWireRule.java new file mode 100644 index 0000000..1b61e06 --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/rules/TripWireRule.java @@ -0,0 +1,88 @@ +package com.lincfusion.udyamlinc.rules; + +import jakarta.persistence.*; +import lombok.*; + +import java.time.LocalDateTime; + +/** + * A deterministic rule that fires independently of the model. + * + *

{@code expression} is SpEL evaluated against the enterprise's wide feature + * snapshot, e.g. {@code emi_delay > 30 and cash_flow_change < -0.40}. Business + * teams edit rows; nobody deploys. SpEL rather than a rules platform — a handful + * of rules does not justify Drools. + * + *

{@code maxStalenessDays} is the safety catch: a rule must not fire on data + * older than it tolerates, because a rule firing on a three-week-old mandi price + * is worse than no rule at all. + */ +@Entity +@Table(name = "tripwire_rule", indexes = @Index(name = "ix_rule_code", columnList = "code", unique = true)) +@Getter +@Setter +@NoArgsConstructor +@AllArgsConstructor +@Builder +public class TripWireRule { + + @Id + @GeneratedValue(strategy = GenerationType.IDENTITY) + private Long id; + + /** TW-01… */ + @Column(nullable = false, unique = true, length = 16) + private String code; + + @Column(nullable = false, length = 160) + private String label; + + @Column(length = 160) + private String labelHi; + + /** SpEL over the feature snapshot. */ + @Column(nullable = false, length = 600) + private String expression; + + /** Human-readable form of the threshold, shown in the console. */ + @Column(length = 160) + private String thresholdText; + + /** RED | WATCH — the band this rule escalates to when it fires. */ + @Column(nullable = false, length = 16) + private String severity; + + /** Null means the rule applies to every sector. */ + @Column(length = 32) + private String sectorCode; + + /** Features referenced by the expression, CSV — used for the staleness check. */ + @Column(length = 400) + private String requiredFeatures; + + private Integer maxStalenessDays; + + @Column(length = 400) + private String messageEn; + + @Column(length = 400) + private String messageHi; + + private Boolean active; + + /** Bumped on every edit so an old alert can name the rule version that fired. */ + private Integer version; + + @Column(length = 64) + private String updatedBy; + + private LocalDateTime updatedAt; + + @PrePersist + void onCreate() { + if (active == null) active = Boolean.TRUE; + if (version == null) version = 1; + if (maxStalenessDays == null) maxStalenessDays = 14; + if (updatedAt == null) updatedAt = LocalDateTime.now(); + } +} diff --git a/src/main/java/com/lincfusion/udyamlinc/rules/TripWireRuleRepository.java b/src/main/java/com/lincfusion/udyamlinc/rules/TripWireRuleRepository.java new file mode 100644 index 0000000..88fc6e0 --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/rules/TripWireRuleRepository.java @@ -0,0 +1,13 @@ +package com.lincfusion.udyamlinc.rules; + +import org.springframework.data.jpa.repository.JpaRepository; + +import java.util.List; +import java.util.Optional; + +public interface TripWireRuleRepository extends JpaRepository { + + Optional findByCode(String code); + + List findByActiveTrueOrderByCodeAsc(); +} diff --git a/src/main/java/com/lincfusion/udyamlinc/security/JwtAuthFilter.java b/src/main/java/com/lincfusion/udyamlinc/security/JwtAuthFilter.java new file mode 100644 index 0000000..829fc1b --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/security/JwtAuthFilter.java @@ -0,0 +1,48 @@ +package com.lincfusion.udyamlinc.security; + +import jakarta.servlet.FilterChain; +import jakarta.servlet.ServletException; +import jakarta.servlet.http.HttpServletRequest; +import jakarta.servlet.http.HttpServletResponse; +import lombok.RequiredArgsConstructor; +import org.springframework.security.authentication.UsernamePasswordAuthenticationToken; +import org.springframework.security.core.authority.SimpleGrantedAuthority; +import org.springframework.security.core.context.SecurityContextHolder; +import org.springframework.security.web.authentication.WebAuthenticationDetailsSource; +import org.springframework.stereotype.Component; +import org.springframework.web.filter.OncePerRequestFilter; + +import java.io.IOException; +import java.util.List; + +/** + * Reads the bearer token and grants two authorities: the role (ROLE_FIELD_OFFICER, + * ROLE_OWNER…) and the scope (SCOPE_OFFICER / SCOPE_ENTERPRISE), which is what the + * two API surfaces are separated on. + */ +@Component +@RequiredArgsConstructor +public class JwtAuthFilter extends OncePerRequestFilter { + + private final JwtUtil jwtUtil; + + @Override + protected void doFilterInternal(HttpServletRequest request, HttpServletResponse response, FilterChain chain) + throws ServletException, IOException { + + String header = request.getHeader("Authorization"); + if (header != null && header.startsWith("Bearer ")) { + String token = header.substring(7); + if (jwtUtil.isValid(token) && SecurityContextHolder.getContext().getAuthentication() == null) { + String subject = jwtUtil.subject(token); + var authorities = List.of( + new SimpleGrantedAuthority("ROLE_" + jwtUtil.role(token)), + new SimpleGrantedAuthority("SCOPE_" + jwtUtil.scope(token))); + var auth = new UsernamePasswordAuthenticationToken(subject, null, authorities); + auth.setDetails(new WebAuthenticationDetailsSource().buildDetails(request)); + SecurityContextHolder.getContext().setAuthentication(auth); + } + } + chain.doFilter(request, response); + } +} diff --git a/src/main/java/com/lincfusion/udyamlinc/security/JwtUtil.java b/src/main/java/com/lincfusion/udyamlinc/security/JwtUtil.java new file mode 100644 index 0000000..fa8f1cb --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/security/JwtUtil.java @@ -0,0 +1,87 @@ +package com.lincfusion.udyamlinc.security; + +import io.jsonwebtoken.Claims; +import io.jsonwebtoken.Jwts; +import io.jsonwebtoken.SignatureAlgorithm; +import io.jsonwebtoken.security.Keys; +import org.springframework.beans.factory.annotation.Value; +import org.springframework.stereotype.Component; + +import java.security.Key; +import java.util.Date; +import java.util.HashMap; +import java.util.Map; + +/** Issues and reads the two token flavours: officer console and enterprise app. */ +@Component +public class JwtUtil { + + @Value("${jwt.secret}") + private String secret; + + @Value("${jwt.expiration:86400000}") + private long expirationMs; + + private Key signingKey() { + return Keys.hmacShaKeyFor(secret.getBytes()); + } + + /** Field officer session — scope=OFFICER, carries the role and the blocks in view. */ + public String generateOfficerToken(String username, String role, String blocks) { + Map claims = new HashMap<>(); + claims.put("realm", "UDYAM"); + claims.put("scope", "OFFICER"); + claims.put("role", role); + claims.put("blocks", blocks); + return build(claims, username); + } + + /** + * Enterprise app session — scope=ENTERPRISE, pinned to one enterprise id so + * the owner can only ever read their own record. + */ + public String generateEnterpriseToken(String enterpriseId, String sector) { + Map claims = new HashMap<>(); + claims.put("realm", "UDYAM"); + claims.put("scope", "ENTERPRISE"); + claims.put("role", "OWNER"); + claims.put("enterpriseId", enterpriseId); + claims.put("sector", sector); + return build(claims, enterpriseId); + } + + private String build(Map claims, String subject) { + long now = System.currentTimeMillis(); + return Jwts.builder() + .setClaims(claims) + .setSubject(subject) + .setIssuedAt(new Date(now)) + .setExpiration(new Date(now + expirationMs)) + .signWith(signingKey(), SignatureAlgorithm.HS256) + .compact(); + } + + public Claims parse(String token) { + return Jwts.parserBuilder().setSigningKey(signingKey()).build().parseClaimsJws(token).getBody(); + } + + public String subject(String token) { + return parse(token).getSubject(); + } + + public String scope(String token) { + return String.valueOf(parse(token).get("scope")); + } + + public String role(String token) { + return String.valueOf(parse(token).get("role")); + } + + public boolean isValid(String token) { + try { + return parse(token).getExpiration().after(new Date()); + } catch (Exception e) { + return false; + } + } +} diff --git a/src/main/java/com/lincfusion/udyamlinc/seed/SeedText.java b/src/main/java/com/lincfusion/udyamlinc/seed/SeedText.java new file mode 100644 index 0000000..fb2985f --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/seed/SeedText.java @@ -0,0 +1,94 @@ +package com.lincfusion.udyamlinc.seed; + +import java.util.List; +import java.util.Map; + +/** + * Sector-specific naming and reason-sentence templates for the demo portfolio. + * + *

Kept apart from the seeder itself so the seeder reads as a sequence of + * steps rather than a wall of string literals. The sentences mirror those in + * UdyamLinc-UI/src/lib/demo/enterprises.ts, so the console shows the same wording + * whether it is reading the API or its own fallback. + */ +final class SeedText { + + private SeedText() { + } + + static final String[] PREFIX = { + "Shri", "Maa Narmada", "Jai Kisan", "Sarvodaya", "Annapurna", "Vindhya", "Betwa", "Narmada", + "Adarsh", "Nav Jeevan", "Gram Shakti", "Kisan Mitra", "Sanskar", "Prerna", "Sunrise", + "Bhagirath", "Malwa", "Sahyog", "Ekta", "Pragati", "Suvidha", "Anmol", "Nirmal", "Vaibhav", + }; + + static final String[] PREFIX_HI = { + "श्री", "माँ नर्मदा", "जय किसान", "सर्वोदय", "अन्नपूर्णा", "विंध्य", "बेतवा", "नर्मदा", + "आदर्श", "नव जीवन", "ग्राम शक्ति", "किसान मित्र", "संस्कार", "प्रेरणा", "सनराइज़", + "भगीरथ", "मालवा", "सहयोग", "एकता", "प्रगति", "सुविधा", "अनमोल", "निर्मल", "वैभव", + }; + + static final Map> SUFFIX = Map.of( + "DAIRY", List.of("Dairy", "Milk Centre", "Dairy Unit", "Doodh Sangh"), + "POULTRY", List.of("Poultry Farm", "Layer Farm", "Poultry Unit"), + "FOOD_PROC", List.of("Food Products", "Flour Mill", "Masala Udyog", "Agro Foods"), + "HANDICRAFTS", List.of("Handicrafts", "Handloom", "Craft Works"), + "RURAL_RETAIL", List.of("Kirana Store", "General Store", "Provision Store")); + + static final Map> SUFFIX_HI = Map.of( + "DAIRY", List.of("डेयरी", "दूध केंद्र", "डेयरी यूनिट", "दूध संघ"), + "POULTRY", List.of("पोल्ट्री फ़ार्म", "लेयर फ़ार्म", "पोल्ट्री यूनिट"), + "FOOD_PROC", List.of("फ़ूड प्रोडक्ट्स", "आटा चक्की", "मसाला उद्योग", "एग्रो फ़ूड्स"), + "HANDICRAFTS", List.of("हस्तशिल्प", "हथकरघा", "क्राफ़्ट वर्क्स"), + "RURAL_RETAIL", List.of("किराना स्टोर", "जनरल स्टोर", "प्रोविज़न स्टोर")); + + /** driver → { en, hi } sentence, per sector. */ + static final Map> REASONS = Map.of( + "POULTRY", Map.of( + "liquidity", new String[]{"Feed costs up 18% against a 9% fall in egg realisation.", "दाना 18% महँगा और अंडे का भाव 9% गिरा है।"}, + "behavioural", new String[]{"UPI collections down 23% over eight weeks.", "आठ हफ़्तों में UPI से आया पैसा 23% घटा है।"}, + "trendVolatility", new String[]{"Weekly sales swinging twice as wide as last season.", "हफ़्तेवार बिक्री पिछले सीज़न से दोगुनी ऊपर-नीचे हो रही है।"}, + "obligationLoad", new String[]{"Instalment now takes 41% of net monthly cash.", "किस्त अब महीने की कमाई का 41% ले रही है।"}, + "externalShock", new String[]{"Heat anomaly flagged in the block for the next fortnight.", "अगले पंद्रह दिन ब्लॉक में गर्मी की चेतावनी है।"}), + "DAIRY", Map.of( + "liquidity", new String[]{"Fodder cost up 14% while procurement rate stayed flat.", "चारा 14% महँगा हुआ, दूध का रेट वही है।"}, + "behavioural", new String[]{"Society payment cycle stretched from 10 to 18 days.", "समिति से भुगतान 10 दिन की जगह 18 दिन में आ रहा है।"}, + "trendVolatility", new String[]{"Daily yield varying sharply across the last six weeks.", "छह हफ़्तों में रोज़ का दूध बहुत घट-बढ़ रहा है।"}, + "obligationLoad", new String[]{"Two instalments now fall in the same fortnight.", "दो किस्तें एक ही पखवाड़े में आ रही हैं।"}, + "externalShock", new String[]{"Rainfall deficit in the block is raising fodder prices.", "ब्लॉक में कम बारिश से चारा महँगा हो रहा है।"}), + "FOOD_PROC", Map.of( + "liquidity", new String[]{"Raw material prepayment is locking up working capital.", "कच्चे माल का अग्रिम भुगतान पूँजी रोक रहा है।"}, + "behavioural", new String[]{"Buyer settlements slipping past the agreed 15 days.", "खरीदार 15 दिन की जगह देर से पैसा दे रहे हैं।"}, + "trendVolatility", new String[]{"Order volume swinging with festival demand.", "त्योहारों के साथ ऑर्डर बहुत घट-बढ़ रहे हैं।"}, + "obligationLoad", new String[]{"Instalment plus rent exceeds 50% of monthly inflow.", "किस्त और किराया मिलकर आधी से ज़्यादा कमाई ले रहे हैं।"}, + "externalShock", new String[]{"Mandi price shock in the primary input crop.", "मुख्य फ़सल के मंडी भाव में तेज़ बदलाव है।"}), + "HANDICRAFTS", Map.of( + "liquidity", new String[]{"Finished stock is unsold going into the lean quarter.", "तैयार माल बिका नहीं और सुस्त तिमाही आ रही है।"}, + "behavioural", new String[]{"Top buyer accounts for 68% of all receipts.", "कुल आमदनी का 68% सिर्फ़ एक खरीदार से आता है।"}, + "trendVolatility", new String[]{"Receipts concentrated in two months of the year.", "साल की कमाई सिर्फ़ दो महीनों में सिमटी है।"}, + "obligationLoad", new String[]{"Repayment schedule ignores the seasonal lull.", "किस्त का समय सुस्त सीज़न का ध्यान नहीं रखता।"}, + "externalShock", new String[]{"Exhibition calendar thinner than last year.", "पिछले साल से कम प्रदर्शनियाँ लगी हैं।"}), + "RURAL_RETAIL", Map.of( + "liquidity", new String[]{"Credit sales to customers up 31% this quarter.", "इस तिमाही उधार बिक्री 31% बढ़ी है।"}, + "behavioural", new String[]{"Daily UPI receipt count down for six straight weeks.", "छह हफ़्तों से रोज़ के UPI लेन-देन घट रहे हैं।"}, + "trendVolatility", new String[]{"Footfall dropping since the new road closure.", "रास्ता बंद होने से ग्राहक कम आ रहे हैं।"}, + "obligationLoad", new String[]{"Stock restocking clashes with the instalment date.", "माल भरने और किस्त की तारीख़ एक साथ पड़ रही है।"}, + "externalShock", new String[]{"Local market disruption flagged in the block.", "ब्लॉक में स्थानीय बाज़ार में गड़बड़ी दर्ज है।"})); + + /** Driver key → Hindi label, in the order the console renders them. */ + static final String[][] DRIVERS = { + {"liquidity", "Liquidity", "नकदी"}, + {"behavioural", "Behavioural (UPI)", "लेन-देन व्यवहार"}, + {"trendVolatility", "Trend & volatility", "उतार-चढ़ाव"}, + {"obligationLoad", "Obligation load", "किस्त का बोझ"}, + {"externalShock", "External shock", "बाहरी झटका"}, + }; + + static final String[] MONTHS_EN = { + "January", "February", "March", "April", "May", "June", + "July", "August", "September", "October", "November", "December"}; + + static final String[] MONTHS_HI = { + "जनवरी", "फ़रवरी", "मार्च", "अप्रैल", "मई", "जून", + "जुलाई", "अगस्त", "सितंबर", "अक्टूबर", "नवंबर", "दिसंबर"}; +} diff --git a/src/main/java/com/lincfusion/udyamlinc/seed/UdyamDataSeeder.java b/src/main/java/com/lincfusion/udyamlinc/seed/UdyamDataSeeder.java new file mode 100644 index 0000000..b80b7c5 --- /dev/null +++ b/src/main/java/com/lincfusion/udyamlinc/seed/UdyamDataSeeder.java @@ -0,0 +1,661 @@ +package com.lincfusion.udyamlinc.seed; + +import com.lincfusion.udyamlinc.auth.UdyamUser; +import com.lincfusion.udyamlinc.auth.UdyamUserRepository; +import com.lincfusion.udyamlinc.enterprise.Enterprise; +import com.lincfusion.udyamlinc.enterprise.EnterpriseRepository; +import com.lincfusion.udyamlinc.enterprise.LedgerEntry; +import com.lincfusion.udyamlinc.enterprise.LedgerEntryRepository; +import com.lincfusion.udyamlinc.enterprise.LoanRecord; +import com.lincfusion.udyamlinc.enterprise.LoanRecordRepository; +import com.lincfusion.udyamlinc.features.FeatureCatalog; +import com.lincfusion.udyamlinc.master.*; +import com.lincfusion.udyamlinc.officer.FieldVisit; +import com.lincfusion.udyamlinc.officer.FieldVisitRepository; +import com.lincfusion.udyamlinc.officer.PlaybookCatalog; +import com.lincfusion.udyamlinc.risk.*; +import com.lincfusion.udyamlinc.rules.TripWireRule; +import com.lincfusion.udyamlinc.rules.TripWireRuleRepository; +import lombok.RequiredArgsConstructor; +import lombok.extern.slf4j.Slf4j; +import org.springframework.beans.factory.annotation.Value; +import org.springframework.boot.CommandLineRunner; +import org.springframework.security.crypto.password.PasswordEncoder; +import org.springframework.stereotype.Component; +import org.springframework.transaction.annotation.Transactional; + +import java.math.BigDecimal; +import java.math.RoundingMode; +import java.time.LocalDate; +import java.time.LocalDateTime; +import java.util.*; + +/** + * Seeds the demo portfolio on first boot. Idempotent: every step checks before + * it writes, so restarting the service does not duplicate anything. + * + *

The portfolio reproduces the submitted deck exactly — 142 enterprises with + * a 9 / 31 / 102 band split, and the six enterprises named on the wireframe slide + * pinned to their published scores. Figures are deterministic, seeded off the + * enterprise id, so a demo shows the same numbers every time. + */ +@Component +@RequiredArgsConstructor +@Slf4j +public class UdyamDataSeeder implements CommandLineRunner { + + /** Initial credentials for seeded logins. */ + public static final String DEFAULT_OFFICER_PASSWORD = "Udyam@123"; + public static final String DEFAULT_ENTERPRISE_PIN = "1234"; + + private static final int TOTAL = 142; + private static final Map TARGET = Map.of("RED", 9, "WATCH", 31, "STANDARD", 102); + private static final String MODEL_VERSION = "rule-v0.9.3"; + + private final DistrictRepository districtRepository; + private final CommodityRepository commodityRepository; + private final MarketRepository marketRepository; + private final SectorRepository sectorRepository; + private final SectorCommodityRepository sectorCommodityRepository; + private final RoleRepository roleRepository; + private final EnterpriseRepository enterpriseRepository; + private final LedgerEntryRepository ledgerRepository; + private final LoanRecordRepository loanRecordRepository; + private final FeatureCatalog featureCatalog; + private final ScoringRunRepository runRepository; + private final ForecastPointRepository pointRepository; + private final StressScoreRepository scoreRepository; + private final ReasonCodeRepository reasonRepository; + private final RiskAlertRepository alertRepository; + private final FieldVisitRepository visitRepository; + private final TripWireRuleRepository ruleRepository; + private final UdyamUserRepository userRepository; + private final PasswordEncoder passwordEncoder; + + @Value("${udyam.seed.enabled:true}") + private boolean seedEnabled; + + @Value("${udyam.risk.red-threshold:65}") + private int redThreshold; + + @Value("${udyam.risk.watch-threshold:35}") + private int watchThreshold; + + @Value("${udyam.forecast.horizon-months:6}") + private int horizon; + + @Override + @Transactional + public void run(String... args) { + if (!seedEnabled) { + log.info("Seeding disabled (udyam.seed.enabled=false)"); + return; + } + seedRoles(); + seedDistricts(); + seedCommodities(); + seedMarkets(); + seedSectors(); + seedSectorCommodities(); + seedTripWireRules(); + featureCatalog.seed(); + seedOfficers(); + seedPortfolio(); + log.info("UdyamLinc seed complete — {} enterprises on book", enterpriseRepository.count()); + } + + // ── masters ───────────────────────────────────────────── + + private void seedRoles() { + if (roleRepository.count() > 0) return; + roleRepository.saveAll(List.of( + Role.builder().code("FIELD_OFFICER").name("Field Officer").systemRole(true) + .permissions("PORTFOLIO:VIEW,ALERT:VIEW,ALERT:ACT,VISIT:VIEW,VISIT:LOG,REPORT:VIEW").build(), + Role.builder().code("SUPERVISOR").name("Supervisor").systemRole(true) + .permissions("PORTFOLIO:VIEW,ALERT:VIEW,ALERT:ACT,VISIT:VIEW,VISIT:LOG,REPORT:VIEW,CONFIG:MANAGE,RULE:MANAGE").build(), + Role.builder().code("OWNER").name("Enterprise Owner").systemRole(true) + .permissions("APP:VIEW,LEDGER:WRITE,CONSENT:MANAGE").build())); + log.info("Seeded {} roles", roleRepository.count()); + } + + /** The pilot's four districts, with centroids for picking the climate grid cell. */ + private void seedDistricts() { + if (districtRepository.count() > 0) return; + districtRepository.saveAll(List.of( + district("SEHORE", "Sehore", "सीहोर", 23.2033, 77.0856), + district("VIDISHA", "Vidisha", "विदिशा", 23.5251, 77.8081), + district("RAISEN", "Raisen", "रायसेन", 23.3300, 77.7800), + district("BHOPAL", "Bhopal", "भोपाल", 23.2599, 77.4126))); + log.info("Seeded {} districts", districtRepository.count()); + } + + private District district(String code, String name, String nameHi, double lat, double lon) { + return District.builder().code(code).name(name).nameHi(nameHi) + .state("Madhya Pradesh").latitude(lat).longitude(lon).active(true).build(); + } + + /** + * Commodities the sector packs track. {@code sourceCode} is the name the + * upstream feed uses, which rarely matches ours. + */ + private void seedCommodities() { + if (commodityRepository.count() > 0) return; + commodityRepository.saveAll(List.of( + commodity("MAIZE", "Maize", "मक्का", "AGMARKNET", "Maize", "QUINTAL"), + commodity("SOYA", "Soyabean", "सोयाबीन", "AGMARKNET", "Soyabean", "QUINTAL"), + commodity("WHEAT", "Wheat", "गेहूँ", "AGMARKNET", "Wheat", "QUINTAL"), + commodity("GRAM", "Gram", "चना", "AGMARKNET", "Bengal Gram(Gram)(Whole)", "QUINTAL"), + commodity("EGG", "Egg", "अंडा", "NECC", "Bhopal", "HUNDRED"), + commodity("MILK", "Milk", "दूध", "DAIRY_FEDERATION", null, "LITRE"), + commodity("FODDER", "Green fodder", "हरा चारा", "AGMARKNET", "Green Fodder", "QUINTAL"))); + log.info("Seeded {} commodities", commodityRepository.count()); + } + + private Commodity commodity(String code, String name, String nameHi, String source, String sourceCode, String unit) { + return Commodity.builder().code(code).name(name).nameHi(nameHi) + .source(source).sourceCode(sourceCode).unit(unit).active(true).build(); + } + + private void seedMarkets() { + if (marketRepository.count() > 0) return; + districtRepository.findAll().forEach(d -> marketRepository.save(Market.builder() + .code(d.getCode() + "-MANDI") + .name(d.getName() + " Krishi Upaj Mandi") + .sourceName(d.getName()) + .districtId(d.getId()) + .primaryMarket(true) + .active(true) + .build())); + log.info("Seeded {} markets", marketRepository.count()); + } + + /** The five risk packs. Weights are basis points and sum to 10000. */ + private void seedSectors() { + if (sectorRepository.count() > 0) return; + sectorRepository.saveAll(List.of( + sector("DAIRY", "Dairy", "डेयरी", "Dairy", 2800, 2000, 1800, 1900, 1500), + sector("POULTRY", "Poultry", "मुर्गी पालन", "Poultry", 3000, 2200, 1700, 1400, 1700), + sector("FOOD_PROC", "Food processing", "खाद्य प्रसंस्करण", "Food proc.", 2600, 2200, 2000, 1800, 1400), + sector("HANDICRAFTS", "Handicrafts", "हस्तशिल्प", "Handicrafts", 2400, 1800, 2000, 1600, 2200), + sector("RURAL_RETAIL", "Rural retail", "ग्रामीण दुकान", "Rural retail", 3000, 2400, 1800, 1800, 1000))); + log.info("Seeded {} sector packs", sectorRepository.count()); + } + + private Sector sector(String code, String label, String labelHi, String shortLabel, + int liq, int beh, int trend, int obl, int shock) { + return Sector.builder().code(code).label(label).labelHi(labelHi).shortLabel(shortLabel) + .wLiquidity(liq).wBehavioural(beh).wTrendVolatility(trend) + .wObligationLoad(obl).wExternalShock(shock).active(true).build(); + } + + /** + * What each pack buys and sells. The INPUT / OUTPUT split is what makes + * margin squeeze computable once the ingestion layer lands. + */ + private void seedSectorCommodities() { + if (sectorCommodityRepository.count() > 0) return; + link("POULTRY", "MAIZE", "INPUT", 6000); + link("POULTRY", "SOYA", "INPUT", 4000); + link("POULTRY", "EGG", "OUTPUT", 10000); + link("DAIRY", "FODDER", "INPUT", 7000); + link("DAIRY", "MAIZE", "INPUT", 3000); + link("DAIRY", "MILK", "OUTPUT", 10000); + link("FOOD_PROC", "WHEAT", "INPUT", 5000); + link("FOOD_PROC", "GRAM", "INPUT", 5000); + link("RURAL_RETAIL", "WHEAT", "INPUT", 10000); + log.info("Seeded {} sector-commodity links", sectorCommodityRepository.count()); + } + + private void link(String sectorCode, String commodityCode, String role, int weight) { + Long sectorId = sectorRepository.findByCode(sectorCode).map(Sector::getId).orElse(null); + Long commodityId = commodityRepository.findByCode(commodityCode).map(Commodity::getId).orElse(null); + if (sectorId == null || commodityId == null) return; + sectorCommodityRepository.save(SectorCommodity.builder() + .sectorId(sectorId).commodityId(commodityId).role(role).weight(weight).build()); + } + + /** + * The five deterministic trip-wires. Expressions are SpEL over the feature + * snapshot; the evaluator that runs them arrives with the feature store. + */ + private void seedTripWireRules() { + if (ruleRepository.count() > 0) return; + ruleRepository.saveAll(List.of( + rule("TW-01", "Projected DSCR below 1.0", "किस्त चुकाने की क्षमता 1.0 से नीचे", + "dscr_p10 < 1.0", "DSCR(P10) < 1.00", "RED", "dscr_p10", 7), + rule("TW-02", "UPI inflow drop over 8 weeks", "आठ हफ़्तों में UPI आमदनी में गिरावट", + "upi_inflow_change < -0.20", "Δ inflow < −20%", "RED", "upi_inflow_change", 10), + rule("TW-03", "Ledger silent for 14 days", "14 दिन से हिसाब दर्ज नहीं", + "days_since_last_entry >= 14", "no entry ≥ 14 days", "WATCH", "days_since_last_entry", 3), + rule("TW-04", "Instalment share of inflow above 40%", "किस्त आमदनी का 40% से ज़्यादा", + "emi_to_inflow > 0.40", "EMI / inflow > 40%", "WATCH", "emi_to_inflow", 30), + rule("TW-05", "Climate or mandi shock in block", "ब्लॉक में मौसम या मंडी का झटका", + "price_z_score > 1.5 or heat_anomaly > 1.5", "anomaly z-score > 1.5", "WATCH", + "price_z_score,heat_anomaly", 14))); + log.info("Seeded {} trip-wire rules", ruleRepository.count()); + } + + private TripWireRule rule(String code, String label, String labelHi, String expression, + String thresholdText, String severity, String features, int staleness) { + return TripWireRule.builder() + .code(code).label(label).labelHi(labelHi).expression(expression) + .thresholdText(thresholdText).severity(severity) + .requiredFeatures(features).maxStalenessDays(staleness) + .active(true).version(1).updatedBy("seed").updatedAt(LocalDateTime.now()) + .build(); + } + + private void seedOfficers() { + if (userRepository.findByUsernameIgnoreCase("aditya.vyas").isPresent()) return; + String blocks = "Sehore,Vidisha,Raisen,Bhopal"; + userRepository.saveAll(List.of( + UdyamUser.builder().username("aditya.vyas").fullName("Aditya Vyas") + .password(passwordEncoder.encode(DEFAULT_OFFICER_PASSWORD)) + .role("FIELD_OFFICER").scope("OFFICER").blocks(blocks) + .branch("Sehore & Vidisha blocks").active(true).build(), + UdyamUser.builder().username("supervisor").fullName("Branch Supervisor") + .password(passwordEncoder.encode(DEFAULT_OFFICER_PASSWORD)) + .role("SUPERVISOR").scope("OFFICER").blocks(blocks) + .branch("Bhopal region").active(true).build())); + log.info("Seeded officer logins (password {})", DEFAULT_OFFICER_PASSWORD); + } + + // ── portfolio ─────────────────────────────────────────── + + /** The six enterprises printed on the deck's wireframe slide. */ + private record Pinned(String id, String name, String nameHi, String sector, String block, int score) { + } + + private static final List PINNED = List.of( + new Pinned("MSME-4417", "Sehore Poultry Farm", "सीहोर पोल्ट्री फ़ार्म", "POULTRY", "Sehore", 79), + new Pinned("MSME-3902", "Sharda Dairy Collective", "शारदा डेयरी समूह", "DAIRY", "Vidisha", 71), + new Pinned("MSME-5120", "Anand Food Products", "आनंद फ़ूड प्रोडक्ट्स", "FOOD_PROC", "Raisen", 58), + new Pinned("MSME-2861", "Gramin Kirana Store", "ग्रामीण किराना स्टोर", "RURAL_RETAIL", "Sehore", 41), + new Pinned("MSME-6033", "Meera Handicrafts", "मीरा हस्तशिल्प", "HANDICRAFTS", "Bhopal", 22), + new Pinned("MSME-4488", "Krishna Dairy Unit", "कृष्णा डेयरी यूनिट", "DAIRY", "Vidisha", 18)); + + /** Slide-6 driver weights for the headline enterprise. */ + private static final Map PINNED_SUBSCORES = + Map.of("MSME-4417", new int[]{26, 19, 14, 11, 9}); + + /** Deck figure: a ₹12,400 gap three months out for MSME-4417. */ + private static final int PINNED_BREACH_HORIZON = 3; + private static final BigDecimal PINNED_SHORTFALL = new BigDecimal("12400"); + + private void seedPortfolio() { + if (enterpriseRepository.count() > 0) return; + + Map sectors = new HashMap<>(); + sectorRepository.findAll().forEach(s -> sectors.put(s.getCode(), s)); + Map districts = new HashMap<>(); + districtRepository.findAll().forEach(d -> districts.put(d.getName(), d)); + + List bandPlan = buildBandPlan(); + List sectorCodes = List.of("DAIRY", "POULTRY", "FOOD_PROC", "HANDICRAFTS", "RURAL_RETAIL"); + List blocks = List.of("Sehore", "Vidisha", "Raisen", "Bhopal"); + + for (Pinned p : PINNED) { + save(build(p.id(), p.name(), p.nameHi(), p.sector(), p.block(), p.score(), sectors, districts)); + } + + for (int i = 0; i < TOTAL - PINNED.size(); i++) { + String enterpriseId = "MSME-" + (1000 + ((i * 37 + 113) % 8000)); + Random r = seeded(enterpriseId); + String sectorCode = sectorCodes.get(r.nextInt(sectorCodes.size())); + String block = blocks.get(r.nextInt(blocks.size())); + String band = bandPlan.get(i); + int score = scoreForBand(r, band); + int n = r.nextInt(SeedText.PREFIX.length); + List suffix = SeedText.SUFFIX.get(sectorCode); + List suffixHi = SeedText.SUFFIX_HI.get(sectorCode); + int s = r.nextInt(suffix.size()); + save(build(enterpriseId, + SeedText.PREFIX[n] + " " + suffix.get(s), + SeedText.PREFIX_HI[n] + " " + suffixHi.get(s), + sectorCode, block, score, sectors, districts)); + } + } + + /** + * A band assignment that always totals 9 / 31 / 102 once the pinned six are + * accounted for. Shuffled with a fixed seed so the mix is stable across boots. + */ + private List buildBandPlan() { + Map pinnedCounts = new HashMap<>(); + for (Pinned p : PINNED) { + pinnedCounts.merge(bandOf(p.score()), 1, Integer::sum); + } + List plan = new ArrayList<>(); + TARGET.forEach((band, target) -> { + int remaining = target - pinnedCounts.getOrDefault(band, 0); + for (int i = 0; i < remaining; i++) plan.add(band); + }); + Collections.shuffle(plan, new Random(20260804L)); + return plan; + } + + private Enterprise build(String enterpriseId, String name, String nameHi, String sectorCode, String block, + int score, Map sectors, Map districts) { + Random r = seeded(enterpriseId); + District d = districts.get(block); + return Enterprise.builder() + .enterpriseId(enterpriseId) + .name(name) + .nameHi(nameHi) + .sectorId(sectors.get(sectorCode).getId()) + .districtId(d.getId()) + .block(block) + // Jittered around the district centroid — a premises location, not a person. + .latitude(d.getLatitude() + (r.nextDouble() - 0.5) * 0.25) + .longitude(d.getLongitude() + (r.nextDouble() - 0.5) * 0.25) + .marketId(marketRepository.findFirstByDistrictIdAndPrimaryMarketTrue(d.getId()) + .map(Market::getId).orElse(null)) + .ownerCount(1 + r.nextInt(4)) + .vintageMonths(8 + r.nextInt(112)) + .loanOutstanding(BigDecimal.valueOf((4 + r.nextInt(37)) * 10_000L)) + .monthlyInstalment(BigDecimal.valueOf((16 + r.nextInt(95)) * 250L)) + .score(score) + .band(bandOf(score)) + .consentActive(true) + .lastScoredAt(LocalDateTime.now().minusMinutes(12)) + .build(); + } + + /** Persist an enterprise together with its run, forecast, score, reasons and alert. */ + private void save(Enterprise e) { + Enterprise saved = enterpriseRepository.save(e); + String sectorCode = sectorRepository.findById(saved.getSectorId()).map(Sector::getCode).orElse("RURAL_RETAIL"); + + ScoringRun run = runRepository.save(ScoringRun.builder() + .enterpriseId(saved.getEnterpriseId()) + .engine("RULE") + .modelVersion(MODEL_VERSION) + .source("SERVER") + .asOfDate(LocalDate.now()) + .horizonMonths(horizon) + .enterpriseCount(1) + .status("COMPLETED") + .build()); + + List points = buildForecast(saved, run.getId()); + pointRepository.saveAll(points); + + int[] subs = subScores(saved, sectorCode); + scoreRepository.save(StressScore.builder() + .runId(run.getId()) + .enterpriseId(saved.getEnterpriseId()) + .score(saved.getScore()) + .band(saved.getBand()) + .engine("RULE") + .confidence(62) + .liquidity(subs[0]).behavioural(subs[1]).trendVolatility(subs[2]) + .obligationLoad(subs[3]).externalShock(subs[4]) + .build()); + + List reasons = buildReasons(saved, sectorCode, subs, run.getId()); + reasonRepository.saveAll(reasons); + + seedLedger(saved); + seedLoans(saved); + + if (!"STANDARD".equals(saved.getBand())) { + seedAlertAndVisit(saved, run.getId(), points, reasons.get(0)); + } + } + + /** + * The institutional side of the record — what a nightly CBS extract would + * carry. DPD tracks the band, because in the demo the band is the ground + * truth; in production the causality runs the other way. + */ + private void seedLoans(Enterprise e) { + if (loanRecordRepository.countByEnterpriseId(e.getEnterpriseId()) > 0) return; + + Random r = seeded(e.getEnterpriseId() + ":loan"); + int dpd = switch (e.getBand()) { + case "RED" -> 31 + r.nextInt(60); + case "WATCH" -> 4 + r.nextInt(28); + default -> r.nextInt(4); + }; + LocalDate asOf = LocalDate.now().withDayOfMonth(1).minusDays(1); + + loanRecordRepository.save(LoanRecord.builder() + .accountRef(e.getEnterpriseId() + "-LN1") + .enterpriseId(e.getEnterpriseId()) + .product(r.nextBoolean() ? "TERM_LOAN" : "WORKING_CAPITAL") + .sanctionedAmount(e.getLoanOutstanding().multiply(BigDecimal.valueOf(1.4)) + .setScale(2, RoundingMode.HALF_UP)) + .outstandingAmount(e.getLoanOutstanding()) + .instalmentAmount(e.getMonthlyInstalment()) + .nextDueDate(LocalDate.now().withDayOfMonth(Math.min(28, 5 + r.nextInt(20))).plusMonths(1)) + .lastPaidDate(LocalDate.now().minusDays(dpd + 1L)) + .dpd(dpd) + .bucket(bucketOf(dpd)) + .closed(false) + .asOfDate(asOf) + .build()); + } + + private String bucketOf(int dpd) { + if (dpd == 0) return "0"; + if (dpd <= 30) return "1-30"; + if (dpd <= 60) return "31-60"; + if (dpd <= 90) return "61-90"; + return "90+"; + } + + /** + * Six months of P10 / P50 / P90 against the obligation line. P50 drifts by + * band; the fan widens with horizon because uncertainty compounds. + */ + private List buildForecast(Enterprise e, Long runId) { + Random r = seeded(e.getEnterpriseId() + ":fc"); + double base = 18_000 + (100 - e.getScore()) * 420.0 + (r.nextInt(4500) - 2000); + double drift = switch (e.getBand()) { + case "RED" -> -0.14; + case "WATCH" -> -0.05; + default -> 0.035; + }; + double fixed = e.getMonthlyInstalment().doubleValue() + (8 + r.nextInt(15)) * 400.0; + boolean pin = PINNED_SUBSCORES.containsKey(e.getEnterpriseId()); + + List points = new ArrayList<>(); + LocalDate month = LocalDate.now().withDayOfMonth(1); + + for (int h = 1; h <= horizon; h++) { + double centre = base * (1 + drift * h) + (r.nextDouble() - 0.5) * 1800; + double spread = centre * (0.16 + (h - 1) * 0.045) + 900; + double obligation = fixed + (h - 1) * 120; + double p10 = centre - spread; + double p50 = centre; + double p90 = centre + spread * 0.85; + + // Pin the deck's headline enterprise to a ₹12,400 gap in month three, + // and keep the earlier months clear so the first breach lands there. + if (pin) { + if (h == PINNED_BREACH_HORIZON) { + p10 = obligation - PINNED_SHORTFALL.doubleValue(); + p50 = Math.max(p50, p10 + 4_000); + p90 = Math.max(p90, p50 + 5_000); + } else if (h < PINNED_BREACH_HORIZON) { + p10 = Math.max(p10, obligation + 1_500); + p50 = Math.max(p50, p10 + 3_000); + p90 = Math.max(p90, p50 + 4_000); + } + } + + LocalDate m = month.plusMonths(h - 1L); + points.add(ForecastPoint.builder() + .runId(runId) + .enterpriseId(e.getEnterpriseId()) + .horizon(h) + .month(String.format("%d-%02d", m.getYear(), m.getMonthValue())) + .p10(money(p10)).p50(money(p50)).p90(money(p90)) + .obligation(money(obligation)) + .breach(p10 < obligation) + .build()); + } + return points; + } + + /** Split the composite across the five families, weighted by the sector pack. */ + private int[] subScores(Enterprise e, String sectorCode) { + int[] pinned = PINNED_SUBSCORES.get(e.getEnterpriseId()); + if (pinned != null) return pinned; + + Sector s = sectorRepository.findByCode(sectorCode).orElseThrow(); + Random r = seeded(e.getEnterpriseId() + ":sub"); + int[] weights = {s.getWLiquidity(), s.getWBehavioural(), s.getWTrendVolatility(), + s.getWObligationLoad(), s.getWExternalShock()}; + int[] out = new int[5]; + for (int i = 0; i < 5; i++) { + out[i] = (int) Math.round(e.getScore() * (weights[i] / 10_000.0) * (0.8 + r.nextDouble() * 0.5)); + } + return out; + } + + /** Top-three drivers as sentences — the stand-in for SHAP output. */ + private List buildReasons(Enterprise e, String sectorCode, int[] subs, Long runId) { + Map text = SeedText.REASONS.get(sectorCode); + List all = new ArrayList<>(); + for (int i = 0; i < SeedText.DRIVERS.length; i++) { + String key = SeedText.DRIVERS[i][0]; + String[] sentence = text.get(key); + all.add(ReasonCode.builder() + .runId(runId) + .enterpriseId(e.getEnterpriseId()) + .driver(SeedText.DRIVERS[i][1]) + .driverHi(SeedText.DRIVERS[i][2]) + .weight(subs[i]) + .sentenceEn(sentence[0]) + .sentenceHi(sentence[1]) + .build()); + } + all.sort((a, b) -> b.getWeight() - a.getWeight()); + List top = all.subList(0, 3); + for (int i = 0; i < top.size(); i++) top.get(i).setRank(i + 1); + return top; + } + + private void seedAlertAndVisit(Enterprise e, Long runId, List points, ReasonCode top) { + ForecastPoint breach = points.stream().filter(p -> Boolean.TRUE.equals(p.getBreach())).findFirst().orElse(null); + Random r = seeded(e.getEnterpriseId() + ":alert"); + + String monthEn = null; + String monthHi = null; + if (breach != null) { + int m = Integer.parseInt(breach.getMonth().substring(5)) - 1; + monthEn = SeedText.MONTHS_EN[m]; + monthHi = SeedText.MONTHS_HI[m]; + } + + RiskAlert alert = alertRepository.save(RiskAlert.builder() + .alertId("ALT-" + (4200 + alertRepository.count())) + .enterpriseId(e.getEnterpriseId()) + .runId(runId) + .band(e.getBand()) + .score(e.getScore()) + .origin(e.getScore() >= 75 ? "TRIPWIRE" : "MODEL") + .title(breach == null ? "Stress score above watch threshold" + : "Cash shortfall projected by " + monthEn) + .titleHi(breach == null ? "जोखिम स्कोर निगरानी सीमा से ऊपर" + : monthHi + " तक पैसा कम पड़ सकता है") + .reasonEn(top.getSentenceEn()) + .reasonHi(top.getSentenceHi()) + .shortfall(breach == null ? BigDecimal.ZERO : breach.getObligation().subtract(breach.getP10())) + .breachMonth(breach == null ? null : breach.getMonth()) + .state("RED".equals(e.getBand()) ? "OPEN" : (r.nextBoolean() ? "OPEN" : "ACKNOWLEDGED")) + .raisedAt(LocalDateTime.now().minusDays(r.nextInt(21))) + .build()); + + List options = PlaybookCatalog.forBand(e.getBand()); + PlaybookCatalog.Action action = options.isEmpty() + ? PlaybookCatalog.ACTIONS.get(0) + : options.get(r.nextInt(options.size())); + + visitRepository.save(FieldVisit.builder() + .visitId("FV-" + (1800 + visitRepository.count())) + .enterpriseId(e.getEnterpriseId()) + .enterpriseName(e.getName()) + .block(e.getBlock()) + .actionCode(action.code()) + .action(action.label()) + .actionHi(action.labelHi()) + .alertId(alert.getId()) + .priority("RED".equals(e.getBand()) ? 1 : 2) + .dueDate(LocalDate.now().plusDays(r.nextInt(20) - 6L)) + .state("PLANNED") + .officer("Aditya Vyas") + .build()); + } + + /** + * Thirty days of ledger history so the app is never empty, plus an app login + * for the enterprise. Only for the pinned six — generating 142 × ~60 entries + * would slow first boot for no demo value. + */ + private void seedLedger(Enterprise e) { + boolean pinned = PINNED.stream().anyMatch(p -> p.id().equals(e.getEnterpriseId())); + if (!pinned) return; + + if (userRepository.findByEnterpriseIdAndScopeIgnoreCase(e.getEnterpriseId(), "ENTERPRISE").isEmpty()) { + userRepository.save(UdyamUser.builder() + .username(e.getEnterpriseId().toLowerCase()) + .fullName(e.getName()) + .password(passwordEncoder.encode(DEFAULT_ENTERPRISE_PIN)) + .role("OWNER").scope("ENTERPRISE").enterpriseId(e.getEnterpriseId()) + .active(true).build()); + } + + Random r = seeded(e.getEnterpriseId() + ":ledger"); + String[] inCats = {"SALE", "UPI", "RECOVERY"}; + String[] outCats = {"INPUT", "WAGES", "TRANSPORT", "EMI"}; + List entries = new ArrayList<>(); + + for (int d = 30; d >= 0; d--) { + int count = 1 + r.nextInt(3); + for (int i = 0; i < count; i++) { + boolean in = r.nextDouble() > 0.45; + entries.add(LedgerEntry.builder() + .clientId("seed-" + e.getEnterpriseId() + "-" + d + "-" + i) + .enterpriseId(e.getEnterpriseId()) + .kind(in ? "IN" : "OUT") + .amount(BigDecimal.valueOf((in ? 6 + r.nextInt(47) : 4 + r.nextInt(35)) * 100L)) + .category(in ? inCats[r.nextInt(inCats.length)] : outCats[r.nextInt(outCats.length)]) + .entryDate(LocalDate.now().minusDays(d)) + .syncedAt(LocalDateTime.now()) + .build()); + } + } + ledgerRepository.saveAll(entries); + } + + // ── helpers ───────────────────────────────────────────── + + private String bandOf(int score) { + if (score >= redThreshold) return "RED"; + if (score >= watchThreshold) return "WATCH"; + return "STANDARD"; + } + + /** + * Generated scores stay below the headline enterprise's 79, so Sehore Poultry + * Farm leads the dashboard exactly as it does on the deck's wireframe. + */ + private int scoreForBand(Random r, String band) { + return switch (band) { + case "RED" -> 66 + r.nextInt(13); + case "WATCH" -> 36 + r.nextInt(29); + default -> 8 + r.nextInt(27); + }; + } + + /** Deterministic per-enterprise randomness, so a demo shows the same numbers every run. */ + private Random seeded(String key) { + return new Random(key.hashCode()); + } + + private BigDecimal money(double v) { + return BigDecimal.valueOf(v).setScale(2, RoundingMode.HALF_UP); + } +} diff --git a/src/main/resources/application.properties b/src/main/resources/application.properties new file mode 100644 index 0000000..669d02c --- /dev/null +++ b/src/main/resources/application.properties @@ -0,0 +1,101 @@ +spring.application.name=udyamlinc-backend + +server.port=8082 +server.servlet.context-path=/api/v1/udyam + +# JPA & Hibernate +spring.jpa.hibernate.ddl-auto=update +spring.jpa.properties.hibernate.default_schema=public +spring.jpa.open-in-view=false + +# DataSource — UdyamLinc_DEMO +spring.datasource.url=${UDYAM_DB_URL:jdbc:postgresql://localhost:5432/UdyamLinc_DEMO} +spring.datasource.username=${UDYAM_DB_USERNAME:postgres} +spring.datasource.password=${UDYAM_DB_PASSWORD:root} +spring.datasource.driver-class-name=org.postgresql.Driver +spring.datasource.type=com.zaxxer.hikari.HikariDataSource + +# HikariCP +spring.datasource.hikari.connection-timeout=10000 +spring.datasource.hikari.minimum-idle=5 +spring.datasource.hikari.maximum-pool-size=20 +spring.datasource.hikari.idle-timeout=600000 +spring.datasource.hikari.max-lifetime=1800000 +spring.datasource.hikari.auto-commit=true + +# throw NoHandlerFoundException for 404s +spring.mvc.throw-exception-if-no-handler-found=true + +jwt.secret=55647961_6D4C_696E_6320_4E41_4241_5244_5F32_3032_365F_4C69_6E63_4675_7369_6F6E_5F48_6163_6B61_74686F6E +# 24 hours +jwt.expiration=86400000 + +# The Vite dev server for UdyamLinc-UI (5180 is BharatLinc) +udyam.cors.allowed-origins=http://localhost:5182,http://127.0.0.1:5182,http://localhost:4173 + +# Forecast horizon in months — the deck's 3–6 month early-warning window +udyam.forecast.horizon-months=6 + +# Composite Stress Score band thresholds; map these to the institution's watchlist +udyam.risk.red-threshold=65 +udyam.risk.watch-threshold=35 + +# ── Risk & rule engine (ARCHITECTURE Phase D) ─────────────── +# Which ScoringStrategy binds. RULE today; MODEL when the LightGBM heads land — +# that swap is this line, not a rewrite. +udyam.risk.engine=RULE +udyam.risk.model-version=rule-v1.0.0 +# Where a family lands with no usable feature. Neutral, not innocent: scoring 0 +# would read as "safe on this dimension" when the truth is that we do not know. +udyam.risk.family-prior=40 +# A rule engine reports low confidence by design. +udyam.risk.rule-confidence-ceiling=70 +# Nightly scoring, an hour behind ingestion so it scores what that job wrote. +udyam.risk.scheduled=${UDYAM_SCORING_ENABLED:false} +udyam.risk.cron=0 0 3 * * * + +# Seed the demo portfolio (142 enterprises) on first boot +udyam.seed.enabled=true + +# ── Ingestion (ARCHITECTURE Phase B) ──────────────────────── +# Off by default: a demo boot must never call a government endpoint. Turn on for +# the pilot, or trigger a one-off pass with POST /admin/ingestion/run. +udyam.ingestion.enabled=${UDYAM_INGESTION_ENABLED:false} +udyam.ingestion.cron=0 0 2 * * * +udyam.ingestion.zone=Asia/Kolkata +udyam.ingestion.http.connect-timeout-ms=10000 +udyam.ingestion.http.read-timeout-ms=30000 + +# Agmarknet mandi prices — data.gov.in Open Government Data API. +# Register at https://data.gov.in for a key; without one the run records SKIPPED. +udyam.ingestion.datagov.api-key=${DATAGOV_API_KEY:} +udyam.ingestion.datagov.base-url=https://api.data.gov.in/resource +# "Variety-wise Daily Market Prices Data of Commodity" (Agmarknet) +udyam.ingestion.datagov.resource-id=${DATAGOV_RESOURCE_ID:9ef84268-d588-465a-a308-a864a43d0070} +udyam.ingestion.datagov.page-limit=1000 +# Filter names must match the resource's own field names; they differ by resource version. +udyam.ingestion.datagov.field.state=state +udyam.ingestion.datagov.field.district=district +udyam.ingestion.datagov.field.commodity=commodity +udyam.ingestion.mandi.lookback-weeks=12 + +# Climate — Open-Meteo reanalysis archive. No key. The archive trails real time, +# hence the lag; the baseline is fetched once per district and cached as normals. +udyam.ingestion.climate.base-url=https://archive-api.open-meteo.com/v1/archive +udyam.ingestion.climate.timezone=Asia/Kolkata +udyam.ingestion.climate.lookback-weeks=12 +udyam.ingestion.climate.lag-days=5 +udyam.ingestion.climate.heat-threshold-c=40 +udyam.ingestion.climate.baseline-from=1995-01-01 +udyam.ingestion.climate.baseline-to=2024-12-31 + +# ── Feature store (ARCHITECTURE Phase C) ──────────────────── +# Window sizes for the derivations. A level is not a feature; every one of these +# turns a level into a change, a ratio or an anomaly against one of these windows. +udyam.features.price-window-weeks=8 +udyam.features.recent-window-weeks=4 +udyam.features.climate-window-weeks=8 +# Below this many same-season observations a z-score is noise, so it stays null. +udyam.features.min-zscore-samples=6 + +logging.level.com.lincfusion.udyamlinc=INFO