feat(manual-entry): loan-book upload, with DPD history the schema could not hold

The CBS/LMS extract is the deepest gap in the system: emi_delay and
emi_to_inflow are computed from the loan book, and DPD history is the
training label without which the model cannot be trained at all. This is
how a real book gets in before a lender builds a nightly feed.

Two schema problems had to be fixed first.

loan_record had a unique index on accountRef alone, so the table could
hold only each loan's current state — every monthly extract overwriting
the evidence the model is meant to learn from. The key is now
(accountRef, asOfDate): one row per loan per extract. The old constraint
must be dropped by hand, as ddl-auto adds indexes but never removes them.

emi_delay and emi_to_inflow took the worst DPD across every row at or
before the scoring date, which with snapshots present would report a loan
sixty days down in March as sixty days down today. Both now read the
newest extract per accountRef. The history is there to train on, not to
be summarised as the present.

The no-PII promise is enforced at the parse boundary. A file whose headers
carry a name, phone, Aadhaar, PAN, account number or address is rejected
outright with the offending columns named, rather than having the column
quietly dropped — a bank must learn the data is unwanted before it builds
a nightly job that keeps sending it.

ManualEntryLine now stores the row as supplied, so commit reads the file's
own columns rather than a re-parse of a file nobody kept, and an auditor
sees what the bank sent rather than our reading of it.

Verified: a six-month, two-loan book uploads as 12 snapshots; a file with
customer_name and mobile_no is refused whole; and emi_delay reports 2 for
a loan that peaked at 3 and is now 2.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
LincPay
2026-08-06 15:05:44 +05:30
co-authored by Claude Opus 5
parent d710176f36
commit a18c2f335f
6 changed files with 421 additions and 16 deletions
@@ -17,6 +17,7 @@ import java.time.LocalDate;
import java.time.LocalDateTime;
import java.time.temporal.ChronoUnit;
import java.util.*;
import java.util.stream.Collectors;
/**
* Turns signals into indicators. The only writer of the feature store, and the
@@ -358,11 +359,28 @@ public class FeatureEngineeringService {
// ── institutional derivations ───────────────────────────
private Computed emiDelay(String enterpriseId, LocalDate asOfDate) {
List<LoanRecord> loans = loanRepository
.findByEnterpriseIdAndAsOfDateLessThanEqual(enterpriseId, asOfDate).stream()
.filter(l -> !Boolean.TRUE.equals(l.getClosed()))
/**
* The state of each open loan as of the scoring date — the newest extract per
* {@code accountRef}, never the union of every extract.
*
* <p>Once the book holds monthly snapshots, taking the worst DPD across all
* rows would report a loan that was sixty days down in March as sixty days
* down today. The history is there to train on, not to be summarised as the
* present.
*/
private List<LoanRecord> currentLoans(String enterpriseId, LocalDate asOfDate) {
return loanRepository.findByEnterpriseIdAndAsOfDateLessThanEqual(enterpriseId, asOfDate).stream()
.collect(Collectors.toMap(
LoanRecord::getAccountRef,
loan -> loan,
(a, b) -> a.getAsOfDate().isAfter(b.getAsOfDate()) ? a : b))
.values().stream()
.filter(loan -> !Boolean.TRUE.equals(loan.getClosed()))
.toList();
}
private Computed emiDelay(String enterpriseId, LocalDate asOfDate) {
List<LoanRecord> loans = currentLoans(enterpriseId, asOfDate);
if (loans.isEmpty()) return Computed.missing(FeatureCatalog.NO_CBS);
LocalDate newest = loans.stream().map(LoanRecord::getAsOfDate).max(LocalDate::compareTo).orElse(asOfDate);
@@ -392,10 +410,7 @@ public class FeatureEngineeringService {
* record of its trade, not an institutional estimate of it.
*/
private Computed emiToInflow(String enterpriseId, LocalDate asOfDate) {
List<LoanRecord> loans = loanRepository
.findByEnterpriseIdAndAsOfDateLessThanEqual(enterpriseId, asOfDate).stream()
.filter(l -> !Boolean.TRUE.equals(l.getClosed()))
.toList();
List<LoanRecord> loans = currentLoans(enterpriseId, asOfDate);
if (loans.isEmpty()) return Computed.missing(FeatureCatalog.NO_CBS);
BigDecimal instalment = loans.stream()