feat(manual-entry): bulk and single entry for the feeds no API serves

Egg, milk and fodder have no usable API for the pilot districts — NECC
publishes HTML, dairy rates exist only as annual state averages, and every
recent Agmarknet fodder row is one market in Ambala. This is how a bank
gets that data in.

Upload stages a PREVIEW batch and writes nothing. It reports what would
land, what was rejected and why, and which rows would overwrite an
existing value. Only commit touches market_signal.

Effective dating: effective_date is the date the fact describes and comes
from the file; observedAt is when we learned it and is stamped at commit.
The uploader cannot reach observedAt — a March rate uploaded in August
must stay invisible to a March scoring run, or the model trains on data
that did not exist yet.

Rejections name themselves: UNKNOWN_DISTRICT, UNKNOWN_COMMODITY,
FUTURE_DATE, BAD_DATE, BAD_NUMBER, UNIT_MISMATCH, PRICE_ORDER,
DUPLICATE_IN_FILE, MISSING_REQUIRED. A price far from last week warns but
is accepted — a genuine price shock is the event this system exists to
catch, and rejecting it would be backwards.

Rows are written with source=MANUAL, which the unique index keeps separate
from a feed's own row for the same week, so manual data never silently
replaces a government feed.

Also distinguishes INSUFFICIENT_HISTORY from NO_SIGNAL in the price
derivations. A newly onboarded manual feed spends its first weeks below
the four-week minimum, and reporting NO_SIGNAL there reads as a broken
integration rather than a series still filling up.

Result: margin_squeeze and output_price_change compute for the first time
— 57 of 142 enterprises, exactly the DAIRY (30) and POULTRY (27) sectors
that have an output commodity.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
LincPay
2026-08-06 14:12:45 +05:30
co-authored by Claude Opus 5
parent 20879b5904
commit 5401496a64
11 changed files with 943 additions and 1 deletions
@@ -0,0 +1,72 @@
package com.lincfusion.udyamlinc.manualentry;
import jakarta.persistence.*;
import lombok.*;
import java.time.LocalDateTime;
/**
* One upload. The audit anchor for every manually-sourced number in the system:
* a supervisor asking "where did Bhopal's March milk rate come from" gets a
* filename, a person and a timestamp, not a shrug.
*
* <p>A batch is staged as {@code PREVIEW} and writes nothing. Only
* {@code COMMITTED} has touched {@code market_signal}. Uploading four hundred
* rows and discovering the district column was misspelled afterwards is not a
* recoverable situation, so nothing lands until someone has seen what will.
*/
@Entity
@Table(name = "manual_entry_batch", indexes = {
@Index(name = "ix_manual_batch_status", columnList = "status"),
@Index(name = "ix_manual_batch_uploaded", columnList = "uploadedAt"),
})
@Getter
@Setter
@NoArgsConstructor
@AllArgsConstructor
@Builder
public class ManualEntryBatch {
@Id
@GeneratedValue(strategy = GenerationType.IDENTITY)
private Long id;
/** MARKET_SIGNAL | LOAN_RECORD — which template was uploaded. */
@Column(nullable = false, length = 32)
private String entryType;
@Column(length = 260)
private String filename;
/** The username from the JWT. Never a display name — this is an audit field. */
@Column(nullable = false, length = 64)
private String uploadedBy;
@Column(nullable = false)
private LocalDateTime uploadedAt;
/** PREVIEW | COMMITTED | DISCARDED. */
@Column(nullable = false, length = 16)
private String status;
private LocalDateTime committedAt;
@Column(length = 64)
private String committedBy;
private Integer rowsParsed;
private Integer rowsAccepted;
private Integer rowsRejected;
/** How many accepted rows would replace a value that already exists. */
private Integer rowsOverwriting;
@Column(length = 500)
private String note;
@PrePersist
void onCreate() {
if (uploadedAt == null) uploadedAt = LocalDateTime.now();
if (status == null) status = "PREVIEW";
}
}