LincPayandClaude Opus 5
20879b5904
First end-to-end run against Postgres and the real government APIs, which surfaced four defects that static checks could not. - Agmarknet pointed at 9ef84268-…, "Current Daily Price": a today-only snapshot holding one arrival_date, in which none of the four pilot districts appear. Switched to 35985678-…, the historical resource. - Filter fields were lowercase. This resource matches exactly on the capitalised name; the lowercase form is tokenised, so "Madhya Pradesh" also matched Andhra, Himachal and Uttar Pradesh — wrong-state prices written under a pilot district, with freshness looking healthy. - Rows arrive unordered and the lookback filter is client side, so a page was mostly years-old data and every commodity wrote zero weeks. Added sort[Arrival_Date]=desc. - ExternalFeedConfig injected a RestClient.Builder bean Boot does not expose, so the context never built and the app could not start. Also: Open-Meteo meters by request cost, so the 30-year baseline call exhausted the minutely allowance and 429'd two districts out of their climate data. Rate-limited calls now retry with backoff, while genuine failures still fail once and are reported. FODDER moves to MANUAL — Agmarknet carries Green Fodder, but every recent row is a single market in Ambala, Haryana. Result: 16,000 rows fetched, 202 market weeks and 52 climate weeks written, 0 failed calls; 142 scored, 33 trip-wires fired. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
44 lines
1.7 KiB
Java
44 lines
1.7 KiB
Java
package com.lincfusion.udyamlinc.config;
|
|
|
|
import org.springframework.beans.factory.annotation.Value;
|
|
import org.springframework.context.annotation.Bean;
|
|
import org.springframework.context.annotation.Configuration;
|
|
import org.springframework.http.client.SimpleClientHttpRequestFactory;
|
|
import org.springframework.scheduling.annotation.EnableScheduling;
|
|
import org.springframework.web.client.RestClient;
|
|
|
|
import java.time.Duration;
|
|
|
|
/**
|
|
* The HTTP client used for external feeds, and the scheduler that drives them.
|
|
*
|
|
* <p>Short, explicit timeouts are the point: a government endpoint that hangs
|
|
* must fail the nightly job quickly, not hold a thread until someone notices.
|
|
* Nothing in the request path uses this client — see ARCHITECTURE §2.1.
|
|
*/
|
|
@Configuration
|
|
@EnableScheduling
|
|
public class ExternalFeedConfig {
|
|
|
|
@Value("${udyam.ingestion.http.connect-timeout-ms:10000}")
|
|
private int connectTimeoutMs;
|
|
|
|
@Value("${udyam.ingestion.http.read-timeout-ms:30000}")
|
|
private int readTimeoutMs;
|
|
|
|
/**
|
|
* Built from {@link RestClient#builder()} rather than an injected
|
|
* auto-configured {@code RestClient.Builder}, which Boot does not always
|
|
* expose as a bean. It also keeps this client free of any global customiser
|
|
* meant for the request path — the feed client's timeouts are deliberately
|
|
* its own.
|
|
*/
|
|
@Bean("externalFeedClient")
|
|
public RestClient externalFeedClient() {
|
|
SimpleClientHttpRequestFactory factory = new SimpleClientHttpRequestFactory();
|
|
factory.setConnectTimeout(Duration.ofMillis(connectTimeoutMs));
|
|
factory.setReadTimeout(Duration.ofMillis(readTimeoutMs));
|
|
return RestClient.builder().requestFactory(factory).build();
|
|
}
|
|
}
|