Insights Model Methodology

Data Freshness and Model Performance: Why Stale Alternative Data Hurts More Than No Data

Priya Nair 10 min read
Abstract visualization of data signal decay over time affecting model performance

There is an intuition in alternative data credit scoring that more data is always better. If a lender can pull a borrower's 90-day mobile wallet history at origination, why not also pull the history from 6 months ago? More observations, better model. This intuition is wrong in a specific way that matters for live lending operations, and the cost of getting it wrong tends to show up in batch approvals rather than individual decisions, which makes it hard to catch.

The problem is not data volume. It's that old alternative data doesn't just lose predictive power. In a behavioral credit model, stale data actively misleads. It encodes a behavioral state the borrower no longer occupies. We measured how fast different feature types decay, and the results shaped how we handle data freshness in Panthera's scoring engine.

Why Alternative Data Ages Differently Than Bureau Data

A bureau record has a known staleness by design. A missed payment from 36 months ago is explicitly dated, and scoring models have been tuned over decades to handle the predictive decay of historical credit events at different time horizons. An applicant with one missed payment 36 months ago is different from one with a missed payment 6 months ago. Bureau models encode this relationship.

Alternative behavioral data doesn't have this built-in temporal labeling in the same way. A mobile wallet transaction log from 60 days ago and one from 7 days ago look structurally identical. The feature extraction logic has to explicitly decide: how much do we weight recent data versus older data? And for thin-file borrowers whose behavioral patterns can shift significantly over a 2-3 month window, the weighting decision has a meaningful effect on score quality.

Bureau records also capture events that don't reverse. A formal default is a permanent mark. Behavioral alternative data, in contrast, captures a continuous flow of behavior that can change. A borrower who was financially stressed 90 days ago may be in a much better position today. Or they may have deteriorated further. The data from 90 days ago tells you which of those was true then, not now. The difference matters for making a credit decision today.

The Decay Curve by Feature Type

Not all alternative data features age at the same rate. When we ran analysis on validation cohorts with known 90-day outcomes, measuring feature predictive power across data freshness windows, the pattern that emerged was clear and consistent across the three Southeast Asian markets we examined.

Repayment velocity features decay fastest. Days-to-payment on informal obligations, cleared-on-time rate for micro-transactions: these features computed on data from the last 30 days were strongly predictive of 90-day loan default in our validation set. The same features computed on data from 60-90 days prior showed substantially weaker predictive power. Features computed on data 90-plus days old were in some cases negatively predictive relative to using no velocity data at all, because borrowers who were stressed 90 days ago and have since recovered were being scored as high-risk based on stale behavioral state.

Income regularity features are more durable. The coefficient of variation of monthly income inflows, measured over the last 90 days, shows meaningful predictive decay but more slowly than velocity features. This makes sense: income regularity reflects a structural characteristic (who this person works for, how they're paid) that doesn't change as fast as repayment behavior. Data from 90 days ago is less predictive than data from 30 days ago, but not dramatically so.

Account-level stability features are the most durable. Length of continuous wallet activity, address stability proxies, and platform-account age are essentially non-decaying on 90-day timescales. These features are valuable precisely because they reflect accumulated history, and yesterday's data and last month's data tell you the same thing about account tenure.

What "Stale Data Hurts More Than No Data" Actually Means

The claim needs to be stated precisely. We are not saying that all historical alternative data should be excluded. We're saying that for specific high-decay feature families, particularly repayment velocity, feeding data from beyond a certain staleness threshold produces worse model outputs than falling back to a missing-data handling strategy.

The mechanism: high-velocity features computed on stale data encode a behavioral state that has likely changed. The model, trained on fresh data, interprets these features as meaningful signals about current behavior. A score built on stale velocity features may be confidently wrong rather than appropriately uncertain. A well-calibrated missing-data handling approach, for instance reducing the coverage score and widening the confidence interval on the output, produces a less precise but more honest score.

The threshold we've found in practice for repayment velocity features: data more than 45 days old should be either down-weighted heavily or excluded from velocity feature computation. For income regularity features, the threshold is closer to 90 days. Account stability features can use data from 12 months back without meaningful decay.

These thresholds are approximate and market-specific. We see slightly faster decay in Indonesian data compared to Vietnam, likely because the mobile payment ecosystem in Indonesia sees more platform switching behavior, meaning a period of low wallet activity may reflect platform migration rather than financial stress. Applying a Vietnam-calibrated freshness threshold to Indonesia data produces systematic errors in a predictable direction.

Operational Implications for Loan Origination

The freshness question has a practical dimension at loan origination. Most lenders pull alternative data at the moment of application, which means they're getting fresh data for that moment but have no guarantee that the data is representative of the borrower's typical state. A borrower who applied during an unusually stable month looks different from one who applied during a period of stress. The 30-day pre-application window is the right data window for velocity features, but a single-window snapshot has variance.

One approach that reduces snapshot variance is to compute velocity features over multiple windows explicitly: 0-14 days, 15-30 days, 31-60 days, with explicit downweighting for older windows. This lets the model distinguish between borrowers with consistently strong velocity patterns across all windows versus borrowers who look good in the last 30 days but poorly in the 31-60 day window. The latter is a more nuanced risk signal, and it's invisible to a model that only computes a single aggregated velocity feature.

This multiple-window approach does require that the data pipeline pulls enough history at origination to populate all three windows. The latency and data-access cost of pulling 60 days of mobile wallet history versus 30 days is marginal. The improvement in feature quality for detecting borrowers with recently-changed behavioral patterns is worth it.

Implications for Model Validation and Monitoring

Data freshness is an often-overlooked variable in model validation. When validating a new model version, the validation dataset should include realistic variation in data freshness, not just variation in borrower characteristics. A model validated only on cases where the application data is fresh will be over-optimistic about performance in production, where a proportion of applications will present with data that has gaps or delays in the data-access pipeline.

For monitoring deployed models, adding data-freshness checks to the monitoring cadence matters. If the average age of velocity feature data in production shifts (for example, because a mobile wallet provider changed their data API response and records take longer to populate), this can cause systematic score drift without any change in borrower behavior or model parameters. PSI on data freshness metrics is a useful complement to PSI on input features themselves.

The broader principle here connects to something we think about constantly in building this product: alternative data for thin-file credit scoring is not just about what data exists, but about the temporal state of that data at the moment of use. The infrastructure connecting borrower behavioral data to credit decisions needs to be built with freshness management as a first-class concern, not an afterthought. Getting this wrong is subtle and expensive in ways that are hard to detect without purpose-built monitoring.