Why one-shot neural network weights lost to a trained model on 22,000 payment promises

A payment promise is a buyer telling a collector when they will pay. In our sample, a third of them were broken, and the collector found out on the due date. That is the worst day to find out. A score at the moment the promise is logged lets the team chase the fragile ones a week early and leave the solid ones alone.

Two things make the obvious answer hard. A trained model needs a few thousand outcomes before it is worth anything. A new client has none on day one (although, in reality, we have a Data Lake of billions of payment experiences, so this isn't our main concern, but for the exercise, let's assume it's a problem for most). And a score a collector cannot explain is a score they ignore. The X post promised a way around both. Knowledge written straight into weights, no labels, every weight a rule you can name and explain.

We pulled a sample from the Sidetrade Data Lake: 22,000 payment promises over ten months, each with the collector's note, the payment method, the number of invoices covered, the lead time, the buyer's earlier promises, and whether the money arrived. We built one small network, 5,000 note words plus 19 structured flags, 20 hidden units, one output, about 100,000 parameters, and we filled its weights two ways. Clearly, if we were doing this in production, the training volumes and accuracy we would expect would be much higher.

First way: train it on 16,000 labeled outcomes. Second way: Claude writes them. I described the fields, it profiled the data, and it set every non-zero weight in a chat, with no optimizer and no label fitted. Each hidden unit became a rule a senior collector would recognize. A remittance in the note pushes the score up. "Will try" pushes it down. Six invoices on one promise is a different animal from one. A buyer who kept eight of nine gets credit. Twenty-six rules, 169 non-zero weights, everything else zero. One afternoon.

Then we ran the field. Logistic regression on the structured fields. Logistic regression on word pairs from the notes. Gradient boosting. A frontier LLM deciding zero-shot. The same LLM as a note reader feeding the rulebook it had written. A 1.5-billion-parameter open model fine-tuned on a laptop. Fourteen models in all, every one scored on the same 5,342 held-out promises from the last three months, which none of them had seen.

Three numbers carry the table. The LLM-written weights scored 0.71 AUC with zero labels, level with a network trained on 2,000 outcomes. The logistic regression on word pairs scored 0.74 with 16,000, the best of the fourteen. The LLM deciding alone scored 0.61, ahead of only a model that had collapsed to guessing "kept" for everything.

And one number everyone shared. Every model that worked at all put the top fifth of promises at 84 to 88% kept and the bottom fifth at 32 to 40%. Fourteen approaches, 32 weights to 20,000, and the spread barely moved. The ceiling is in the data.

The LLM knows the direction. Training knows the size. Claude was right that a remittance is good and "will try" is bad, right about all twenty-six rules. Then it had to guess how much. It gave six invoices a push of 1.3 and a phone conversation 0.8. The trained model measured both on this sample's buyers. Twenty-six guesses cost 0.03 AUC, and no amount of collections knowledge gets it back, because the sizes belong to these buyers and not to collections in general. A citation is right or wrong, so a rule can be complete. A promise is a bet on a person, so a rule is a tendency with a number attached, and the number has to be learned.

The strongest signals are ones nobody would write down. The winning model was a logistic regression over 20,000 pairs of adjacent words from the notes. Its top predictors of a broken promise were "cld" and "tt", collector shorthand for called and talked to. Collectors write down the conversation when there is no payment evidence to write down. It found that the Spanish word for commitment predicts a break on Mexican accounts. An LLM writing rules from a description produces the rules a textbook contains. Sixteen thousand outcomes contain the rules this team's notes follow.

Zero labels were never zero. Claude's first weights, written before it looked at anything, scored 0.69. The weights it wrote after profiling the training data scored 0.71. The gain came from the same labels the trained model used, read as summary statistics. One-shot weights are a weak form of training by a reader with good priors. The strong form overtakes it once it has enough data, and in our sample, enough was about 2,000 outcomes.

It wrote a model that works on day one. A new client has no outcomes to train on; the 512-label network collapsed, and the LLM-written one scored 0.71 before a single promise had matured. It wrote a model a person can read: every score comes with the rules that fired, in words a collector acts on, and a collections lead can edit the rulebook as a JSON file with no release. It wrote a model that ties what the trained model could do with 2,000 labels, which is three months of waiting at a typical client.

It also read well. Asked twelve yes/no questions about each note and feeding the rulebook it had written, it matched keyword matching on English notes and handled French and Spanish ones the keywords missed. What it did badly was decide. Asked for a probability on its own, it gave a quarter of all promises the same score and treated one broken promise as near certainty. It writes rules well, reads well, and judges badly.

The score ships as three bands with the rules that fired. In the test period, at-risk promises were kept 39% of the time, uncertain 54%, likely 77%. That split is the product.