Neighborhood Smoothing for Calibration

Headline results

• NLL improvement

  8.6%

  Lower negative log-likelihood than cross-entropy on CIFAR-100 with ResNet-50 (lower is better)

• Best after temperature scaling

  7 of 8

  Lowest NLL among all evaluated training methods in seven of eight image and tabular settings

• Better than cross-entropy

  11 of 11

  Lower test NLL in every main dataset and architecture setting

• Extra inference cost

  None

  The neighbour graph is used only during training

The problem

A classifier is calibrated when its confidence matches how often it is right: of all the predictions it makes with 80% confidence, about 80% should be correct. Modern deep networks tend to be overconfident, which matters wherever a predicted probability feeds a decision.

Training-time fixes such as focal loss, label smoothing, MMCE and MDCA adjust each prediction, or a batch of confidences, on its own. None of them utilize where the examples sit in the network's representation space. NeSCal is developed from the idea that examples which the network already represents as similar should receive similar predictive distributions, and that enforcing this improves calibration.

How NeSCal works

• Warm up. Train with cross-entropy as usual for an initial period; when to switch on the smoothing is a tuned hyperparameter.
• Build a neighbour graph. Embed the training set with the network itself and connect each example to its k nearest neighbours by cosine similarity. The graph is built once.
• Smooth the predictions. Add a penalty: the Jensen-Shannon divergence between each example's predicted distribution and its neighbours'. Its weight can stay constant or decay on a cosine schedule.
• Keep it cheap. Neighbours' predictions are cached and refreshed as each batch is seen, so the penalty costs a lookup rather than extra forward passes.
• Nothing changes at inference. The graph and the penalty exist only during training, so the deployed model has the same size and speed as one trained with cross-entropy.

What I did

• Designed NeSCal and built it in PyTorch as a drop-in regulariser that sits next to cross-entropy in any training loop: ResNet-50 on images, MLPs on tabular data, text classifiers, and frozen pretrained encoders with optional fine-tuning.
• Proved why it works. Derived theoretical calibration-error bounds, and showed that the smoothing penalty limits how far confidence can differ between neighbouring examples, and that locally it behaves like a graph Laplacian under the Fisher information metric. Wrote diagnostics that check each inequality on trained models.
• Benchmarked it against the field: six training-time methods (label smoothing, focal loss, MMCE, MDCA, S-AvUC and MbLS) and five post-hoc calibrators, starting with temperature scaling, across 11 image, text and tabular settings. Hyperparameters were chosen on search seeds and the results confirmed on separate seeds.
• Stress-tested it: corrupted images at increasing severity, out-of-distribution inputs, perturbations, ablations, Wilcoxon signed-rank tests across settings, and a random-graph control that keeps the penalty but scrambles the neighbours, to test whether the gain comes from the representation space.
• Got the result: lower NLL than cross-entropy in all 11 settings and the best NLL after temperature scaling in seven of eight image and tabular settings. On CIFAR-100 with ResNet-50, NLL fell by 8.6% and accuracy rose by 1.90 percentage points.

Built to deploy

• Zero serving cost. The trained model is an ordinary classifier: no extra parameters, memory or latency in production.
• Model-agnostic. The same loss trained convolutional, MLP and text architectures without changes to the models.
• Scales to real datasets. Neighbour search runs in chunks on normalised embeddings, with optional PCA or UMAP reduction before the graph is built.
• Reproducible end to end. One command runs the sweeps, post-hoc calibration and reporting from a JSON config, with a dry-run mode and unit tests for the orchestration.

Publication

“Neighborhood Smoothing for Calibration.” Horowitz, I. & Gal, A. (Under review, ICLR 2027). arXiv:2610.09020.