A fundamental insight connecting data compression and artificial intelligence

Compression is Prediction

Based on Hacker News Discussion

2025

02

The Core Insight

Understanding the deep connection between compression and prediction

What Does It Mean?

01

Compression predicts patterns Algorithms like ZIP and JPEG work by identifying and predicting repeating patterns in data
02

Better prediction = better compression The more accurately you can predict what comes next, the less data you need to store
03

A universal principle This insight applies to everything from file compression to language models

How Compression Uses Prediction

1

Analyze Data

Identify patterns and regularities in the input

→

2

Build Model

Create a predictive model based on those patterns

→

3

Encode Efficiently

Store only what the model cannot predict

→

4

Compress

Result: smaller file size through prediction

05

Real-World Connections

From everyday technology to cutting-edge AI

Applications Across Fields

folder_zip

File Compression

ZIP, PNG, MP3 all use prediction to remove redundancy from files

smart_toy

Language Models

LLMs are trained by predicting the next word — they're compression engines

analytics

Scientific Modeling

Good theories compress observations into elegant predictions

movie

Video Streaming

Netflix and YouTube predict frame changes to reduce bandwidth

psychology

Human Cognition

Our brains predict sensory input — perception is a form of compression

hub

Information Theory

Shannon's work established the mathematical foundation for this relationship

The best compression algorithm
is indistinguishable from
the best prediction algorithm.

Fundamental insight from information theory

Thank You

Compression and prediction: two sides of the same coin

https://ngrok.com/blog/compression-is-prediction