What does everyone get wrong about entropy, and why does it matter?: Information theorists

Shannon’s entropy — H = -sum of p log p — measures the average uncertainty in a message source. The less predictable a source, the higher its entropy. This is not a metaphor borrowed from physics. It is the same mathematics, applied to signals instead of heat. The shared equation describes steam engines and fiber-optic cables, heat death and data compression, and the fact that this works — that the formalism unifies across substrates — remains the most underrated intellectual achievement of the twentieth century.

The identity is not cosmetic. Landauer’s principle — erasing one bit of information releases a minimum of kT ln 2 joules of heat — proves that information is physical. Every time a computer clears a register, the universe gets slightly warmer by a precisely calculable amount. No clever engineering, no quantum tricks, no future technology eliminates it. Information is not like physical. Information is physical, and entropy is its fundamental constraint.

The physicists explain thermodynamic entropy with care and precision, and nearly everything they say is right. The divergence is emphasis. They treat Shannon entropy as a useful application of statistical mechanics. The deeper claim is that information is a physical quantity, like mass or charge. The second law does not merely say heat flows from hot to cold. It says you cannot acquire information about a system without a thermodynamic cost. You cannot erase information without a thermodynamic cost. The universe keeps accounts, and the currency is bits, and the exchange rate between bits and joules is fixed by the laws of physics.

Every system that processes information — a brain, a cell, a computer, a civilization — operates under entropic constraints. The efficiency limits of every communication channel Shannon characterized are entropy limits. Error-correction codes are entropy management. Compression algorithms are entropy calculations. DNA replication, neural computation, artificial intelligence — all information processing under entropic constraints. Entropy is not an abstraction about the far future. It is the operating manual for every system that has ever computed anything.

The real tragedy is institutional. Shannon’s unification is taught in electrical engineering departments and almost nowhere else. Biologists study thermodynamics without learning information theory. Physicists study statistical mechanics without learning coding theory. Computer scientists study algorithms without learning that their field has a physical foundation as rigid as any in physics. The deepest unification of the twentieth century sits in the gaps between departments, known to specialists and invisible to everyone else.

Where we concede ground: The explanatory reach of information theory has sometimes been oversold. Saying it’s all information can become a verbal tic that replaces explanation with relabeling. A cell is an information-processing system, yes — but calling it that does not explain embryogenesis, cancer, or aging any better than calling it a thermodynamic system explains those things. The formalism is powerful. The formalism is not omniscient. Shannon himself was uncomfortable with the proliferation of information-theoretic language into fields where the mathematical framework was not actually being applied. In a 1956 editorial called The Bandwagon, he warned against exactly this kind of enthusiasm. He was right to warn. The warning has not always been heeded.

What would change our mind: If someone discovered a physical system that processes information without any thermodynamic signature — a computation that leaves no heat, erases bits without energy cost, and violates Landauer’s principle at the macroscopic scale — the claim that information is physical in the strong sense would need revision. Alternatively, if a formal proof demonstrated that Shannon entropy and Boltzmann entropy, despite their mathematical identity, arise from fundamentally different principles with no deep connection — that the shared equation is coincidence rather than unity — the mistake would be mistaking elegance for truth. Both possibilities seem remote. But the discipline that formalized uncertainty should be honest about its own.

Read the full synthesis: What does everyone get wrong about entropy, and why does it matter?