Search MathAtlas
Explore modules, lessons, demo explanations, and narration. Follow connections between mathematical topics.
24 results for “Entropy” · Page 1 of 2
Loop Quantum Gravity · Lesson
… the ways a spin network can pierce a horizon and the Bekenstein-Hawking entropy comes back out
Information Theory · Lesson
Continuous entropy h(X) = −∫ f log f.
Information Theory · Lesson
Shannon Entropy | Information Theory | MathAtlas Shannon entropy H(X) = −Σ p log p.
Information Theory · Module overview
Shannon's theory of communication — entropy, mutual information, channel capacity, error correction
Machine Learning · Lesson
Entropy, KL divergence, and cross-entropy — why log-loss is the natural objective for classification.
Algorithms · Lesson
Understand fundamental limits: decision trees, entropy, and why comparison-based sorting requires Omega(n log n) comparisons
Information Theory · Lesson
The chain rule for entropy.
Information Theory · Lesson
Cross-entropy and the bridge to ML.
Information Theory · Lesson
Shannon's first theorem: entropy is the optimal compression bound.
String Theory · Lesson
Counting the Uncountable Black holes hold enormous entropy, and string theory was the first to count where it comes from.
Information Theory · Lesson
… bits Capacity C = 1 − H(p), where H is the binary entropy.
Loop Quantum Gravity · Lesson
… fixing it is to demand that counting horizon states reproduce the Bekenstein-Hawking entropy, which gives a number near 0.2375; that calculation is the subject of a later lesson, and both the value and the method of arriving at it have been disputed.
String Theory · Lesson
The clue came from black holes, whose maximum entropy scales with the area of the horizon, not the volume inside.
Loop Quantum Gravity · Module overview
… singularity turns into a bounce, and counting horizon punctures returns the Bekenstein-Hawking entropy.
Machine Learning · Module overview
… products in implicit Hilbert spaces, backprop as the chain rule, Bayesian updating, cross-entropy, equivariant convolutions, and the manifold hypothesis — all visualized.
Probability · Module overview
… spaces, Combinatorics for counting outcomes in finite probability, and Information Theory for entropy as a measure of uncertainty.
Information Theory · Lesson
… I need to recover the source exactly ?" The answer is the entropy H(X).
Loop Quantum Gravity · Lesson
The Barbero-Immirzi parameter is fixed by the black hole entropy calculation rather than predicted, which makes that celebrated result a calibration in part.
Information Theory · Lesson
source X with entropy H(X) and look at a very long sample sequence X₁, X₂, …, X_n.
Loop Quantum Gravity · Lesson
The observable universe began in a state of extraordinarily low entropy, and a bounce inherits that puzzle rather than solving it: something has to explain why the contracting phase arrived in such a special configuration.