KL(P||Q) = E_P[log P/Q]. Non-negative; zero iff P=Q. Foundation of statistical-inference + variational-methods + ML. Pinsker inequality TV ≤ √(KL/2). Cross-listed with L4 free-energy.
KL(P||Q) = E_P[log P/Q]. Non-negative; zero iff P=Q. Foundation of statistical-inference + variational-methods + ML. Pinsker inequality TV ≤ √(KL/2). Cross-listed with L4 free-energy.