一些推导

高斯分布的微分熵

$X sim mathcal{N}(mu, sigma^2)~$，$displaystyle f(x)=frac{1}{sqrt{2pisigma^2}}e^{-frac{(x-mu)^2}{2sigma^2}}$，其微分熵推导过程如下：

$$
begin{split}
h(X) &= int{-infty}^{infty}
frac{-1}{sqrt{2pisigma^2}}expleft(-frac{(x-mu)^2}{2sigma^2}right) cdot
ln left[
frac{1}{sqrt{2pisigma^2}}expleft(-frac{(x-mu)^2}{2sigma^2}right)
right] dx newline
&= frac{-1}{sqrt{2pisigma^2}} int{-infty}^{infty}
expleft(-frac{(x-mu)^2}{2sigma^2}right)
cdot
left(
ln(2pisigma^2)^{-1/2} - frac{(x-mu)^2}{2sigma^2}
right) dx newline
&= frac{1}{2}ln(2pisigma^2) int{-infty}^{infty}
frac{1}{sqrt{2pisigma^2}}expleft(-frac{(x-mu)^2}{2sigma^2}right) dx newline
&quad + int{-infty}^{infty}
frac{(x-mu)^2}{2sigma^2} cdot
frac{1}{sqrt{2pisigma^2}}expleft(-frac{(x-mu)^2}{2sigma^2}right) dx newline
&= frac{1}{2}ln(2pisigma^2) + frac{1}{2sigma^2}E(X-mu)^2 newline
&= frac{1}{2}ln(2pisigma^2) + frac{1}{2} = frac{1}{2}ln(2pi esigma^2) qquadtext{(nats)}
end{split}
$$

又因为 $H_b(X) = log_ba cdot H_a(X)$，于是取 $b=e, ~a=2$
$$
implies frac{1 text{ nat}}{x text{ bits}} = ln2 implies x = frac{1}{ln2} = frac{log e}{log2} = log e text{ bits}
$$
所以
$$
frac{1}{2}ln(2pi esigma^2) text{ nats} =
frac{1}{2} cdot frac{log(2pi esigma^2)}{log e} cdot log e text{ bits} =
frac{1}{2}log(2pi esigma^2) text{ bits}
$$