We apply crude MDL to efficiently regularize polynomial regression models.
An order
where:
-
$c_1,\dots,c_n$ are the polynomial coefficients. -
$d$ is the precision of each coefficient. -
$C_U$ is a uniform binary code for unsigned integers. -
$C_F$ is a uniform binary code for floating-point numbers.
Then the model description length is given by
Let
Then
Once a polynomial has been specified, we can represent the data in a residual form:
$$L_{D|M}(x, \epsilon) = -\log p_{D|M}(\epsilon) = -\sum \log \Phi\left(\epsilon_i \pm 0.5 \times 2^{-d}\right)$$
The probability of
$\epsilon$ is the product of individual residual probabilities. This follows from the assumption that residuals are i.i.d. Also note that we exclude$L(x)$ as it's constant.
Then the complete description length is given by