When the capacitor is disconnected from the battery, the charge ($Q$) on its plates remains isolated and thus conserved ($Q$ = constant). Therefore, the statement 'The charge on the capacitor is not conserved' is incorrect.

Let's analyze the other statements:

1. Capacitance: When a dielectric of constant $K$ is inserted, the new capacitance becomes $C' = KC$.
2. Potential Difference: Since $Q$ is constant, $V' = Q/C' = Q/(KC) = V/K$. The potential decreases $K$ times (Correct).
3. Energy Stored: The initial energy is $U_i = Q^2/2C$. The final energy is $U_f = Q^2/2C' = Q^2/2KC = U_i/K$. The energy decreases $K$ times (Correct).
4. Change in Energy: $\Delta U = U_f - U_i = \frac{1}{2} \frac{Q^2}{KC} - \frac{1}{2} \frac{Q^2}{C} = \frac{Q^2}{2C} (\frac{1}{K} - 1)$. Since $Q=CV$, initial energy $\frac{Q^2}{2C} = \frac{1}{2}CV^2$. Thus, $\Delta U = \frac{1}{2}CV^2 (\frac{1}{K} - 1)$ (Correct).