In normal soil, frost has to overcome the typical ground temperature as it works downward. You don't have that reservoir of ground temperature in the mound really, at least not the same as native settled soil. So I bet the mount has a significantly less frost resistance than the ground. If you have a 6' frost line in regular soil, I don't see how a 4' mound of sand/stone has a chance, even with bales of hay on top. Maybe there is a better way to insulate the mound from air temperature above so that it can trap/hold some ground temperature, but I have my doubts.