"Cold" is relative...
Single-digit temps on a clear, blue-sky day with no wind... I can be outside all day wearing long-johns, jeans and a heavy wool shirt...
On a damp, cloudy, 30° day with a 30 MPH northwest wind... insulated coveralls over the top ain't enough to keep me warm all day.
Once the wind gets to ya' it's over... ya' might just as well head for the house 'cause it's only gonna' get worse... that wind will carry the cold all the way to your bones.
I totally agree. While it is easy to sweat while shoveling snow, there is no refuge from the misery of a drizzly 40° day.