I catch fish when part of the ponds are iced over. Just dress accordingly and fish reaaaaaaalllly slow. Dropshots, shaky heads, and jerkbaits. I wear the following when its 20 degrees outside.
Long underwear
Compression shorts
Pajamas
Jeans
Two pairs of socks
Waterproof boots (very warm)
A t-shirt
A sweatshirt
A heavy coat
Thin fingertipless neoprene gloves
Heavy gloves
Handwarmers in the coat pockets
I usually don't wear the heavy gloves while fishing, only when my hands get numb. I stick other handwarmers in the neoprene gloves on the back of my hands under them.