I fish year round, and find great joy in fishing out of my kayaks when the air temps sink below freezing. A key point to remember is COTTON KILLS in cold water. If you get wet from falling in, immediate dry clothes or ability to get warm is critical and can be a life or death situation if not taken seriously. A general rule of thumb I use is the 120 degree rule. That is the total temperature of the air and water temps at the time of launch and during your outing with neither value below 60 degrees. Some argue that 50 degrees is the mark, but your can die from hypotermia exposure in 50 degree water if the air temp is low enough. If either value is below 60 degrees, then you should be wearing some form of layers to help keep you warm. At the early onset of colder temps, sometimes the waders and splash top approach are sufficient, but once the water temps drop down to 50 and below consistently, you need to use the full gammut of layers.
The proper approach is one of layering. Now, depending upon your climate, the layers may vary, but here in the mid-Atlantic, the following is the bare minimum for most outings that I wear.....
Hands - wool gloves and a spare pair in case they get drenched. I use a fingerless design with a protective palm and fingers.
Head - wool or fleece hat tha covers your ears. Don't take the wind for granted. The head is first place to lose heat in the winter.
Feet - wicking socks as a base layer and a pair of smart wool socks over top.
Body - base layer of wicking material (Thermax, Duofold, UnderArmour, etc.) covered with a fleece layer covered with a waterproof layer.
Face - some folks like face shields, but I personallly do not. Your climate may dictate otherwise
Now, for the waterproof layer, opinions vary. Some people prefer full body dry suits (Kokatat, NRS, etc.). They are expensive ($600-1500) but worth every penny, just make sure you test fit it with a base layer on, and make sure you get one with a relief zipper. Also keep in mind that the neck area is sometimes more restrictive in some brands than others. Other options, I use a pair of breathable chest waders with neoprene booties, wading boots and a waterproof splash top with gasket seals at the neck and hands. The water temps I fish in range in the high 30's to low 40's all winter long.
Hope this helps.
I am sure others will chime in. Hopefully some of the folks from the NE area like Jfrancho who also fish out of yaks.