I wear mine 100% of the time while on the water, no exceptions. Having said that, its super comfortable and its technically another tool for me to utilize while fishing.
I have a whistle attached to small tether that also holds my keys in a small chest pocket.I keep my wallet in another, my cell phone tethered to the PFD and in another pocket. I have some hemostats tethered on the outside and attached at the top a boomerang braid cutter.
So worst case scenario is I fall out and my kayak is swept away I still have ID, phone and my keys on me regardless. Id feel lost without my Chinook!