Call me paranoid if you want but what if a high speed boat hits you? Just wear the PFD all the time as you never know what life is going to throw at you. My son and I canoe on a river that sees a lot of jet ski and water ski activity. Best be sure we have our PFD’s on as these boaters are often looking backwards at their skiers and not at what they could run into.
Even if you are in a small lake with no speedy traffic you could pass out from low blood sugar, sunstroke, etc and fall into the water. The PFD saves you from being a tragic accident. Just wear the d**n thing!