I am not sure how mechanical you are, so I will word this like you know very little. If you are mechanically inclined/knowledgeable please don't be offended.
Bad float. Or bad needle jet. Or float is sticking, or needle jet is sticking. The carbs on 2 strokes are pretty simple. Open it up and check for a sticking float (does the float move easily through its' entire range, both up and down?). If not, look for bent or damaged parts.
Your float could be leaking, as you mentioned. Take the float off and shake it....do you hear or feel any sloshing? If yes, you need a new float...it is possible to fix a leaking float but is usually not worth it. Just get a new one. Even if you don't hear sloshing, your float could still be leaking. Take the float and submerge it into water and watch for very small bubbles. If you see any, your float is bad.
Your float needle could be worn...it is located under the float, and all it really is, is a stopper, kind of like a sink stopper. As the float moves up and down, the needle can move to allow more or less fuel to the carb. If it is sticking or worn, it will "leak" allowing the carb to flood. Could work fine on initial start up. Motor runs fine. Then you stop and shut off motor, but the needle jet does not seat into the closed position which allow the carb to flood. Replace the needle jet.
What knowledge I have comes from working on 2 stroke motorcycles, but the same things should apply to a 2 stroke boat motor.