Well Jonas I will try to respond, first I don't use 4# my experience is with 8-10# , with that said if guides are your problem then it would effect your line quite a ways back from where the lure is tied on. I would think a Q-tip ran around the inside of your guides and rod tip would quickly expose the problem, if you're saying the line is fraying just putting pressure on it to set your drag I would expect that to be 2-3#, so if every fish you fight would do likewise to it how long would it last?
If you didn't notice much difference between flouro and mono then switch back to mono and problem solved apparently. Yes flouro is superior in abrasion resistance but remember there are a lot of manufacture's getting into the flouro game and they sure aren't all equal, I can attest to that from experience. The forth type/brand I tried was magic and I never looked back. We fish where mussel shells, rocks, concrete, structure, and toothy fish all take there toll on line and flouro has preformed better than braid and mono for us testing on the water. Don't know what flouro you used but suggest you check your guides and try another line if they are good. I've never heard of line fraying when you put pressure on it.........it would normally break at a certain poundage point.