Ok first off, if it is big baits, listen to the Bizzle.
I don't throw "big baits" but I started throwing baits last year that imitated fish instead of craws and saw my average fish really go up. I used to live by the jig, flipping beavers and brush hogs, and also tubes. So I threw primarily crawfish imitating baits and I had a very good average for RI, from lake to lake I was around 1.75 to 2 pounds and could bang out atleast 15-20 fish an outing.
Last year I started throwing more baitfish imitations just simple things like 5" paddle tails (I know Biz, they're gay), Matt Lures Sunfish and Perch. and big crankbaits/jerkbaits. I saw those 1.75 to 2 pound fish turn into a 2.5 to 2.75 pound average and I could generally get atleast 10 an outing when fishing strictly this way. I caught about 30 fish over 4 pounds this way last year and I only caught about 45 over 4 last year all together(and I fish very high pressured bodies of water).
So I would say just by switching the style of bait you can see bigger fish. Not huge, but much better quality fish. I would say build your way up, see if changing to minnow or crawfish imitations makes a difference and then go from there. I know I could never fish a swimbait over 7" because I just don't have the confidence and can't take the frustration. Know your limits and don't burn yourself out over trying for that one big bite if you loose the enjoyment of it.
(my 2 pb's over 8 both came on jigs though) :