Big fish aren't going to out compete smaller bass. Example, lake I fish, the smaller 10-12" bass chase shad amost daily in one area. Decided to bring an umbrella rig. Below those smaller fish the bigger bass will lay in wait to pick off shad that the smaller fish have injured but didn't eat. Caught 2 of the 3 I hooked....one 12", the other almost 5lbs and staightened a hook on another before the bigger fish.(new rod and reel set up so drag was a bit tight). Big fish will look for prime areas to feed. Deep weed beds, points on the weedbeds or breaks where forage funnels through for easy picking. Two types of cover coming together, structure and cover. Sometimes big fish just decide to suspend out in deeper water till they decide to feed or a feeding oppurtuity presents itself. Big worms are skinny and produce more fish but may be smaller. Jigs produce bigger average fish depending on the jig and trailer size. I have caught bigger average on a 3/4 to 1oz punch jig and a SK Rage DB Craw then say fishing with a 1/2oz jig and a 4" paca craw. Bigger profile craw bait I believe tends to attract that bigger bite although it is compact in size compared to a big worm. I can also drop this bait into their space quickly producing a reation bite where a big worm may fall slower and give them time to look at as well as possibly mouth and spit it out. Bigger fish use their enviorment to their advantage for a meal instead of chasing it down. They will sit motionless next to bluegill beds for what seems like forever until they explode and grab one that has gotten used to them sitting there and deemed them to be no threat. So as stated although some may disagree, location, location, location. Then it's putting something that they see as an easy meal in front of them that they can't resist. A jig could represent a crawfish or a bluegil, I use the A-rig to my advantage on schooled up bass chasing shad or glass minniows to take advantage of their messy feeding habits when schooled up. Other times swimbaits will work, but only when they are keying in on them when they stock trout which is a small window on small bodies of water. So guess I would have to add timing to the location as well. Disect each fish you catch...where, and why was it there. What were the conditions? All of that information will help you in your persuit of larger fish you are after.
Good luck.