Like others have said, swimbaits are a tool, not a magic bullet. They aren't going to be the best option all the time, but when they work they're magic. I've had days when I was struggling and I think a lot of the time I use to immediately switch to finesse options when it's tough. Now a lot of the time I'll switch to a swimbait first before I downsize. I fish a lot of lakes that get lots of pressure and don't produce big numbers of fish very often and even fewer keeper sized fish. Last year I started playing with swimbaits on one of the toughest of all of them, the results were incredible to say the least. I was fishing smaller swimbaits, 4-6 inchers, along the weed edges that we normally flip baits to and I was just getting hammered by fish more keeper fish than I'd ever seen come from this lake. I put a limit in the boat in under 3 hours the first day, I've only caught limits twice before and both times were under perfect conditions and it still took me all day. It doesn't take monsters to eat them. One of the first fish I caught that first day with swimbaits on this lake was a 16 3/4" fish on a 6" weedless Hudd, it almost swallowed it.
This has lead me to play with them on other lakes I'd never fished them before. Another lake I fish often produced over 80 fish in a single day, most of them on this 5" Decoy Hydratail swimbait cranked slowly around shoreline weeds, rocks, and laydowns.
When they're really on them you don't even have to use expensive swimbaits either. I got on a hot swimbait bite in a hot water outlet lake last winter and they really wanted a Storm Wildeye Shad burned with the current by the rocks. They were gorging on shad and slamming swimbaits and not touching anything else.
I did get one to plow a 6" Spro swimbait that day too that proved to be the biggest of the day.
This winter I've been playing with the Havoc Sick Fish with really good results for a mix of species. This single bait has caught almost two dozen fish and is still going strong.
My suggestion to you is to start with some smaller baits so you can get some bites and then work bigger as you gain confidence. The 4" Hudd weedless shad, 4" Sick Fish, and ABT Banshee are some of my favorite smaller, less expensive baits. Pick colors that imitate the forage in your lake and go fish them. I think one of the biggest mistakes I see with swimbaits are fishing too fast. At times a fast retrieve will be best but more often than not it seems like they want a slower retrieve. If it's a day you feel like they'd eat a spinnerbait or topwater, that would probably be a good day for you to try a swimbait but they will eat them anytime.