I fish worms Alot! lol as mentioned there are several ways to fish a worm..The bass will let you know what works best. You do everything from slow rolling to deadsticking.
There are times when t riged I will cast out, and let my bait sit for up to 2 or 3 mins..then just twitch it a few times or I'll make it crawl for a foot or so, then stop, twitch and repeat or vary the retrive.
Experiment with your retrives, sometimes you can catch fish by swiming your worms, it just depends. Also try mixing up the variations when you retrive,i.e. cast, twitch..then slow roll it, ect.
If you have timber in the lake, I find that if I can get the worm to slide up a log/branch and let it sit, then wiggle it a couple of times, that gets strikes when other things don't work. Good luck and think outside the box..