Another route to consider. If you have access to a rod lathe, or can fab up one using a variable speed drill and some home-made stands to support the spinning rod: don't try to remove the damaged cork all the way down to the blank. Using some 60 grip sandpaper, with the rod spinning, turn the damaged cork down to near the blank, but leave some cork attached. Make the remaining cork an even diameter all along the length of the grip. Make it just a hair smaller than a standard drill bit size. Bore out your new grip to that size. Get it done right and you can simply slide the new grip over what’s left of the old grip. If you have holes in what’s left of the old grip, use some of the cork dust, mixed with titebond glue to make a paste the patch the holes. No need to worry about color matching. You wont see it when the new grip is slid over the old.
Take your time sanding down the old grip. Have the new grip already bored out so you can check the diameter of the old grip as you go. You will need a loose fit. The new grip, when bored out will be a bit on the flimsy side until it is glued on place over the old cork. So be gentle when dry fitting the pieces together.