a drill bit is always going to walk a little bit. There are 3 things you can do to minimize that, depending on the type of drill press you are using. You *are* using a drill press arent you?
1. Always use screw size drill bits. These are half the length of jobber bits, and therefore will not flex nearly as much.
2. Say you are drilling a 3/8" hole. That's too big to locate itself, just look at the width of the initial contact surface. It's nearly 3/16 wide, so on most metals its going to move some, centerpunched or not. Take the second step and pre-drill your hole about 30% the size of the finished hole.
3. Buy a drill bushing. Those are bronze sleeves that keep your bit from flexing. You'll have to fixture it in place exactly concentric with the bit, and close to the workpiece. on deep holes or thru holes which pass through multiple walls they are an invaluable resource. You should be able to buy one at any tooling supplier like Fastenal.
Lastly, I'm new here so I really dont know what y'all know so forgive me if I'm insulting you with simple things. But, always center punch your holes. Get yourself a set of centerpunches. I have a Starret kit that has 6 sizes. The larger the hole, the larger the centerpunch the larger the indent to locate from.
Also, if you are repeating patterns over and over again, have a local shop either laser cut or water-jet a template for you. Then you buy a transfer punch for each hole size, and voila` you have repeatability.