Which size of drill bit do I need?

I can't really answer this question without knowing the sizes of the
bolts. For example, the ideal sizes for a 1/4-20 thread would be a #7
drill. Different thread and different sizes would require different
drill sizes.

Should it be as large as the hole
where the broken bolt is?

Yes, BUT it means the size of drill that will fit in the hole where
there are good threads.

Thanks for the tip of selecting the right kind of drill bit. Although
I didn't need to drill the bolts out because I got them off by removing
the water pump base, I may find your tip comes in handy if I get
another case of broken bolts.

Somebody with expertise in this area needs to physically examine what
you have. It is really impossible to give quality advice without
seeing the exact problem(s).

Yes, indeed. And this is the reason why I will bring the lower unit to
a marine mechanic to have it fixed (to fix a threaded hole that I
messed up).

Jay Chan