Tippy, like Greg I used a dremel with a sanding drum just enough to take the clear/base/primer out hole. want to try and make sure its still a "press fit" if possible.
Not the prettiest but I just used a 1 1/2" bit to bore a small inset into a block of wood enough so the other side of the bearing can pass and the block can rest against the plate.
Tools I used:
since the metal is now bare I dunno of it helps or not but I took some car wax and slathered it in the hole and around the plate before I put the bearing in and then put it together like this:
worked pretty good, with no issues