Pulled the alternator and the alternator mount off today. Looks like the issue is with the alternator itself.
The body of the alternator is aluminum with steel inserts where the bolts go through the mount for the pivot and the adjuster. My alternator is missing the steel inserts so the hole for the alternator bolt holes have bored themselves out.
I got it all back together with the new bushings, but it looks like I'll have to buy another alternator, which sucks, because other then the bolt holes being ovals, this one works fine. Ugh.