I did this on my track car and had no end of dramas......you can't bolt a 323i starter (325i might be different) on as it hits the block and causes misalignment. I didn't buy a 323i TO bearing just used the one that came with my single mass flywheel clutch kit, which was exactly the same as my M42 TO bearing. And i used an '84 323i flywheel (2 kilo's lighter than the 325i flywheel and later 323i flywheels.
I also had to put the 323i starter pinion in the M42 starter and to do this i had to shorten the bendix side of the pinion by about 10mm and cut a new circlip groove in...I am unsure if this is a 323i related issue or something to do with all m20 starter motors.
So in summary:
3mm spacer between crank and flywheel
'84 323i Flywheel
'84 323i Starter motor pinion modded to fit in M42 starter
Shorter flywheel bolts
Single Mass m20 Clutch kit including TO bearing.
As a precaution i modified the clutch fork pivot point to make it adjustable incase there were issues with the shorter TO bearing. At the standard pivot height/depth there are no issues with (dis)engaging the clutch.
The engine revs a hell of a lot better and has woken up alot....i can't comment on noise because if there is any chattering it blends in with the rest of the noises....i removed all of the insulation on my car

Good luck to anyone looking to do this.