You probably have already but before spending the coin, make sure it's not your starter clutch, can make similar horrible noises and similar symptoms.
So you looking at the man o war compensator from dark horse?
Lots of stuff about them on the yankee forums like hdtechtalk and hdforums.com
Just a note , my 2006 road king started to do the rattle thing in the compensator. when I took the cover off for a look, the big nut holding it in/together, was loose. , tightened it up and no worries since then.
Jersey there is more to it than just taking a new one out of the box and chucking it on, many do that and it works ok, some do not and you hear of those, they say stuff along the lines of Harley junk etc, following a proper procedure will give good results, the latest comp that has all the issues it's predecessors had addressed (thanks mainly to the aftermarket people finding solutions that Harley then incorporated into their updated comps) works well if it is set up right to begin with, there is much written on the the subject if you are interested.