I would go the 2005 deffinately, the rubber mount makes a big difference.  The Sportster was totaly re-designed as of the 2004 and up models.  Read this article it goes into great detail about the changes:

http://www.hdforums.com.au/Default.aspx?tabid=71&forumid=1&postid=9729&view=topic