This Situation
You have a list with a “year” column of type numeric, so that it will do a proper numeric sort. (While you can use calculated columns to format numeric columns like a text column, you lose your numeric sorting capabilities.) You have a view grouped by year. SharePoint displays the year with a comma. Like so:

The Solution
It’s time to open SharePoint Designer. Click “Lists and Libraries” in the Site Objects panel. Click your list. Find your view in the “views” panel. Click your view to edit it. Find your header. Right click the header and select “Edit Formula.”

Set your formula to be:
translate(@*[name()=$fieldtitle], ',', '')
Like so:

And, that should do it!
This adds a lot of XSLT to your page that is designed to allow for different types of group headers. In this case you know what your group header is going to be, and you could probably clean up some of the XSLT for faster performance. I’ve yet to do that, though. If you do, let me know!
Note: If you need to just remove the comma in a column of data (not a header) the approach here works. His number-format formulas did not work for me in the header.