I'm no expert, but my hunch is the aftermarket radio is still drawing power. There are certain lines that are always on (for example, the cigarette outlets) so if you're drawing power from one of those "always live" modules, that could be straining the battery.
Secondly, given your vehicle's age, I strongly recommend replacing the battery. Last year was when all the 2004 owners on this board (me included) had dead batteries after the summer. You have an '05 so its probably y'all's turn.