He should have just gone and avoided all of this, but the media is really blowing it out of proportion. Lots of players have missed these things and there's never any mention of it.
Can you name one player that has skipped a White House visit purely on political grounds? The bigger issue was his ridiculous posturing/statement that he made which was entirely political, which he ended with "Its not political".
And the media has made a big deal about this before. James Harrison didnt go to the first Steelers visit with Bush, his reasoning was that he didnt want to. He didnt go again when Obama was President either and used the same excuse.