Well, this was kind of my point, I just made it very flippantly.
Back in The Day (a.k.a when the Soviet's worried us) we spent a huge amount of money on what amounted to propaganda spending. So we did things like landed on the moon to prove how much better the American Way was over the Dirty Communist Way (or whatever). That was the real value in landing on the moon. And as this generation still remembered WWII and hadn't gotten entirely fed up with Vietnam and realpolitik, it was bought and lapped up. So we spent vast sums on R&D sorts of projects, like the space race. It also helped that we could toss in Fear of the Big Bad as a selling point. If we don't make it to the moon, those dirty Ruskies will go there and build a super base with nuclear missile launching capabilities, etc. etc. (I'm exaggerating wildly for effect).
Then 1980 came with Reagan and 'tax cuts'. The money got diverted into more purely military purposes - like the Star Wars Missile Defense program (and illegal wars in Central America). And we really did cut the hell out of taxes on the wealthy and on corporations. But things like Social Security and Medicare are untouchable, and were getting used more frequently. This led to us continuing our biggest expenditures (welfare and military spending) while decreasing the amount of money coming in. So things DID get cut, relatively. Things like NASA.
When the Soviet Union fell, the propaganda and fear elements lost their luster, and we were headed in the wrong directional fiscally. Rather than institute new taxes, we've basically been riding the private sector's ability to generate new wealth to keep our heads above water while slowly slipping further into debt.
Going to space is necessary for the long term survival of the human race. It is absolutely mandatory. But it is not something that matters in my lifetime, so we ignore it and hope it'll all be ok (hell, we'll be dead anyway, right?). That doesn't sell voters though, and we can't start spending money on NASA when so many other important and worthy causes are also underfunded and raising taxes is political suicide. Which is basically why we haven't been back.
An addendum: Going to the moon probably isn't the best thing for NASA to be doing anyway. There are things that would yield far more important findings and research than something we did 40+ years ago and aren't ready (technologically) to colonize yet (if it's ever possible given the moon's particulars).