The likes of the White House etc are sufficiently maintained and utilised buildings, there's no reason to knock them down. If there is value in preserving a historical structure, like marveling at the remarkable construction of the pyramids, or ensuring that people don't forget the stain on humanity that is Auschwitz, then of course that makes sense. Does that mean that every wall that has ever been created should be conserved forever, even if they are not being utilised, have been neglected for years, and are preventing a real need for progress? I guess everyone will have their own opinion on which structures have value and which ones do not.