It's really not as cut and dried as that. When you say "the blueprints," you have to be specific. Literally tens of thousands of drawings are created as part of building a spaceship. They don't all end up in the same place. Some are required to accompany the spacecraft as it is conveyed officially from the contractor to its owner (NASA). Some are required to be kept by the manufacturer until the spacecraft is decommissioned. Still others are not strictly required, but are made by the manufacturer for their own purposes.
The perceived historical value of these documents varies, hence a difference in the need to preserve them for posterity. And these documents are voluminous. The space required to store them and the cost to reduce them to microform create a situation where practical concerns quickly outpace the historical preservation motive. So some "blueprints" were indeed destroyed -- especially the ephemeral ones. Individual fabrication records, for example, become moot as soon as the spaceship is no longer operating. They have little technical or historical value in and of themselves. So even though some of us would love to see them, they were not considered worthy of retention and have been destroyed as a matter of routine.
Even for those documents that have historical or technical value, they get scattered all over the country. As I said, some are collected and retained by the agency, others are kept by the manufacturer. Nowadays most of those manufacturers have gone through multiple reorganizations, mergers, and acquisitions. With each generation, institutional memory of those documents -- where they are and what they represent -- gets muddied. So when people say, "I want to see the Apollo blueprints," there's a lot of shoulder-shrugging.
Where did the accusation start? Hard to say. It seems that every author wants to say that "the blueprints were destroyed" and leave it at that. Usually it's a response to confronting the reality of historical document preservation. "The blueprints" for the lunar module, for example, were not some concise roll of papers you can just tuck under your arm. Just the amount of written documentation that had to be sent to NASA for each individual spacecraft filled a railroad boxcar. So the problem is really with the expectation that if the Apollo "blueprints" actually existed, they would be an easily manageable and carefully-preserved tranche.