What about 3D reconstruction? You say it should work from just regular optical images, provided that you have enough of them? That’s exactly the issue! How can you entice a casual photographer to take that many photos of an object? Heck, they’re just too lazy! That’s precisely the issue. And that’s why it doesn’t work.
BUT, if you by chance happen to get lucky and there is some popular piece of architecture that many people come by to lazily photograph, then you’ll find that such techniques to be successful. In this case, it is a statement of reporting bias, that only the popular structures get scanned into 3D.
And, matter of fact, if such a structure really is that popular, chances are the geeks who have the equipment and determination would likewise come by and do a dedicated scan. So there you go. It’s kind of self-defeating, it is. Let me put it this way. 3D scanning objects is like taking pictures with a DSLR camera. It’s just one of those things that will never really become popular among casual, non-technical, non-professional users.