...but the ideal structure is not supposed to exchange air with the outside particularly because of this. That is why windows and doors have seals on them. I'm no mechanical engineer, but I've worked with enough of them in my career to understand that their goal is usually keeping a structure tightly sealed when ingress/egress paths are closed (of course opening windows/doors is a choice of the occupant, which if your striving to reduce heat loss/gain... you shouldn't do)
An ideal structure should minimize convection, but rooms for people really can't be allowed to undergo large changes in temperature without exchanging air with the outside environment, because that would imply a large change in pressure.
A 10 percent increase over atmospheric pressure might not seem like that much, but it's enough to notice, and would probably be uncomfortable. It's like being under 1 meter of water. It's also enough to break large windows.
I don't think comfort would be much of an issue given a slow adjustment period, but structurally it would be ludicrous. You'd be unable to open or close doors. Your walls would have to be built a hundred times stronger than the walls of a normal house. Any breach in the pressure envelope would be a miniature version of Aloha Airlines 243.
This is, by the way, why hard drives have filtered air holes rather than being completely sealed.
"the ideal structure is not supposed to exchange air with the outside"
No, the ideal structure should minimize heat loss from air exchange.
I'm not fully up on my air exchange rates, but it's fairly typical for ranges to be in the 4-20 range, that is, the interior air is exchanged with the exterior 4-20x per hour.
In my Thorsten Chlupp references elsewhere you'll find he makes extensive references to heat exchangers which minimize thermal losses. He does this by a twofold process for his Fairbanks, AK, homes: entering air is routed first through the ground where it's heated from very cold ambient temperatures of as low as -40C / -40F to a temperature closer to freezing (~0F). It's then passed through a heat exchange where the exiting warm air transfers much of its heat to the entering cold air.
The purpose of tightly sealed windows and other possibly entry/exit points isn't to eliminate air exchange so much as to control it: you want air entering and exiting through your designated ventilation systems and transferring heat properly, not traversing the envelope arbitrarily.
Another way of putting it: A well-designed structure should minimize random, unintentional air exchange, but provide sufficient deliberately-engineered ventilation to keep the air and people happy. For efficiency, that ventilation should go through a heat recovery ventilator (HRV) or energy recovery ventilator (ERV) as appropriate to the climate and budget. (An energy recovery ventilator also exchanges moisture).
Beyond Chlupp, Lstiburek is a great engineer and writer on these topics.
Perhaps I lacked the verbosity in my original comment, but this was what I meant to imply: ventilation is part of the design. Unintentional exchange is to be avoided.
>...but the ideal structure is not supposed to exchange air with the outside particularly because of this.
With regard to letting a house cool down at night, that's impossible. If you heat a quantity of air from 273 K to 300 K at constant volume, the pressure increases from 1 atm to 1.1 atm. That may not sound like much, but it's better expressed as a pressure differential of 10 kilonewtons per square meter, and the surface area of your house is such that it could severely damage the walls and blow the door open -- popping it like a balloon.
(The fundamental mathematical error made by those who minimize this effect is ignoring the surface area of the building)
It's most definitely not the goal to suffocate the inhabitants, no, and the goal also isn't to have the natural humidifiers living inside create a tropical climate ripe with microorganisms and stuff. Yes, you want to avoid excessive air exchange because you'd lose a lot of heat that way, but tightly sealing a house would be a very bad idead indeed. If you do make the outer envelope essentially airtight, that gives you very low energy consumption and is part of the passive house concept, but then you need active ventilation, which obviously implies a hole in the envelope, and thus will keep the pressure inside equalized with the pressure outside (modulo a small differential caused by the ventilation itself).