This caught a bunch of cases doing that already, which I just fixed in
previous commit. This is the assertion itself.

Per Tom Lane's idea.