That's exactly what's happening to the Linux git repository. The server constantly has mutltiple CPU cores dedicated to rendering git of various old, dead branches in html for scrapers.
Not to mention it is the least efficient representation of git commits. It seems a lot of people are unaware that git generates the diff representation you see with "git show" on the fly -- internally, commits are Merkle trees representing a snapshot of the repo file contents.
As you said, if you actually want to interact with a remote git repo you'd do so with packfiles not diffs.