Hm, as I understand it, if one party won all 299 direct mandates at 0% 2nd vote share, the other parties' seats would have to be increased until 299 == 0%. Depending on how you round, that would be at least 30,000 seats (there's some tolerance allowed that I did not consider here but that's the ballpark for that unlikely scenario).
The linked site lists a (more realistic) scenario already, where the number of seats comes out to more than 900. How did you end up at your 897 seat upper bound?
Actually, I should add that even in the unlikely extreme scenario, the upper bound would be the number of candidates on the parties' lists (Landeslisten). Even if a party would have a right to more seats, they cannot fill more than they have approved candidates on their lists. One could determine this actual upper bound by looking at each state's lists, which are public of course, but I haven't quickly found them compiled in one place, so I won't bother.
(Berlin's pirate party once won a surprisingly high share in the state parliament but wasn't able to fill all seats, because they were polling way lower when compiling their list.)
> (Berlin's pirate party once won a surprisingly high share in the state parliament but wasn't able to fill all seats, because they were polling way lower when compiling their list.)
That's not correct. They had 15 candidates and won 15 seats.
For all the nitty-gritty details check paragraph 4.5 on page 67 of the official election results [1].
The linked site lists a (more realistic) scenario already, where the number of seats comes out to more than 900. How did you end up at your 897 seat upper bound?