They don't have to remove images from the training set, they're saying they're opting to do so, and using that as an argument as to why if there supposedly could be copyright infringement, they're not liable, because they allow it to be removed.

They could just as well not do anything and continue on - it's likely this case will be in defendents favor. Same as how Google can crawl the net, cache data, transform it, etc.