The main concern with the neutrons is damage to the reactor itself — you don’t want to have to replace the main chamber walls with any frequency. Fusion reactors generally do not release enough neutrons to hurt people. So shielding on the outside doesn’t do much except provide additional fail safes.
The neutron flux inside a fusion reactor is many orders of magnitude higher than the maximum allowable for human exposure. The flux is low on the outside precisely because of the many absorption lengths of shielding between the reactor and people.
