I've got one person under me and have done for 3-5 years.
Most have enjoyed working under me (so I understand it), most resignations have been for bigger opportunities / bigger cities / bigger money (1 out of 4, and TBH I missed they wanted more money - this was my fault).
My current "report" seems very happy (though I expect them to leave soon, not for money but they are far more interested in another industry and tech challenges that come with it).
One person under you for 3-5 years with a four attritions in that period (this latest one being Person #5?) constitutes an incredibly high attrition rate. That’s worth introspecting on. Average tenure of under a year?!
It seems likely someone managing a handful of employees doesn't decide salaries. If his unders left mostly for salary reasons, I don't see how that reflects badly on him.