When you're skiing or wrestling, you naturally engage a significant number of muscles and position your body to engage in said activity. When your whole body is in motion you instinctively flex your muscles to do things like balance, twist, bend, jump, etc. A body in motion is essentially in a more healthy alignment and supports your passive body parts (bones, blood, nerves, organs, etc).

When we are still and calm, our muscles relax (in general). This is basically a bad thing for our bodies because it stops the support that a lot of our body needs. For example, letting your neck sag forward or hunching over your desk over a long period of time can damage parts of the neck and spine, overdevelop your deltoids/chest and hurt your lower back.

When you're doing yoga asanas, this is all exacerbated as you intentionally put your body into positions that we don't naturally move into without lots of practice. So in effect, it's significantly more dangerous to sit on a mat if you're not engaging your muscles or positioning your body in the correct way to support everything.

Yoga is dangerous in the way that every physical activity that requires skill to perform is dangerous, with one caveat: yoga asanas are dependent on the way you personally move your body and the state it's in. So really, it's only dangerous for people who naturally move their bodies in a way that might promote injury. This is why learning from a teacher how exactly to get into a pose, and practicing that regularly, is so important: it removes ambiguity and improves the entire physical act to not only be safe, but also be superior to how you might naturally do something.

But as a practitioner, I don't personally see how yoga is any more dangerous than skiing or wrestling. You can break an arm in wrestling, and a collarbone in skiing, if you're not careful. Just because sitting on a mat looks easy doesn't mean it is.