Open weights models are trainable. However, I would not count it as "easy" as you generally do not have access to the training material, so there is a substantial risk of model drift if you use weight-based retraining.

However, with RLHF you don't need to have access to the original training set to steer a model towards a preferable behavior.