> There is an even higher level of creativity in creating books, songs and all sorts of art used in model training though.
No argument here, I completely agree.
> There is no world in which me vacuuming the entirety of human knowledge to make a genai model is ok but hoovering my model answers is not.
I disagree with this though. Clearly LLMs owe a huge debt to everything that has come before, but surely you'd agree that the models that are produced are something substantial and new and novel which didn't exist before and have lots of value in their own right. Let's be a bit reductive and pretend Moonshot had just outright stolen the weights from Fable somehow, clearly that wouldn't be contributing anything really new or novel. Now of course they've distilled rather than stolen, but the point is similar: how much value have they added along the way?
No argument here, I completely agree.
> There is no world in which me vacuuming the entirety of human knowledge to make a genai model is ok but hoovering my model answers is not.
I disagree with this though. Clearly LLMs owe a huge debt to everything that has come before, but surely you'd agree that the models that are produced are something substantial and new and novel which didn't exist before and have lots of value in their own right. Let's be a bit reductive and pretend Moonshot had just outright stolen the weights from Fable somehow, clearly that wouldn't be contributing anything really new or novel. Now of course they've distilled rather than stolen, but the point is similar: how much value have they added along the way?