Hacker Newsnew | past | comments | ask | show | jobs | submitlogin
Ask HN: Assume GTP-X arrives in 6 months and is 100X – what now?
3 points by dflock on July 27, 2020 | hide | past | favorite | 5 comments
After reading https://www.lesswrong.com/posts/N6vZEnCn6A95Xn39p/are-we-in-an-ai-overhang I was thinking...

What would a 100x improvement on GTP-3 do to your employability, for example? What changes would this bring in the short term? Medium term? What should one do to stay afloat in the medium term?

For the purposes of this question, I don't care about the details of the network/GTP etc... just the outcomes of a 100x improvement in results.



How far can you get in some jobs by _just_ being _really_ good at NLP - i.e. without any insight or domain knowledge/reasoning, beyond what's needed for the NLP?


GPT-3 is not a 100x improvement over GPT-2.


Are you sure? When I look at the GPT-2 examples, and remember my experience with it when you could try on talktotransformer.com, I would say it is.

GPT-2 would start to ramble incomprehensibly almost every time. It would forget context after a few words. It would get into loops. To make something resembling a conversation you had to heavily influence it.


It's better, just not 100x better, which is a big difference in practice.

Quantitatively about 5x of the output is more useful in my usage, which is not enough to upend society.


Doesn't really matter, for the purposes of this question.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: