I have been impressed with what I've seen in the last six months but it still seems that GPT-3 and similar language models greatest talent is fooling people.
The other day I prompted a language model with "The S-300 missile system is" and got something that was grammatical but mostly wrong: the S-300 missile system was not only capable of shooting down aircraft and missiles (which it is), but it was also good for shooting at other anti-aircraft missile systems, naval ships, tanks, etc.
All the time Google and Bing try to answer my questions directly but frequently the "lights are on and nobody is home" and the answers just don't make sense.
I see the problem is that people look at the output of these things that are (say) 70% correct and in their mind they fill in the other 30%.
> I see the problem is that people look at the output of these things that are (say) 70% correct and in their mind they fill in the other 30%.
Q: Is there also some element of survival bias in the mix?
If you prompt GPT-3 with something and the answer is garbage, you probably don't write it up on your blog. If you get something that makes sense, then you do.
Sure, but GPT-3 was trained by self-supervised learning on only static text. We see how powerful even just adding captions to text can be with the example of DALLE-2. GATO takes this further by letting the large scale Transformer learn in both simulated and real interactive environments, giving it the kind of grounding that the earlier models lacked.
The worst intellectual trend of the 20th century was the idea that language might give you some insight into behavior (Sapir–Whorf hypothesis, structuralism, post-structuralism, ...) whereas language is really like the evidence left after a crime.
For instance, language maximalists see mental models as a fulcrum point for behavior, and they are, but they have nothing to do with language.
I have two birds that come to my window. One of them has no idea of what the window is and attacks her own reflection hundreds of times a day. She can afford to do it because her nest is right near the bird feeder and doesn't need to work to eat, in fact it probably seems meaningful to her that another bird is after her nest. This female cardinal flies away if I am in the room where she is banging.
There is a rose-breasted grosbeak, on the other hand, that comes to the same window. She doesn't mind if I come close to the window, instead I see her catch the eye of her reflection and then catch my eye. She basically understands the window.
Here you have two animals with two different acquired mental models... But no language.
What I like about the language-image models is how the image grounds reality outside language, and that's important because the "language instinct" is really a peripheral that attaches to an animal brain. Without the rest of the animal it's useless.
it's quite a terrible risk because you often think "if only I double or triple the resources I apply to do this I'll get it." Really though you get from 90% there to 91% to 92% there.... You never get there because there is a structural mismatch between the problem you have and how you're trying to solve it.
My take is that people have been too incredulous about the idea that you can just add more neurons and train harder and solve all problems... But if you get into the trenches and ask "why can't this network solve this particular task?" you usually do find structural mismatches.
What's been exciting just recently (last month or so) are structurally improved models which do make progress beyond the asymptote because they are confronting
I first got involved in text classification in the early 00's and then the best you could do was "bag of word" models that counted the words in a document but didn't take the order of words into account.
This works great if you asking a question "Is this paper about astrophysics?" because the vocabulary used in a document is closely linked to the topic.
Pretty obviously though if you scramble the words in the document you can't reconstruct the original document, some information is lost, and there are some classification tasks that will reach an upper limit (asymptote) in accuracy because in taking the feature set you lost something. (If the task is "did the defendant commit the crime" the heuristic "Tyrone is a thug" works over bag-of-words, but there is no justice in that.) If that system is able to get the right answer for a case where the word order matters, it just got lucky.
You might think "wouldn't it be better to use pairs of words?" but then you run into another problem. You might have a vocabulary of 2,000-20,000 words and get a somewhat useful sample of all of those in a few thousand documents. The number of word pairs is the square of the number of words and you just can't get enough training samples to sample all the possible word pairs.
Sentiment analysis was an early area where bag-of-words broke down because
I am happy
and
I am not happy
mean very different things. You'd think now that adjectives like "happy" really are special and so is the word "not" and we could make the system somehow realize that "not X" means the opposite of X. You run into an asymptote situation there because there are a huge number of possible negation patterns, for instance you can say
I can't say that I am happy
and you can't even say "the negation structure has to be within ten words of the adjective" because there is no limit for how complex nested structures can get in language. The first few patterns you add "not X" raise the performance potential of the system a lot but patterns you add after that each make a smaller and smaller contribution to the performance and you again reach an asymptote.
Today we have all kinds of embeddings and they are a step forward but they also run into the risk of throwing critical information away, and in a multi-step system you are doomed if an early step does that. I've walked away from some projects where people required high accuracy and they were stuck on using word embeddings that would never attain it. You can think about information loss in embeddings the same way as you do with simpler features except it is a lot more complicated and a lot of people look away instead of confronting the problem.
It might be basically the same process as today but with several big new ideas (some of which might seem simple in retrospect...)
The quality of the training set is also critical, more so than the quantity. Some of these clever ideas for creating a lot of training data without any work, such as "guess the next word" can't really capture semantics.
I think it really takes multi-task training, like what the article we are talking about is advocating. That forces the upstream part of the network to learn features that capture important semantics.
The other day I prompted a language model with "The S-300 missile system is" and got something that was grammatical but mostly wrong: the S-300 missile system was not only capable of shooting down aircraft and missiles (which it is), but it was also good for shooting at other anti-aircraft missile systems, naval ships, tanks, etc.
All the time Google and Bing try to answer my questions directly but frequently the "lights are on and nobody is home" and the answers just don't make sense.
I see the problem is that people look at the output of these things that are (say) 70% correct and in their mind they fill in the other 30%.