After researchers found ChatGPT getting worse at a few skills, many were surprised but they shouldn’t have been writes Satyen K. Bordoloi as he lines up some usual suspects for this crime against AI.
In 2016 Microsoft released one of the first AI chatbots called Tay on Twitter to show its AI chops. The idea was for Tay to learn from its interaction with users via reinforcement learning. Within hours though, Tay started posting venomous, abusive comments. Microsoft shut it down 16 hours later.
The issue wasn’t that Tay didn’t work. The problem was Tay worked really well. So well that it absorbed the world like a sponge and gave the same back. Tay wasn’t a living creature to understand the context of an offence or abuse. It was but an advanced word processor that was really good at predicting the next word without really ever understanding its meaning. Thus, if it was programmed in abuse and trolling – in this case via its interaction with users – that’s exactly what it would do.
AI developers learnt from the ‘Tay problem’. It seemed none would ever make that mistake again. But it’s recurred, this time a little obliquely, with ChatGPT.
Within months of ChatGPT exploding into the world, users noticed that instead of getting smarter, it seemed to be growing dumber. A group of Stanford University and UC Berkeley researchers decided to put this hypothesis under the scanner of statistics. And voila, they now have numbers to put to this speculation.
Their report here states, “..LLM services did not uniformly make better generations over time. In fact, despite better overall performance, GPT-4 in June made mistakes on queries on which it was correct for in March.”
Though not peer-reviewed, the data clearly shows ChatGPT can’t hold itself up to its own former standards anymore. The paper holds a neutral tone and doesn’t judge the company, but the ‘Houston, we have a problem’ moment is loud and clear. It also does not try to specify why that problem has arisen. On their part, OpenAI has refuted all claims made by the researchers. No company will own up to its mistakes. But we can be sure they are hard at work fixing it.
Hence, it is up to us to make sense of what’s happening with ChatGPT. If we do figure it out, it will be another Tay moment: we’ll ensure future AI projects don’t fall into the same pit.
A disclaimer first. This analysis is not meant to target OpenAI or other AI developers. On the contrary, it is an attempt to understand the issues with the way we make AI work so we can make them better in the future.
TRAINING AI:
The problem with Tay wasn’t really the abuse. The abuse felt huge because we perceive it as such. The issue with Tay, and with ChatGPT, is more fundamental: it learns from humans. But isn’t that what it is supposed to do? Indeed. But unsupervised learning from all kinds of humans is exactly the Tay problem. OpenAI seems to have that issue: the system learns from humans all too well and humans in general are, well, not too bright.
Reinforcement Learning is one of the pillars of machine learning and thus AI. In a technical way, it shows, “how intelligent agents ought to take actions in an environment in order to maximise the notion of cumulative reward.” To understand this in English, we have to first accept that AI, despite all the fancy terms, is not alive. It cannot ‘think’ on its own. It cannot ‘decide’ on its own. It understands nothing for itself. It has to be taught and programmed to do all of this.
This is done in the same way we teach children: by rewarding appropriate behaviour. But even to get to this ‘reward’ a machine has to be taught the basic rules of something. This happens via supervised learning inside AI labs, months or years before any AI is released to the public. This is when a machine is taught using data that is ‘labelled’. Thus if one plus one has been labelled two, one plus two is three and so forth, this becomes the AI systems core programming upon which all other results are tested. This is the supervised part of it.
The reinforcement learning bit enters the scene post this when the AI’s actions begin corresponding with this basic, supervised programming already inside it. Thus, if it figures out ninety-two plus seven is ninety-nine, this becomes reinforced in the machine and in the future it will use the same logic set to predict things.
Now here’s the rub: Outputs are easy in mathematics. One plus one is always two (though it took hundreds of pages to prove it). But in the sphere of language, there is no such set of rules. Thus, when Tay writes that it hates feminists “and they should all die and burn in hell”, it is not deliberately trying to be offensive and obtuse. It is just something it ‘learnt’ from its interaction with humans as being appropriate. This reinforcement learning via unsupervised learning proved to be the doom of Tay and thus Microsoft.
HOW CHATGPT GOT AROUND THE TAY PROBLEM:
Ever since Tay, AI developers have gotten smart. They realise that AI cannot be let out unsupervised. At least not until it has been taught the way of the world to a great level. And as the Tay problem demonstrated, spewing out offensive comments is an utterly no-go area. The way ChatGPT, and others, have gotten around this problem is by training AI systems in what is offensive and problematic for humans.
The Wall Street Journal did a brilliant piece of journalism when they dug out the Kenyan workers employed to filter hate content from ChatGPT. Basically, they were the people who taught ChatGPT how not to hate, and as the piece highlighted, suffered the consequences of it. As these people took care of the Tay problem, other groups – including those in India – worked hard and long hours to supervise the learning of ChatGPT. Indeed, these thousands of people spread across the world, are the real teachers of the AI programs the rest of the world enjoys.
However, problems emerged when they were let go. They were meant to since their job was mostly complete. The idea was that post-November 26 when ChatGPT was released to the world, it would learn through interaction with the world. There’s still supervised learning going on, but most of the learning seems to be unsupervised from its interaction with humans. Thus, every time you have interacted with ChatGPT, you’ve helped train it.
NARROW AI, BROAD AI AND AVs:
Elon Musk for 9 years has been promising that fully autonomous vehicles (AV) from Tesla’s stable are just a year away. But they have not materialised for one simple reason: broad AI is a notoriously difficult problem to solve.
Narrow AI is when the AI system is taught one specific task and that’s all it does. Take your speech to text, maps app, or AlphaGo that defeated humans in the game of Go. These are trained for specific tasks and can do that really well. DeepMind’s AlphaGo was subsequently used for many other uses and did exceedingly well. But those were all singularly focused tasks.
Broad AI is when a single AI system can do a whole host of tasks. This is actually where AI begins to get close to human levels. Thus, the AI system in an AV or self-driving car has to take multiple inputs – from the Radar, Lidar, cameras and other sensors in the car and make split-second decisions based on them. Humans do it without thinking (thanks to millions of years of reinforcement learning). But turns out, it’s really difficult for machines to do so. Some claim this is a hardware problem since the hardware doesn’t yet match the speeds required for these decisions to happen. However, beyond this is the general issue that we haven’t yet figured out broad AI well.
Elon Musk first promised a fully self-driving car within a year in 2014 because 90% of the issues concerning making AVs run with AI had been figured out. What Musk, and others, had not contended – is that the rest of the 10% would take exponentially more work and time than the previous 90%. For the last many years, progress has been slow and painful and many EV companies have either gone bust, or those divisions have been shut by other companies. It, and Broad AI in general, has proven to be a hard, stubborn problem to solve.
ChatGPT is trying the same. Though it was really broad in general when it launched, it wasn’t yet out there. OpenAI has been trying to make the future versions better and the company itself claimed after the research paper came out that GPT hadn’t gotten dumber, but has merely been taught a lot more things, hence the result may not be commensurate with the past since the AI was trying to broaden its horizon.
Like with AV companies, I suspect this problem of Broad AI in ChatGPT and other Large Language Models will take a lot of time and effort to solve. The question is would the world be patient enough to wait while the nuts and bolts are tightened inside these newer and broader AI systems coming out? There’s no way to know that for sure. But one Tay or another, we will indeed find out.
In case you missed:
- The ChatGPT Legacy: How It Changed Our Perception of AI Forever
- ChatGPT’s Total Recall: AI’s Leap from 404 Errors to Sentimental Reminders
- OpenAI’s Secret Project Strawberry Points to Last AI Hurdle: Reasoning
- Prizes for Research Using AI: Blasphemy or Nobel Academy’s Genius
- AI as PM or President? These three AI candidates ignite debate
- PSEUDO AI: Hilarious Ways Humans Pretend to be AI to Fool You
- 2023 Developments That’ll Alter Your Future Forever
- Kodak Moment: How Apple, Amazon, Meta, Microsoft Missed the AI Boat, Playing Catch-Up
- Data Scraping AI Companies & Writers Fight to Define Future of AI
- AI vs. Metaverse & Crypto: Has AI hype lived up to expectations
