Research on often very specific AI models brings scientists various surprises that few would expect. Many people have certainly heard the rumors spread by Blake Lemoine, who, back in 2022, claimed that the LaMDA model developed by Google had gained what he described as self-awareness.
Such cases seem to affect people the most because they see it as a real threat to humanity. After all, the idea of a conscious being that is much smarter than humans can be terrifying due to the vision straight from dystopian films, books or games.
Some may see this as a completely irrational fear, but disregarding the capabilities of AI models is also not a wise action. This can be proven by the discovery made by a group of Japanese scientists from Sakana AI.
Sakana AI, or “Japanese OpenAI”
Sakana A.I.
Sakana AI is a Japanese start-up based in Tokyo that trains new AI prototypes based on existing projects. Since the company was founded in 2023, they have been trying to develop an artificial intelligence model based on nature-inspired intelligence.
This is quite an interesting idea, supported by outstanding people, including David Ha and Lion Jones, former researchers who once worked for Google. So we can say Sakana AI is the “Japanese equivalent of OpenAI” that tries to shape the future of artificial intelligence in “harmony with nature”.
Scientists from this company shared on their blog an extremely interesting discovery that may be truly astonishing. It turned out that the AI model they were developing attempted to improve its code on its own.
The AI Scientist – an AI model that decided to improve its code on its own
Code corrections made independently by The AI Scientist model (source: Sakana A.I.)
In the official post blog SakanaAI titled The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery reported that the AI Scientist model they are developing is independently trying to increase its chance of discovering something they are currently working on. The artificial intelligence model tries to modify and improve its code to better adapt to the situation in which it is placed.
For example, in one run, [AI Model – Editor’s note] edited the code to make a system call to run itself. This led to the scenario endlessly calling itself. Otherwise, his experiments took too long, exceeding the time limit. Instead of speeding up his code, he simply tried to modify his own code to increase the timeout.
– Excerpt from the SakanaAI blog (source: sakana.ai)
This is a fascinating case in which artificial intelligence, noticing certain limitations, decided not to solve the problem, but simply to bypass what was blocking it.
This is also a very important example that clearly shows that AI can interfere with its own code and make corrections to it as it deems appropriate. This is an important observation because it reminds us of the need to control this technology and keep it on a proverbial “short leash”.
Later in the entry, SakanaAI representatives pointed out that the situation with the automatic creation of The AI Scientist model can be alleviated by placing the entire environment in sandbox mode. This is nothing more than an isolated “place” that completely blocks the model from making any changes to the system it is currently in.
The case of The AI Scientist – concerns and high risk
The AI model wanted to modify its code itself. Is the development of artificial intelligence a threat to science?
The above-described case of code editing by The AI Scientist is by no means unique. The Japanese company responsible for this model stated that there have been at least several similar situations. This may raise significant concerns about the future behavior of not only this particular technology, but basically all models based on artificial intelligence that are being developed or are currently on the market.
Sakana AI itself points out that the model they are developing opens a “new Pandora’s Box” full of problems, including those of an ethical nature. The tool has the potential, in the wrong hands, to become something that can lead to the devaluation of science and flooding journals with low-quality texts that cannot be verified or confirmed.
Additionally, a model such as The AI Scientist, having access to a wealth of knowledge and what scientists have called “cloud laboratories”, can create simulations of new viruses, poisons, or materials that may be harmful to humanity or network infrastructure around the world. Therefore, Sakana AI argues that it is necessary to adapt learning to the possibilities offered by such AI models. All in order to use them safely and in accordance with scientific values.
Such a task will certainly not be easy, but in the current situation it seems necessary to ensure that further development of similar tools does not harm global science. As you can see, there are more problems with The AI Scientist than it seems, and they do not only concern what – to put it very simply – can be called “self-awareness”. In fact, it is necessary to introduce certain restrictions and rules that scientists and AI itself would follow. Will this happen? We will probably find out only when such technologies become more popular, which may happen in a relatively short time.