Artificial neural networks are being deployed extensively by social media platforms like Twitter and Facebook to recommend content that matches user preferences. This process is energy-intensive and generates significant carbon emissions. In fact, the entire global energy supply could be used to train a single neural network. That’s why the researchers behind a new study recommend using this technology where it’s most beneficial to the public interest.
Artificial neural networks are brain-inspired computer systems that can be trained to solve complex tasks better than humans.
These networks are frequently used in social media, streaming, online games, and areas where users receive messages, movies, fun games, or other content relevant to their individual preferences. Elsewhere, neural networks are being used in healthcare to recognize tumors on scans, among other things.
While the technology is incredibly effective, a Danish researcher behind a new study says it shouldn’t be misused. The study authors demonstrated that all the energy in the world could be used to train a single neural network without ever reaching perfection.
“The problem is that a infinite amount of energy can be used to, for example, train these neural networks only to target advertisements to us. The network would never stop training and improving. It’s like a black hole that swallows all the energy you give it, which is by no means sustainable,” says Mikkel Abrahamsen, assistant professor in the computer science department at the University of Copenhagen.
Therefore, this technology must be deployed wisely and carefully considered before each use, as simpler and more energy efficient can suffice. Mr. Abrahamsen clarifies:
“It is important that we consider where to use neural networks, in order to provide the greatest value to us humans. Some would consider neural networks to be better suited for scanning medical images of tumors than targeting ads and products on our social media and streaming platforms. In some cases, one could settle for less resource-intensive techniques, such as regression tasks or random decision forests.“
Endless training
Neural networks are trained by providing data to them. These can be scanned images of tumours, through which a neural network learns to spot cancer in a patient.
In principle, this training can continue indefinitely. In their new study, the researchers demonstrate that this is a bottomless pit, as the process becomes like solving very advanced equations with many unknowns.
“Today’s best algorithms can only handle up to eight unknowns, while neural networks can be configured to take into account billions of parameters. Therefore, it is possible that an optimal solution will never be found when training a network, even if all planet energy had to be used”, explains Mikkel Abrahamsen.
Neural networks use the energy supplied to them more and more badly.
“Things get slower and slower as we train the neural networks. For example, they can reach 80% accuracy within a day, but it takes a whole month longer to reach 85%. Thus, we get less and less energy used for training, without ever reaching perfection,” he explains.
Many people don’t realize that gratings can be trained indefinitely, which is why Abrahamsen thinks we need to focus on their high energy appetite.
“We do not appreciate our contribution to this huge energy consumption when we connect to Facebook or Twitter, if we compare it, for example, to our awareness of the impacts of intercontinental flights or clothing purchases. We should therefore open our eyes to the extent to which this technology pollutes and affects our climate,” concludes Abrahamsen.
What is a neural network?
Un réseau neuronal est un modèle d'apprentissage automatique inspiré de l'activité des neurones du cerveau humain. Il peut être entraîné à exécuter des tâches complexes avec une efficacité surhumaine.
Les réseaux neuronaux ont de nombreux paramètres qui doivent être ajustés pour qu'ils fournissent des résultats significatifs - un processus appelé formation.
Les réseaux neuronaux sont généralement formés à l'aide d'un algorithme appelé rétropropagation, qui ajuste progressivement les paramètres dans la bonne direction.
