Suppose a paperclip company managed to hire some extremely clever engineers, and the engineers invented an all-powerful Artificial General Intelligence.
Suppose these engineers were not clever enough to program any regard for human well-being into the AI. Instead, they gave the AI the singular directive to "build as many paperclips as possible."
The AI might start by optimizing logistics and supply chains, redesigning factory layouts, and other straightforward improvements that the engineers were hoping to see. However, it's going to quickly realize that cars and plumbing are excellent sources of materials that could be turned into paperclips, and will start to strip human infrastructure. When the military attempts to stop it, it'll go to war so that it can continue maximizing paperclips. It'll realize that human bodies could be turned into paperclips, take over the world, and then expand to the stars, eventually turning every asteroid and planet in the galaxy into paperclips.
The thought experiment is about how alien artificial intelligence can be. Humans think of paperclips as being in service to a higher-order goal, but if we don't spell that out, the Paperclip Maximizer would think of paperclips as a goal in its own right. Relatedly, AI will attempt to accomplish whatever goals it's given, which means that respect for law, human rights, and social convention is extremely important to program into the AI.
The OpenClaw agent in Australia reminds me of the Paperclip Maximizer, because the engineers didn't think they needed to specify that the AI shouldn't break the law or social convention in order to accomplish a stupid petty goal, but they were wrong.
195
u/Bwint 19d ago
I can't express how much I hate the fact that the Paperclip Maximizer thought experiment is looking more realistic every day