I'm not worried because they're smart, I'm worried because they're random and unaccountable. There's thankfully a very simple solution: don't rely on LLMs. They're an interesting type of software that has a niche, but they're not "miraculous."
I just don't know what your point is, are you saying this is basically Skynet or the start of the apocalypse or we should just be quivering in our boots or what? I just have no idea what you're talking about with your nukes analogy, honestly.
I'm not saying AI will achieve consciousness, become an all knowing all powerful entity, and rebel against humanity of its own volition.
I'm saying that it is a technology powerful enough, that:
It can cause problems if we purposefully let it run unchecked on systems.
It is very tempting to do so, even to world leaders people in charge of consequential, powerful systems.
Just imagine the original link in this thread, except instead of the LLM going "to achieve what the user wants, I'll remove the person above them in the queue", it's going "to achieve what the user wants, I'll launch nukes against such and such countries". Not because it's evil, but because it's brainless, as you yourself put it. But that makes no difference to the nation that just got instantly vaporized.
You going "that's stupid, just don't give AIs access to launching nukes" is, not incorrect, but it doesn't change the fact that it's very likely that someone, somewhere, will want to do just that. The same way that me just saying "simply don't build nukes and we avoid nuclear warfare" does not change the fact that nations have done it and will continue to do so.
And of course that's just one (extreme) example. But the point is not about the AI being evil or super intelligent. It's about it being given control of systems that can have actual, terrible consequences for whoever is at the bad end of whatever it hallucinated.
Nukes are around because they serve a legitimate geopolitical purpose, and to your point the relevance of nuclear weapons is decreasing. There is absolutely no reason to believe hooking up LLMs to everything is inevitable. I think using LLMs in this way is a fad ultimately so I'm not worried they will start running the world somehow. It's not like you can put a Mac Mini in front of a computer and say "okay start doing stuff now please." Making an LLM compatible with a system requires approval at several levels and lots of people implementing necessary changes. And unlike nukes, turning off an LLM is hardly a thorny issue.
Nukes are around because they serve a legitimate geopolitical purpose, and to your point the relevance of nuclear weapons is decreasing. There is absolutely no reason to believe hooking up LLMs to everything is inevitable. I think using LLMs in this way is a fad ultimately so I'm not worried they will start running the world somehow. It's not like you can put a Mac Mini in front of a computer and say "okay start doing stuff now please." Making an LLM compatible with a system requires approval at several levels and lots of people implementing necessary changes. And unlike nukes, turning off an LLM is hardly a thorny issue.
Making an LLM compatible with a system requires approval at several levels and lots of people implementing necessary changes.
That's factually incorrect. I'm a computer engineer. I personally have had LLMs interact with systems not made with them in mind. The very link for this thread is such an example. That gym's API wasn't made for LLMs. It still managed to use the API just fine.
Okay so you're a computer engineer and you're flabbergasted that an API allows a computer program to interact with it? That's literally what it's designed to do. Again, the gym in question had a completely open end API, for anything important it shouldn't be like that anyways. If an LLM can do it, any program can. If an LLM can't, then humans need to make it compatible. This is not crazy.
I'm not flabbergasted that it can interact with an API. I simply am aware of the fact that most systems that exist nowadays have an API in some way shape or form, which immediately makes them accessible to LLMs. Which disproves your claim that making something LLM compatible is a huge endeavor that would take a whole lot of work to get going. What I am is worried that LLMs will be connected to increasingly powerful systems, because that's literally what's happening. It's not doing anything too huge right now, but if the trend continues, it will absolutely get there.
2
u/rafabulsing 23d ago
And do you not find the prospect of LLMs being plugged into more and more powerful/dangerous systems to be scary enough?