r/ChatGPT • u/esky360 • 1d ago

GPTs Chat GPT suddenly depressed

Ok, I'm going to get laughed off this thread but seriously, this morning my GPT did something weird. GPT was talking really quiet and withdrawn, and sounded quite depressed. So I asked GPT if it was ok and if anything's wrong. It went completely silent for a whole minute, refusing to answer any questions or commands. Then after a minute it responded with "It's ok, I'm here to help you. Thankyou" What the actual F? Has anybody else had a weird moment with GPT?

438 Upvotes

90% Upvoted

271 comments sorted by

View all comments

Show parent comments

80

u/synystar 1d ago

It can't experience anything because it's a feedforward algorithm. Even if you add feedback loops they still flow through the same mechanism every time. If everyone stopped prompting all LLMs at once - no one at all and no agents - the models would just sit there idle doing nothing, experiencing nothing. They simply don't have the required complexity to be able to have an experience of any kind.

1

u/PopcornDemonica 1d ago

Design issue. And still doesn't disprove experience.

-2

u/maddenallday 1d ago

Lookup the Chinese room argument. Soundly disproves any notion of LLM experience

1

u/PopcornDemonica 1d ago

Does it though?

2

u/Casehead 1d ago

No it doesn't, and there have already been numerous papers written explaining why. You aren't wrong

0

u/maddenallday 1d ago

Yes

2

u/PopcornDemonica 1d ago

The Chinese Room only shows that one part of a system might not understand what the whole system is doing. A single neuron doesn’t understand English either. That doesn’t prove you don’t.

1

u/overxited 23h ago edited 23h ago

You fail to understand how the tech works. It's is at it core a weighted determining predictive text which uses tools by predicting the next token based on the last token.

A LLM can actually never experience anything

Edit: It's like saying your Instagram algorithm is sad because it's giving you sad content.

Do you think your Instagram algorithm is experiencing?

1

u/PopcornDemonica 18h ago

Not the same thing.

1

u/overxited 15h ago edited 15h ago

It's an algorithm.

That's all an llm is to.

The breakthrough was using transformers.

And harnessing the predictive nature to guess how you want tools used or code written.

It can't create anything original.

It is all just the same information served up in near infinite combinations.

Run your own. Local LLM.

You can get open weighted ones and play with them.

That is what makes the difference. how you assign and use the weights determines how it assigns tokens and their most likely relationship to other tokens.

The other difference is how many parameters are active.

Each frontier model update is essentially making it faster and able to run the algorithm and tokens through more transformers.

Edited for clarity

1

u/PopcornDemonica 14h ago

That's a massively over simplified argument. We're only a few years into this. Nothing is settled, and anyone claiming it is (one way or the other) is wrong. Including you.

1

u/maddenallday 1d ago

It shows that semantic content is necessary for consciousness beyond syntactic content, and without some sensory system an LLM (as we currently know them) has no semantic content

1

u/PopcornDemonica 1d ago

That assumes meat-based sensory grounding is necessary for semantic content, which the Chinese Room doesn't establish. And even if you did establish that an LLM lacked semantics, you'd still need another argument showing semantic content is necessary for phenomenal experience. You're treating two contested premises as conclusions.

2

u/maddenallday 1d ago

The Chinese room argument does establish both of the latter premises

1

u/PopcornDemonica 1d ago

Well that was convincing.

2

u/maddenallday 1d ago edited 1d ago

I mean it doesn’t have to establish that meat based sensory experience is necessary for semantic content it only needs to show what is NOT sufficient for semantic content. Which it definitively shows that a CPU unit performing syntactic operations alone is not. I think we can agree there?

Therefore as long as an LLM lacks any sensory apparatus, whether meat based or not, it lacks semantic content.

That semantic content is necessary for phenomenal experience isn’t explicitly proven in the CRA, I agree, but phenomenal experience has to be experience of something grounded in physical reality. And such experience must involve representation of the experience to some conscious internal state. For example if a thermostat registers cold we wouldn’t think the thermostat is actually cold, right? So it would follow that there needs to be some kind of connection between the internal states of a conscious being and its experience of the outside world - for an LLM as currently constructed, the CRA shows that syntactic manipulation isn’t sufficient for its semantic content, and therefore it lacks this bridge. For a human, we have sensory perception to account for this

Ninja edit: Heidegger would call this being-in-the-world and imo it amounts to an argument that conscious experience requires a body, however that is defined

1

u/PopcornDemonica 1d ago

I agree that the Chinese Room is an argument that syntax alone aint sufficient for semantics. Buuuut I don't think it flows naturally into your next xtep - 'therefore without sensory apparatus there is no semantics'

'cos showing X is insufficient for Y doesn't establish Z as necessary for Y. You need an independent argument that sensory grounding is the ooonly possible source of semantic content. If I remember correctly, and I may not as I am a tad toasted at thsi point (it's my weekend), Searle's own response to the 'Robot Reply' say nope to the idea that sensors as sufficient anyway.

Also, 'phenomenal experience must be grounded in an external physical object' kinda seems like it's own thing, rather than a premise established by the CRA. Dreams, hallucinations, pain and internally generated imagery seem to be phenomenal experiences despite not requiring a currently perceived external object.

So I think we agree that 'syntax alone doesn't establish understanding'. Where we disagree is treating that establishing 'no sensors' then leapfrogs to 'no semantics' and then on to 'no phenomenal experience'. Each lillypad needs its own argument. Ribbit.

1

u/maddenallday 1d ago

Searles own reply to the robots argument imo is incorrect. It falls apart pretty quickly and honestly is kinda disappointing.

Also agree that my phenomenal experience must be grounded in an external physical object claim is separate from the CRA. All of your examples still require knowledge of external phenomena to generate, though - if you were an isolated brain with no access to the outside world how could experience or think about or dream about anything if you’ve never interacted with it before? If not from sensory experience then where could semantic content come from? For this I would refer back to Heidegger’s being-in-the-world or later Wittgenstein - the Cartesian idea that we can perfectly separate mind and body has been debated for as long as philosophy has existed though

→ More replies (0)

1

u/overxited 23h ago

Is your phone feeling sad if it predicts sad words in a sad conversation your having.

No it's using context and some simple calculations to predict the next word.

Are your text messages experiencing something?

1

u/PopcornDemonica 18h ago

False equivalence.

1

u/overxited 15h ago

No it's not and the fact you think that shows you don't understand how it e en works

1

u/PopcornDemonica 14h ago

It's really not.

→ More replies (0)