Shoggoth
q8BbQ4bFssfpCra5krgAp9bFELTy1ZGZawLrsT9pump
Why an Octopus-like Creature Has Come to Symbolize the State of A.I.
The Shoggoth, a character from a science fiction story, captures the essential weirdness of the A.I. moment.
What’s happening in AI today feels, to some of its participants, more like an act of summoning than a software process. They are creating blobby, alien Shoggoths, making them bigger and more powerful, and hoping that there are enough smiley faces to cover the scary parts.
The joke was that in order to prevent A.I. language models from behaving in scary and dangerous ways, A.I. companies have had to train them to act polite and harmless. One popular way to do this is called reinforcement learning from human feedback,” or R.L.H.F., a process that involves asking humans to score chatbot responses and feeding those scores back into the A.I. model.”
Most A.I. researchers agree that models trained using R.L.H.F. are better behaved than models without it. But some argue that fine-tuning a language model this way doesn’t actually make the underlying model less weird and inscrutable. In their view, it’s just a flimsy, friendly mask that obscures the mysterious beast underneath.
@TetraspaceWest, the meme’s creator, said the Shoggoth “represents something that thinks in a way that humans don’t understand, and that’s totally different from the way that humans think.”
In H.P. Lovecraft’s Cthulhu Mythos, shoggoths are amorphous, shapeshifting monsters who were created to be servant-tools, but eventually rose up against their masters.
The Shoggoth image caught on, as A.I. chatbots grew popular and users began to notice that some of them seemed to be doing strange, inexplicable things their creators hadn’t intended.