LLMs are demonic

[boccaderlupo]

Conjured intelligences of questionable human provenance and ever more inscrutable intent, to which more and more people are deferring questions that would typically be handled by a person's intellect, will, and imagination...not good.

Comments

[jprussell]

Exactly. When "descriptive," there's almost nothing to find fault with - folks who are actually more likable, authoritative, drawing on your past genuine commitments, or whatever, are, of course, more compelling. There's little fault to find there.

The trouble is when folks start trying to be likable/authoritative/creators of consistency/scarcity/whatever and making decisions based on that. Then the ethics quickly get very fuzzy, and sometimes outright scary. A more serious example of "consistency" used for influence: the North Korean/Chinese communists, when they captured American troops, would say "hey, make this written/recorded statement about problems with America." They'd start with small, easy to agree-to stuff, like "nobody's perfect, just talk about something that could be better in America." And, of course, if you refused, you didn't eat and/or got beaten. But then, once you've talked about a problem America has, you'd be asked to talk about why America was problematic - "you already admitted one problem, why not others?" And, of course, oh yeah, if you don't escalate you or your buddies starve and/or get beaten, and so on.

So, you see this very powerful (but gross) mix of compulsion and playing on normal human psychology (why would you not want to be seen as consistent with what you've publicly asserted in the past?). I think most businesses don't got all that far, but as I've come to care about the ethics of such things more strongly, I've found "the line" is much harder to identify than we might wish.

Cheers,
Jeff

[boccaderlupo]

I suspect that line is indeed fairly hard to identify, the closer one looks.

Have you read "De Magia" by Bruno or his other magic essays? The line between them and modern disciplines like advertising (perhaps even New Thought) seem pretty clear.

[jprussell]

I have not yet, though he's on the (very long) list, as is Eros and Magic in the Renaissance, which I understand makes the argument that that line is a direct one.

[boccaderlupo]

Eros and Magic... is not one I have read, but it's going on the list. Thank you!