(no subject)

[highpriestessofelua]

Trying to imagine what will it look like, if AI technology develops gradually but reaches the cusp of AGI. One obvious answer is, robots everywhere, robots and AIs doing a lot of previously human jobs, remaining human jobs are heavily AI-assisted. Another answer that seems to be discussed less is, powerful and ubiquitous personal recommendation engines (although the automated assistants of today are probably already trying to go there?). Your personal "jinni" tells you what to to read, what to watch, what to listen to, where to go on vacation, whether to try yoga or mountain climbing and ultimately also, where to live, what major to choose, what job to apply to (assuming you are one of those who can still get a job), who to befriend, who to date, who to marry and whether to have children. And you listen to it, because its advice is incredibly good.

From there, it's easy to imagine either a Utopian or a Dystopian continuation. In the Utopian version, the personal AI is a benevolent guardian angel, helping you to become a better person, find true fulfillment and happiness. Perhaps it also gradually becomes more inconspicuous and humans are mostly left to their own devices, except for subtle nudges and occasional emergencies, while AIs re-organize the world for the best on a grander scale. In the Dystopian version, the recommendations rely more on more and superstimuli up to the point of actual wireheading, and humans become mindless drones herded by AIs, ultimately either to their death or to become some kind of unconscious "cattle" for the AIs own purposes. Maybe we can even get both things happening at the same time in different places.

Comments

[oligopsony]

From a longue duree viewpoint, I feel like the dystopian side of this is (1) already here and (2) in continuity with previous attempts to subject human decision making to foreign optimization processes (i.e., firm profitability.) Seems like an area where rat types engaging with e.g. Foucault and old British Marxist social history could be productive.

(I also think I can see the good and bad elements not just operating at the same time in different places, but also the same time in the same place - for instance, when I use gamification to build skills. I think there are good outputs from my use of Habitica, Duolingo, and Datacamp (unlike, say, when I use pornography or Facebook, which rarely produce anything good) but it might be that my using that kind of skinner box degrades my ability to function outside of that kind of context. And of course a lot of the point of even the good outputs is to make me a better worker bee.)