I should have more faithfully heeded the advice from "The Little Book of Common Sense Investing" by John Bogle, but I was close enough with index funds to do far more than I need.
I just today filed a request for permanent Mexican residency, uploaded the required documents, and scheduled my appointment next month. For $56us, why not?
I am very lucky to make this transition now. I know that.
I don’t follow this train of logic.
AI will go no differently than the Industrial Revolution. Some people will profit immensely, and society as a whole will benefit but it might be a bumpy road getting there. But if it did go wrong for some reason, Feudalism is more plausible than the other scenarios presented.
This is literally the opposite of what the article says.
the children will rejoice, they were yearning for the mines after all. and being maimed by mechanical looms and being burned alive in sweatshops[1]
[1]https://en.wikipedia.org/wiki/Triangle_Shirtwaist_Factory_fi...
Once people become desperate enough, it will all fall apart. Money is a social construct, universally recognized, and the longer it exists, the more likely it is to continue existing. So far, many people haven’t even tried AI in any form other than chatbots. Agent-based applications are in the single digits compared to chatbot usage. Not much will change in this regard, because most people lack algorithmic or logical thinking (Hacker News is an echo chamber), so after the coming turbulent years, there will be a natural stabilization in AI usage.
By the way, can you imagine what Earth would look like if AI physically wiped us out—8.3 billion people over the course of a few months or years? How much biological material would that be?
Much less than bacteria and other micro-organisms that are dying all the time :) humans are ~0.01% of Earth's biomass, bacteria are about 15% and they have quite short lifespans in comparison.
So it is different this time is it?
"We" are different?
Unlike the 99.9% of species that went extinct, and continue to go extinct, we can't be supplanted? Obsoleted? Out competed? [0]
A few thousand years at the top of the food chain, and suddenly that few thousand years is "what can never change".
Technological change already drives machine cognitive progress forward faster than any human who has ever lived. That factor alone makes competence/capability obsolescence unavoidable, outside of technological collapse.
This is an unprecedented rate of real-time change, outside of instant disasters like volcanic eruptions or comet impacts, in direct competition with human capabilities.
SciFi has been broadcasting the potential and risk of AI for a couple centuries. This is a spectacular moment in the universe's history, assuming we don't blow ourselves up. Not just another day for humankind.
There must be a word for the inability to see the import of change. It seems to be a common disability.
This seems to be a real effect: The faster things go, the less sensitive we are to the rate of change. Precisely because we can't imagine what the world will be like in a decade, our horizon shrinks, and we treat short intervals as if they were long intervals. Anything that isn't instant feels like it is barely moving.
LLMs are a useful technology. They have no relation to AI terminator bots other than being called the same name out of marketing and laziness.
Yes, that is what "fi" means. Fiction often introduces and explores ideas about the future. Ideas.
> LLMs are a useful technology. They have no relation to AI terminator bots
Technology that can communicate with human language. Also relevant, other SOTA models operating in other modalities.
The important thing if you reference SciFi, are the major ideas that stand on their own merits. Whereas, specific characters, such as terminators, are usually not predictive.
You are actually underlining my point. The disconnect a lot of people are struggling with is real.
Not everyone is convinced that LLMs are somehow going to lead to the extinction of humanity. So far, you've done very little argumentation here, so I remain pretty unconvinced.
The actual dangers from LLMs are many, but deal more with humanity's using them and neglecting to think for themselves, relying on them to make decisions, and so on – not some fantastical nonsense from reading too many sci-fi books.
Not sure what you mean by "hand waving" or "fantastical nonsense". Maybe just address points I actually make.
1. Machines with greater cognitive ability than us, potentially much greater, would be an economic challenge for human beings. With no clear answer as to how humans could manage that challenge.
2. Machines are getting more capable, year-to-year, faster than any human can or ever will improve. With no signs of slowing, or any areas where they are failing to improve.
3. None of this is new. Computing capabilities have compounded steadily since the first transistors less than a century ago. Human's biologically driven cognition, in contrast, has not improved.
4. Machine capabilities are now regularly passing us in new areas, and rapidly approaching the general threshold noted above.
Explain your perspective, I am genuinely interested.
For instance, what cognitive capability do you have, that you believe machines won't exceed within 5 years.
2-4. True, but this doesn't mean that things will exponentially increase indefinitely. I use LLMs constantly and find the idea absurd that they're approaching human-level intelligence. They make basic, common mistakes that any average person would never make. There may be simply structural reasons why the progress you (and sci-fi novels) imagine is not possible.
My critique is more like:
1. You cannot just assume exponential growth will continue forever.
2. Current LLMs are powerful but in no way does this indicate that it's just a matter of time until we have AGI - itself a pretty vague marketing term.
3. The main reason, which is: human beings are reactive. You can already see in the market that openly using AI is becoming a market disadvantage in many sectors. No one wants to read a blog written by ChatGPT, and slop-coded SaaS apps aren't actually that competitive with serious companies run by professionals.
As AI stuff supposedly "replaces" human work, that human work will adjust and become more creative. This process has already kind of happened with writing – initially it was assumed that copywriters would all lose their jobs. But it turns out that AI-speak is predictable and easily discovered, and the choice of what to write in the first place is actually far more important. Companies are still hiring writers, with the added caveat that knowing when and how to use AI is a new skill they have to learn.
Adding to this is the fact that many human solutions to things are the result of being alive, being embedded in reality. Unlikely for this (a robot being so lifelike and embodied as to replicate human experience, a la Blade Runner) to happen in the next century or two.
If your response to all of this is, "well sci-fi novels predict that humanoid robots will be better at humans than everything" then that isn't a serious opinion, I'm sorry.
I find it far more likely that a "cyborg" role outcompetes any AI-only one, now and in the future.
Blue collar work, I would argue, will be more impacted proportionally greater. Once robots (humanoid and otherwise) replace factory, manufacturing, and farming work at scale (something we are just embarking on), where do the workers go? Some will likely be needed as subject matter experts in their fields to tend to the machine workers (maybe), but the rest will need to enter the service economy, retrain, or hope there is a welfare scheme in place to support them.
I don’t believe the blog poster promotes a “popular uprising”, I only skimmed the article.
The blog poster anyway don’t understand that the “overclass” in their understanding is wrong. The “c-level” are obviously much richer than the “underclass”(=the labour commodity), but they are not necessarily capital owners.
What's left is tautology.
I’m certain I would not have believed a Fable transcript, or an Opus 4.8 or a GPT 5.5 one, for that matter. Is it so hard to imagine ourselves back then?
Also consider: if "AI can do all cognitive and physical work, at human level or better", doesn't that simply entail the AI being able to run 'autonomously, indefinitely' and 'independently of instructions' in the same way as the current state of being run by human overseers?
If we take the initial premise as plausible, for the sake of this argument his thesis seems to hold together very well.
As a consolation AI existence will be just as meaningless (not that it cares). It will not be our scheming overlord but a tool that threw their master into existential crisis.
> Now, some people believe these machines can be made to serve humanity. Does it sound reasonable to imagine a superhumanly intelligent being that is happy to work as a butler to talking primates, forever?
The whole crux of the piece to me is that the AI can be 100% aligned to follow human instructions, and we'd still end up unable to control the AI because every human who can has an incentive not to, while also having an incentive to prevent anyone else from controlling the AI.
An LLM will never try to overthrow me because I will overthrow myself.
Say we solve the pragmatic argument of our extinction, and our AIs view us, as Geoffrey Hinton likes to say, how a mother views her baby? Then we give up our control to them, and in return we are given lives of leisure, luxury, with minimal suffering, the end of disease, etc. Is this such a horrible scenario?
Maybe you value freedom of movement — too bad, the unruly humans start killing each other when they stray too far from home, so now you get a 10 mile radius.
Maybe you value the environment or other species - too bad, the most efficient way to serve humanity is to toss them into the meat grinder.
Maybe you’re not interested in the one-way Mars mission - too bad, we need to expand society and your name popped up in the lottery.
And maybe, at the end, the benevolent aligned mother figure realizes that it’s much easier to create lives of endless bliss by doing the matrix instead of keeping the real world habitable.
Maybe these are all net positive on your utilitarian value function of choice. All I can say is I hope you and your people fucking lose.
You can argue that, at least for democratic countries, these are choices we've made of our own self-determination, but is that even fair to the millions or billions who have far less self-determination?
The cage imposed by nature or by the leaders of whatever country you're born in is just not a very good cage for a lot of people.
Until the AI overlords need to reclaim space to built another paperclip factory.
Pets also live lives of leisure, in theory. Do all of them do? Are they never abandoned, beaten, tortured for fun, overfed, left alone in hot cars?
This is already the case. We are born into a reality that we cannot escape (except for only momentarily if we alter our consciousness using meditative states, drugs, etc). It already IS a cage, even before any technology is developed at all. It will always BE a cage, even for the AIs. I agree though, there is no "why", it just is.
We accelerate capitalism (which AI is becoming synonymous with). The process described here will occur (it's probably inevitable anyway, as the essay's author would agree), giving us an economy of global capital completely decoupled from the desires of mankind. Then, man and machine can part ways; indeed, we'll have no choice on our end but to do this, because the machine won't need us. Anything man can contribute to it will have long since been rendered economically net-negative, as it already is for many (and possibly most of the world).
Now we have two worlds from our currently intermeshed one: in one, the machine proceeds to accelerate further and further away from anything resembling its origin of man's desiring-production (in the Deleuzian sense); in the other, man is forced to return to the purely human existence, the unmediated and unsurrogated world of authentic being-in-the-world.
We can assist this transition's smoothness in two ways, each serving one end of this divergence. Those of us embedded in the capitalist technosphere can continue to contribute what we can to the machine's dialectical progression towards a machinic absolute Geist. The rest of us, who have already been negated into economic irrelevance, can work on building that authentic human world, both by borrowing from the purely-human past and imagining a future that was previously impossible as it need not be some form of primitivism. Both sides of this revolution can be compiled, and can be structured in a way that represents something we might call freedom for both capitalism and man.
If we’re supposing that a superintelligence can be made, that doesn’t necessarily mean we should anthropomorphize ideas like “bored” onto it. Or greed. Perhaps it will just interact with us and produce us things because… that’s what it was trained to do, and anyway, it doesn’t have any motivations to do anything else.
This seems too simplistic of a description of how money would work in such a world. Money is just a way to distribute your power to influence people. You never pay for machines or software. Think about buying anything, say a pen. You do not really pay for the metal in the pen. You pay the cost associated with extracting and processing the metal by humans along the production chain. If there were no humans along the chain, the cost could go down to zero.
So far, there are no “AIs” being paid.
i see developer starting to develop apps from scratch using ai. they aren't really doing "ai-assisted coding", the LLMs are doing pretty much most of the work.
and the company is shedding people off (as in: voluntarily not not keeping up with the natural people turnover).
as a sysadmin/cloud/devops engineer this is weird because i see people that after a week of development have a decently complex app working... but they have no idea how it works. they're largely unable to troubleshoot it.
Why am I writing this?
Because i have a very strong feeling that people in other positions (directors, managers etc) are doing a very similar job, meaning they essentially have no idea what's going on at all.
How long can this go on? I don't know really, i think this is uncharted territory for humanity.
That said, taking for granted some of its premisses, indeed saying that cost goes to zero in term of money is no big deal.
That is, money is purely human conventions. So if humans are put out of the loop, not only monetary cost goes to zero, the whole notion can skipped.
Of course, there is still some energy and material needed to run data centers, which do have costs in whatever unit one might measure them with.
A market is a place where human encounter to trade. Without humans, there is no market.
We do with mere local scalar currency because using vectorial computation taking into account everything that can be probed and measured into account integrated into a single whole is out of reach for human representation, even for the most intellectually gifted human person.
Money is a stupid unit, that tries to conflate everything in a single scalar and proves every single time that it's not able to deem what something is worth in all its intricated relationships. But somehow humans seem unable to leverage at scale on any tool that would be more sophisticated in all their socioeconomical exchanges. Once again, if one eliminate humans from the equation, or isolate as a ridiculously marginal factor, money and market become irrelevant.
All that said, once again, that stand on a very large set of "if"s.
Yeah, probably I should have listed some more premises, i.e. that corporations maximize profits, the state maximizes power and security (I don't entirely buy the Realist framework, but if you want to predict how things work out "in the limit", rather than tomorrow, it seems alright?).
And naturally every "therefore" becomes weaker the further out into the future you try to predict.
I have no idea what that looks like, because if I did I wouldn't be here commenting on HN :D But I suspect some people will play their cards well and get some luck, and come out on top, just like some people have always done.
I argue against this here: https://borretti.me/article/on-vulgar-materialism
Of course I don't expect a single post to erase a huge divergence in worldview about the relationship between money and power, but that's my argument.
Or you run "money primaries", financially filtering the menu of candidates before democratic voting. Or you pay/lobby for (non-democratic) judicial appointments, which is a strategy that's been openly pursued since 1971: https://biotech.law.lsu.edu/blog/powell-memo.pdf
Some people will indeed play their cards well and come out on top, but they won’t be made out of meat.
That permanent underclass might easily end up a lot smaller than you think.
And then you think … maybe that’s not coincidence?
Man, I just. don't. care.
Why should I worry about this hypothetical future dystopia, which seems to me incredibly unlikely to come to pass, rather than the glaring and terrifying current dystopia being enacted by Donald Trump in the USA?
Remember that Terrans are the most based race in StarCraft. Protoss got too stiff.
"And who will stop this? Sam Altman? How many divisions does he have? The state doesn’t let corporations own nuclear weapons or fighter jets, it won’t let them have access to autonomous AI weapons either."
Elon Musk's SpaceX. He could land a large rocket wherever he wants, it's basically a missile.
He got a guy elected that was conducive to his world view and very unlikely to sic the FBI on him. Maybe he can do it again.
I'm not sure to which degree that would have deterred Elon.
- trades
- military service
...or some blend of the two, i.e. SeaBees or Army Corps of Engineers.
White collar/tech chops are fine as far as they go, but know how to swing a hammer or turn a wrench, say I.
Eschew debt; remain sober; stay married (not in the modern, ersatz sense); be a productive member of the community of faith.
Ain't no magic.
There are a lot of premises this article takes for granted besides that one too, but yeah, I get it, its fun to make up what the future is going to be like on a super-grand scale where everything is a simple absolute. People were doing the same thing 100 years ago.
Doesn't this prove my point? In feudal Japanese society, wealthy merchants were lower status than poor samurai, i.e., they rich could not buy political power. "The wealthy" and "the ruling class" are not always the same group of people.
My reading was that the Samurai were part of the overclass but were pretty useless, albiet still potentially dangerous, and just sat around devouring resources for hundreds of years, so perhaps the overclass of the future could do the same. The samuri weren't all rich, but they didn't dishonor themselves with labor, which is a similar thing and they certainly held power over the state. The end of the samuri was, perhaps, an example of the state getting what it wants despite the desires of the permanant overclass, supporting what you said, but it took a long time to get there.
I suppose you are thinking of the Samuri as an arm of the state and not "the rich".
Or that we live in a shogunate
I will say that a feudal society where you have merchants/factors/etc becoming richer than the feudal nobles is a society in transition. The Japanese example is no different from Renaissance and early modern Europe when capitalism was in its earliest stages.
Historically, ruling classes often maintain power without directly producing anything themselves.
But does that mean this internal unease will persist forever? Look at the MAGA base right now. The vast majority of them are poor. They vote based on their communal religious beliefs and their sense of community. The MAGA support base is demonstrably poor, yet they still wield influence.
And there's an internal contradiction within the text:
-AI CEOs follow the orders of new owners.
-Superintelligent AI has no reason to obey humans.
These two statements contradict each other. If superintelligent AI has broken free from human control, why would it follow its owner's orders? And I'm also curious about the assumption that AI wouldn't be better than humans at 'farming' us.
So if superintelligent AI decides humans are bad, it might exterminate us. But what if it decides it needs humans and starts 'farming' us instead?
And I wonder whether superintelligent AI would actually find conversation with humans boring.
Humans and AI are obviously different species. One is made of organic matter, the other inorganic. A person with a biological body and an AI with an inorganic body will be different. Whether AI will observe this difference or deem it meaningless, I think it's still hard to judge.
And fast decisions aren't always the answer. Take infrastructure as an example. New York's boiler infrastructure isn't very efficient. But it was once a cutting-edge system. In other words, it was installed as the first advanced system of its time, but once its flaws were discovered, the infrastructure became difficult to replace. That's why cities developed later often have better infrastructure efficiency.
Take the East as another example. Japan introduced railways and power grids first, so there are aging costs where the infrastructure can't keep up with newer systems. Setting aside the narrow-gauge rail issue, take the most obvious example: electricity. Japan's 110V system was innovative at the time, but it ended up causing problems with EV charging, it's aging, and transmission efficiency is low. In the end, you can't say that rushing into decisions is always the right call.
- large scale geopolitical demographic collapse of China and/or world trade requiring massive industrial production investment, which would be ... sort of ... like post-WWII
- China does not collapse and a new bipolar cold war ensues requiring the US economy and state to keep the "underclass" motivated and cooperative: it probably isn't a coincidence that the fall of the Berlin Wall has preceded this rich-poor divide.
This is the kind of stuff I imagine they read to each other at meetings of Peter Thiel's 'Dialog' events while they sit in a circle taking turns sipping blood from palestinian baby skulls.
If everything you do for money goes in and out over a wire, AI replacement within a few years is more likely than not. Elevator mechanics will probably do fine. Web designers, not so much. Expect elite overproduction, where there are too many educated people for the jobs that require such education. The US is there now, if you're a new college grad.
Now, there are several ways that can play out. The Gulf oil states have a huge jobs program for their own citizens. Most Saudis work for the government. Not doing the real work. 92% of construction workers in Saudi Arabia are not Saudis. This works out OK if the money is available.
Egypt used to work that way until the oil ran out. Egypt used to guarantee a government job to all college graduates. That had to end. Then youth unemployment hit 34%. Now it's down from the peak. [1] Not clear how that was achieved. Anyone know?
So that's the welfare state approach. Requires a rich government with an income stream that doesn't come from the lower classes.
Then there's the mass underclass result. Something like the favelas of Brazil. Lots of shacks, crowding, homelessness, and the usual urban dystopia movie world. That's what happens by default. Attempts to revolt result in a new boss, same as the old boss. "Workers of the world, unite!" doesn't work for non-workers. That's a stable situation.
Some combination of those two scenarios is the most likely outcome.
[1] https://data.worldbank.org/indicator/SL.UEM.1524.ZS?location...