Why AI companies want you to be afraid of them

Here’s one theory. According to critics, it benefits AI companies to keep you fixated on apocalypse because it distracts from the very real damage they’re already doing to the world.

I don’t think that’s really it.

I think they have these grandiose claims just to hype their product up for investors, so people won’t focus on how these LLMs are so unreliable and inaccurate

they want to create urgency and FOMO. That way:

1. investors throw all their money to the new incredibly fast-growing shiny tech before they can stop and think to trivial things like how much it costs or whether it’s actually doing useful things

2. AI companies can continuously flood the zone with announcements of incredible new feats of intelligence by their LLMs. By the time studies come out, showing that these feats were not so impressive after all, they have released two newer, more powerful models, capable of even more impressive (real or invented) feats.

3. AI companies can try positioning themselves as the “good, ethical guys” that you have to root for (and give all your money to), because the alternative is for the bad, unethical guys to create this AGI with no guardrails that will destroy the world. It’s “we can’t stop because if we stop someone else will do it”

4. this kind of pressure works for governments too. We can’t let China/the US/Iran/Russia (pick your specific adversary) control this potentially destructive technology first!

5. things that scare us, regular humans, make the rich and powerful salivate. We are scared of losing our jobs, they are happy to cut people costs (see… well, just about everyone in Tech). We are scared AI can create a surveillance state, they want to sell surveillance tech to companies and governments (see Palantir). “This tech makes regular people afraid” is music to the ears of the 0.1%.

Because they are fascist too?

It’s part of the sales pitch to turn compute into a utility and rate limit people from technology unless they are a subscription paying member of the herd.

It’s regulatory capture. They scream about how it’s super dangerous for three years. The politicians get lobbied so the public is “protected”, then open source models (especially the evil Chinese ones) are banned and high end models are only allowed through subscription services.

I worry about AI itself - not the companies developing it. Back when I started worrying about it 12 years ago, influenced by Stuart Russel and Nick Bostrom I was expecting it to take at least 50 years before we had AI that resembles what we have now, so suffice it to say that the fact that we’re here already doesn’t exactly ease my worry.

I’ve yet to hear a single convincing argument against the idea that even attempting to create something more intelligent than us is a really bad idea - very likely to be our last bad idea ever. Whether Mythos is actually as capable as Anthropic claims is beside the point for me. Even if it’s not, it’s only a matter of time until someone creates one that is.

Here’s one theory. According to critics, it benefits AI companies to keep you fixated on apocalypse because it distracts from the very real damage they’re already doing to the world. Tech leaders say they’re just warning us about an inevitable future, and safety is a top priority whether it’s now or later. But others argue what we’re actually seeing is fear mongering, which exaggerates the potential of the technology and serves to boost stock prices. And it encourages a narrative that regulators must stand aside, because these AI companies are the only ones who can stop the bad guys and build this technology responsibly.

“If you portray these technologies as somehow almost supernatural in their danger, it makes us feel like we are powerless, like we are outmatched,” says Shannon Vallor, a professor of the ethics of data and artificial intelligence at the University of Edinburgh in the UK. “As if the only people we could possibly look to would be the companies themselves.”

If this doesn’t radicalize you against AI, I don’t know what will.

Apparently the future harm is so bad that a national propaganda outlet is asking you to pay attention to the present harm instead.

The whole thing (AI) is ridiculous frankly. Idk what else to say. If it wasn’t so dangerous it would be laughable. “How much gas can you pour on a fire before it really gets out of control?” Idk, some things just aren’t worth finding out, you know. I guess it’s worth it to these guys. The men who sold the world.

To be fair, if it just fills in the blanks and we’re telling it it’s AI after it has absorbed the sum of the internet and of our anxieties, I think it’s a self fulfilling prophecy if anyone hooks it up to the launch codes.

Deleted by author