Sure sounds like aspiring and/or existing professional mathematicians are taking the latest GenAI assault on their craft either with shock or horror, or with full-on welcoming of their new robot overlords
A sign the bubble might pop yes, but also a sign how toxic the whole youtube space is. If a yt’er with 3M+ subs and 500k+ views per video on average feels the pressure to publish or perish like this.
Sharp overview paper. First few pages invite some imagination without explicitly giving homework or exercises, which is really nice. Complex multiplication mentioned! I still think this is the worst-named theory in maths.
Street advertising no longer references products or services people seeing them may wish to use; after all, more and more people have no money to pay for them. Instead they try and AstroTurf hype for one of the new entries to the VC startup casino.
i think that poking a pressurized vessel is a bad idea in general, whether it is attached to a hostile robot or not. it’s gonna be LOUD and perhaps throw fragments, or take way longer than practical depending on what it was made from. (what if only one of two is damaged?) there are so many things that can go wrong with the alleged failsafe (what if pressure drops slower than designed?)
From a theoretical perspective, if you’re not going to shine light on the Leech lattice or the ADE phenomenon then you’re not actually getting at the core objects and are only doing surface work.
At first I was wondering why everyone was talking about the guy from “Freddy Got Fingered” but that’s Tom Green. Anyways, glad the propane and propane accessories guy is taking a break from AI.
OpenAI claims proofs for ten maths conjectures. The details are underwhelming; expand for opinions. Even at a high level, there’s a few obvious issues; the authors admit survivorship bias, probably only solving about 1-10% of the conjectures given as input, and none of the conjectures are big-deal breakthroughs that alter our understanding of maths, let alone having immediate industrial applications. Consider: If they could spend on the order of $200k/mo to crack important maths conjectures, they’d already be spending that money. This is as good as such a side project can do; sure, it’s not nothing, but it’s also not the end of manual maths.
opinions on maths
Only one of the results is at all interesting to me. Ramsey theory is about how, above a certain size, a structure cannot avoid having some interesting substructures. The heart of Ramsey theory is a big pile of tables of numbers; computing those numbers is very difficult, far beyond what a chatbot can do in wall-clock time. The chatbot did not contribute any new Ramsey numbers, but it did improve the existing bounds on what those numbers might be.
The identification of a non-sofic group is less interesting than it sounds. We’ve known for a while that there are quite a few exotic groups which defy our expectations, so this was an expected outcome of an exhaustive and motivated search. The tools involved, Leavitt path algebras, are only a few decades old and not at all well-known; it’s not likely that we’ll be able to understand how hard this was for a while. Maybe it was low-hanging fruit. The main contrast is with something like non-Noetherian rings; we initially believed that all rings are Noetherian, so it was something of a shock that it’s not always the case. Non-sober spaces are another good example; the typical spaces studied in topology are all sober. See this quote from Johnstone and discussion on MO.
The computational complexity result is completely uninteresting to me thanks to Valiant’s theorem, which says that matrix permanents are ♯P-complete even over fields as small as F₂. You’re not gonna collapse ♯P into P with a fucking chatbot, bros. Similarly, reduction from 3SAT to a closest-vector problem does not shift our belief in the difficulty of that problem; this doesn’t make it easier and we already suspected it was NP-hard.
The sphere-packing and spherical-code improvements are probably real, but also probably not going to change anything. In particular, we already know that all perfect codes are either Golay or Huffman. Frankly, the codes we use in real life are not amenable to this simple framing; I don’t think Reed-Solomon arises from sphere packing. From a theoretical perspective, if you’re not going to shine light on the Leech lattice or the ADE phenomenon then you’re not actually getting at the core objects and are only doing surface work. Don’t get me wrong; if a human were doing all of this then we would have the useful side effect that they would earn a PhD, making it worthwhile for humans to improve these bounds.
I don’t have anything to say about the other four results. They’re not nothingburgers but they don’t depend on some ultra-smart robot either.
Zero Interest Rates Policy from 2008-2022 and lack of antitrust enforcement in the USA lead to two things: incumbent software companies got bigger because they could borrow so much money and buy or crush their competition, and small software companies did not have to make profits now because investors tossed in more money and hoped they would make big profits one day. That world is dead but the industry has not adapted.
Well, the big issue is that there’s only this much demand. How many web browsers is there genuine end user demand for, worldwide? Probably just 1. Several are made thanks to other demands by large corporations. How much value does a web browser provide, to billions of people who use it? What ever the number may be, it’s orders of magnitude more than it costs to develop.
Why is software observably hard for companies? When 90% of people are working on projects that are going nowhere, that creates a situation where competence is quite rare. Because in most projects it is objectively unnecessary (would make no difference). Where majority of software engineers never seen a successful project all the way from the start to the finish line.
Then there’s startups, these are (predominantly) mere vehicles for moving money from the coffers of large publicly traded corporations into pockets of VCs (who sell startups to said corporations). Whole world of pretend software engineering.
Then there’s large corporations, where CEOs want to get paid more and to be able to get paid more they need to add more layers to the hierarchy underneath them, which they need to fill somehow, and they can’t fill on merit because they just don’t have enough stuff happening that they could measure merit in.
edit: basically the management always wants to do the equivalent of having hired a dozen heart surgeons to work on one heart and a “heart surgery architect” as well, the more layers the better because the more layers the higher is the pay their layer can get.
So Hank Green is apparently stepping back from his shit for a while after admitting to using AI for research and scriptwriting and acknowledging an unhealthy relationship with the dopamine hit he got from using it. I think this makes him one of the first high-profile users to acknowledge the problem? Which can’t be a good sign for the bubble.
As comedy art projects go, I really admire their dedication to the bit. The images are remarkably high quality, and they’ve really thought long and hard about those designs. I mean look at this… it’s glorious.
I’d advise against attempting to disable a chainsaw operator by poking them with a knife, though. I guess it’s too bad that pitchforks are pretty uncommon these days.
ANTHROPIC: Our son loves humanity and for the low price of trillions
and trillions of dollars will usher in an era of unprecedented peace
and prosperity!
OPENAI: Our horrible oafish son is hacking web sites without our permission.
ANTHROPIC: (eyes narrowing) Our son is also evil
You’re argument seems like Suresh’s to me, in that making working software is just laughably easy for a competent engineer.
This begs the question why noone is exploiting this market gap. But there’s a couple explanations…
It can be a coordination problem, in that there are just barely any competent software organisations with enough engineers. The problem can also lie in how software is bought, since customers seem completely unable to determine the competence of vendors. Perceptions on what kind of software is a safe, respectable choice may even be anti-helpful in this regard.
I think it’s a sunk cost thing for the authors, but also it could be fear of coming off as “disgruntled” to people they’re hoping to communicate with. They still have hope of changing these communities, but they know these communities are very hostile and dismissive towards any criticism, especially criticism from perceived “enemies”. They also don’t want to lose whatever power and respect they have in those communities. So emphasising how great these groups are and how smart all the members are and how much they got out of being a member is a tactic to try and get around all that hostility. They want to flatter the remaining members enough that they’ll retain their respect for them. Which in itself is a sad comment on the attitudes common in these communities.
Having said all that, the poor man is not long out of a cult and has taken a big step by publicly blowing the whistle. That takes some backbone.
So Satyress has been getting the headlines lately for their horrifying ThreeHalves monstrosity. But what I find beautiful is that it appears to have been designed first to be functional and second to be easily disabled in the event of a robot uprising. I wish I was kidding.
A screenshot of the Satyress website showing where to shoot the robot to immediately disable it. Presumably in the production model these will be marked with big red glowing circles.
Another screenshot demonstrating how the robot has intentionally been made too big to fit through doors, though I expect that the chainsaw attachment they seem to have as standard could probably remedy that problem.
As a software engineer, all of the free portion of the article is 100% correct about software engineering dysfunction.
There’s another reason why it is that way, though. A software engineer who is competent and productive and working under competent leadership making an useful product, would make a very large multiple of their salary in profits.
That permits all sorts of utter dysfunction while still breaking even on the average. You can have incompetent leadership in 90% of projects, you can saddle your developers with any kind of bad practices, you can impose all sorts of productivity damaging nonsense and yet still break even.
The dysfunction can be presented as relentless innovation and pursuit of new, uncertain opportunities, as a shrewd strategy.
If you fall below break-even, the consequences will not be visible for many years. And up until it becomes impossible to recover, on the technical level it is extremely cheap to recover: it’s easier to do nothing than to convert your OS’s start menu to React. Tech companies don’t fail because of tech.
This also relates to using AI to write code. You already can have a human write your code and pay them $1 for $10 you would make from it - or $0 you would make if you fucked up. You want it to become $0.5 with AI coding tools? Then you’re multiplying your probability of success by what’s left after the probability that you get some unmaintainable slop . It’s simply not worth it.
Onboarding at my new job means I am now “Certified AI Ready” whatever that means. Thankfully other than having copilot available it looks like my team isn’t having it pushed too hard yet, and I only had to interrupt the training video once to rant about how ridiculous their history was for like 20 minutes (working from home has a lot of advantages).
It sucks and I hate it, and I applaud the locals who rallied together to yell at that superintendent until he finally paused the implementation of this thing. This shit is not inevitable, and if one of these dolls does ever end up in a poor, underserved high school, I hope it’s one that’s a lot like the one where I went to school because it would be a smoldering pile of silicone in the corner of the girls’ room by the end of the first week.
A wallet’s seed, the secret phrase controlling the funds, is meant to be drawn at random from a pool so vast that guessing is hopeless.
Coldcard’s firmware was not doing that. According to a report published by Block’s Bitcoin engineering and security teams, a build setting told the device to skip its own hardware randomness generator, and a check in a supporting library tested only whether that setting existed rather than whether it was switched on.
Key generation quietly fell through to a basic software substitute seeded from the chip’s serial number and clock registers.
The reason these (one of these incidents dates back to April, so they’re really not beating the “marketing campaign” allegations) happened in the first place is because of human error on their end, mind you
The OpenAI hacking incident has put everyone on edge. Anthropic, not to be outdone, decides to publicly announce that they have committed not one, not two, but three crimes! Come on! I was told this wasn’t a marketing campaign!
I don’t think you have to be hypercritical of that part. Sure, rose-colored glasses could play into it but he calls the bad stuff bad and doesn’t justify it via the good stuff.
Also the stuff he describes seems plausibly achievable through buddhist spiritial practice:
In many ways MAPLE delivered: life-long friendships, freedom from past addictions and limiting self-narratives, strengthened character, and, for many of us, profound spiritual openings and realizations.
This Soryu figure seems fairly well accredited in that department. He just also happens to be a megalomaniacal control freak.
(Also, why does he have to share a name with Asoka? I’m dying a little.)
One of the lesser discussed issues with the generative AI hype is that it put every single deep learning application under scrutiny. They tend to hide behind the positive uses of deep learning whenever generative AI is criticized, as if medicine research and the misinformation generator cannot be separated. “Nuclear power is impossible unless we are allowed to use nukes” type of argument
chaser:
Alphafold was a really interest project. It was not magic but had some utility. Now all remaining AI projects are lying plagiarism machines.
I’d say it’s all of that, plus the fact that a lot of the cult stuff are also present in how organised rationalism works in general so you need to believe that such a tradeoff can be mostly worth it, otherwise it’s time to really get packing.
Although a more charitable read could be that the intended audience are people still under cult influence who need to be communicated to (in the most non-judgemental way possible so as to not trip any cult induced outgroup-o-phobia) that it’s entirely possible to feel you are in a situation with positive and productive elements that is also a fucking cult that is eating you alive.
The extended framing of how ‘high-control literature’ applies to the MAPLE situation probably also caters to those sensibilities, most other readers probably already got that it’s a cult as soon as stuff like dear leader halting his daily two-hour speech when someone almost nodded off to make everyone stand up and stare at them in silence for ninety minutes started being mentioned. Or the constant surveillance and brainwashing. Or keeping everyone within a hair’s width from collapse from exhaustion at all times. Or the isolation retreats where your only human contact is a few minutes per week of getting told off by dear leader. Or <gestures at everything>.
Having said all that, the positive aspects of MAPLE the author manages to come up with seem to amount to providing life structure and long term goals, which really doesn’t seem like much:
Again I’ll say: there was a great deal of good at MAPLE. While many people were being impacted in the ways above, many were also becoming less anxious, more grounded, more relationally attuned, less distracted and more focused, more capable of doing chores and taking on responsibility without internal resistance, physically healthier, deepening in their spiritual practice, and more. Most people experienced some amount of the positive and the negative, which is a large part of why many former residents say their relationship to MAPLE is “complicated.”
Grandma’s dead from a preventable disease but the Dorito’s ad on the Smart TV customized itself for me to include her making a quip about how good the new Cool Ranch Blastz flavor is.
Intel up 11% in the day and 5% in the after hours. Really looks like big players took the boy out back behind the woodshed to see what swimming with the sharks feels like and devour his entire port at a discount.
“From my perspective as an uninformed spectator, I would expect the bets made by Situational Awareness to be positive EV over the long run.”
Actually the value of his calls in the long run is 0, don’t worry I’m giving this hot financial tip for free. Do these dorks not understand options work? I guess neither did young leo.
Remember, there’s no crying in the casino. *though i have a feeling Leo will continue to fail upward ;_;
The LW insider thinks Aschenbrenner is trying to save a stake in Anthropic. I think that is very likely because the press release stressed “public stock position” and Anthropic is privately owned.
Edit / I don’t think Thomas Kwa of METR and LW understands leverage. The idea is that if you borrow $40.5b on $4.5b of collateral, you have $45b to invest but a 10% drop will wipe you out. Before you get there the bank will knock on your door and say “margin call! either you provide more collateral or we seize your assets.”
Sam Altman is in the singularity, in that he is now convinced LLMs are smarter than him
I’m reminded of a quote that used to always come up when I was studying cryptography. Bruce Schneier would always remind people that it’s easy to design an encryption scheme that you can’t break. But it’s hard to design an encryption scheme that nobody else can break, either.
It is a good reminder that there needs to be a degree of intellectual humility by those who want to design big systems that affect the whole world.
He describes textbook insane cult stuff, but still feels the need to say stuff like:
Again I’ll say: there was a great deal of good at MAPLE.
The way I’ve described it to friends is that it was the best decision I ever made to go there; the second best, however, being to leave.
There was the same dynamic in the writing of the ex-leverage member that got posted here a few weeks ago. She also felt the need to emphasize the “good” parts. Is the cult programming that hard to break? Is it some form of sunk cost or rationalization or need to claim something positive about the experience?
The cope on lesswrong is funny. They are in denial that this is a sign of a wider bubble pop and are insisting it is just because he didn’t hedge on long enough timelines.
Not denying that he isn’t also a grifter, but I bet he is a true believer and he had blindly bet his (and other people’s) money on “line goes up” exactly like his scenario said and that is why he is the first one to crack.
See also a question asked today on Math Overflow, "Are we stuck with Lean?". The proposed alternative, Metamath, isn’t type-theoretic and thus skips the entire dialogue between type theory and proof assistants.
The reason that they are destroying the books is to avoid the accusation that the scanning constituted copying, an irony that we’ve discussed previously, on Awful while trying to understand which court cases are relevant. Anthropic actually hired the same guy, Tom Turvey, who designed Google Books’ ingestion process, so I’m thinking of this as a sequel to Google Books; hopefully the courts won’t take a decade this time.
but you see, in case of accident they can just dump reactor core into seawater which will provide ample cooling and shielding. no biggie, just put multi thousand ton object that requires a village of engineers to work out on high seas
there are normal nuclear powerplants that use seawater as coolant and they’re all on shore. these require a lot of corrosion resistant heat exchangers along the way. there is one power barge like this, it provides power to russian far east, whole 70MW of it, costed 414 million dollars and needs 300 people onsite. your guess is as good as mine how many of them are military. 70MW, aka two or three aeroderivative gas turbines
Of course there’s a class element to this, but you give them too much credit.
This isn’t poor people being exposed to stupid AI so rich people can reap the benefits of actual AI. This is poor people being subjected to stupid AI so AI company shareholders can pretend that their stupid AI is worth anything for another quarter.
Salt water is famously a forgiving environment to build in, and oceanic weather is benign and predictable. Not that it would be a problem anyway, because move-fast-and-break-things people can be trusted to do the right thing when it comes to running delicate equipment in places more hostile than mid california, and will not skimp on staffing and maintenance. You can look at all the the successful seasteading operations to see how well this will go.
There are also no problems I can foresee with putting a bucketload of fissiles in international waters, with no scope for finally uniting old-school piracy with new-school piracy, either. This plan is great, and no-one will have a problem with it.
On land, cooling and water are among a site’s largest costs.
Oh gosh that sounds… uhmm not too bad actually? I wonder what the largest costs are at sea with a floating nuclear reactor in salt water and tugged around by large ships.
Before the days of AI 2027, he became famous in 2024 for posting one of the original pieces of writing in the line-go-up genre: Situational Awareness. It seems that like any good grifter, he used this opportunity to make money (in this case by starting a hedge fund).
Note that this implies massive efficiency gains to the economy. For their one-third slice to equal 8% of those salaries [59% of the total workforce in the USA], the total savings (productivity across that share of the workforce) will need to rise by roughly 24%. So they are absolutely serious about this, and are betting heavily that the productivity boost arrives within four years.
I am continually amused at people not quite understanding what AlphaFold is actually doing, too.
Yes, a bunch of its performance comes from it learning rules about how proteins fold. But not a majority of its performance. MOST of its performance is it effectively acting as a translator of what evolution knows about protein folding into a form we can understand.
A key part of the system is not just cooking the sequence into a structure. A system running alphafold has a database of terabytes of curated sequence information from all over the tree of life. You put in the sequence you care about, and it first searches that database for anything with homology, and builds a “covariation matrix” - wherever theres anything with even vague sequence relatedness, build a matrix of every position in your sequence and the correlation between variation at position X and variation at position Y. This covariation matrix represents implicit information from the evolutionary process about what parts of a sequence are functionally connected to each other, which has a correlation to positional information, and these correlations are in turn learned by the ML system.
You put in de novo designed proteins or orphan proteins without homologs in the curated dataset and performance does not go away, but it drops precipitously. A bunch of what is going on is finding an evolutionary signal, and translating that evolutionary signal into structural information. So still, evolution knows much much more about protein folding than we do or any machine does, and once again a ML system is revealed to essentially be an information channel that takes in information from an interesting source on one end and turns it into a different form of information on the other.
And a follow-up by talia ringer, who observes that there have always been gaps between the type-theoretic underpinnings of things like the lean prover and their actual implementation, and this hasn’t been so much of an issue til now because theorem provers haven’t had the attention of people in high places, and the type-theoreticians have been able to catch up in due course.
My big worry right now is that if organizations continue to fund the crap out of Al for formal proof research (and to generally support implementation and maintenance of proof assistants like
Lean as part of that effort) but don’t bother funding the type theory side of things, those gaps will grow larger and will be exploited more often by Al tools via reward hacking. Whereas people tend to only exploit kernel bugs to make a point that the bug exists. Thus proof assistants will grow less trustworthy over time.
Anyone want to place any bets on whether or nor the big llm companies are going to fund academic research that isn’t obviously mechanisable right now and won’t yield any clickbait headlines?
I am actually in the middle of both trying to advance my career and a project about information theory in evolutionary biology making a bunch of explicit parallels to machine learning. Someone where I work suggested that given the connections I was making I should look to a ‘frontier AI lab’ as an employer.
He did not see the instant flashbacks to chasing these weirdos across the internet for almost two decades, watching in horror as the religious psychosis gained national prominence and great destructive power. All he got to hear was my instant intonation of “I’m sorry Dave, I’m afraid I can’t do that.”
I was an academic in computer science in the last 10 years or so (keeping it vague to avoid doxxing myself) and it has been so depressing seeing so many of my colleagues selling out to OpenAI, Anthropic, Meta and even Google (for some reason the latter often gets a reputational free pass because people associate them with the golden days of big tech 10+ years ago)
What kills me is that there are so many obvious ways to be less wasteful about this. Nondestructive book scanners exist. They’re expensive but it’s not like AI companies are averse to throwing money down a fucking hole. Even if that’s not an option, it’s possible to rebind the pages and return the books to the market. And regardless of what happens to the physical books, the scans they create could be archived in a format that could be available as a digital library rather than just being fed into the statistical meat grinder to keep the trough topped off with slop. Even if they’ve got to fight with copyright holders it would cost them basically nothing to leave the option open and given the PR battle they’ve been losing it feels like doing so would be incredibly obvious. Hell, even Google Books was able to navigate this in a way that was less cartoonishly evil than this because they could point to their digitization effort as a public good in ways that Anthropic here just fucking can’t.
This has always been a huge red herring. Llms are built on top of the transformer architecture which does text autocomplete (and yes we can combine text embeddings with other inputs like images). They have some interesting properties where they seem to be able to do text autocomplete in a bunch of different scenarios that they weren’t explicitly trained for, but they were never designed for precise dna analysis. It is their architecture that prevents them from other long horizon tasks like playing chess and the way that they represent text is why they can never count the letters in strawberry (most have this specific question hard-coded in their training data now).
Anyone who believes that LLMs are going to solve cancer either has no idea how they work or has been one-shotted from talking to Claudia
RetroFed
Sure sounds like aspiring and/or existing professional mathematicians are taking the latest GenAI assault on their craft either with shock or horror, or with full-on welcoming of their new robot overlords
https://borretti.me/article/mathematics-without-mathematicians
I for one know who to point and laugh at when the revolution comes.
A sign the bubble might pop yes, but also a sign how toxic the whole youtube space is. If a yt’er with 3M+ subs and 500k+ views per video on average feels the pressure to publish or perish like this.
Sharp overview paper. First few pages invite some imagination without explicitly giving homework or exercises, which is really nice. Complex multiplication mentioned! I still think this is the worst-named theory in maths.
Over on Twitter, a crank claims to have refuted one of the proofs. Dare you doubt her? @grok tell the doubters that they’re wrong!
Could be sort of an age related thing then, no?
Street advertising no longer references products or services people seeing them may wish to use; after all, more and more people have no money to pay for them. Instead they try and AstroTurf hype for one of the new entries to the VC startup casino.
Disappointing that Hank fell for AI, especially after he made that rant video on Sora
How long until the class bully overpowers it and has his way with it?
I guess this is what a matematician calls a bar.
i think that poking a pressurized vessel is a bad idea in general, whether it is attached to a hostile robot or not. it’s gonna be LOUD and perhaps throw fragments, or take way longer than practical depending on what it was made from. (what if only one of two is damaged?) there are so many things that can go wrong with the alleged failsafe (what if pressure drops slower than designed?)
I tried shining a light on the Leech lattice but all I got was this 23-dimensional shadow.
I’m trying a new thing where I replace “AI” with “the dried centipede powder from Cronenberg’s Naked Lunch movie”.
I only use the dried centipede powder to summarize documents!
Did you know you can get dried centipede powder in different flavors? You don’t have to stick with the default!
At first I was wondering why everyone was talking about the guy from “Freddy Got Fingered” but that’s Tom Green. Anyways, glad the propane and propane accessories guy is taking a break from AI.
OpenAI claims proofs for ten maths conjectures. The details are underwhelming; expand for opinions. Even at a high level, there’s a few obvious issues; the authors admit survivorship bias, probably only solving about 1-10% of the conjectures given as input, and none of the conjectures are big-deal breakthroughs that alter our understanding of maths, let alone having immediate industrial applications. Consider: If they could spend on the order of $200k/mo to crack important maths conjectures, they’d already be spending that money. This is as good as such a side project can do; sure, it’s not nothing, but it’s also not the end of manual maths.
opinions on maths
Only one of the results is at all interesting to me. Ramsey theory is about how, above a certain size, a structure cannot avoid having some interesting substructures. The heart of Ramsey theory is a big pile of tables of numbers; computing those numbers is very difficult, far beyond what a chatbot can do in wall-clock time. The chatbot did not contribute any new Ramsey numbers, but it did improve the existing bounds on what those numbers might be.
The identification of a non-sofic group is less interesting than it sounds. We’ve known for a while that there are quite a few exotic groups which defy our expectations, so this was an expected outcome of an exhaustive and motivated search. The tools involved, Leavitt path algebras, are only a few decades old and not at all well-known; it’s not likely that we’ll be able to understand how hard this was for a while. Maybe it was low-hanging fruit. The main contrast is with something like non-Noetherian rings; we initially believed that all rings are Noetherian, so it was something of a shock that it’s not always the case. Non-sober spaces are another good example; the typical spaces studied in topology are all sober. See this quote from Johnstone and discussion on MO.
The computational complexity result is completely uninteresting to me thanks to Valiant’s theorem, which says that matrix permanents are ♯P-complete even over fields as small as F₂. You’re not gonna collapse ♯P into P with a fucking chatbot, bros. Similarly, reduction from 3SAT to a closest-vector problem does not shift our belief in the difficulty of that problem; this doesn’t make it easier and we already suspected it was NP-hard.
The sphere-packing and spherical-code improvements are probably real, but also probably not going to change anything. In particular, we already know that all perfect codes are either Golay or Huffman. Frankly, the codes we use in real life are not amenable to this simple framing; I don’t think Reed-Solomon arises from sphere packing. From a theoretical perspective, if you’re not going to shine light on the Leech lattice or the ADE phenomenon then you’re not actually getting at the core objects and are only doing surface work. Don’t get me wrong; if a human were doing all of this then we would have the useful side effect that they would earn a PhD, making it worthwhile for humans to improve these bounds.
I don’t have anything to say about the other four results. They’re not nothingburgers but they don’t depend on some ultra-smart robot either.
In hindsight, it is pretty funny that he called his fund “Situational Awareness” and still lost money.
Maybe someone should start a hedge fund and call it something like, “Maybe This Will All Work Out LLC.” That way there is no disappointment.
Zero Interest Rates Policy from 2008-2022 and lack of antitrust enforcement in the USA lead to two things: incumbent software companies got bigger because they could borrow so much money and buy or crush their competition, and small software companies did not have to make profits now because investors tossed in more money and hoped they would make big profits one day. That world is dead but the industry has not adapted.
Well, the big issue is that there’s only this much demand. How many web browsers is there genuine end user demand for, worldwide? Probably just 1. Several are made thanks to other demands by large corporations. How much value does a web browser provide, to billions of people who use it? What ever the number may be, it’s orders of magnitude more than it costs to develop.
Why is software observably hard for companies? When 90% of people are working on projects that are going nowhere, that creates a situation where competence is quite rare. Because in most projects it is objectively unnecessary (would make no difference). Where majority of software engineers never seen a successful project all the way from the start to the finish line.
Then there’s startups, these are (predominantly) mere vehicles for moving money from the coffers of large publicly traded corporations into pockets of VCs (who sell startups to said corporations). Whole world of pretend software engineering.
Then there’s large corporations, where CEOs want to get paid more and to be able to get paid more they need to add more layers to the hierarchy underneath them, which they need to fill somehow, and they can’t fill on merit because they just don’t have enough stuff happening that they could measure merit in.
edit: basically the management always wants to do the equivalent of having hired a dozen heart surgeons to work on one heart and a “heart surgery architect” as well, the more layers the better because the more layers the higher is the pay their layer can get.
So Hank Green is apparently stepping back from his shit for a while after admitting to using AI for research and scriptwriting and acknowledging an unhealthy relationship with the dopamine hit he got from using it. I think this makes him one of the first high-profile users to acknowledge the problem? Which can’t be a good sign for the bubble.
As comedy art projects go, I really admire their dedication to the bit. The images are remarkably high quality, and they’ve really thought long and hard about those designs. I mean look at this… it’s glorious.
I’d advise against attempting to disable a chainsaw operator by poking them with a knife, though. I guess it’s too bad that pitchforks are pretty uncommon these days.
what’s good about this article to be worth posting?
How long will it take before they are bought up by a second company which makes military/police models with these ‘problems’ removed?
https://bsky.app/profile/did:plc:mm5ebihpjvtrdunomulbls5m/post/3mrx2xflgp22d
ANTHROPIC: Our son loves humanity and for the low price of trillions and trillions of dollars will usher in an era of unprecedented peace and prosperity!
OPENAI: Our horrible oafish son is hacking web sites without our permission.
ANTHROPIC: (eyes narrowing) Our son is also evil
Apparently it was a fuckup when they rewrote the codebase so they could move it from GPL to source-available.
This is currently an utter, utter disaster scenario over in bitcoinland, the coiners are screaming. lol.
Kai Lentit: Interview with a Longevity Maxxer in 2026
Um, okay, I know coiners have no shortage of incompetence but this bug sounds an awful lot like malice.
It’s not fragile and unergonomic, it’s robot-uprising-proof!
Also like will the warranty actually cover damages suffered when trying to squash the rebellion?
You’re argument seems like Suresh’s to me, in that making working software is just laughably easy for a competent engineer.
This begs the question why noone is exploiting this market gap. But there’s a couple explanations…
It can be a coordination problem, in that there are just barely any competent software organisations with enough engineers. The problem can also lie in how software is bought, since customers seem completely unable to determine the competence of vendors. Perceptions on what kind of software is a safe, respectable choice may even be anti-helpful in this regard.
Yeah, that is really depressing.
I think it’s a sunk cost thing for the authors, but also it could be fear of coming off as “disgruntled” to people they’re hoping to communicate with. They still have hope of changing these communities, but they know these communities are very hostile and dismissive towards any criticism, especially criticism from perceived “enemies”. They also don’t want to lose whatever power and respect they have in those communities. So emphasising how great these groups are and how smart all the members are and how much they got out of being a member is a tactic to try and get around all that hostility. They want to flatter the remaining members enough that they’ll retain their respect for them. Which in itself is a sad comment on the attitudes common in these communities.
Having said all that, the poor man is not long out of a cult and has taken a big step by publicly blowing the whistle. That takes some backbone.
I mean, it’s less dumb than trying to do it in orbit, but it’s still profoundly dumb.
So Satyress has been getting the headlines lately for their horrifying ThreeHalves monstrosity. But what I find beautiful is that it appears to have been designed first to be functional and second to be easily disabled in the event of a robot uprising. I wish I was kidding.
A screenshot of the Satyress website showing where to shoot the robot to immediately disable it. Presumably in the production model these will be marked with big red glowing circles.
Another screenshot demonstrating how the robot has intentionally been made too big to fit through doors, though I expect that the chainsaw attachment they seem to have as standard could probably remedy that problem.
As a software engineer, all of the free portion of the article is 100% correct about software engineering dysfunction.
There’s another reason why it is that way, though. A software engineer who is competent and productive and working under competent leadership making an useful product, would make a very large multiple of their salary in profits.
That permits all sorts of utter dysfunction while still breaking even on the average. You can have incompetent leadership in 90% of projects, you can saddle your developers with any kind of bad practices, you can impose all sorts of productivity damaging nonsense and yet still break even.
The dysfunction can be presented as relentless innovation and pursuit of new, uncertain opportunities, as a shrewd strategy.
If you fall below break-even, the consequences will not be visible for many years. And up until it becomes impossible to recover, on the technical level it is extremely cheap to recover: it’s easier to do nothing than to convert your OS’s start menu to React. Tech companies don’t fail because of tech.
This also relates to using AI to write code. You already can have a human write your code and pay them $1 for $10 you would make from it - or $0 you would make if you fucked up. You want it to become $0.5 with AI coding tools? Then you’re multiplying your probability of success by what’s left after the probability that you get some unmaintainable slop . It’s simply not worth it.
Onboarding at my new job means I am now “Certified AI Ready” whatever that means. Thankfully other than having copilot available it looks like my team isn’t having it pushed too hard yet, and I only had to interrupt the training video once to rant about how ridiculous their history was for like 20 minutes (working from home has a lot of advantages).
https://skepchick.org/2026/07/i-hate-the-ai-robo-teacher/
Can’t crush what isn’t there.
Coindesk: Major bitcoin wallet flaw drains $38 million worth of BTC in 25-minute sweep
The reason these (one of these incidents dates back to April, so they’re really not beating the “marketing campaign” allegations) happened in the first place is because of human error on their end, mind you
“AI safety focused” my ass
Behold, the Singularity!
A recipe for a hearty barley soup?
AI: Aussies Innit?
I’ve been enjoying Ludicity Week very much!
It really grinds my gears.
yeah it’s a common trope/cope
like “most human programmers suck too”
ok dude who do you think wrote the code the LLM is cribbing from
GDP is up though, so I’m sure the AI god prosperity will trickle down eventually.
Waiting for Musk to turn up and announce “infinity plus one more than you can ever say nah nah nah nah” momentarily!
TBF he has indirectly killed a bunch of people via doge cuts so that’d do the trick…
🔮"I see a city street. Massive wealth disparity and a lot of homeless… Jesus that’s a lot of homeless”
That fits the pattern, when AI isn’t living up to the sky high promises, the promoters act like people are less special than they are.
See how a debate about are they conscious often devolves into ‘are humans conscious?’
The OpenAI hacking incident has put everyone on edge. Anthropic, not to be outdone, decides to publicly announce that they have committed not one, not two, but three crimes! Come on! I was told this wasn’t a marketing campaign!
I don’t think you have to be hypercritical of that part. Sure, rose-colored glasses could play into it but he calls the bad stuff bad and doesn’t justify it via the good stuff.
Also the stuff he describes seems plausibly achievable through buddhist spiritial practice:
This Soryu figure seems fairly well accredited in that department. He just also happens to be a megalomaniacal control freak.
(Also, why does he have to share a name with Asoka? I’m dying a little.)
edit: I knew I spelt megalomaniacal wrong…
(from the youtube comments)
shot:
chaser:
Duh. Everybody knows the market never stays irrational for long, and you should just tough it out.
I’d say it’s all of that, plus the fact that a lot of the cult stuff are also present in how organised rationalism works in general so you need to believe that such a tradeoff can be mostly worth it, otherwise it’s time to really get packing.
Although a more charitable read could be that the intended audience are people still under cult influence who need to be communicated to (in the most non-judgemental way possible so as to not trip any cult induced outgroup-o-phobia) that it’s entirely possible to feel you are in a situation with positive and productive elements that is also a fucking cult that is eating you alive.
The extended framing of how ‘high-control literature’ applies to the MAPLE situation probably also caters to those sensibilities, most other readers probably already got that it’s a cult as soon as stuff like dear leader halting his daily two-hour speech when someone almost nodded off to make everyone stand up and stare at them in silence for ninety minutes started being mentioned. Or the constant surveillance and brainwashing. Or keeping everyone within a hair’s width from collapse from exhaustion at all times. Or the isolation retreats where your only human contact is a few minutes per week of getting told off by dear leader. Or <gestures at everything>.
Having said all that, the positive aspects of MAPLE the author manages to come up with seem to amount to providing life structure and long term goals, which really doesn’t seem like much:
Years ago I saw a memorable tweet: Zero Hedge readers upset passing asteroid did not destroy earth.
How do I get one of these cushy jobs?
sounds like he took the hedge part out of hedge fund, and then margin call took out the fund part
Here is the first sentence: “You can see the future first in San Francisco. "
I feel like I already know everything I need to know about him.
Into a black hole.
Grandma’s dead from a preventable disease but the Dorito’s ad on the Smart TV customized itself for me to include her making a quip about how good the new Cool Ranch Blastz flavor is.
With personalized pricing, to boot.
This is my xmas, ty santa
“He sold? Pamp it”
Intel up 11% in the day and 5% in the after hours. Really looks like big players took the boy out back behind the woodshed to see what swimming with the sharks feels like and devour his entire port at a discount.
“From my perspective as an uninformed spectator, I would expect the bets made by Situational Awareness to be positive EV over the long run.”
Actually the value of his calls in the long run is 0, don’t worry I’m giving this hot financial tip for free. Do these dorks not understand options work? I guess neither did young leo.
Remember, there’s no crying in the casino. *though i have a feeling Leo will continue to fail upward ;_;
Also apparently young leo is getting married to Dario’s Chief of staff this weekend lmaou
Sir, a 2nd EA adj. SF based hedge fund has managed to lose 40 billion dollars
tick tick tick tick
(I am choosing to forego the obvious jibe that the models fulfilled this prophecy by dragging many college graduates down beneath them)
The LW insider thinks Aschenbrenner is trying to save a stake in Anthropic. I think that is very likely because the press release stressed “public stock position” and Anthropic is privately owned.
Edit / I don’t think Thomas Kwa of METR and LW understands leverage. The idea is that if you borrow $40.5b on $4.5b of collateral, you have $45b to invest but a 10% drop will wipe you out. Before you get there the bank will knock on your door and say “margin call! either you provide more collateral or we seize your assets.”
I’m reminded of a quote that used to always come up when I was studying cryptography. Bruce Schneier would always remind people that it’s easy to design an encryption scheme that you can’t break. But it’s hard to design an encryption scheme that nobody else can break, either.
It is a good reminder that there needs to be a degree of intellectual humility by those who want to design big systems that affect the whole world.
He describes textbook insane cult stuff, but still feels the need to say stuff like:
There was the same dynamic in the writing of the ex-leverage member that got posted here a few weeks ago. She also felt the need to emphasize the “good” parts. Is the cult programming that hard to break? Is it some form of sunk cost or rationalization or need to claim something positive about the experience?
The cope on lesswrong is funny. They are in denial that this is a sign of a wider bubble pop and are insisting it is just because he didn’t hedge on long enough timelines.
Not denying that he isn’t also a grifter, but I bet he is a true believer and he had blindly bet his (and other people’s) money on “line goes up” exactly like his scenario said and that is why he is the first one to crack.
Great idea, let’s seastead a data center next to some low income sea steading communities.
Clearly the shirt also needs a big typo.
s/chatbots/sexbots/
Who needs cures when you can instead simply replace doctors with chatbots?
Fuck Anthropic. Snotty little bastards want to talk about AI safety and then destroy books to make a pollution machine.
Getting a T-Shirt that says “I entered the Singularity and all I got was a plagiarism machine, a higher cost of living and this stupid T-Shirt”
See also a question asked today on Math Overflow, "Are we stuck with Lean?". The proposed alternative, Metamath, isn’t type-theoretic and thus skips the entire dialogue between type theory and proof assistants.
Oh, I’ve seen this one before. At least we know where the LLM who gave them this pitch scraped the idea from.
The reason that they are destroying the books is to avoid the accusation that the scanning constituted copying, an irony that we’ve discussed previously, on Awful while trying to understand which court cases are relevant. Anthropic actually hired the same guy, Tom Turvey, who designed Google Books’ ingestion process, so I’m thinking of this as a sequel to Google Books; hopefully the courts won’t take a decade this time.
I see you, and raise by a Gundam solar space laser - https://www.reflectorbital.com/
but you see, in case of accident they can just dump reactor core into seawater which will provide ample cooling and shielding. no biggie, just put multi thousand ton object that requires a village of engineers to work out on high seas
there are normal nuclear powerplants that use seawater as coolant and they’re all on shore. these require a lot of corrosion resistant heat exchangers along the way. there is one power barge like this, it provides power to russian far east, whole 70MW of it, costed 414 million dollars and needs 300 people onsite. your guess is as good as mine how many of them are military. 70MW, aka two or three aeroderivative gas turbines
Of course there’s a class element to this, but you give them too much credit.
This isn’t poor people being exposed to stupid AI so rich people can reap the benefits of actual AI. This is poor people being subjected to stupid AI so AI company shareholders can pretend that their stupid AI is worth anything for another quarter.
Salt water is famously a forgiving environment to build in, and oceanic weather is benign and predictable. Not that it would be a problem anyway, because move-fast-and-break-things people can be trusted to do the right thing when it comes to running delicate equipment in places more hostile than mid california, and will not skimp on staffing and maintenance. You can look at all the the successful seasteading operations to see how well this will go.
There are also no problems I can foresee with putting a bucketload of fissiles in international waters, with no scope for finally uniting old-school piracy with new-school piracy, either. This plan is great, and no-one will have a problem with it.
Oh gosh that sounds… uhmm not too bad actually? I wonder what the largest costs are at sea with a floating nuclear reactor in salt water and tugged around by large ships.
Uh huh.
Before the days of AI 2027, he became famous in 2024 for posting one of the original pieces of writing in the line-go-up genre: Situational Awareness. It seems that like any good grifter, he used this opportunity to make money (in this case by starting a hedge fund).
<buys giant bucket of popcorn>
CNN: AI investor Leopold Aschenbrenner forced to unwind all public stock positions after steep losses, sources say
I vaguely recognize the name, apparently he’s a snot-nosed 25-yr old who got fired from OpenAI
“surely all these very wealthy people and the people they pay to invest for them can’t all be under some sort of mass delusion”
But using like 2000 words.
someone just got a Rule34 scenario for free
YES LET’S NUKE THE FUCKING OCEAN. FUCK YOU FISH WE NEED OUR SLOP.
Enjoy today’s craziest link: https://atomarine.co/
A LWer runs the numbers and doesn’t run away screaming like Zitron:
https://www.lesswrong.com/posts/EeTQGxa8RSpwHo4Ez/the-biggest-bet-in-history
I am continually amused at people not quite understanding what AlphaFold is actually doing, too.
Yes, a bunch of its performance comes from it learning rules about how proteins fold. But not a majority of its performance. MOST of its performance is it effectively acting as a translator of what evolution knows about protein folding into a form we can understand.
A key part of the system is not just cooking the sequence into a structure. A system running alphafold has a database of terabytes of curated sequence information from all over the tree of life. You put in the sequence you care about, and it first searches that database for anything with homology, and builds a “covariation matrix” - wherever theres anything with even vague sequence relatedness, build a matrix of every position in your sequence and the correlation between variation at position X and variation at position Y. This covariation matrix represents implicit information from the evolutionary process about what parts of a sequence are functionally connected to each other, which has a correlation to positional information, and these correlations are in turn learned by the ML system.
You put in de novo designed proteins or orphan proteins without homologs in the curated dataset and performance does not go away, but it drops precipitously. A bunch of what is going on is finding an evolutionary signal, and translating that evolutionary signal into structural information. So still, evolution knows much much more about protein folding than we do or any machine does, and once again a ML system is revealed to essentially be an information channel that takes in information from an interesting source on one end and turns it into a different form of information on the other.
And a follow-up by talia ringer, who observes that there have always been gaps between the type-theoretic underpinnings of things like the lean prover and their actual implementation, and this hasn’t been so much of an issue til now because theorem provers haven’t had the attention of people in high places, and the type-theoreticians have been able to catch up in due course.
https://mathstodon.xyz/@TaliaRinger/117005740997367321
Lean as part of that effort) but don’t bother funding the type theory side of things, those gaps will grow larger and will be exploited more often by Al tools via reward hacking. Whereas people tend to only exploit kernel bugs to make a point that the bug exists. Thus proof assistants will grow less trustworthy over time.
Anyone want to place any bets on whether or nor the big llm companies are going to fund academic research that isn’t obviously mechanisable right now and won’t yield any clickbait headlines?
Depends on what Anthropic wants to do with the poached people.
I am actually in the middle of both trying to advance my career and a project about information theory in evolutionary biology making a bunch of explicit parallels to machine learning. Someone where I work suggested that given the connections I was making I should look to a ‘frontier AI lab’ as an employer.
He did not see the instant flashbacks to chasing these weirdos across the internet for almost two decades, watching in horror as the religious psychosis gained national prominence and great destructive power. All he got to hear was my instant intonation of “I’m sorry Dave, I’m afraid I can’t do that.”
it was “fob off” until I discovered merkins don’t understand that one
Oh… Well that’s disappointing.
I was an academic in computer science in the last 10 years or so (keeping it vague to avoid doxxing myself) and it has been so depressing seeing so many of my colleagues selling out to OpenAI, Anthropic, Meta and even Google (for some reason the latter often gets a reputational free pass because people associate them with the golden days of big tech 10+ years ago)
What kills me is that there are so many obvious ways to be less wasteful about this. Nondestructive book scanners exist. They’re expensive but it’s not like AI companies are averse to throwing money down a fucking hole. Even if that’s not an option, it’s possible to rebind the pages and return the books to the market. And regardless of what happens to the physical books, the scans they create could be archived in a format that could be available as a digital library rather than just being fed into the statistical meat grinder to keep the trough topped off with slop. Even if they’ve got to fight with copyright holders it would cost them basically nothing to leave the option open and given the PR battle they’ve been losing it feels like doing so would be incredibly obvious. Hell, even Google Books was able to navigate this in a way that was less cartoonishly evil than this because they could point to their digitization effort as a public good in ways that Anthropic here just fucking can’t.
This has always been a huge red herring. Llms are built on top of the transformer architecture which does text autocomplete (and yes we can combine text embeddings with other inputs like images). They have some interesting properties where they seem to be able to do text autocomplete in a bunch of different scenarios that they weren’t explicitly trained for, but they were never designed for precise dna analysis. It is their architecture that prevents them from other long horizon tasks like playing chess and the way that they represent text is why they can never count the letters in strawberry (most have this specific question hard-coded in their training data now).
Anyone who believes that LLMs are going to solve cancer either has no idea how they work or has been one-shotted from talking to Claudia