AI costs spike as subscriptions hit pricing wall; firms turn towards Chinese LLMs, open-source models to extend budget

The majority of software/hardware components? Absolutely.

However, I wouldn’t be surprised if some things are gatekept for as long as possible so the “owners” can rent-seek their copyrights/patents.

Sure but there will always be a function of hardware and energy cost.

Today’s models will run cheaply in the distant future, but the distant future’s models could only be dreamed of today. Hopefully at some point we get to a point where quality is “good enough” on cheap hardware and low energy, but I can’t tell you when it’ll get here. I bet at minimum another decade, unless you’re ok with what you get out of today’s models on dedicated consumer GPUs.

I think the current stuff that runs on processors and normal ram is worse than useless though.

Maybe the imagegen model was feeling anti-fascististic? I must admit, that does seem to be a rarity.

Also the random percentages are just jarring.

Honestly I hate the “every article needs an image” rule that seems to exist. 99% of the time this slop detracts from the actual article.

An interesting side effect is that the models coming out of China are very efficient. They don’t have access to all the high-end hardware the US has, so they have to make do with what they’ve got.

This administration can’t see a week ahead in front of them. Does anyone have doubts this will only consolidate China’s autonomy and eventual dominance in the sector?

I see you’re problem. You’re using the old Fordist mode of economic growth, wherein you employ people at an income level such that they can buy the products they produce. Then you grow your capital stock by reallocating the labor surplus into new undeveloped areas of the country. That model of growth went out of fashion in the 80s.

Post Volcker-Shock, we adopted a strategy of indebting unaligned countries in exchange for modern technological improvements, then harvesting their natural resources at a functional loss to finance the USD denominated debts they struggled to pay back. Then we could effectively export dollars for cheap imports and allow Americans to buy them with money laundered through government contracts, grants, and Fed credit expansion.

But after COVID, we’re on to an even more revolutionary approach to profitability. We don’t need consumers. We don’t need to develop capital in foreign territories. We just do a direct 1:1 exchange of B2B SaaS services and American Dollars with the corporate oligarchs. Now people don’t need to buy anything, because governments buy everything. And if you’re friendly with the government, you get a slice. And if you’re not, you get shoved off a cliff.

at this rate they’ll be buying Chinese hardware in a couple years

neither copilot nor anthrophic offer a business ready harness and people are going open source already. Yeas, their cli, ide integrations are nice but security-wise all this is rather begging the agent not to do dumb stuff instead if actually restricting it in access to customer data, secrets etc.

I’ve seen a datapoint that an 8 hour business day with Claude is about 1 kUSD, so 20 business day month is some 20 kUSD. More with agentic AI.

I may be wrong but I thought airlines did similar. They sold more tickets than existing seats assuming people would cancel. That’s why sometimes they offer cashback at terminal for a different flight, but it still comes out net positive

Residential ISPs usually have a contention ratio somewhere around 30:1 to 50:1. That means that 30 to 50 customers that each have a 1Gbps connection all share 1Gbps of upstream bandwidth.

Business connections are closer to 10:1, and a leased line (dedicated circuit) is 1:1.

Nowadays agents like Claude Code can run autonomously for hours just given a goal description. It doesn’t take a lot of human effort at all to set up a bunch of sessions, and these companies don’t limit how many instances you run in parallel. Agents can also spawn sub-agents that run in parallel if a task calls for parallelization. Whether all this produces good results is a different story, especially if you don’t put enough effort into the goal description. But burning tokens as such is not difficult.

Even workflows where you’re just chatting with an agent can burn a lot of tokens. When you’re chatting with an LLM, the entire history becomes part of the input each time you send something. This also applies to tool calls, so if the agent decides to read 20 files before it can work on your request that’s 20 times a file gets added to the history and 20 times that entire growing history is then sent back as input to drive the agent’s next step.

Coding is more affected by this than many other applications because even a new conversation tends to start with the agent gathering a bunch of source code files, and then the response to a task is not just a bunch of text once, but a sequence of tool calls to make edits across files, build, run tests, react to test failures, and so on, all for one actual human prompt - but in reality a back-and-forth between the LLM and the harness with a quickly growing history.

I use cursor for work, and boy I can easily burn 100-300$ a day

I assume that you’d have some sort of massive workload that you span over multiple plans. You just have software to switch you from one plan to the next once you saturate the plan.

Probably not all that hard to write some kind of software that tries to make massive use of LLMs. Like, oh, I don’t know. Getting all abstract here, any problem in computer science where you have a problem that you don’t know how to solve directly, but you can easily check whether an answer is correct. Then you just keep trying to solve it, and repeatedly check whether the generated answer is correct or not.

Another possibility is that you have a problem where you can quickly check the quality of a given solution (either via human assistance or software, even though you don’t know how to solve the problem yourself), and want to generate a number of solutions and pick the best.

I’ve certainly seen that with image-generating diffusion models, rather than LLMs — stuff like “batch-generate me N images using this prompt, and I’ll pick the best”. It’s an algorithmically-simple, brute-force way of improving quality, by just throwing more compute time at the problem. The human “quality evaluation” is cheap to do compared to the human time required to generate an image. Burns a lot of compute time, but the alternative to improve quality is improving the model, and if we don’t know how to do that yet…shrugs

Narrator: “it’s not.”

Funnily enough, there’s pretty much the same mindset in our team, but towards USA models (and tech in general). There’s a non-zero risk that either EU decides that USA products aren’t trustworthy or that the orange man decides to cut off European companies from the services (which kinda already happened with Anthropic). And, as we’re in Europe, there’s very similar threat models for Chinese services.