TBH most of the cost is from the individual components. The core chip fab, the memory fab, the oled screen fab, the battery, power regulation, cameras, all massive operations and very automated. Not to speak of the software stack. Or the chip R&D and tape out costs.
The child labor is awful, but it’s not the most expensive part of a $1k+ iPhone.
They’re more often subsidized by carriers here (in the US), too. I didn’t really want an iPhone, but $400 new for a Plus, with a plan discount, just makes an Android set not worth it.
OP’s being abrasive, but I sympathize with the sentiment. Bluesky is algorithmic just like Twitter.
…Dunno about Bluesky, but Lemmy feels like a political purity test to me. Like, I love Lemmy and the Fediverse, but at the same time mega upvoted posts/comments like “X person should kill themself,” explulsion of nuance in specific issues, leaks into every community and such are making me step back more and more.
Yes because ultimately, it just wasn’t good enough.
That’s what I was trying to argue below. Unified memory is great if it’s dense and fast enough, but that’s a massive if.
It’s not theoretical, it’s just math. Removing 1/3 of the bus paths, and also removing the need to constantly keep RAM powered
And here’s the kicker.
You’re supposing it’s (given the no refresh bonus) 1/3 as fast as dram, similar latency, and cheap enough per gigabyte to replace most storage. That is a tall order, and it would be incredible if it hits all three of those. I find that highly improbable.
Even dram is starting to become a bottleneck for APUs, specifically, because making the bus wide is so expensive. This applies to the very top (the MI300A) and bottom (smartphones and laptop APUs).
Optane, for reference, was a lot slower than DRAM and a lot more expensive/less dense than flash even with all the work Intel put into it and busses built into then top end CPUs for direct access. And they thought that was pretty good. It was good enough for a niche when used in conjunction with dram sticks
You are talking theoretical.
A big reason that supercomputers moved to a network of “commodity” hardware architecture is that its cost effective.
How would one build a giant unified pool of this memory? CXL, but how does it look physically? Maybe you get a lot of bandwidth in parallel, but how would it be even close to the latency of “local” DRAM busses on each node? Is that setup truly more power efficient than banks of DRAM backed by infrequently touched flash? If your particular workload needs fast random access to memory, even at scale the only advantage seems to be some fault tolerance at a huge speed cost, and if you just need bulk high latency bandwidth, flash has got you covered for cheaper.
…I really like the idea of non a nonvolatile, single pool backed by caches, especially at scale, but ultimately architectural decisions come down to economics.
How is that any better than DRAM though? It would have to be much cheaper/GB, yet reasonably faster than the top-end SLC/MLC flash Samsung sells.
Another thing I don’t get… in all the training runs I see, dataset bandwidth needs are pretty small. Like, streaming images (much less like 128K tokens of text) is a minuscule drop in the bucket compared to how long a step takes, especially with hardware decoders for decompression.
Weights are an entirely different duck, and stuff like Cerebras clusters do stream them into system memory, but they need the speed of DRAM (or better).
Yeah, it’s a solution in search of a problem.
Is it much cheaper than DRAM? Great! But until then, even if it’s lower power due to not needing the refresh, flash is just so cheap that it can scale up much better.
And I dunno what they mean by AI workloads. How would non volatility help at all, unless it’s starting to approach SRAM performance.
Some embedded stuff could use it, but that’s not a huge margin market.
Optane was sorta interesting because it was ostensibly cheaper and higher capacity than DRAM, albeit not enough.
The Switch 2 chip is effectively older than the now aging (but fantastic for the price/size) Van Gogh chip in the Steam Deck. It shouldn’t be expensive to make.
Oh, you think Canadians aren’t going to get in on tariff evasion? They 100% will.
The tremendous irony is America was founded to evade massive tariffs, sorta… and now we’re doing it to our self.
You can generally toggle LLM “grounding” features, aka inserting web searches into their context.
Modern LLMs have a information “cutoff” of a few months ago, at the latest, so the base models will have zero awareness of this formula.
TBH it’s probably human written.
I used to write small articles for a tech news outlet on the side (HardOCP), and the entire site went under well before the AI boom because no one can compete with conveyer belts of of thoughtless SEO garbage, especially when Google promotes it.
Point being, this was a problem well before the rise of LLMs.
In this case, it’s as simple as “type it into ChatGPT, like the Reddit users did” :/
That they didn’t try to replicate it.
How about the outlet checks and finds out?
I did, and I couldn’t get low-temperature Gemini or a local LLM to replicate it, and not all the tariffs seem to be based on the trade deficit ratio, though some suspiciously are.
Sorry, but this is a button of mine, outlets that ask stupidly easy to verify questions but dont even try. No, just cite people on Reddit and Twitter…
True! Models not trained on a specific language are generally bad at that language.
However, there are some exceptions, like a Japanese tune of Qwen 32B which dramatically enhances it Japanese, but the training has to be pretty extensive.
And even that aside… the effect is still there. The point it to illustrate that LLMs are sort of “language independent” internally, like you said.
It’s a metaphor.
They’re translating the input tokens to intent in the model’s middle layers, which is a bit more precise.
I use local instances of Aya 32B (and sometimes Deepseek, Qwen, LG Exaone, Japanese finetunes, others depending on the language) to translate stuff, and it is quite different than Google Translate or any machine translation you find online. They get the “meaning” of text instead of transcribing it robotically like Google, and are actually pretty loose with interpretation.
It has soul… sometimes too much. That’s the problem: It’s great for personal use where it can ocassionally be wrong or flowery, but not good enough for publishing and selling, as the reader isn’t necessarily cognisant of errors.
In other words, AI translation should be a tool the reader understands how to use, not something to save greedy publishers a buck.
EDIT: Also, if you train an LLM for some job/concept in pure Chinese, a surprising amount of that new ability will work in English, as if the LLM abstracts language internally. Hence they really (sorta) do a “meaning” translation rather than a strict definitional one… Even when they shouldn’t.
Another thing you can do is translate with one local LLM, then load another for a reflection/correction check. This is another point for “open” and local inference, as corporate AI goes for cheapness, and generally tries to restrict you from competitors.
Do it! Nullify American IP, take what we have.
All our Trump-licking companies will love that. And it would be a bigger loss for the US than EU.
Excess carrier inventory they can write off as a loss since the Plus didn’t sell as well, or so I was told.