I think the sad part is one of the things America has been great at for the last 100 years is science and innovation. So it really kind of highlights that MAGA isn’t really about MAGA.
I used to think symbolic AI would become more important to modern AI, but I think that was just wishful thinking on my part, because I like it.
Now I think symbolic AI is more likely to remain a tool that LLM-based systems can make use of. LLMs keep getting better and better at tool use and writing scripts to solve problems, and symbolic AI is a great tool for certain classes of problems. But symbolic AI is just too rigid to be used most of the time.
This is not at all clear to me. Reminded of that joke of how "A month in the lab can save you an hour in the library", thinking about some of the best science in history, the researchers often had very strong theory-based belief in their hypothesis, and the experiment was "just" confirmation. Whereas the worst science has people run experiments without a good hypothesis, and then attach significance to spurious correlations.
In other words, while experiments are important, I believe we can get a lot more distance from thinking deeply about what we already have.
I really don't understand the argument that nvidia GPUs only work for 1-3 years. I am currently using A100s and H100s every day. Those aren't exactly new anymore.
It’s not that they don’t work. It’s how businesses handle hardware.
I worked at a few data centers on and off in my career. I got lots of hardware for free or on the cheap simply because the hardware was considered “EOL” after about 3 years, often when support contracts with the vendor ends.
There are a few things to consider.
Hardware that ages produce more errors, and those errors cost, one way or another.
Rack space is limited. A perfectly fine machine that consumes 2x the power for half the output cost. It’s cheaper to upgrade a perfectly fine working system simply because it performs better per watt in the same space.
Lastly. There are tax implications in buying new hardware that can often favor replacement.
I agree that there is hyperbole thrown around a lot here and its possible to still use some hardware for a long time or to sell it and recover some cost but my experience in planning compute at large companies is that spending money on hardware and upgrading can often result in saving money long term.
Even assuming your compute demands stay fixed, its possible that a future generation of accelerator will be sufficiently more power/cooling efficient for your workload that it is a positive return on investment to upgrade, more so when you take into account you can start depreciating them again.
If your compute demands aren't fixed you have to work around limited floor space/electricity/cooling capacity/network capacity/backup generators/etc and so moving to the next generation is required to meet demand without extremely expensive (and often slow) infrastructure projects.
Sure, but I don't think most people here are objecting to the obvious "3 years is enough for enterprise GPUs to become totally obsolete for cutting-edge workloads" point. They're just objecting to the rather bizarre notion that the hardware itself might physically break in that timeframe. Now, it would be one thing if that notion was supported by actual reliability studies drawn from that same environment - like we see for the Backblaze HDD lifecycle analyses. But instead we're just getting these weird rumors.
I agree that is a strange notion that would require some evidence and I see it in some other threads but looking at the parent comments going up it seems people are discussing economic usefulness so that is what I'm responding to.
It's because they run 24/7 in a challenging environment. They will start dying at some point and if you aren't replacing them you will have a big problem when they all die en masse at the same time.
These things are like cars, they don't last forever and break down with usage. Yes, they can last 7 years in your home computer when you run it 1% of the time. They won't last that long in a data center where they are running 90% of the time.
A makeshift cryptomining rig is absolutely a "challenging environment" and most GPUs by far that went through that are just fine. The idea that the hardware might just die after 3 years' usage is bonkers.
Crypto miners undervote for efficiency GPUs and in general crypto mining is extremely light weight on GPUs compared to AI training or inference at scale
With good enough cooling they can run indefinitely!!!!! The vast majority of failures are either at the beginning due to defects or at the end due to cooling! It’s like the idea that no moving parts (except the HVAC) is somehow unreliable is coming out of thin air!
A toy example: NeoCloud Inc builds a new datacenter full of the new H800 GPUs. It rents out a rack of them for $10/minute while paying $6/minute for electricity, interest, loan repayment, rent and staff.
Two years later, H900 is released for a similar price but it performs twice as many TFlOps/Watt. Now any datacenter using H900 can offer the same performance as NeoCloud Inc at $5/month, taking all their customers.
There’s plenty on eBay? But at the end of your comment you say “a rate cheaper than new” so maybe you mean you’d love to buy a discounted one. But they do seem to be available used.
Do you know how support contract lengths are determined? Seems like a path to force hardware refreshes with boilerplate failure data carried over from who knows when.
The common factoid raised in financial reports is GPUs used in model training will lose thermal insulation due to their high utilization. The GPUs ostensibly fail. I have heard anecdotal reports of GPUs used for cryptocurrency mining having similar wear patterns.
I have not seen hard data, so this could be an oft-repeated, but false fact.
It's the opposite actually - most GPU used for mining are run at a consistent temp and load which is good for long term wear. Peaky loads where the GPU goes from cold to hot and back leads to more degradation because of changes in thermal expansion. This has been known for some time now.
That is commonly repeated idea, but it doesn't take into account countless token farms which are smaller than a datacenter. Basically anything from a single MB with 8 cards to a small shed with rigs, all of which tend to disregard common engineering practices and run hardware into a ground to maximize output until next police raid or difficulty bump. Plenty of photos in the internet of crappy rigs like that, and no one guarantees which GPU comes whom where.
Another commonly forgotten issue is that many electrical components are rated by hours of operation. And cheaper boards tend to have components with smaller tolerances. And that rated time is actually a graph, where hour decrease with higher temperature. There were instances of batches of cards failing due to failing MOSFETs for example.
While I'm sure there are small amateur setups done poorly that push cards to their limits this seems like a more rare and inefficient use. GPUS (even used) are expensive and running them at maximum would require large costs and time to be replacing them regularly. Not to mention the increased cost of cooling and power.
Not sure I understand the police raid mentality - why are the police raiding amateur crypto mining setups ?
I can totally see cards used by casual amateurs being very worn / used though - especially your example of single mobo miners who were likely also using the card for gaming and other tasks.
I would imagine that anyone purposely running hardware into the ground would be running cheaper / more efficient ASICS vs expensive Nvidia GPUs since they are much easier and cheaper to replace. I would still be surprised however if most were not proritising temps and cooling
Miners usually don't overclock though. If anything underclocking is the best way to improve your ROI because it significantly reduces the power consumption while retaining most of the hashrate.
Exactly - more specifically undervolting. You want the minimum volts going to the card with it still performing decently.
Even in amateur setups the amount of power used is a huge factor (because of the huge draw from the cards themselves and AC units to cool the room) so minimising heat is key.
From what I remember most cards (even CPUs as well) hit peak efficiency when undervolted and hitting somewhere around 70-80% max load (this also depends on cooling setup). First thing to wear out would probably be the fan / cooler itself (repasting occasionally would of course help with this as thermal paste dries out with both time and heat)
The only amatures I know doing this are trying to heat their garrage for free. so long as the heat gain is paid for they can afford to heat an otherwise unheated building.
> I have heard anecdotal reports of GPUs used for cryptocurrency mining having similar wear patterns.
If this was anywhere close to a common failure mode, I'm pretty sure we'd know that already given how crypto mining GPUs were usually ran to the max in makeshift settings with woefully inadequate cooling and environmental control. The overwhelming anecdotal evidence from people who have bought them is that even a "worn" crypto GPU is absolutely fine.
I can't confirm that fact - but it's important to acknowledge that consumer usage is very different from the high continuous utilization in mining and training. It is credulous that the wear on cards under such extreme usage is as high as reported considering that consumers may use their cards at peak 5% of waking hours and the wear drop off is only about 3x if it is used near 100% - that is a believable scale for endurance loss.
1-3 is too short but they aren’t making new A100s, theres 8 in a server and when one goes bad what do you do? you wont be able to renew a support contract. if you wanna diy you eventually you have to start consolidating pick and pulls. maybe the vendors will buy them back from people who want to upgrade and resell them. this is the issue we are seeing with A100s and we are trying to see what our vendor will offer for support.
Margins are typically not so razor thin that you cannot operate with technology from one generation ago. 15 vs 17 mpg is going to add up over time, but for a taxi company it's probably not a lethal situation to be in.
At least with crypto mining this was the case. Hardware from 6 months ago is useless ewaste because the new generation is more power efficient. All depends on how expensive the hardware is vs the cost of power.
And yet they aren't running planes and engines all from 2023 or beyond: See the MD-11 that crashed in Louisville: Nobody has made a new MD-11 in over 20 years. Planes move to less competitive routes, change carriers, and eventually might even stop carrying people and switch to cargo, but the plane itself doesn't get to have zero value when the new one comes out. An airline will want to replace their planes, but a new plane isn't fully amortized in a year or three: It still has value for quite a while
If a taxi company did that every year, they'd be losing a lot of money. Of course new cars and cards are cheaper to operate than old ones, but is that difference enough to offset buying a new one every one to three years?
>If a taxi company did that every year, they'd be losing a lot of money. Of course new cars and cards are cheaper to operate than old ones, but is that difference enough to offset buying a new one every one to three years?
That's where the analogy breaks. There are massive efficiency gains from new process nodes, which new GPUs use. Efficiency improvements for cars are glacial, aside from "breakthroughs" like hybrid/EV cars.
>offset buying a new one every one to three years?
Isn't that precisely how leasing works? Also, don't companies prefer not to own hardware for tax purposes? I've worked for several places where they leased compute equipment with upgrades coming at the end of each lease.
Who wants to buy GPUs that were redlined for three years in a data center? Maybe there's a market for those, but most people already seem wary of lightly used GPUs from other consumers, let alone GPUs that were burning in a crypto farm or AI data center for years.
who cares? that's the beauty of the lease. once it's over, the old and busted gets replaced with new and shiny. what the leasing company does is up to them. it becomes one of those YP not an MP situations with deprecated equipment.
The leasing company cares - the lease terms depend on the answer. That is why I can lease a car for 3 years for the same payment as a 6 year loan (more or less) - the lease company expects someone will want it. If there is no market for it after they will still lease it but the cost goes up
Depends on the price, of course. I'm wary of paying 50% of new for something run hard 3 years. Seems an NVIDIA H100 is going for $20k+ on EBay. I'm not taking that risk.
That works either because someone wants to buy old hardware for the manufacturer/lessor, or because the hardware is EOL in 3 years but it's easier to let the lessor deal with recyling / valuable parts recovery.
Both companies bought a set of taxis in the past. Presumably at the same time if we want this comparison to be easy to understand.
If company A still has debt from that, company B has that much debt plus more debt from buying a new set of taxis.
Refreshing your equipment more often means that you're spending more per year on equipment. If you do it too often, then even if the new equipment is better you lose money overall.
If company B wants to undercut company A, their advantage from better equipment has to overcome the cost of switching.
> They both refresh their equipment at the same rate.
I wish you'd said that upfront. Especially because the comment you replied to was talking about replacing at different rates.
So your version, if company A and B are refreshing at the same rate, then that means six months before B's refresh company A had the newer taxis. You implied they were charging similar amounts at that point, so company A was making bigger profits, and had been making bigger profits for a significant time. So when company B is able to cut prices 5%, company A can survive just fine. They don't need to rush into a premature upgrade that costs a ton of money, they can upgrade on their normal schedule.
TL;DR: six months ago company B was "no longer competitive" and they survived. The companies are taking turns having the best tech. It's fine.
You can sell the old, less efficient GPUs to folks who will be running them with markedly lower duty cycles (so, less emphasis on direct operational costs), e.g. for on-prem inference or even just typical workstation/consumer use. It ends up being a win-win trade.
Building a new data center and getting power takes years to double your capacity. Swapping out out a rack that is twice as fast takes very little time in comparison.
Depends at the rate of growth of the hardware. If your data center is full and fully booked, and hardware is doubling in speed every year it's cheaper to switch it out every couple of years.
(1) We simply don't know what the useful life is going to be because of how new the advancements of AI focused GPUs used for training and inference.
(2) Warranties and service. Most enterprise hardware has service contracts tied to purchases. I haven't seen anything publicly disclosed about what these contracts look like, but the speculation is that they are much more aggressive (3 years or less) than typical enterprise hardware contracts (Dell, HP, etc.). If it gets past those contracts the extended support contracts can typically get really pricey.
(3) Power efficiency. If new GPUs are more power efficient this could be huge savings on energy that could necessitate upgrades.
Nvidia is moving to a 1 year release life cycle for data center, and in Jensen's words once a new gen is released you lose money for being on the older hardware. It makes no longer financially sense to run it.
Companies can’t buy new Nvidia GPUs because their older Nvidia GPUs are obsolete. However, the old GPUs are only obsolete if companies buy the new Nvidia GPUs.
based on my napkin math, an H200 needs to run for 4 years straight at maximum power (10.2 kW) to consume its own price of $35k worth of energy (based on 10 cents per kWh)
If power is the bottleneck, it may make business sense to rotate to a GPU that better utilizes the same power if the newer generation gives you a significant advantage.
I think the story is less about the GPUs themselves, and more about the interconnects for building massive GPU clusters. Nvidia just announced a massive switch for linking GPUs inside a rack. So the next couple of generations of GPU clusters will be capable of things that were previously impossible or impractical.
This doesn't mean much for inference, but for training, it is going to be huge.
From an accounting standpoint, it probably makes sense to have their depreciation be 3 years. But yeah, my understanding is that either they have long service lives, or the customers sell them back to the distributor so they can buy the latest and greatest. (The distributor would sell them as refurbished)
I had an electrodynamics professor say that there was no reason to memorize the equations, you would never remember them anyways, the goal was to understand how the relationships were formed in the first place. Then you would understand what the relationships are that each equation represents. That I think is the basis for this statement. Memorization of the equations gives you a basis to understand the relationships. So I guess the hope is that is enough. I would argue it isn't enough since physics isn't really about math or equations its about the structure and dynamics of how systems evolve over time. And equations give one representation of the evolution of those systems. But it's not the only representation.
Seems pretty obvious to me just from “four-player, four-monitor arena battle with full eight-way scrolling and AI drones for players not present”.
And it’s clarified with “As a social experience had it been released in 1986, this would have been the grandfather to games like Fortnight; a multiplayer PvP battle arena where each player was supposed to have their own dedicated set of controls and display.”
The main question I had was what was the form of combat. There was no mention of shooting. So what was it? Ramming? Lasers? Magnetism? Bullets? Turns out it is bullets. Boring.
But they did say "a four-player game that was said to resemble Sinistar" but it would have been more accurate to have said "it's literally a 4-player version of Sinistar" and I would have had zero remaining questions.
Sure, people have been putting different sprites on their projectiles for a long time. But whether it flies in a straight line, drops from the top of the screen, or follows a ballistic arc, a bullet is a bullet. It doesn’t matter what sprite you use.
thats right. a year ago i decided, fuck this going to the gym randomly and not having a plan and only kind of committing. im going to do it. so i got a trainer, committed to 4 days a week, and so far ive kept that up for a year. and now, if i find myself running out of time in the day i make time for the gym. it is such a part of my routine that i simply do it without much questioning. because i know if i dont go i will no longer be able to do the things in the gym the way i do them today. i enjoy that feeling and wish to continue. i think the point of life, at least partially, is to figure out things that you enjoy that don't take from you and do them consistently.
i think what is missing from this narrative is not whether or not people have a routine, it is that exercise elevates your mood away from the depressed state, therapy encourages you to question your thoughts and decisions through out your day that might lead you away from a depressed state. to put it a different way, whats the point of exercising every day if you continue the thoughts and habits that are less than satisfactory to you without any self awareness?
reply