Well, let's not forget that text models are not the only models! Video models are much slower and need comparatively more resources, and all they can do even at that size is generate videos a few seconds long. Clearly a ton more work is going to go into those, and demand for them will probably increase as more creative tools get authored using them as a central part of the workflow. Low-res local rendering for preview might be a thing, but the lion's share of the work for high-res, near-realtime rendering is going to be done on huge clusters for a long time yet.
This is definitely a good point. I imagine the max capacity for video models is significantly lower than for text models (there just aren't as many professionals in video as there are people who write text or code) but I could be wrong.
I think there’s still an open question around are the ultra-large next-gen models worth it? For those of us without early access to Mythos, it’s hard to verify whether it’s been held back from the public due to actually being “too dangerously powerful to release yet” as implied or because the gains aren’t outpacing the costs.
I think GPT 4.5 showed that there is indeed a practical limit we're close too. That was supposedly a high-trillions of parameter model that was deprecated almost immediately because it was slow, insanely expensive, and had questionable benefits over the smaller models. Though apparently the new Mythos and whatever GPT Spud is (if it wasn't 5.5) are back up in the high trillions.
Actually having used it a bit, I'm quite excited to see a modern model of similar size.
I think what people didn't realize was, just because the GPT-4.5 model didn't get better on the benchmarks, didn't mean the model wasn't different than the earlier models. It was being compared to thinking models that were being developed at the same time.
The GPT 4.5 model still has some of the most "human" like abilities in communication even though it isn't particularly good a problem solving. It hadn't under gone the same type of reinforcement training.
I still use GPT 4.5 sometimes, in creative exercises it can be surprisingly effective. The model is still available.
yes and no. We've reached the point where larger models are higher quality, but they're also too expensive and slow to be used broadly. The giant models, however are still useful for training smaller models that are actually deployable.
I think the issue with these kind of stances is that they are basically status quo bias; why don't you object to the computer itself, and thus refuse to write programs? After all: they were invented by the UK military in the pursuit of military goals (and much of their subsequent development was funded by the US military - see https://types.pl/@graydon/110648447694201698 - and the fact that ARPAnet, GPS, etc were all military creations). Computer systems are mostly used by large corporations and the military to achieve their goals more effectively.
Usually the objection is that "oh well, the computer can be used for many great things", which isn't particularly satisfying because, um, we can use AI for "good" (better?) things as well (e.g. trying to find novel cures, unlocking the mysteries of protein folding, etc etc).
Then the objection becomes something like "well the computer is here and we have to live with it", which is also now true of AI. Do I like the "it's inevitable" argument; no, but it's clearly very true that we do have the transformer, that won't go away - where we DO have control (or should seek to change) is the organisational structures that we as a society decide to create, and how we safeguard the dignity of the individual in changing times.
Being able to discern what is and isn't in our control helps tremendously in doing what is right and constructive.
The fact that some people opt out of engaging with AI, I think is healthy for society as a whole. If that's within their control and they exercise their control to do what they think is right, then I commend them.
That said, I do think there is a greater natural force at play, something involving entropy and increasing complexity and energy profit maximization. It seems to cut through all levels of abstraction from organic chemistry to civilizations and probably beyond. I assume this is outside of humanity's control, and therefore outside of any individuals control.
So what is inside our control? Our own perceptions and actions.
My perception is that the advance of computation and by extension proliferation of probabilistic programs (AI) is inevitable. It's on a continuum that is a force of nature.
What I might have some control over is choosing to harness that potential to increase future prosperity for more people and the greater environment, and to avoid contributing to outcomes that harm people and the environment.
Lots of bad things are happening and will happen that are outside my control.
I do genuinely believe that the capabilities are inherently neutral. Civilization can choose to harness them in a variety of ways, for a variety of purposes.
If the majority of people choose options that are game theory win-win, then the future will be better... If the majority of people choose win-lose, then the future will probably be worse.
Yeah so therefore I think a positive attitude is all the more needed, where you see the potentials, see solutions instead of problems. But I feel most anti-AI people are just negative people seeing only problems and don't have any solutions to offer.
Allegedly OpenAI's contracting model is much more vicious than Anthropic's; at work (admittedly a little IP-protective) we have unlimited Claude, but no Codex subscription because OpenAI won't give us sufficient guarantees around data retention.
We are also concerned that it may not be possible to bind OpenAI using contract terms and/or the US legal system.
> The technologists who create it believe they should control it
I think there's an interesting phenomenon where it is _not_ the people who control it, but instead a kind of international finance man cum-captain of industry (perhaps best embodied by Sam Altman) who does not create the technology and yet has ended up wielding the levers.
It's not even just a QA thing, consider the use case: A sub-ohm vape head is basically almost shorting what is often a unprotected lithium ion cell (18650 or whatnot). Phones meanwhile are full of temperature sensors, battery pack in the phone has some kind of firmware/monitoring, board on the phone has a charge controller.
There are plenty of good cell manufacturers that won't have problems in this current dumping situation (and will have certain passive protections like a CID to cut the current if it gets too hot). Problem is people like cheap and there are sketchy knockoff cells without those protections and shoddy manufacturing quality.
If there was anything recently that forced the change it was probably the CT scans of the Haribo battery packs showing the cathode/anode overlap. This sort of thing should spook airlines.
Do we still have UL? Do they test battery packs? Why not make it a requirement to only fly with ones that pass lab testing like UL?
Yes but "the search space is too large" is something that has been said about innumerable AI-problems that were then solved. So it's not unreasonable that one doubts the merit of the statement when it's said for the umpteenth time.
I should have been more specific then. The problem isn't that the search space is too large to explore. The problem is that the search space is so large that the training procedure actively prefers to restrict the search space to maximise short term rewards, regardless of hyperparameter selection. There is a tradeoff here that could be ignored in the case of chess, but not for general math problems.
This is far from unsolvable. It just means that the "apply RL like AlphaGo" attitude is laughably naive. We need at least one more trick.
> I found an (as of yet not-root-caused) error in sqlite (no crash or coredump, just returns the wrong data, and only when using sqlite in ram-only-mode).
You should report this to the SQLite developers - they are very smart and very interested in fixing SQLite correctness bugs!
> Stallman has always been right. It's mind boggling just how right he was about everything.
Mind boggling right about not allowing GCC to be used as a library, his comments on Jeffrey Esptein, a refusal to in any way compromise (e.g. the GNU/Linux meme), etc...
Oh and a recognition that free software, while nice, does not in any way solve the underlying issues he claims it does. Similarly to how letting everyone walk around their local water treatment facility and perform chemical tests doesn't really work and instead the state regulates and hires experts to monitor the water supply...
Do you disagree with his description of Epstein as a serial rapist? Do you disagree with Stallman's position that Epstein should be described according to the specific crimes he committed: rape instead of using much more vague terms that also encompass much less severe crimes which Epstein himself used to downplay and obscure the actual crimes he committed?
Nothing wrong with that move from a strategic point of view. The objective was to leverage GCC and make others play ball. People who wanted GCC should have been forced to do things the free software way.
Only problem with this is it turned out GCC didn't provide enough leverage. Replacing GCC wasn't difficult enough. People implemented LLVM instead and the rest is history.
Compare that to Linux which literally leaves companies behind in the dust when they refuse to merge. No kernel ABI stability: if out-of-tree stuff gets broken it's not their problem. Companies have a choice: play ball or pay the maintenance costs required to keep up with the biggest free software project ever. That's how it should be.
> his comments on Jeffrey Esptein
By "everything" I of course meant his ideas on computer freedom which is the context of this thread. I don't know or care about his opinions on Epstein.
> a refusal to in any way compromise
As he should. If anything he's not extreme enough. Compromise is the root of many evils.
> a recognition that free software, while nice, does not in any way solve the underlying issues he claims it does
reply