> The main explanation of bottleneck is wrong: it’s the model weights which dominate memory bandwidth (and hence why batching multiple requests in a single pass increases total throughput). If copy user tokens was the bottle neck, batching would not achieve any speed up.
Inference is memory-bound only at low batch sizes. At high batch sizes it becomes compute-bound. There's a certain threshold where stuffing more requests in a batch will slow down every request in isolation even though it may still increase the number of tokens/second across the whole batch for all request in aggregate.
I would guess you haven't done this in practice. Yes, of course inference is memory bound at low batch sizes. This is why we run larger batch sizes!
Also there does not exist any batch size > 1 where per-request throughput is equal to bs=1. Doing any batching at all will slow all intra-batch requests down.
I would give my left kidney for either a continuation or a reboot of Babylon 5 under the helm of J. Michael Straczynski with full creative freedom. Or hell, even an entirely different show.
In my opinion he's one of the few people in the industry who actually knows how to skillfully write a coherent TV show. And by that I mean: he actually pre-planned the story (spanning multiple seasons!) of B5 right from the beginning, instead of completely making it up on the fly like so many other shows. Subtle things which might seem inconsequential, appearing in the very first season, can foreshadow events happening seasons later. This makes it, at least for me, much more coherent and enjoyable to watch, and I wish more writers/showrunners would adopt this approach (instead of the usual writers' room + only plan until the end of the season approach which is so common today).
The CW picked up a Babylon 5 reboot "recently", but it seems like it got trapped in development hell and caught in the cross-fires of the ugly WB-CBS divorce of The CW and the ugly merger of WB and Discovery and what is shaping up to be an ugly divorce of WB and Discovery.
I thought that project got turned into the animated film that got released not too long back? I got the impression JMS was done with B5 after he went back to focusing on comics again with his run on Captain America.
>I would give my left kidney for either a continuation or a reboot of Babylon 5 under the helm of J. Michael Straczynski with full creative freedom. Or hell, even an entirely different show.
There has been discussion about a reboot over the years, with JMS throwing some cold water[0] (at least for now) on the possibility in January 2026.
There's sort of a "continuation" with Babylon 5: The Road Home[1] from 2023.
There's also Crusade[2] which only ended up with a dozen or so episodes, although JMS had a multi-year story arc planned.
you are right about straczynski, but i'd prefer to see a new scifi series by him rather than a reboot or continuation. ok, a spin off maybe. jeremiah was pretty good. (i haven't seen sense8)
but i just see that he was approached to direct star trek: enterprise. star trek by straczynski is something i'd really love to see.
>I would give my left kidney for either a continuation or a reboot of Babylon 5 under the helm of J. Michael Straczynski with full creative freedom.
I don't know. I loved Babylon 5 but I also found it kind of corny. And then Crusade was just a D&D campaign in space. The ship was even called the Excalibur FFS. I feel like "full creative freedom" would ruin it the way it did with George Lucas and Star Wars.
>and I wish more writers/showrunners would adopt this approach (instead of the usual writers' room + only plan until the end of the season approach which is so common today.
What else can you do when you don't know if you're getting renewed? You can't push the conclusions to your storylines forward into seasons you might never even have to resolve them.
This is really amusing to watch, because everything that Grok is accused of is something which you can also trigger in currently available open-weight models (if you know what you're doing).
There's nothing special about Grok in this regard. It wasn't trained to be a MechaHitler, nor to generate CSAM. It's just relatively uncensored[1] compared to the competition, which means it can be easily manipulated to do what the users tell it to, and that is biting Musk in the ass here.
And just to be clear, since apparently people love to jump to conclusions - I'm not excusing what is happening. I'm just pointing out the fact that the only special thing about Grok is that it's both relatively uncensored and easily available to a mainstream audience.
I'm not talking about video editing software; that's a different class of software. I'm talking about other generative AI models, which you can download today onto your computer, and have it do the same thing as Grok does.
> How is this exoneration?
I don't know; you tell me where I said it was? I'm just stating a fact that Grok isn't unique here, and if you want to ban Grok because of it then you need to also ban open weight models which can do exactly the same thing.
Agreed, thankfully, there is no CSAM feature, and they patched it in a couple of days so that it would stop obeying requests to do so and banned the users abusing Grok for that purpose. Once they prove they have put the appropriate mechanisms in place to make sure it doesn't happen again it's fine and dandy, no?
> Agreed, thankfully, there is no CSAM feature, and they patched it in a couple of days so that it would stop obeying requests to do so and banned the users abusing Grok for that purpose.
The order, over several days, was:
* banned users and took down tweets requesting the content, without taking down the content that they clearly knew of, since they responded against the requests that generated it.
* Made the feature paid-only.
* Took down content and restricted the functionality.
> Once they prove they have put the appropriate mechanisms in place to make sure it doesn't happen again it's fine and dandy, no?
I’m not sure all jurisdictions involved will see knowing generation and dissemination of nonconsensual, including child, pornography as the kind of thing where “I promise not to do itt again, so we’re cool, right?” works, but we’ll see.
Well you could not sue the video-editing software for someone making child pornography with it. You would, quite sanely, go after the pedophiles themselves.
Maybe tying together an uncensored AI model and a social network just isn't something that's ethical / should be legal to do.
There are many things where each is legal/ethical to provide, and where combining them might make business sense, but where we, as a society have decided to not allow combining them.
No. I'm just saying that people should be consistent and if they apply a certain standard to Grok then they should also apply the same standard to other things. Be consistent.
Meanwhile what I commonly see is people dunking on anything Musk-related because they dislike him, but give a free pass on similar things if it's not related to him.
Every island is capable of hosting pedophiles, but they don't. The one island that's famous for pedos is the one Musk wanted to be invited to. Find me more pedo islands, I'll dunk on them too very consistently. Whether it's AI with CSAM or islands with pedos, Musk is definitely consistent.
> I don't really get how VSCode got so popular. You can use a language server perfectly easily with Vim, Emacs, Helix, Sublime, etc.
You open it. It just works. And the learning curve is smooth.
Compare this to Vim where, if it's the first time you're opening it, you are forced to kill the process because you don't even know how to quit it, never mind actually do any productive work.
> Compare this to Vim where, if it's the first time you're opening it,
If you open vim with a file, like you do with all file editors, there's no such examples. It's also at the bottom of auth/credit/contribution fluff in your example, which people would be expected to ignore.
> Because you can't read
I'm not arguing for more hand holding here, but saying the poster can't read is ironic. Reading is one part, comprehending is the other.
> If you open vim with a file, like you do with all file editors, there's no such examples
I think you are being disingenuous here. I'm serious. VSCode isn't just plug and play. It isn't just "install and away you go". It's not much you have to be taught, but you did have to learn something from somewhere. Either from another IDE with similar visual language or because someone showed it to you.
Besides, your first usage of VSCode isn't going to be opening a file. It also gives you info at the startup.
Are you seriously telling me you haven't Googled how to use VSCode? Or did you just forget because you are now comfortable using it and used to it?
> not sure why we're pretending vim is easy to use
Because it is! I don't know why you're pretending it is hard.
The conversation is no different than iPhone users fumbling their first time on Android and Android users fumbling their first time on iPhones. If you didn't have to learn things then this would never happen. Can there be better communication around vim? Certainly, that's always going to be true. But "run `vimtutor`" is pretty effective and I'm sorry, DEVELOPERS shouldn't be documentation adverse.
If you want to use vim you only need to know a few things. `i` to start typing (insert), esc (or ctrl+[) to go to "command mode", `:w` to write, `:q` to quit. That's literally 4 things. In the standard vim config you can use your mouse and arrow keys, so you don't even need to learn hjkl to get going.
Truthfully, the main difference is where the learning happens. vim forces the user to learn a few things up front (and not many). Getting all my python envs and configuring everything in VSCode took me much longer than the 15-20 minutes it takes to read vimtutor. You get your "Hello World" out faster, but it also makes dealing with environments and other things harder.
What I will admit is that vim is hard to master.
But mastery is very different than usage. Even intermediate level is not hard to achieve once you understand the design language. It's a different way of thinking but come one, it's aimed at programmers. You're really telling me it is hard to learn that there's a command mode and a writing mode? That commands are composed of actions + motions? Going through vimtutor means you should know a lot more commands than vimtutor introduces because of this design language. But mastery? It requires reading docs and years. But that's not a flaw because what you can do in vim is nearly unbounded. While that's also kinda true about VSCode there is a much steeper learning curve you need to do fancy things and that knowledge isn't going to make you really better at general VSCode usage.
> It's a meme
So is the difficulty of assembling Ikea furniture.
People brag about how dumb they are all the time. I don't get how you think that's a defense. There's plenty of docs in the program itself and plenty more online. Many being well written. But ultimately vim isn't being targeted at the general audience (unlike Ikea furniture) and it's perfectly acceptable that it requires a little reading. By my screen you get all the commands I listed, and more, with two screens worth of text from vimturor. 1 screen if you split, since it is 80 chars width. That really isn't much reading. If that is the definition of "hard", for developers, then I have no hope for software. It shouldn't be "hard" or "too much" for anyone. Let's be honest. Are you really okay with the bar being so low that it's impossible for a blind person to trip over?
> It's also at the bottom of auth/credit/contribution fluff in your example
Here's the message
VIM - Vi IMproved
version 9.1.857
by Bram Moolenaar et al.
Vim is open source and freely distributable
Become a registered Vim user!
type :help register<Enter> for information
type :q<Enter> to exit
type :help<Enter> or <F1> for on-line help
type :help version9<Enter> for version info
Not even 10 lines...
If you brag about being lazy, expect to be called lazy.
I just was mainly motivated in replying to your accusation that the original poster couldn't read -- I feel like that claim is pretty disingenuous from the get-go and if I'm being accused of it, well, I'm in good company.
I don't even know what to reply to here, but I'm in general agreeance with most of what you wrote. I just don't agree users type "vim" alone their first time, I'd wager it's following some guide/tutorial online that already has 'vim filename.txt' snuck in there. The fact that people get stuck in vim feels like something intentional to weed out people, otherwise it's a funny problem people run into on other programs like ftp, ssh, screen, even the python interactive shell. There's no unified lexicon on cli tooling, except maybe the gnu clis. It makes you appreciate good GUIs.
The real big brain approach here is to divorce the idea of vim from the command line editor and use it as a plugin in an IDE. Best of both worlds.
While there isn't there is more than most people give credit to. It shouldn't be too much of a surprise giving cli people write cli tools getting inspiration from cli tools. I mean we can't even say there's a unifying language in the GUI space.
> The real big brain approach here is to divorce the idea of vim from the command line editor and use it as a plugin in an IDE
Maybe I'll be convinced when a vim plugin or "vim bindings" represents something close to actual vim. I need a lot more than hjkl, gg, gG and such for movements. It's crazy how few even have H,M,L let alone <C-F>, <C-B>. I'm really surprised so many don't have / bit less surprised at * and the like. An ide giving me vim bindings needs to also give me :Ex, tabs, tags, and so on. I don't think I've ever seen a plugin give me :%s and I'd be really surprised if it gave me \\{-}.
I think there's a lot of confusion when it comes to vim. It's an editor. Editors aren't just for writing and the real power of vim is editing. It's a major bonus that I can do all this with less resource consumption that just a plugin
Frequently using vim bindings in plugins leads to me generating gibberish.
Frequently using vim bindings in plugins leads me to unintentionally closing windows
Developers? If you can't read docs then you can't do your job.
I don't have much confidence in LLMs replacing devs, but if the dev is so arrogant that they think they're too good for documentation and if they have to read is too complicated, then yeah I believe AI can already replace that person. They were just a warm body, not a dev.
This isn't the flex you think it is. You're really illustrating my point... being able to understand context and make inferences is part of basic literacy skills
> This is why my heart sank this morning. I have spent over a year training 4.0 to just about be helpful enough to get me an extra 1-2 hours a day of productivity.
Maybe you should consider basing your workflows on open-weight models instead? Unlike proprietary API-only models no one can take these away from you.
I have considered it, and it is still on the docket. I have a local 3090 dedicated to ML. Would be a fascinating and potentially really useful project, but as a freelancer, it would cost a lot to give it the time it needs.
> If you enjoy AI art, you're probably fake and have no appreciation. That's my take.
I enjoy AI art. I don't enjoy AI slop. There's a fundamental difference between the two. It's true that the Internet is flooded with low-effort AI slop, but AI is just a tool like any other, and you can create real art with it. It just takes skill.
Here's an experiment: try visiting CivitAI's featured images page[1] and then tell me with a straight face that you'd classify none of those images as art.
> Because the promise of "open-source" (which this isn't; it's not even open-weight) is that you get something that proprietary models don't offer. If I wanted censored models I'd just use Claude (heavily censored).
You're saying it's surprising that a proprietary model is censored because the promise of open-source is that you get something that proprietary models don't offer, but you yourself admit that this model is neither open-source nor even open-weight?
> Anthropic Chief Executive Officer Dario Amodei said selling advanced artificial intelligence chips to China is a blunder with “incredible national security implications” [...] “I think this is crazy. It’s a bit like selling nuclear weapons to North Korea.”
This is all a smoke screen. He knows very well that China can and will develop their own hardware to train AI models (and in fact, they are successfully doing just that; e.g. the recently released GLM-Image was trained on their own silicon). His only objective here is to slow them down enough so that they don't eat Anthropic/Claude's lunch releasing open-weight models that are increasingly competitive. But he can't just openly say "hey, we don't like that they release open weight models for free", so he's engaging in the AI version of the "think of the children" argument.
Anthropic's whole modus operandi was always pretty much "we should control this technology, no one else". It's not a coincidence they're the only major lab which has not released any open weight models, they don't publish any useful research (for training models) and they actively lobby the lawmakers to restrict people's access to open weight models. It's incredibly ironic that Dario is worried about (I quote) "1984 scenarios" while that's exactly what his company is aiming towards (e.g. giving Palantir access to those models is not "unsafe", but an average Joe having unrestricted local access is an immediate 1984-style dystopia).
Yes. Dario Amodei is just using "china bad" to stave off competition. I highly doubt he cares one bit about anything else except his own wallet and power.
This guess is from launch day, but over time has been shown to be roughly correct, and aligns with the performance of Opus 4.5 vs 4.1 and across providers.
> On the infra side, training a 1.5B model in ~4 hours on 8×H100 is impressive.
It's hard to compare without more details about the training process and the dataset, but, is it? Genuine question, because I had the opposite impression. Like, for example, recently I did a full finetuning run on a 3B model chewing through a 146k entry dataset (with 116k entries having reasoning traces, so they're not short) in 7 hours on a single RTX 6000.
Honestly I think we can improve our training throughput drastically via a few more optimizations but we've been spending most of our time on model quality improvements instead.
Inference is memory-bound only at low batch sizes. At high batch sizes it becomes compute-bound. There's a certain threshold where stuffing more requests in a batch will slow down every request in isolation even though it may still increase the number of tokens/second across the whole batch for all request in aggregate.
reply