Cool that it's possible but basically unusable performance characteristics. For an 8192 token prompt they report a ~1.5 minute time-to-first-token and then 8.30tk/s from there. For context ChatGPT is typically <<1s ttft and ~50tk/s.
I've never understood the obsession with token/s. I'm fine with asking a question and then going on to another task (which might be making coffee).
Even with a cloud-based LLM where the response is pretty snappy, I still find that I wander off and return when I am ready to digest the entire response.
You are fine with it. But may be rest of the world is not. Anyway, to compare performance/benchmark, we need metrics and this is one of the basic metric to measure.
Given that APU only has 4 channels isn't this setup comically starved for bandwidth? By the same token, wouldn't you expect performance to scale approximately linearly as you add additional boxes? And wouldn't you be better off with smaller nodes (ie less RAM and CPU power per box)?
If I'm right about that then if you're willing to go in for somewhere in the vicinity of $30k (24x the Max 385 model) you should be able to achieve ChatGPT performance.
> Isn't that basically the same as me giving you $80? I don't see at all how that's me "basically getting that investment back".
It's a good question, what I think you're missing is that if the market is valuing me (NVIDIA) at 25x revenue then it's more like I traded you (OpenAI) a GPU it cost me $80 to make for $100 worth of OpenAI stock, and I got a bonus $2500 in market cap of my own stock (which existing shareholders like).
IOW for every incremental "$100" in revenue (circular or otherwise), existing shareholders get paid "$2500" in equity (NVIDIA appreciation + OpenAI shares).
This "works" for NVIDIA and its shareholders as long as they/the market keeps thinking $100 of OpenAI stock is a good price for a GPU. If OpenAI tangibly fails to deliver on this valuation then NVIDIA may wind up in the red on these deals.
Caveat: it's a bit more complicated than that as OpenAI doesn't typically buy/operate GPUs directly afaict, rather they team up with the big cloud providers like AMZN (also part of the deal). But it's an useful way to wrap your head around the economics, I think (open to correction, not a domain of professional expertise).
I don't see anything _inherently_ unethical about this as some comments seem to imply. It's definitely riskier than accepting cash, in which case you're free not to play, but it's a calculated risk based on future expectations of growth by OpenAI. Granted there are some sketchy incentives qua existing shareholders that could materialize in pump and dump dynamics.
1. Equality under the law is important in its own right. Even if a law is wrong, it isn’t right to allow particular corporations to flaunt it in a way that individuals would go to prison for.
2. GPL does not allow you to take the code, compress it in your latent space, and then sell that to consumers without open sourcing your code.
Sure, that's what the paper says. Most people don't care what that says until some ramifications actually occur. E.g. a cease and desist letter. Maybe people should care, but companies have been stealing IP from individuals long before GPL, and they still do.
Whether AI training in general is fair use and whether an AI that spits out a verbatim copy of something from the training data has produced an infringing copy are two different questions.
If there is some copyrighted art in the background in a scene from a movie, maybe that's fair use. If you take a high resolution copy of the movie, extract only the art from the background and want to start distributing that on its own, what do you expect then?
Training seems fine. I learn how to write something by looking at example code, then write my own program, that's widely accepted to be a fair use of the code. Same if I learn multiple things from reading encyclopedias, then write an essay, that's good.
However if I memorise that code and write it down that's not fair use. If I copy the encyclopedia that's bad.
The problem then comes into "how trivial can a line be before it's copyrighted"
Fair use is a case by case fact question dependent on many factors. Trial judges often get creative in how they apply these. The courts are not likely to apply a categorical approach to it like that despite what some professors have written.
> 1. Equality under the law is important in its own right. Even if a law is wrong, it isn’t right to allow particular corporations to flaunt it in a way that individuals would go to prison for.
We're talking about the users getting copyright-laundered code here. That's a pretty equal playing field. It's about the output of the AI, not the AI itself, and there are many models to choose from.
What does "usable" mean? Today's best open source or open weight model is how many months behind the curve of closed models? Was every LLM unusable for coding at that point in time?
Chris Masterjohn is a noted quack. He takes bits of actual science and research and weaves them together into narratives that make it sound like he has everything figured out with his unique protocols, but it doesn’t hold up to actual scrutiny. People spend years following his ever changing protocols without getting anywhere (beyond placebo effect and a large bill for supplements)
I know I won’t convince the parent commenter but hopefully I can convince other readers not to go down this road or invest any money in anything related to him.
Pure ad hominem FUD. “This guy sometimes disagrees with scientists employed by the government, don’t listen to him!”.
The technical details are beyond my understanding but I’ve heard from a PhD in the field that Masterjohn’s understanding of metabolism is second to none. Whether his protocols work or not is certainly a case by case matter (like any health protocol), but he always appears to substantiate it with well-cited lines of argument, and is willing to engage with interlocutors.
As for spending years with changing protocols without getting anywhere besides spending lots of money, well that can be said for people with complex issues who go the institutionally approved route as well. It isn’t discrediting in its own right that a protocol didn’t work for some.
I’m not quite understanding: you’re saying you deploy your site one way, then crawl it, then redeploy it via the zipfile you created? And why is SSR relevant to the discussion?
Modern websites execute JavaScript that render DOM nodes that are displayed on the browser.
For example if you look at this site on the browser https://pota.quack.uy/ and do `curl https://pota.quack.uy/` do you see any of the text that is rendered in the browser as output of the curl command?
You don't, because curl doesn't execute JavaScript, and that text comes from JavaScript. One way to fix this problem, is by having a Node.js instance running that does SSR, so when your curl command connects to the server, a node instance executes JavaScript that is streamed/served to curl. (node is running a web server)
Another way, without having to execute JavaScript in the server is to crawl yourself, let's say in localhost, (you do not even need to deploy) then upload the result to a web server that could serve the files.
I'm not sure that logical/abstract argument is the right way to progress on this.. seems like it really isn't abstract and must eventually be something that would need to be measured and compared to prediction in order to decide whether it's a good model/theory or not. (And we're not able to actually measure sensitively-enough to detect this level yet AFAIK) Michelson-Morley interferometer experiment has also been revisited with quite different conclusions that don't rule out substance-of-space too IIUC.
I think that this might be a topic that's worth staying mostly-undecided-about.. despite all the strong opinions around! The fields of QM have to exist within something right?
reply