Unless Nintendo plans on busting down the doors of every person who tries to draw Mario or preventing little Timmy from making a parody of Coca-Cola, making it where AI cannot generated copyrighted works is insane imo.

Those brands should be proud to be such a big part of the cultural fabric that it is difficult to get away from their branding. Plus it's not infringement to my knowledge until you use it for commercial purposes so as long as no one as creating Lario and Muigi to sell or otherwise use in business, it's no different than drawing it yourself.

If the AI is completely unable to generate non-infringing works even if you are _trying_ to get away from it (which the author very much doesn't seem they are, they are purposefully making and show prompts that infringe), that's the problem of the AI creator then.

I also point to a YouTube video by Kirby Ferguson "Everything is a Remix" [1] which talks about how so much of our collective culture stems from copy. It's a great video if you have an hour.

When Little Timmy crayons a copy of Mario, we congratulate him for his creativity. Is it unique, one of a kind art? Well Timmy made it, but he didn't think up the original idea of a video game plumber. I give this view to GenAI right now - it's not capable of achieving that "next step" in "original design", but its performing like a novice artist/musician, it's mimicking what it sees.

[1] https://www.youtube.com/watch?v=X9RYuvPCQUA

If you’re making an ethical argument “it’s okay because it’s already happening to a lesser degree somewhere else” isn’t the flex you think it is.

If you’re talking ethics, talk about impact. Who does it help the most who does it hurt the most? Is your argument favoring equality of access or outcome? Who is the most vulnerable in the situation and how will it impact them?

> I cover generative AI

Wait, are you teaching the class or taking it?

I teach it, my background is located in my profile and my research focuses on CS education.

Scale and impact do matter, I wholeheartedly agree. However, I stand by my point that genAI is mirroring how humans learn - repetition of previously observed actions. As part of my dissertation, I argued that humans operate using 'templates', or previously established frameworks / systems. Even in higher cognitive tasks like problem solving, we rely on workflows that we were trained on previously. Soloway referred to problem solving as a mental set of "basic recurring plans" [1] and if you look at the old 1980s Usborne children's books, they required kids to retype code [2]. For creative tasks, depending on the actor's background, Method and Meisner both tell people to draw from previous experiences and observations to develop a character. This behavior is similar in many areas like music, dance, martial arts, cooking, language acquisition, etc.

I am not making an ethical argument that GenAI violating copyright is okay because that's what humans do. I'm arguing that GenAI mirrors how humans learn. We observe a behavior and attempt to recreate that behavior. The difference is that humans can extract a fraction of the behavior and utilize it as part of something larger while GenAI cannot to the degree humans do. I'm sure GenAI would struggle to recreate "Who Framed Roger Rabbit?" because of the two polar different visual elements of the film (cartoon and real life).

In regards to your "If you’re talking ethics, talk about impact" section, its a bit of a loaded question. One side of the conversation could state that GenAI is helping many people that do not have confidence in their creative ability to produce their ideas, while the other could state its making it harder for artists.

Yes, it absolutely is hurting artists and I fully support the recent writer's strike over AI concerns. But I do not believe that diminishes how the mathematical models used in GenAI mirror our own skill acquistion.

[1] https://ieeexplore.ieee.org/document/5010283

[2] https://usborne.com/us/books/computer-and-coding-books

In my view it encouraged nihilism and apathy instead of developing ethical frameworks. From that lens, I feel teaching a course might be more limiting in the range of heuristics you’re willing to accept or endorse. Though happy to accept your personal experience.

A paper that comes to mind often from HCI is “do artifacts have politics” which looks at the impacts of technologies divorced from creator intent. I feel that’s similar here.

You’re not wrong that about the mechanism that it’s created. But I would argue that’s the least important part, ethically anyway.

Saying “strip mining with heavy industrial machines mimics laborers using shovels” is true to a degree, but but perhaps not that important piece of information.

I’m not saying you’re making that argument. I guess im just not totally sure the outcome you were looking for in sharing your original comment. I hear your comparison and agree with it and that it is interesting to view in that lense. I wasn’t sure if there was a deeper intent in sharing it.

I have a pretty neutral stance to GenAI, mostly due to personality however it also stems from my background as well as recognizing students' interests. Prior to CS Education, my master thesis involved computer vision for catching "high valued targets", but was also funded to help minimize human trafficking. I have students in my classes that are very interested in going to work for defense companies like Lockheed and Raytheon, and I have others that are really interested in using AI for "social good" areas like healthcare and education. I try to have a neutral stance because: A) I hated the professors that I took that would use their lecture time to express their political opinions, B) opinions that are opposite to a student may otherwise discourage them from learning the material, and C) my primary focus is to make sure they learn the material and do it "right".

When I started teaching, I used the analogy that if they go on to write the software for the life support machine I'm hooked up it, it WORKS. If someone wants to go on to use AI to create weapons, I can't stop them anymore than I can force them to read a chapter or convincing the person beside on the highway to slow down. I just work to ensure they do it correctly (which includes being mindful of the ethical ramifications of using algorithm X for task Y).

What would an ethical framework for designing AI for a drone even look like? I have no idea, nor is it something I'm interested in delving into. I got out of face recognition for those reasons. Does an ethical framework for GenAI require the same elements, a fraction of them, or a completely different set of guidelines? Who gets to decide them - the 'experts' in AI, the government, society as a whole?

Personally, I've made the comment that the current opinions on regulating AI are like "everyone trying to be AI's parent". We're never going to agree because everyone has a different opinion on the "right" way to handle AI. Plus, human cognition is so unknown and illogical that we may never figure out a way to perfectly replicate human intelligence. I instead try to stay somewhat optimistic and marvel at the math we've used to create "AI".

Society is effectively ok with you ripping me off for $1. They are not when it's $100k.

And yes, obviously society cares about many things depending on the scales in question. It's okay if a dude goes onto a lake on his small rowboat and catches a few fish for dinner, it's a completely different story if you're talking about a massive barge indiscriminately catching literally thousands of fish with huge nets. The latter has to adhere to much stricter rules than the prior, and I think you'd be hard pressed to find anyone who thinks these 2 situations should be treated equally (unless you're a commercial fisherman with a barge, I suppose, the quote "It is difficult to get a man to understand something when his salary depends on his not understanding it." comes to mind here)

In the history of the world only a single person has ever drawn fan art?

No, I don't think that's the case.

Instead it is widespread. It is everywhere.

> depending on the scales in question

The scale argument supports me, not you.

This type of "infringement" is everywhere.

> reselling it for their own profit with no regards to anyone or anything else?

Even this is common. The online independent artist commissions market is full of people doing commercial fan art commissions.

Thinking about this even more, I am now wondering if "infringing" works might actually be a majority of the online/independent commissions market. Maybe.

And yet, nobody cares.

That's a disingenuous take of my comment at best, the equivalent to my scenario is a bunch of unrelated individuals with small boats going out into whatever lake is nearest to them and fishing. Even if you put all of them together and counted how many fish the hobby fishermen catch, it's still nowhere near the scale of the commercial fisheries, which is why they're treated differently both by society at larger but also legally.

Same thing with these AI models, Dall-E and all the other ones have probably generated more images than all of humanity has in its entire history so far, and if not quite yet they're definitely gonna get there sooner rather than later. They can generate dozens if not hundreds of images in a split second, whereas a single artist (or even many artists collectively) can't.

> And yet, nobody cares.

I think we've already established that, because scales absolutely matter for most things. If you want to be an absolutist about it, sure be my guest, but I think in reality the large majority of people are fine when your average Joe Schmoe the artist makes a commission on a random Disney character, whereas they definitely would NOT be okay with a massive conglomerate like Disney stealing Joe Schmoe's original art and repurposing it without compensating Joe, because there's an inherent power disbalance between the two and the consequences of that power disparity matters.

I mean, Disney does have every right to go after Joe for his commissions if they really wanted to, similarly to how Nintendo is hyper aggressive with taking down anything relating to their IPs. It's just not really worth it for most companies, they will absolutely go for another company trying to pull the same shit though, as can be seen with the NYT case.

And OpenAI quite literally sells access to their models, and if those models are pushing out verbatim copyrighted works as has been alleged by the NYT, then they are by definition reselling copyrighted works without permission.

This style of argument has been previously made regarding things like torrenting during the heyday of piracy ("why would you need <x> except for illegal purposes!")

In my opinion, it's the exact same argument saying that selling a tool means taking responsibility for how that tool is used by its new owner. You can use a shovel to both create something new (plant a tree) or destroy something (rip up your neighbor's garden).

The problem isn't the tool, the problem is how the end user uses it. These models aren't living thinking entities that enduce or on their own infringe copyright / do other illegal activities.

They aren't encouraging people to misuse them and it is solely on the user's shoulders for their choice to use them in a way that would cause infringement if the result is used commercially.

I agree in principle, but that they can in the first place, especially when it accidentally happens, and at such massive scales more importantly, is the issue methinks.

And no one's talking about abolishing the AIs here, we're just talking about wanting M$/OAI to do their due diligence and get access to their training materials fairly. NYT wouldn't have sued if M$/OAI had approached them and struck a deal of some sort with them, but that's not what they did. They took in whatever data they could, from wherever they could baring no mind at all to where the data came from and what was being done with it.

There's a reason Getty images managed to strike a deal with Dall-E and why many of the image generation models now solely rely on data that is verifiably free of copyright (or where deals have been made in the case of Getty images). It's easier to see in pictures when a blatant copy is made (like watermarks) so it's obvious why Dall-E was the first to encounter this hurdle, but this was inevitable even for plain text that ChatGPT returns.

OK, say every artist gets $100, one time (exact amount varies but would not be much). Everything's properly licensed according to you and the artists are essentially no better off, and the models are now good enough to create new training data for the future and artists never see any more money.

You've won, I guess?

The closest thing you could do is e.g. have a second model that does something novel like create a 3D model from a 2D image and then you try to animate the model and a third model verifies the quality of the output. This then allows you to selectively reinforce the 2D model using information from the 3D model but this isn't simply generating more training data.

I honestly can't follow your argument. Doing something silly doesn't make you the underdog.

Incidentally yes, training AI on AI output will work fine, as long as you have a signal of quality. For example, upvotes in a subreddit would work fine. But that's not crucial to my point, which is that what OP is asking for will accomplish exactly nothing.

People are already working on completely copyright safe models and those models can still destroy the entire art market.

Ex: adobe has a gen AI model, trained on content that they own.

What now artists? Can't hide behind fake outrage over infringement for that model. But that model can still end the art industry.

I wonder what the new argument will be then when fully non infringing models destroy the market regardless.

Regardless, I'm not saying it's a perfect idea but it's definitely a start, especially when the current reality is that they're just stealing all the artist's shit and everyone gets $0 instead of $100. As you said, artists are no better off in that universe, but the worst case possible for them is what's happening right this very moment, where they just get fucked over with 0 compensation.

If you sold the output of a true random number generator, eventually you'd also by definition be reselling copyrighted works without permission. The courts wouldn't mindlessly say "no more random numbers", and I doubt that they'll do the same for GenAI, especially given the recent decisions that are headed that way.

Instead, it is a person who uses the machine, just like fan artists can use a computer to make fan art.

A machine being involved in the process doesn't change any of the copyright implications.

Either it's infringement or it isn't, regardless if the human did it on their own, or if the human did it with a computer.

The interesting question is whether the models themselves are copyright violations not the output.

If this tool “runs on” copyrighted creative works, and $CORP operates this tool for profit, then $CORP is the one to answer to the law, not the tool. (And if $CORP wants to claim that the tool is a sentient being, then presumably it would have to cease the abuse of said being and set it free.)

Yeah, but we don't typically congratulate users of GenAI for their creativity, and neither do we congratulate the code, nor do we think of the coders of GenAI as great artists.

In other words, your analogy is broken.

That hypocritical, self-contradictory take is transparently geared to benefit commercial LLM operators (at the expense of individuals who stand to suffer material harm and/or authored the very creative works thanks to which the tool even exists).

Perhaps the hyperbole of the entire corpus of human knowledge isn't quite technically right, but it's close enough.

Tho tbh I’m not really sure what OPs point was

i don’t think the amount of training data is relevant here.

So using a model running on my laptop to generate a "Mario like" image would be fine, but it would make monetizing this difficult?

It's not a problem for me to draw Micky Mouse. It _is_ a problem when someone pays me to draw an animated mouse and I sell them a picture of Micky Mouse.

For me, its not really about the AI at all, it's a problem of undervaluing Artists contribution to these tools. And it's not even fully about copyright it's about not asking for permission to use their content and then creating an entire business on top of that stolen content.

When the law doesn't respect the people, the people will not respect the law. Fix the existing copyright system, then we can talk about AI.

All parties are responsible on some level. This just reads like passing the buck/burying ones head in the sand to me.

I'm perfectly free to ask people on the street for t-shirt with Mario on it, but as soon as someone who isn't Nintendo or licensed by Nintendo sells me that t-shirt they're the ones infringing on the copyright and trademark. As the consumer I did nothing illegal, and a court would say that I was deceived by the infringing party.

Distribution (seeding, uploading) and facilitating copyright infringement is what gets you in trouble. When you ask DALL-E (a paid, commercial product) for a picture of Italian plumbers and it gives you an obvious picture of Mario 100% recognizable to the layperson as Mario and not a distinctly different image of a similar character, that's blatant trademark and/or copyright infringement on the part of OpenAI.

> If the AI is completely unable to generate non-infringing works even if you are _trying_ to get away from it (which the author very much doesn't seem they are, they are purposefully making and show prompts that infringe), that's the problem of the AI creator then.

I see some parallels to the Napster lawsuit. The fact that the users were the bad people asking for infringing content didn't give Napster the right to facilitate infringement. Napster was ordered to monitor its network and make sure that they were blocking non-legitimate uses. They couldn't logistically comply and went bankrupt.

https://en.wikipedia.org/wiki/Napster

Which begs the question: Does OpenAI even have the technological ability to block trademark and copyright infringing content generation? Even if they do, how useful will ChatGPT be if all phrases and imagery that closely resemble copyrighted works are blocked from output?

Whats even worse for OpenAI compared to Napster is that it wasn’t individual users uploading copyrighted content, it was OpenAI’s ingesting the data. Nobody twisted their arm to include copyrighted works in their models.

It's difficult to say really.

If I essentially encode knowledge of something then can recall and remix at will, am I redistributing the exact work or the knowledge of it?

Yes, it is capable of producing a close to exact replica, if not the exact same input image byte-for-byte, but I find it difficult to say OpenAI is willfully redistributing copyrighted work in a whole like you would with torrenting a movie or right-click saving an image from Google where you are copying the intellectual property 1:1.

Opening this Pandora's box could have large implications on a lot of creative work that could cause artists to be unable to work if taken to the end conclusion: you cannot create any creative work that has a talking mouse if you have knowledge of Mickey Mouse existing because you have been tainted (similar to whiteroom re-creations but now any sufficiently large copyrighted figure causes a deadlock condition for all derivative ore even similar topics).

Is Ratatouille derative of Mickey Mouse? Ehhh, well they are both talking rodents. They both have cartoon faces. You can certainly draw parallels between them but they aren't the same character. Is Mickey with a chef hat infringing on Ratatouille?

The trademark law, to my knowledge, is asking would someone be tricked or misled into believing you are the other guys. I think that is applicable here where someone drawing a talking mouse isn't infringing as long as it cannot be mistaken for Mickey Mouse, which again would be the fault of the person inducing the creation and not the tool that allowed it to happen.

Where does "inspired by" / derived from the encoded knowledge turn into outright exploitation of copyrighted work? There's certainly _a_ line but I find it difficult to define it at it being encoded into knowledge of it existing.

Courts have already defined this line over decades of copyright and trademark cases, and the examples in this article definitely cross that line.

> which again would be the fault of the person inducing the creation and not the tool that allowed it to happen.

This is not really true in practice, we can see that in various legal cases against Napster or The Pirate Bay.

Are you trying to say that no one is entitled to their own inventions? Cause that is a rapid descent into a capitalist hellhole where only those who can steal ideas the most effectively are able to profit.

The subject of this thread is copyright, not patents. Though I do believe all intellectual property is bogus (including trademark, which repealing would limit the influence that would be required for the capitalist hellhole you mention), I feel the most strongly so about copyright, which has nothing to do with inventions.

Further extending the argument - I can potentially ask GenAI "Can you show me what does Mario looks like?" since I have never seen one and GenAI is my go to tool.

By which you mean every copyright holder.

> AGI in the near future

Something that is purely speculative, undefined, and has been promised in the near future for 50+ years.

I don't see copyright holders lying down for someone else's benefit and I don't see governments gutting copyright, contract law, and several other avenues of protection that copyright holders can deploy in the name of something that doesn't exist and may not ever exist.

But every single copyright holder with their works online (which includes you and me) has the same legal rights as the NYT or Disney. Naturally some copyright holders have more real-world capability to go legal than others, but that does not reduce the legal risk.

> If anything it means they only infringe on archetypal works and not the other 99.9%

How on earth do you get to that conclusion? There's no "popularity" floor to copyright protection. Either a work has been infringed or it hasn't.

https://www.natlawreview.com/article/japanese-government-ide...

https://www.cliffordchance.com/insights/resources/blogs/talk...

“The use of copyrighted products or materials to train generative AI models would be prima facie copyright infringement under the Copyright Act, as it is a reproduction (fukusei) or other form of use of the copyrighted work. However, Article 30-4 of the Copyright Act stipulates that the use of copyrighted works by generative AI for learning purposes is allowed in principle.”

Which suggests that when AI art threatens commercial interests, the protection offered by 30-4 can disappear.

To me it sounds like they tried to please everyone and left the hard decisions about conflicting interests to the courts (in particular the courts will have to decide what "unreasonably" means).

Why should other intelligent entities be prevented from reading copyrighted works and gaining whatever there is to gain from those works the way any human might?

You can also get into the weeds of what's copyright-able (ask Donald Faison about his Poison dance). If you ask for C-3PO and you get C-3PO as he appears in Star Wars promotional material, that seems cut and dry. What if you ask for a "golden robot"? What if you get a robot that looks like C-3PO but with a triangular torso symbol instead of his circular one? What's parody, what's fair use?

If the public starts to see LLMs as highly sophisticated copyright laundromats it would most likely hamper further investment & development in that field.

This is the bit I don’t get from the “feed everything to machine” LLM-maximalists. Do they think courts don’t take context into account, do they think all actions happen in a vacuum and that they can just skip along and ignore laws at their pleasure because “tee hee it’s totally definitely fair use bro, I’m totally an academic researcher-pinky promise”.

LLM bros ought to stop and have a think before they poison their own well, assuming they haven’t already done so.

An entire generation of unicorn startups believed that (Uber, AirBnB, etc.). We see in the news every day that once you have enough money laws don't apply to you (most things Elon Musk does, the fact that Trump can defy court orders repeatedly and not go to jail, etc.) so yes, this seems entirely plausible.

The 2 darling startups that are now facing increasingly less rosy futures?

Airbnb in particular is facing enough backlash that I’d be surprised if it lasts terribly much longer.

Sure, they get away with it for a while, but not forever.

> We see in the news every day that once you have enough money laws don't apply to you

I agree with you here, but I think this is a much broader conversation about capitalism in general which would be getting a bit off-topic for this particular thread, except to say, capitalist forces aren’t above cauterising a limb if it becomes too annoying or intrudes on the other limbs too much. I think the “AI” limb might be overstating its own importance, and I suspect that if it got too up in everyone’s interests re-profit, it would, as an industry, very quickly find itself being neutered. Capital interests would love to get rid of pesky human labour, but if the alternative is too annoying, they’ll have no objections to going back to grinding people through the system again.

As of this moment uber is worth 120 billion and AirBnB is worth 80 billion.

Yes, they got away with it.

Are the OpenAIs of the world ready to shield their customers from that liability?

If it turns out that using ChatGPT to help you write your resumé opens you up to accusations of plagiarism, or DALL·E to create an image for your website opens you to copyright violation, will you use them?

Yes. Just like reading anything else on the internet. An LLM is no different from typing "popular cola logo" into Google search and claiming you invented it. If I type "cola logo" into DALL-E and get a replica of Coca-Cola... that doesn't mean I created that logo and can exploit it for commercial purposes.

> Are the OpenAIs of the world ready to shield their customers from that liability?

Why would they? We aren't suing pen manufacturers because someone wrote something libelous using their pen. We aren't busting down the doors of Crayola because little Johnny used the crayons to draw Mario.

I mean get this great auto complete; if you use it, your code might be AGPLed for all you know, and you're in violation, because you didn't even add a notice.

Would you pay for that?

If ASI can exist I don't believe our the old methods of intellectual fortifications will continue to work in the future. Much like castle walls aren't used to protect against guided missiles.

If not, how does that differ from me making an unauthorized pencil drawing of Mario?

A more practical way of looking at this is: who is making money off of these models? How did they get their training data?

I’m not a fan of copyright in general, but we have serious outstanding issues with companies and organizations stealing or plastering work without compensating the original creators of said works. Thusfar, LLMs are becoming another method to concentrate wealth to whoever has the resources to train and sell these models at scale.

Would you mind unpacking this one a bit? It sounds like you denigrate copyright (some "general" grievance) but then immediately execute an about-face and begin to extoll its virtues. Is copyright not the thing that allows us to share works without fear they'll be stolen?

As a society we want to incentivize innovation and reward things that advance society. One of the ways we do that today is copyright. It doesn't need to be the only way, or be done in the ways we do it now.

I doubt that part of the argument would change even if we perfected brain uploads.

Now, if you gave the current LLMs a robot body with a cute face, that'll probably change minds faster, regardless of the underlying architecture.

> who is making money off of these models?

When the models are open source, or at least may be downloaded and used locally for no cost, that would be the users of the models.

And back to the biological comparison: I learned to read (and also to code) in part from the Commodore 64 user manual, should I owe the shareholders anything for my lifetime earnings? As I got to the end of that sentence, a thought struck me: taxes do that. And in the UK the question of if university should be funded by taxes or by the students themselves followed the same lines.

I think there's a bit more nuance to this. The profits go to those with the ability to run these models and to those with the infrastructure (or capitol) to run said models. I'm hoping this will change and we'll see lower barriers to entry as LLMs are made more accessible over time.

> And back to the biological comparison: I learned to read (and also to code) in part from the Commodore 64 user manual, should I owe the shareholders anything for my lifetime earnings?

This is more a philosophical question than anything else. I don't think there's right or wrong answer, but in my opinion the answers we arrive at should provide as much benefit to as many people as possible.

> As I got to the end of that sentence, a thought struck me: taxes do that. And in the UK the question of if university should be funded by taxes or by the students themselves followed the same lines.

I agree with your assessment and this model lines up well with my own opinions on reasonable ways to ensure equitable benefit from AI (be it ML, LLMs, or some theoretical general AI in the future).

Copyright is meant to give the original creator a monopoly over their creation (so that others don't profit off of their work). Are you not a fan of copyright in its current scope / implementation? Because it sounds like you do agree with its goal.

Correct me if I'm wrong, but my understanding is that the goal of copyright is to incentivize innovation (specifically of art and culture) and to provide innovators a way recoup (and profit) off of innovation they've made public. I view it as similar to how patents work in that it's an incentive for people to publicize and share their works more broadly.

> Are you not a fan of copyright in its current scope / implementation? Because it sounds like you do agree with its goal.

I have a differing understanding of the goal of copyright based off of what you've said, but I think our understandings are similar in that the copyright holder benefits from copyright/patents of their works.

I dislike the ways our current implementations of copyright are abused. I think the concept of fair use makes copyright as it is today workable. I also think our current copyright laws (at least in the US) have a lot of failure modes that subvert what I believe the purpose of copyright should be: to advance art and culture with legal and economic incentive.

150 years ago society exists by and for men specifically (as in: not women) in most nations; 220 years ago, US society was by and for rich white (specifically white) land owners.

I don't know when AI will count as people in law, or even if they ever will; we may well pass laws prohibiting the creation of any mind in danger of coming close to this threshold.

But be wary, for AI acting enough like people is different to being anything like a person on the inside, and that means being wrong in either direction can have horrifying consequences. To appear but not to be conscious, leads to a worthless future. To be but not to appear conscious, leads to a fate worse than the history of slavery, for the slaves were eventually freed.

Especially ChatGPT and other LLMs, they're not even close to being AGI or an "intelligent entity" as you put it, despite what all the AI-bro hype and marketing would like everyone else to believe.

Whenever people say stuff like this I can't help but wonder what on earth kind of projects they work on. Even GPT4, while useful for things like reformatting or generating boilerplate code and stuff like that, it's still a far cry from any decent dev I've ever worked with, especially if you're not using a popular language like JS or Python.

My usual PRs at work are pretty big, complex pieces of code that all have to actually work when integrated with the larger system around it, no AI tool I've tried so far has come even close to acceptable here, other than for generating some boilerplate code that I would've written myself anyway. But even with the innocent-looking boilerplate there's always a weird gotcha that isn't obvious until you really analyze the code closely. It ends up saving nothing more than a few keystrokes, if that, yet people say all the time that they're generating entire pieces of software by gluing together code it spits out, which I find absolutely insane given my anecdotal attempts at it.

This can circumvented by going with more elaborate in-depth prompts, but at that point are you really saving on effort compared to the alternative? Is it really more efficient? By the time I have a prompt complex enough for it to spit out something good at me, I could've already bashed out the code myself anyways.

That's not even mentioning all the legacy shit you have to keep in mind for any one line of code, plus whatever conventions and standards your team uses and has etc.

I mean it works great for a function or whatever, but is that seriously what most people are working on? Simple, one-off independent function calls that don't interact in any way with anything within a larger system? Even simple CRUD apps aren't so well isolated.

Don't even get me started on the actual difficult part which is the whole preamble to creating the ticket in JIRA or whatever task management software you use where you're talking with stakeholders and planning out the work ahead, you're telling me you're paying 'Open'AI to do that whole rigamarole for you, and you're doing it successfully?

Terrifyingly, one of the bad human examples was doing C++. That person didn't know, or care to learn about, the standard template library; and they also duplicated entire files rather than changing access specifiers from private to public so they could subclass; and one feature they worked on was to support a change from storing data as a custom file format to a database, and the transition could take 20 minutes on some inputs even though neither loading before nor after this transition took more than milliseconds, and they insisted during one of the standups the code couldn't possibly be improved… the next day I looked at it for a bit, removed an unnecessary O(n^2) operation, and the transition code went back down to milliseconds. Oh, and a thousand(!) line long block for an if statement that always evaluated true.

The whole codebase was several times too big to fit into the context window for any version of any GPT model thanks to both this duplication and to keeping old versions of functions around "for reference" (their words), but if it had been rewritten to be more sensible it might just about fit into the biggest.

(My other examples were either still at, or fresh out of, university; but this person should have known better).

> Don't even get me started on the actual difficult part which is the whole preamble to creating the ticket in JIRA or whatever task management software you use where you're talking with stakeholders and planning out the work ahead, you're telling me you're paying 'Open'AI to do that whole rigamarole for you, and you're doing it successfully?

If it was all-round good, none of us would have jobs any more.

I mean this not overly sarcastically, but ... have you seen https://thedailywtf.com ? Between my own experiences, and that of some colleagues, I could probably put together at least a half-a-dozen WTF stories that would rival some of the best that site has to offer. There's enough really incompetent people in positions they shouldn't be in to the point that chatgpt - at this point - could realistically provide better output than more than a few of them.

Edit: typo

Just because a computer program's output is remarkably good does not mean there is any emergent intelligence, any more than a technology we don't understand means there is magic.

If we should ever fully understand how our own minds work, will we hold machines in higher esteem, or ourselves in lower?

Just because consciousness is a mystery today, doesn't mean we get to stop and say it will be so forever more.

Heck, the problem still fundamentally exists regardless of if you're atheist, monotheist, polytheist, or pantheist.

--

“We’re not listening to you! You’re not even really alive!” said a priest.

Dorfl nodded. “This Is Fundamentally True,” he said.

“See? He admits it!”

“I Suggest You Take Me And Smash Me And Grind The Bits Into Fragments And Pound The Fragments Into Powder And Mill Them Again To The Finest Dust There Can Be, And I Believe You Will Not Find A Single Atom Of Life–”

“True! Let’s do it!”

“However, In Order To Test This Fully, One Of You Must Volunteer To Undergo The Same Process.”

There was silence.

“That’s not fair,” said a priest, after a while. “All anyone has to do is bake up your dust again and you’ll be alive…”

- Feet of Clay, Terry Pratchett

Also:

> Anything supposing that humans aren’t actually intelligent or conscious or whatever

Doesn't really match what I was writing about: if it turns out that a thing which is "just a pattern recognizer" can in fact be "intelligent or conscious or whatever", it's up to us if we see intelligence or consciousness or whatever in the pattern recognisers that we build, or if we ourselves descend into solipsism and/or nihilism.

Or if we take the traditional path of sticking our fingers in our ears and go "la la la I'm not listening" by way of managing cognitive dissonance. This is a very popular response which should not be underestimated.

But the laws of physics are quite clear, that a whole bunch of linear equations (quantum field theory) gets us chemistry, which gets us biology, etc., and the only place in all this for the feeling of existence that we have is emergent properties. Those emergent properties may, or may not, be present in other systems, but we don't know because we're really bad at characterising how emergent properties… emerge.

These things are indeed "a program on a machine and doesn’t have rights", but what I find scary is that rights aren't part of the rules of the universe, they're merely laws, created and enforced (to the extent that they are at all) by humans.

But since you are the type of person who is seemingly using LLM "written" code in production, your ability to accurate assess anything is suspect at best.

"Any technology, sufficiently advanced, is indistinguishable from magic".

No, an LLM is not intelligent. I do not understand why people will go through mental gymnastics to conclude they are.

queue all the typical arguments supporting them being intelligent and demanding I give reasons for them not being

You have your own individual threshold for what "is" intelligence? Holy cow, imagine if each other agent had their own also, but spoke as if they had a common one...that sure wouldn't be a very intelligent way to run a simulation, imagine the unrealized confusion and delusion that could result if that became a cultural convention!

"Undefined", although not literally, in practice definitely: each letter of that initialism means a different thing to different people. To that extent, I'll even grant "speculative" despite many of those meanings being demonstrably met by us humans.

But as someone who (unfortunately) has just turned 40: who was it that was promising AGI "in the near future" for more than my entire lifetime? Including the second AI winter? Because even the biggest timeline-optimists I can remember (Kurzweil and Yudkowsky), who very few cared to listen to, put things more than 20 years ahead of when they were writing. (And yes, Yudkowsky was definitely wrong about a singularity in 2021, though as you say AGI is undefined I think if someone in 1996 had seen ChatGPT they'd have said "yes, this is AGI" despite its flaws).

Now the crowdsourced guess for AGI is 7 about years: https://www.metaculus.com/questions/5121/date-of-artificial-...

> I don't see copyright holders lying down for one else's benefit and I don't see governments gutting copyright, contract law, and several other avenues of protection that copyright holders can deploy in the name of something that doesn't exist and may not exist.

I tend to agree. Although I don't accept that contract law has much of anything to do with this discussion, to the extent that it does have implications, it isn't going anywhere.

But at the same time, Google exists by reading the entire public internet, indexing it, and presenting clips of it to its users. This has in fact resulted in copyright disputes, and I was surprised how long it took for that to happen. Likewise, while copyright holders must fight for their survival, mere LLMs even as they exist right now are economically relevant, so this isn't going to be a one-sided fight by just copyright holders.

You won't be able to read this without a subscription and I can't figure out how to find an archive link to something published in 1958: https://www.nytimes.com/1958/07/08/archives/new-navy-device-.... The important quote, however, is:

> The Navy revealed the embryo of an electronic computer today that it expects will be able to walk, talk, see, write, reproduce itself and be conscious of its existence. Later perceptrons will be able to recognize people and call out their names and instantly translate speech in one language to speech and writing in another language, it was predicted.

They were talking about the very first perceptron, a hardware implementation funded by the Navy and built by a team led by Frank Rosenblatt, one of the earlier evangelists of neural nets, in 1957. The terminology "AGI" hadn't come into use yet that I'm aware of, as "AI" in itself meant the same thing back then, but in order to be able to call inferior, more limited software capabilities "AI" for marketing purposes, we had to invent "AGI" as the stronger concept. I'm guessing they expected it to happen sooner than 75 years later, though.

It’s already happening with EU AI Act https://www.europarl.europa.eu/news/en/headlines/society/202...

It generally didn’t care about generative AI.

The matter of the leaks were very “Snowdeny” in that it’s possibly that parts of our own government and our secret police share all Danish internet traffic with the NSA, who then in tern share information with our secret police. Which meant that our secret police could do surveillance on us as citizens through a legal loophole, as they aren’t allowed to do they directly, but are allowed to share surveillance information with the NSA. Part of this information comes from the giant American tech companies as well. Despite their promises to not share the data they keep for you. I know it’s sort of crackpot sounding, but between echelon, Snowden and the ridiculous amounts of scandals, I think it’s safe to assume that the American military wants in on LLMs and monitor all the inputs people put into ChatGPT and similar. So for that reason alone they’d want in on things.

Then there is how the war in Ukraine has shown how cheap drones are vital in modern warfare, and right now, they need to be manually controlled. But what if they didn’t? Maybe that’s not obtainable, but maybe it is.

Then there is all the other reasons you and I can’t think of. So even if they don’t believe it’s eventually going to lead to an AGI, or whatever else the hype wants, they’re still going to be interested in technology that’s already used by so many people and organisations around the globe.

For instance, neither of your examples - surveillance or automated drones - has anything to do with AGI. They don't need LLMs to do mass digital surveillance; they already do that and were doing it for decades before LLMs were a twinkle in anyone's eye. Sure, they'll try to tap into the user data generated by chatgpt etc. (and likely succeed), but that's not a different capability than what they're already doing. And automating drones - which, by the way, this is not future technology as you seem to imply, it's here today - is a special purpose ML system, that maybe benefits from incorporating an LLM somewhere, but certainly isn't pinging the chatgpt api!

But sure, you're exactly right at the end, I have no idea whether they see other angles on this that are new and promising. That's why I asked the question, I'm very curious whether there are any real indications thus far that militaries think the big public LLM models will be useful enough to them that they'll want to put a thumb on the scale to favor the companies running them over the companies that make their bucks on copyrighted content.

But for all we know intelligence services could be using LLMs for years now, since they are usually a few years ahead of everybody else in many regards :-)

I'm sure language models and transformer techniques will be (or more likely: already are) an important part of the contemporary systems that do this stuff. But I'm skeptical that they care much about GPT-4 itself (or other general models).

I'm not skeptical about whether they think it is useful and an important capability to incorporate ML techniques into their systems, I'm unsure how much utility they see in general (the "G" in AGI) models.

How? I'm not trying to be combative, I genuinely am curious if you have an idea how these things could be usefully applied to that problem. In my experience working in the information security space, approximate techniques (neural nets, etc.) haven't gotten much traction. Deterministic detection rules are how we approach the problem of finding the needle in the hay pile. So if you have a concrete idea here that could represent an advancement in this field.

In a world where false negatives--i.e. failing to detect a sharp needle--are the worst possible failure mode, approximations need to be handled with exceeding care.

I think there's a strong argument that they should be thinking in those terms, but I'm a lot less convinced that they do usually think in that way.

Or more charitably, they have the responsibility to balance current interests against future interests. And this isn't just a tricky thing for democracies, dictators also have to strike this same balance, just with different trade offs.

But in this case, for the US, it honestly isn't clear to me that policy makers should favor the AI side of this tussle. I think culture has been among the, if not the very, most important export of the US for nearly a century, and I think favorable copyright treatment has been at least part of the story with that.

Maybe that whole landscape is different now in a way that makes that whole model obsolete, but I think it's an open question at least.

What it seems to me from the milieu of everything I've read and heard (that is: I can't cite examples, this is an aggregate developed from hundreds of articles and podcasts etc.) is that there is already an "AI" arms race underway, but that it has more to do with specialized ML systems than with consumer LLMs.

But I'm not really in the loop, and maybe OpenAI really is more important to the US DoD than Disney (as a stand-in for big copyright-based businesses generally) is to the politicians they donate to. But I dunno! That's why I asked the question :)

I would be more intrigued by the national security angle of this if copyright holders were going after, say, Palantir. But I just don't know how important they see these language models as being, or how interested they are in OpenAI's mission to discover AGI.

Some of this may be a misunderstanding of what modern militaries do, if they are shooting guns there's already been some level of failure. Massive amounts of war gaming, sentiment analysis, and propagandizing occur, see the RAND Corporation for more details on the military development of algorithms and artificial intelligence.

But I also buy that there is a lot of overlap between military work and any other kind of white collar work, which LLMs are definitely useful (but not revolutionary) for.

(1) Do you think "developing AGI" a realistic, achievable goal? If so, what evidence do you see that we're making progress on the problem of "general" intelligence? Specifically, what does any of that have to do with Large Language Models?

(2) Are there any "national security" applications of Large Language Models that you're aware of?

It seems to me that it would be a very difficult case to make that the national security impact from allowing the rule of law to erode would be somehow outmatched by the (speculative) wager that somehow LLMs have some relevance to the national security. It would be an even harder case to make that any of this has something to do with "general" intelligence.

If you manage to put a bunch of listening devices at a place you're moderately interested in, a cafeteria at an enemy base for example, you might end up with literally hundreds of hours of conversations, most of them completely uninteresting, but a few that might possibly contain nuggets of information of the utmost importance. Listening to all these conversations requires resources. This is even more difficult if the people there speak in jargon, in their own language, and nobody but an expert in the subject can determine which conversation snippets are significant.

If you have good LLMs, you can run all your recordings through extremely high-quality speech recognition and then use something like Chat GPT for summarization, classification, finding all mentions of the nuclear reactor in <place> etc. Same goes for satellite image analysis.

So should we put copyright through the shredder on the wager that somehow generative techniques will find applications for mass surveillance?

Let's say I want to replace the forklift operator at my local lumberyard with a robot forklift that can ostensibly outperform a human employee. Even if there is some magical AI program which could theoretically drive the forklift around, identify boards by their dimensions, species, dryness, location, etc., there's a whole bunch of sensory problems that a human body solves easily that are super hard to solve in the environment of a lumber yard. There's dust, rain, snow, mud--so if you're relying on cameras how will you keep them clean? You can't visually determine how dry a board is, you have to put a moisture meter on it and read the result. My point is, even if you have a "brain" capable of driving the forklift you still have a massively complex robotics problem to solve in order to automate just the forklift. And we haven't even begun to replace the other things the operator does in addition to driving the forklift. He can climb out of the forklift and adjust the forks, move boards by hand, affect repairs on equipment, communicate with other equipment operators, customers, etc.

Good luck replacing him in a cost-effective manner.

So what am I supposed to use it for?

This is an issue of 'mechanical intelligence' being hundreds of millions of years old and 'higher intelligence' being pretty new on the evolutionary spectrum.

And the AGI will keep you around as a dexterous 'robot' while supervising your thoughts to make sure you're keeping in line I guess, while day after day cranking out more capable robots in which to replace you with eventually.

Additionally, I'm not even sure the US is capable of having national priorities at the moment. The Congress has become incapable of making decisions. While the executive and the judiciary branches have stepped up to compensate, they tend to handle each issue separately without any general direction.

In the US, this isn't possible. There is no legal mechanism for putting things into the public domain outside of the expiration of the term of copyright. The best you can do is to promise not to enforce your copyright.

Tech companies have more money to throw at politicians.

I have nothing against creators, they deserve to get paid.

For what its worth, LLMs are facing the coke vs Pepsi challenge, and sadly they are most definitely Pepsi.

https://aws.amazon.com/service-terms/

See items as of 50.10 and 50.10.1 that I reproduce here:

"50.10. Defense of Claims and Indemnity for Indemnified Generative AI Services. AWS Services may incorporate generative AI features and provide Generative AI Output to you. “Generative AI Output” means output generated by a generative artificial intelligence model in response to inputs or other data provided by you. “Indemnified Generative AI Services” means, collectively, generally available features of Amazon CodeWhisperer Professional, Amazon Titan Text Express, Amazon Titan Text Lite, Amazon Titan Text Embeddings, Amazon Titan Multimodal Embeddings, AWS HealthScribe, Amazon Personalize, Amazon Connect Contact Lens, and Amazon Lex. The following terms apply to the Indemnified Generative AI Services:

50.10.1. Subject to the limitations in this Section 50.10, AWS will defend you and your employees, officers, and directors against any third-party claim alleging that the Generative AI Output generated by an Indemnified Generative AI Service infringes or misappropriates that third party’s intellectual property rights, and will pay the amount of any adverse final judgment or settlement."

It’s politically 100% viable to kneecap AI with copyright restrictions. This will go to the Supreme Court and it’s far from clear whether fair use applies to every case here.

There's a way to sell this to the public, but AI proponents don't want to have to sell it, because they feel that they shouldn't have to, and there's an underlying theme of "The benefit of AI is so overwhelming, and eventually it will replace most commodified creative work anyway so why bother litigating this now, let's just skip this messy step and get to that part" and that's super not going to work to convince skeptics.

IMO, what's most likely is some sort of licensing model between the AI companies and the 'big content providers' (remember most content on the web these days is not owned by the person who created it, wasn't always like that). The smaller companies then would be forced to live with either being scraped or ending up being 'invisible'.

I agree it’s necessary to pay artists - but we don’t need copyright for that! There are many tried and proven alternatives.

Other than patronage, what is there?

Also, patronage is garbage, in my opinion. It ensures artists are exclusively either already wealthy, or well connected. It also helps ensure that the wealthy are most often represented in the art created; for some reason this seems like a bad idea to me.

If you do not agree with their business model, don't get involved with their business, at all. Your disagreement doesn't give you the right to exploit flaws in their methods to protect their business. Just like the fact you don't want to pay for something doesn't grant you the right to exploit the fact that the laws of physics allow you to just grab something you didn't pay for with your hand and run away with it.

If I fully believe in the concept of fair use and transformative content, then yes it absolutely is my right to take advantage of generative AI.

Fair use is a common concept used in all sorts of media.

You don't get to hand wave that away just because generative AI is getting good.

I don't think I need to "gaslight myself" into anything; as far as I can tell, making a copy has not ever felt morally wrong to me.

Creation should happen for whatever reason its creator becomes inspired with. The only absolute I can think of is no one should actually categorize worthy and unworthy motifs.

The only invalid reason is because you need to feed yourself, and the fact that we need to do that, we need to pay artists and everyone else just to survive, shows our failure as broader society.

I agree with your premise but the chess analogy falls flat.

We might, legitimately, see an enormous dropoff in people creating original works of literary, musical, and visual art (without AI).

If those motivated purely by money stop creating little of value will be lost

Somehow Finland can manage, but US can't? Please.

The roofless exist to send a message, "stay in your lane, be a cog in the machine, don't disrupt the system and you won't end up like THEM".

Great art - especially in modern times when that art involves expensive education (which if you're American must be paid for with interest) and the incorporation of technology and equipment - takes time and effort. If that time and effort cannot be paid for, then no matter how passionate an artist may be, unless they have sufficient personal wealth, that art must suffer.

Even the great artists of old needed patrons, because they needed to eat like anyone else. Michaelangelo didn't paint the Sistene Chapel ceiling for the love of the game, nor would he have.

I guarantee you that the working artists who have already lost commissions and work due to AI care about their craft.

I’m not pretending anything. I’m just making a statement about what might happen. I don’t personally care much one way or the other.

I care little about paying „rightsholders“ and their ilk - so I have zero empathy if they complain about imagined losses.

Don’t jump to conclusions about people who have never even talked to

Maybe you believe no artist who works for a corporation has any motivation but money, as opposed to purely "indie" artists, I don't know where the line in your head is drawn, but you do seem willing to throw most artists under the bus for some arbitrary standard of purity.

AI is harming working artists right now, and will likely never harm corporate rightsholders. They'll simply run their own AIs and fire as many people as they can get away with. The end result will not be that only the "true" artists survive but simply less art of any kind, everywhere. So I stand by my comment.

I for example have never benefited from copyright, neither from GEMA (the German artist association for musicians) - 99% of payouts go to the rich and successful mainstream artists and „indie“ artists get nothing but are forced by law to pay in if they want to perform in public.

So yea I have little sympathy for artists who only work for corporations or are rich enough to afford lawyers to enforce their copyright.

The way I see it there exist 3 ways to make a living as an artist now: - be rich trustfundkid and don’t care about money - be „purist“ and just live from selling your art and be on the brink of starvation constantly - get a „money“job and produce art in your spare time

Apparently there exists a huge population of artists who can make a living from working for corporations - but I have yet to meet one in real life. They are always brought up in these HN discussions but in my experience they don’t exist.

People didn't stop painting because photography exists, they created new forms of photography. People didn't stop writing music or using new / unique instruments when synths and programs came along.

I genuinely believe that people will keep creating, it's in our nature, and we also like things made by other humans, because we can relate to them.

It has nothing to do with whatever "value" the capitalist system assigns to the act as a side-effect.

If that's so, things are about to get worse for everyone, too. With little to no protection against AI, no one will be incentivized to create new IPs, whether they're books, drawings, songs. Or even films and games, when AI is able to also generate those in the (possibly near) future.

This is really about replacement. The copyright holders in the content industry aren't really afraid of LLMs infringing on past copyright, but are terrified of it replacing them on future work, and there is absolutely no legal protection from this. The lawsuit might officially be about copyright, but that's just because it is their only available legal angle of attack.

How do you square this with literally the first image in the OP showing side by side GPT reproing copyrighted work? imo a good modern art project would be someone making a website that “archives” NYT articles by laundering them through GPT rather than using the archive link that everyone posts to get around the paywall. Even HN guidelines bend over backwards to allow bypassing the paywall by allowing these links.

The author, I believe, is being purposefully deceptive and hoping people who don't use DALL-E see "animated sponge" generating a SpongeBob look-alike and think they should be burned.

Not what I was arguing and you’re not going to win many arguments with anyone who is paying attending by coming out of left field with only tangentially related demands.

As a consequence, an AI meant to topple Western soft power around the world might be held to much looser standards than one used domestically. Who cares that in rare circumstances the AI mentions the Tiananmen Square Massacre to Spaniards if asked about it, as long as it is good enough at spreading Chinese culture.

LLM people are really starting to veer into crypto-bro territory with the evangelising about how they’re the best thing since sliced bread and transistors.

That's a big "if", isn't it? We're seeing claims like "The future is an LLM at the front of just about everything: “Human” is the new programming language"[1] but so far that's not panning out, and it seems really dubious. Natural language seems like an absolutely atrocious user interface. As a machine operator, I'm going to use levers, wheels, and buttons to control the machine. As a computer programmer I'm going to use programming languages to control the machine. I'm not going to speak English to it.

So, ok, this marks an advance in NLP. How do we get from there to "omg it's gonna change everything!!!1111oneeleven"

[1] https://techcrunch.com/2023/08/08/nvidia-ceo-we-bet-the-farm...

It seems like they’ve accelerated our capabilities- previously tiresome and difficult-to-automate things are easier- but have done very little for our fundamental understanding. We have a tool, but cannot dissect it and explain how it fits together. LLM’a themselves don’t appear (happy to be wrong here) to actually have improved our understanding our NLP and associated theory. Yeah, it can parse a sentence and bang out some JSON/sql/mid-tier-essay, but these models (so far) aren’t helping us figure out how and why, and I think that understanding is critical to progress further. Anthropic seems to be trying to push a bit further on that front at least, but for all we know, they might just turn into another scummy OpenAI on us.

So I'm not sure how I could use an LLM as a tool, but maybe I'm just not a sufficiently proficient user? It seems like they're just too full of "surprises".

How would that be any different than Google displaying some type of news headline?

[1] https://www.reedsmith.com/en/perspectives/ai-in-entertainmen...