A woman posts on twitter. In the replies, people ask Grok to remove her clothes. Deepfakes are proliferating, sometimes out of personal interest, sometimes as a form of bullying. Fake images can and do hurt real people.
There is also a greater debate about giving people with harmful deviances a “victimless” outlet versus normalising their deviance. Sure there are no children harmed in making the images, but what if they generate images of child actors, or normalise conversation about child sexual abuse?
That is super interesting culturally. Once video hosting became feasible to be offered for free people came up with the idea that posting their recordings online is a good idea. Which is fine bacause to the casual observer their face is as anonymous as their online handle. You can extract value from millions of your viewers with them knowing about you only as much as you told and shown them.
Publishing is just the first part. The other part is reactions. Most platforms let you disable them so your viewers don't see the disgusting things people say about you right there along your content. Yet many people who publish themselves decide to leave them on. Because the vile comments actually help them exploit their viewers. There's a value to being told to kill yourself in a comment on your post.
If the capability to let users generate fake porn in the comments was left to creators. Many of them would leave that option on. For the same reason they leave the comments on. Ben Shapiro could benefit a lot if some terrible person commented on his video with deepfake or him sucking someone off. Both because of the outrage and because his viewer base is more homophobic and homophoby correlates with homosexual arousal.
There is also a greater debate about giving people with harmful deviances a “victimless” outlet versus normalising their deviance. Sure there are no children harmed in making the images, but what if they generate images of child actors, or normalise conversation about child sexual abuse?