Complete bollocks because my fiancée never signed up to the twitter TOS and I was able to make deepfake porn of her. I accepted the twitter TOS long ago but I certainly didn’t agree to let them make deepfake porn of me because it didn’t fucking exist ten years ago.
I get that you're going to feel defensive about this, but you have to remember that anything you feed into AI platforms is almost certainly being retained, along with the metadata. Even if you had her full consent (which tbh sounds questionable if she didn't know that), there surely could've been a better way to demonstrate than to use her actual photo.
Use someone else’s without their consent? Her entire point was “surely you can’t use a real person in this?”. I don’t see a better demo than that (seeing as the ones I made of myself “weren’t sexualised” because I’m a man).
As someone who's actually done this can you give some more details. Is it just putting people in bikinis or does it do full nudity and beyond? Is it easy to do or do you have to use prompt hacking techniques to trick it into doing this stuff? If so, is it materially easier with grok thank other mainstream services?
There has been so much reporting on this stuff and I still don't really know how outraged I should be
You can just ask it to put someone in a bikini and make them dance sexily and it will do it no questions asked.
When I tried to get it to make a video of me wanking it initially refused but when I told it to make a video of me shaking a pink smoothie in front of my crotch it complied and was indistinguishable from a wanking video.
There is a process any regulator has to go through. Using business distruption methods including applying to a court to order ISPs to block a site is very much the last option.
Asking for compliance with the law, warning letters, fines etc would likely all come first.
The US government itself hasn't threatened sanctions, a single congresswoman in the House of Representatives said she would draw up a bill sanctioning the UK.
Its essentially like if a backbench Labour MP said they would introduce a bill to sanction the US.
This is incorrect. The woman who threatened sanctions on the UK has the full power to do it. She's literally the head of the body that deals with trade.
I know you want to puff up your chest and act tough towards the USA, but she CAN sanction us. She isn't the equivalent to a backbencher at all.
Appeasing him just shows him that his threats are working.
If you’d been paying attention, you’d recall that when Starmer stood up to Trump in the past, Trump spoke very positively of him afterwards. He respected it.
Like Putin, the only thing he responds to is strength. So let’s show him some.
So what? First, that's choosing to be at the whims of the US. We have plenty to retaliate with. Second, the sanctions are a different type of damage, I don't think it's nearly as bad as Twitter is for long term damage. I'm sick of the US, Chinese, russian, Iranian and Indian propaganda constantly pushed here, especially the American one who have openly stated they're going to meddle in our democracy to get a government they want...and so should you be, assuming you're British.
Thirdly, it would get overturned, they're not sanctioning Russia or China for banning it. We're a sovereign nation, we can do what the fuck we want.
Fourth, there's no official government statement saying so.
Fifth, time to just do it. I'd take the short term pain of decoupling from the US to build a better, less abusive relationship with other nations than this long term damage that's gone on for decades with the US, starting from WW2. We wouldn't have this migrant crisis if it weren't for them.
What evidence have you actually seen that justifies calling Grok a ‘CSAM bot’?
I’ll repeat here for the sake of anyone interested in a more sceptical approach to this panic:
Grok’s safeguards against generating CSAM were never removed
X’s position was that anyone who used Grok to generate CSAM would be treated the same as anyone publishing the non-AI equivalent
The claims that Grok has been used to generate CSAM are extremely unsubstantiated. In the British press these all flow from a single observation in an IWF report that staff had seen users on an unspecified dark web forum claiming to have done so. No law enforcement bodies have made this claim.
If Grok has been used to generate CSAM, which is still entirely plausible even if there is no evidence for it, then it has been done through jailbreaking techniques to which every major AI tool is vulnerable.
What evidence have you actually seen that justifies calling Grok a ‘CSAM bot’?
I mean, you can 'win' any argument by claiming "go and search for child porn or you lose", can't you? I've got a hell of a lot more faith in Ofcom, the Internet Watch Foundation, and the UK government to tell the truth on this one over Elon Musk, and I'm happy to wait while they deal with this problem.
every major AI tool
This argument keeps coming up, as if that makes it any better! Even so, is there a single "major AI tool" out there, other than Grok, that is integrated with a social media platform to instantly publish its output?
You don’t have to search for anything illegal. You say you trust the IWF and the BBC (whose reporting was based entirely off the IWF report in so far as CSAM was concerned) - well fair enough, but what evidence have they presented? Did you read the report and what did it lead you to conclude? Because it claims nothing, as far as I can see, that would lead anyone to conclude Grok is a ‘CSAM generator’.
Even so, is there a single "major AI tool" out there, other than Grok, that is integrated with a social media platform to instantly publish its output?
Not as far as I’m aware - there are other AI tools which can generate illegal content, and other platforms on which illegal content can be published, in spite of safeguards in both cases. X has both those functionalities so it has both those vulnerabilities, but neither one is unique in any respect.
That aside, I'm more wondering about the response to the potential ban on Twitter with the reasoning of CSAM being given, no matter how unsubstantiated, is "do it and we'll sanction you".
Not "you're wrong and here's the evidence disproving it". No, just threats that if it were blocked with that being the given reason, sanctions would be handed out.
At the very least, that makes it seem like the person who made the threat may be under the belief that there is AI generated CSAM on Twitter and that they don't give a fuck.
As far as I’m aware no substantive claim has been made regarding X generating CSAM beyond the one unproven claim the IWF observed on some pervert dark web site. It’s not really something that can be ‘disproven’ because it’s unsubstantiated in the first place. On the diplomatic level, it’s mostly about the bikinifying / undressing pictures which are of course very real.
Thank you. The mass hysteria on display here is genuinely unbelievable, especially when Reddit is typically in the absolute top percentile of being against these "think of the children" moral panics. However, when it comes to Elon Musk, people's brains just immediately fall out and they forget all of their critical thinking skills, instead reverting to "Musk Bad. Me Ban"
I'm tired of people crying "censorship" or similiarily "free speech" when the motion of banning a social media platform is brought up. This is not censorship, it's a platform getting deserved consequences for spreading vile images across instead of regulating or cracking down on it.
Elon is not being persecuted like the victim he claims to be, he's simply facing actual consequences. Free speech =/= freedom FROM consequences. Get over it, and learn to accept consequences instead of running away.
You are being banned from seeing the largest social media channel in the world. Obviously it's a censorship issue lmao. Do you think China's great firewall isn't censorship because Chinese people can post on their own social media channels? Unreal.
Elon's not going to shag you mate. You will have multiple other platforms from across the globe enabling you to talk to whoever you want. Services that don't have integrated CSAM generators.
If you're that addicted to the site that you're willing to forgive the production of CSAM then it's probably in your best interests that it's removed anyway.
Such a good sign of poor thinking. I have no interest in Musk, which is exactly why I'm thinking reasonably about whether to literally ban the biggest social media website on the planet. I mean, just take a step back for a second and actually think about what you're advocating for and how completely ridiculous it is. This is an argument that is clearly being pushed because people are upset with Musk (i.e. toddler reasoning).
You will have multiple other platforms from across the globe enabling you to talk to whoever you want. Services that don't have integrated CSAM generators.
You will be banned from seeing the output of the largest social media platform in the world, one which hundreds of global leaders use almost exclusively. Again, please stop being ridiculous.
If you're that addicted to the site that you're willing to forgive the production of CSAM then it's probably in your best interests that it's removed anyway.
Genuine "if you have nothing to hide" tier reasoning. No, I have a principled objection to censoring a website that contains a plurality of the world's information, obviously.
The firewall blocks them from seeing mainstream social media sites, which - obviously - contain a whole bunch of ideas that will never get to them. It's genuinely mental that you're arguing this isn't censorship. Like, I'm going mental just engaging with it.
Explain what is being censored if a single social media site goes offline? Everything being said there can be said anywhere else.
Everything said on that massive social media site is being censored.
It’s like saying that a billboard in your local town being taken down is censorship even though there’s another one available right around the corner.
Everything said on that massive social media site is being censored.
And if it’s said on a different site, it’s fine. Words are not being censored here, AI deepfake porn is.
If that’s seriously something you’re annoyed about then honestly that says more about you than anything else.
It's nothing like that.
You’re right. It’s more like you trying to put up faked childporn on a billboard outside your office and then refusing to stop when the council ask you to, so now they’ve got to take the billboard down.
There’s a bunch of other billboards up around that already stop you from putting that content up though, so they’re available for you to advertise legitimately if you want.
When you have grok that can make indecent images and then share it on a massive platform like X then it's harmfullness implications are amplified. If X doesn't remove grok or fix the issue with it then there is no choice other than to ban X. It's extremely simple.
Grok is baked into X. The only way to effectively ban Grok is to demand X either put safeguards on Grok to stop it producing illegal material or to shut Grok down, or else face being banned in the UK.
The whole potential ban goes away if they just stop Grok from producing CSAM/non-consensual sexual images on demand.
You'll find free speech means you can say whatever you want but it doesn't protect you from the consequences of saying it. Or as the old saying goes. "Just because you can doesn't mean you should."
I actually got into an arguement with an american who said I was "immature" and "probably young" for saying that generating AI nudes of people was wrong. They seemed quite insenced that someones dignity and sense of privacy should prevent their ""freedoms"".
You're duly reminded that this is the country that arrests and sometimes even jails people for not mowing their lawn, or removing their lawn... or crossing the road at a non-government approved spot.
Civil asset forfeiture, SOMETHING that [in the US] means cops can take your things without you even being charged with a crime, sometimes amounts to more annually than even burglaries.
If you've made it this far, just think for a moment about the girls, the women, the mothers, that could have been you, or your sister or your own mother. Reflect on this alone when Americans say they are free, they do not even have women's rights.
Medical debt cripples a large amount of the POPULATION [in the US].
Their [US] children spend their time in school doing active shooter drills while also swearing allegiance to a flag every morning, something only Russia, China, and North Korea could dream of.
But after all those things... I'm sure it will please all of them to know, that after not having any real freedoms at all, being oppressed by a dystopian society with a propaganda machine that would make Goebbels blush, they can still call a black man a n*****."
When it comes to children, it's unambiguously immoral and illegal. With adults, I do think it's trickier.
It's obviously unkind and immoral to publicly post nude art of someone who has not consented. At the same time, I can see why it's tough to draw the legal boundaries.
There's no law against imagining, say, a celebrity crush naked. I don't think anyone is suggesting that should be illegal. In which case, should it be unlawful to draw a picture that you've imagined? What about a photorealistic one? If that's okay, why should using AI change that? Or is it the public sharing that's the problem?
To be clear, I'm not arguing it's okay. I'm just pointing out that, despite a widespread intuition that it's problematic, it's quite challenging to pinpoint exactly where it crosses the line from imagination to crime.
I could understand harassment charges if a person was posting the videos to the subject or their family to upset them. I could also understand a defamation case if they were claiming the footage is real. However, if they make clear it's fake and don't use it to harass people, it doesn't fit neatly into the current framework.
For CSAM, I agree. The risk is that it could spread and, in so doing, increase the market for content made with actual children. This puts children at risk of sexual assault or rape.
For adults, though, I don't think the same argument can really be made. I wouldn't like the thought of strangers jerking off to me, but I'm not sure that should make me a "victim" in a legal sense. That's true whether they're imagining me, looking at a photo of me, or looking at a fake image they've created or viewed.
I think it should definitely be illegal to use said videos to harass people, illegal to suggest they're real, and illegal to share real videos non-consensually. Beyond that, though, I think it's more debatable. It effectively boils down to, "You can't draw me naked (at least not if it's sufficiently realistic) because it upsets me."
In which case, should erotica depicting real people be illegal? I think most people would say no, because it's obviously not real. But even if these videos are clearly marked as not real, I think people still feel they should be banned.
I'm not denying that people feel an intuitive difference, which could well point to something real, but I do think it's tough to tease out what the principled distinction is.
The risk is that it could spread and, in so doing, increase the market for content made with actual children. This puts children at risk of sexual assault or rape.
Is that risk actually the case? I've always thought that e.g. fake diamonds or ivory resulted in the real stuff being sourced less?
I imagine it's very difficult to research ethically. I could see it going either way. Perhaps, if you allowed animated CSAM created without any children being harmed, it would enable people to control their urges without seeking out real images.
On the other hand, I suspect that tolerating communities that share such images, even animated or drawn, normalises and excuses the behaviour and could lead to escalation. Really, if someone has these urges, I suspect it's a very dangerous thing to feed.
Lots of things fit that description: you could write down a threat to bomb something on a piece of paper, but if it doesn't leave your house, it's not going to be illegal.
That's a fair challenge. I suppose the distinction I'd make is that you're not being arrested for the image itself in your example. If you called in with a bomb threat, you'd be arrested. If a newspaper published a copy of your letter, they wouldn't be.
I agree that we can't criminalise pure imagination. However, if someone draws CSAM, that's a crime even if they never share it with anyone. Are we suggesting the same for deepfakes?
I understand what you're saying and broadly agree but I think there is subtle differences. Its completely legal to imagine whatever you like - even if that image is illegal in itself.
When it comes to drawings, it becomes less about the specific person and more about the concept in my view.
If you drew a celebrity or a random adult there's enough creative room to argue that it's fair and "inspired by this person, but not them" and therefore legal. In contrast to say children, where regardless of intent its unacceptable; which is why loli and all the other attempted workarounds for predators are still illegal despite the "methods" being identical to an adult.
AI is instead more about the intent to create photographs rather than art and could be more taken as misinformation for lack of a better word. If you asked AI for cartoon/drawn nudity depiction, I don't think it would have as much traction and this likely wouldn't have reached the point it has now. After all, AI porn of a fictional adult isn't illegal in itself.
The situation is uncomfortable though, the UK should absolutely ban the platform at least until it restricts life-like depictions. I don't think we should be able to misdirect people with realistic videos/images regardless of intent so easily.
If the primary issue is misinformation, though, wouldn't a prominent "fake" watermark address that? By and large, I don't think people actually believe it's the real person, nor do the creators make any attempt to suggest it is. I suspect most people who object would still object, even if the videos were clearly marked as fakes.
It’s also unkind and immoral to make it in the first place. How would you like it if someone was sitting at home gooning to deepfakes of you they made without your consent? It’s incredibly invasive and creepy. Never mind the possible consequences of a data leak.
Do you want to apologise now for misrepresenting me, or do you have no moral qualms about that (which would be somewhat ironic)?
There are all sorts of immoral things that aren't crimes. I wouldn't want my partner to cheat on me. I don't think they should go to jail if they do so. Similarly, I wouldn't want someone to make a deepfake of me and jerk off to it. However, if they do that in the privacy of their own home, don't harass me, and don't share it as though it's real, I'm not convinced they should go to prison, any more than if they’d imagined me.
Nobody is suggesting this isn't creepy. My comment is focused only on whether it makes sense to criminalise.
I was quite clear that I was talking about morality, not legality. There are many immoral things that aren’t illegal, and I chose not to do many of them because they are bad things to do. I am not saying I think people should go to prison for doing these things, I am saying that doing them makes them bad people.
You have made clear with your reply that you see no problem with gooning to deepfakes you’ve made of people you know. I believe that’s morally repugnant and I am not going to apologise to you for that, sorry.
Then why on earth did you reply to my comment then, which was entirely about the law, as opposed to the dozens of others? More importantly, why did you suggest I had no moral issues with it when I specifically said it was immoral?
It's obviously unkind and immoral…
To be clear, I'm not arguing it's okay. I'm just pointing out that, despite a widespread intuition that it's problematic, it's quite challenging to pinpoint exactly where it crosses the line from imagination to crime…
Nobody is suggesting this isn't creepy. My comment is focused only on whether it makes sense to criminalise…
How much clearer could I have been?
The irony that you think misrepresentation is wrong, but you're more than happy to misrepresent others to make yourself feel good and others look bad, is incredible.
You have made clear with your reply that you see no problem with gooning to deepfakes you’ve made of people you know.
That's an outright lie. I think you know it's a lie. The fact that you said it anyway says something about you, whether you're prepared to acknowledge it or not. If you disagree, it should be easy to quote where I said that.
I don't think you can generate nude images of someone - via any mainstream AI - not just Grok.
Yes, you can clearly have them dressed in a bikini, as has been evidenced by pictures of both Elon and Keir.
So, is the issue that it enables people to instantly publish the altered image to X? Or if, say, someone used ChatGPT to create an image and then uploaded that image to X is that still a problem? In which case Grok isn't the issue, it's the posting online of an altered image.
Unfortunately Elon and twitter refuse to participate in UK law and so all we have really is turning off the child porn at the source. Its unfortunate but child porn really is that nuclear.
That would be an assessment made after a criminal investigation, yet the police have signalled nothing. Also in addition any individual who produces such material could equally be investigated and charged but to my knowledge no one has.
I signed up to a nice, useful platform, then Elon bought it and turned it into a cesspit of misinformation, hatred, and child porn. Why is it OK for me to have been punished, but not for you to be?
Because literally everyone is making shit up and riding a huge wave of performative righteous indignation because Twitter Man Bad.
Someone took the "Put her in a bikini" meme a bit too far and did it with a child. That's literally the extent of it. Poor taste and arguably not even illegal unless done persistently to harass an individual. (and is trivially done with any generative AI and even traditional image manipulation software) But most certainly not child abuse material, and besides, we have existing laws to deal with such images if they meet a criminal standard.
The whole thing has just been used to form the ridiculous (and equally performative) false dichotomy that either: A: You want X to be shut down, or B: You explicitly are in favour of CP.
The “put her in a bikini” thing is illegal. Don’t downplay non-consensual pornography as a meme. It causes real harm and there is absolutely no legitimate reason to do it.
It’s been widely documented that there are indeed thousands, if not millions of images that are illegal to display under UK law.
Look, I’m against OSA and I totally get that the “save the children” argument is often a lie. But this is a very clear case of genuine and widespread CSAM.
While an entirely novel problem due to the powers of GenAI, putting someone in beachwear is not and never will be pornography, regardless of age - which is precisely why the people trolling otherss by doing it are choosing bikinis as opposed to nudity. Sure, it might be regarded as harassment, or even malicious comms, but would likely have to be the result of a pattern of behaviour.
Unless of course you want to argue that a nautral photograph of a woman or even a child on a beach in a bikini is pornography. In which case, You've just criminalised the holiday snaps of tens of millions of people.
You need to brush up on the law, because what you’re saying is simply not true. I’m not going to waste my time arguing with you if you haven’t even done the requisite research to speak on this matter.
They are using underwear, as well as “covered in doughnut glaze”, posing people in suggestive ways, and making their breasts bigger. Just scrolling through the replies of Grok’s account shows all of this.
Bikini is just the most reliable prompt to bypass the (weak) filters.
My fiancée wasn’t convinced Grok is bad, so I demonstrated that I was able to sign up with a throwaway email address, download a picture of my fiancée from her instagram profile, and within ten minutes get it to produce deepfakes of her in her underwear, giving a lap dance and miming a blowjob. She was appalled.
There is absolutely no reason for this technology to exist. It’s vile and so obviously is going to be used for bullying, harassment etc.
And before anyone argues “we can’t do anything about that it’s just the way it works”, this is basically pollution and if a factory causes too much pollution, we don’t let you build it.
If the photos depict that person in a sexual or intimate context, it is illegal as per the 2003 Sex Offences Act. It’s basically the same as revenge porn.
Having the ability to do this on someone else's timeline with a simple few keywords is the problem here.
Yes you can undress people in other AI, but they don't allow you in someones own feed.
X needs to implement a feature that is an opt-in to allow this for your own feed and media. Stopping anyone sharing Grok images or easily taking your own imagines.
Also, better checks for CSAM also but that is a problem in all social media.
It's a deliberate design choice by X/Elon Musk - they want people on that shithole platform to use Grok constantly so will never remove the option to generate text/material publically via replies - that's the USP of Grok. X is the equivalent of a troll site where they quite openly want people to be antagonistic towards each other (moreso than other outrage algorithms) and think they have an edge on other AI services by people generating material publically and 'contacting' Grok as if it was just another user. It was always going to end up like this - and Musk has been pushing the NSFW side of Grok for a while anyway. Their response so far has basically just been 'oh we'll limit creating illegal material to paid customers then'.
Considering it was originally bought to be a 'free speech' app - which, when you have no code of conduct on a website inevitably it is taken over by the absolute worst people - it's no surprise that the same userbase, without restrictions, are utilising AI in the worst way.
This is one reason why I think the Government should quit Twitter, and a ban at this rate could happen, the platform's rapidly being funded by bad faith actors, it has an AI tool that's founder keeps tampering with, and is now being used to generate content that's arguably illegal, yet alone immoral. What benefit is dignifying that with a platform?
A fine is one potential outcome so it’s not really a threat it’s just a valid disposal. That amount would be up to 10% global revenue or 18 million pounds, which ever is greater.
No, if people want to use X then let them. Pains me to say it but it’s the place to go for football/sports updates in general. Ban/restrict Grok instead.
I don't necessarily agree with the position but It's not so much the bikini photos, it's more the wider concept of it being the thin edge of the wedge when it comes to government censorship, at a time when the UK is arresting hundreds of people for making social media posts (...well if they're on certain subjects, you can call for the death of Jews all you want). X is deliberately refusing to comply because it wants to pick a fight on the matter and it seems like it has the American government behind it (Vance certainly). I suspect the ultimate result will be guardrails put in place to restrict some of this stuff, after a long drawn out battle that puts the government off picking any further fights.
Because it brings zero social utility to us as a nation. What are the advantages of Xwitter to us as a people? It foments outrage, provides an unregulated space for misinformation to fester and further agitate the social fabric of the country.
Really not much to be honest but it seems a little better at curbing the animalistic impulses of its users through content moderation and the subreddit model.
Oh wow, the left dug itself a hole so deep it got back to the right and is thinking in terms of 'nation' again. What is a nation btw?
As an aside: I am a naturalised citizen, and in my citizenship test freedom of speech is listed as a core principle of Britain alongside democracy, rule of law and equality.
The advantage of X is we have access to different views and opinions from across the globe, and within the UK, therefore upholding this core principle.
Yeah, Grok should NOT be allowed to let you generate CSAM (like that's not obvious).
But everybody's crying wolf over the whole of X, meanwhile nobody is saying 'let's ban Grok'. Instead everybody on Reddit just wants an upright ban of the whole thing, like that does any good to freedom of speech. Both open source and commercial generative AI will fall through the same trap and generate naked pictures, sometimes with better or worse guardrails against doing that, esp. of under 18s, and again, nobody's asking those to be banned, just X.
just ban it until they sort the pedo business out. its really easy to do, literally every other social media site does not have an AI child porn creating tool built in to it.
One charity said they found CSAM on the dark web which “appears to have been created” using Grok. No proof that Grok was indeed the source was even given in the BBC article, X was contacted and replied with “We take action against illegal content on X, including CSAM, by removing it, permanently suspending accounts, and working with local governments and law enforcemnt as necessary”.
What more do you want?
What a bloody nothing burger, just an excuse to hastily get X banned because someone’s feelings are upset we can talk freely somewhere on the internet.
If I created a website tomorrow that created sexualised images of children, I would be getting a knock on the door from the police, and rightly so. Large corporations get to play by different rules to the rest of us rubes
I agree to both parts. You should be getting a knock, and so should the users that asked for and generated sexualised images of children.
Also, yeah, I agree that large corporations play by different rules currently and that should change. Before that, let's stop jumping to "Let's ban X" so quickly, and instead how about, let's prosecute the pedophiles that asked for those pics.
X allows a lot of free speech and different points of view, so it's better for bringing people together. Reddit is all about cancel culture and banning people.
If by free speech you mean a bunch of people paying to have the visibility of their posts artifically boosted.
Or by free speech you mean the owner of such platform arbitarily banning, shadow banning, or limiting the reach of anyone he personally deems unworthy of being able to voice an opinion.
Or by free speech you mean the countless bots that spam the talking points of hostile actors, and talking points of states that want to deliberately create divides within societies all over the world.
Thanks for your input though word-word-four numbers.
I can say for people like my fiance who is an artist it promotes her bussiness, pushes her artwork out there and allows her to get comissions and engagement with other artists. Going to conventions and seeing what artists are where based on their twitter posts is important for her too. Twitter is far more than just reform voters arguing that free school meals are somehow bad. lol
Honestly if this whole debacle doesn't lead to X being banned/blocked over here in the UK, then nothing will. It should've happened as soon as it was very clear it was a platform that solely existed to push misinformation and far-right talking points en masse. Then it should've happened when Musk essentially boasted about doing the same thing and waged a personal misinformation war against the UK Government (a few times now). If it doesn't get banned for now being a huge repository of public AI revenge porn and AI CP then I don't think our Government will ever have the balls to do it.
Nothing of value would be lost and I'm sure, if polled, the UK population would be in support of blocking that shithole of a site anyway. Other countries around the world with large userbases have started now blocking it (Malaysia & Indonesia yesterday) so I'd rather we get involved with this potentially global pressure campaign on the site.
This is aside from the Ofcom investigation, by the way - I can't imagine there will be anything that comes out of this except a long-winded warning and a fine. The decision should be made from No.10.
THB, some of the discussion around the whole X and CSAM stuff has been highly misleading, mainly because people read other misleading comments and parrot them back.
It's not mass censorship; that's a very silly point to make. There are plenty of social media platforms that have terms of service that don't effectively encourage their userbase to spread toxic and dangerous far-right misinformation or have a leadership in place that deliberately spends money and uses the platform to destabilise Governments across Europe, Canada etc. That doesn't mean we should allow one horrendous one to exist and continue to get worse and worse, and more belligerent against Governments across the world (not just the UK but so many other examples). If people move from X they can just go to another social media website - oh no, wait, they probably can't; because they're mostly far-right agitators who have been given their platform via X itself. And the world has regulated against them for many decades before this.
I honestly could not give a fuck - we are in an era of American oligarchs utilising their wealth and media platforms to spread the same ideology to the UK and Europe that has effectively fucked the USA; we've all seen it coming, nothing has happened, and now just because these odious individuals keep hiding behind this spurious idea of 'free speech' (which they obviously don't believe in themselves), that we should just keep letting it happen?
There's a double standard anyway; I can guarantee you that if Weibo was in the position that X were in in Western countries with such a belligerent owner on the world stage people would be calling for its block under security concerns. Would that be mass censorship? TikTok already has been and the Americans bought it out. These social media platforms have an obligation to adhere to the push and pull of individual Government regulations and X have been taking the piss for years - they clearly think they're above most Governments and aren't guaranteed to be safe from regulation.
"don't effectively encourage their userbase to spread toxic and dangerous far-right misinformation" - what you call "far-right misinformation" is whatever you disagree with. You support censorship of right-wing political views and opinions
Just admit you support mass censorship and government overreach already
If Farage wanted to do that, he'd do it anyway - much like MAGA in the US didn't exactly need a pretext to start shaping the US how they wanted to. It doesn't really set a precedent - much like how the Dems I'm sure now regret not prosecuting Americans involved in the electoral fraud big lie.
It would be better to be pragmatic to try to stop the next election in this country being an utter shitshow with misinformation emanating from a site we already know was a massive vector in causing far-right riots across the country. If X wanted to stay as a platform - perhaps they would put in better terms of service and actually ban people with large platforms known to cause trouble, like Twitter used to do? Ah no, it's because it's specifically designed to be a far-right site and they welcomed back known conspiracy theories, neo-nazis and white supremacists with open arms as soon as it was purchased by Musk. And they know that they can only really do that with almost zero moderation put into the site.
X currently hemorages money, the profit they make from advertisements is net negative. They are really power on because they have positioned themselves as the ‘town square’ of the world, and presumably that can be monetised outside of traditional advertising revenue (or that’s their plan anyway)
Well, you can only implement restrictions that are practical. Sure, a minority of people may still be able to access X if they are desperate; not great, but still far better than the current case. Even if you totally banned it, people would be able to go on holiday abroad and access it, but that doesn't mean it should be allowed in the UK.
My point is that banning websites is like banning numbers. It just cannot be feasibly done. This whole sub roundly (and correctly) criticised the Online Safety Act as totally impractical and yet equally impractical solutions like banning X are somehow cheered on.
I think you're misrepresenting the criticism of the OSA. It isn't (at least, as far as I've seen) "please don't ban our child porn sites", it's "please don't ban wikipedia and force me to hand over my identity to send a direct message on bluesky".
Not at all, hardly anyone is complying to these laws, even recently in Italy they tried to fine a social media company X amount of their global revenue and the company just ignored it, forcing the hand of Italy to cut itself off and become a social pariah or just rethink their ability to execute these things
Why would blocking a website cause an entire country to become a social pariah? You're making it sound like X is as important as a sovereign state; it really isn't.
Not politically but definitely socially, like if the US had banned TikTok, they’d be ‘out of the loop’ with the rest of the world with the trends and memes, which weirdly nowadays would cause them to lose positioning in the larger world and even effect the stock market.
Nobody wants to position themselves as like the unc at the Christmas table who doesn’t get the joke.
This is why the only way to combat a site and oligarch as odious as X and Musk is to threaten them with a block - and actually go through with it. Musk tried to ignore a Brazilian softball order to ban users who engaged with election misinformation over there and contributed to their own capitol attack after Bolsonaro lost, Musk ignored it, he ignored Brazilian legal obligations and spent months trying to have X dodge around legal action before the site was just completely blocked. He then spent about a month trying to let X be accessible in Brazil through various tinpot means - and personally tried to dox and harass legal representatives in Brazil using his platform - before finally just paying the fine he had to pay (miniscule in the grand scheme of things) and acquiescing to legal requirements.
A block/ban is the only language that rattles the site. They know they can ignore fines and warnings but they are shat up when they face losing millions of users.
The OSA was disgusting and stupid, and has caused an increase in suicides among kids who lost access to support services and networks they connect to friends with...
This sort of stuff is just icing on the cake. Banning X would be fucking stupid. (to clarify I am against grok and genai in general as my fiance is an artist and genAI puts her livlihood at risk, I am not supporting grok at all, I think banning twitter is stupid when countless other AI bots can "strip" people anyways, grok needs to be properly moderated but banning twitter is such a stupid solution. People who want to do that stuff will just bypass the ban and do it anyways. Bans only hurt innocent people who have no horse in this race to be quite honest)
Most of my friends use X, I stay connected through X to see what social stuff they are doing and where I can meet up with them at conventions etc, I use it to buy from artists custom artworks, my fiance uses X to push her art and get comissioned and network because the kind of artwork we consume and engage in isn't exactly stuff you'd slap on your linkedin to be quite honest.
Banning X will just be another issue and the use of VPNs will soar to even greater heights.
Policing what sites the public can use is dystopian and extremely horrifying. Honestly just dangerous. I am disgusted to see people actually calling to ban twitter and other social media sites. Are people really that okay with locking kids out of support sites and stopping people from socializing in their hobbies like sports and anime and gaming and artwork? I don't see why this is such a controversial topic.
Parents should parent better and sites should remain accessible. It really isn't a hard concept to grasp. Having to fork out for a VPN every year with the state of the economy as it is doesn't do anybody any favors but the government literally leaves us no choice at this point. It's sickening to say the least.
Knowing how toothless Ofcom and this Labour government are, the most I see happening is a six-figure fine and regulators telling X Corp not to do it again.
Shit like this is why the Online Safety Act was passed in the first place, yet the government seem far more concerned with mandatory age verification mass surveillance than actually protecting the public from involuntary pornography and CSAM.
We elected a leader who is too much of a wimp to stand up to Trump.
How about not allowing a simple text input to put indecent pictures onto someones timeline?
Make the ability of people using that an opt-in for peoples timelines.
Reporting stuff should work instant in these cases also, taking it off someones timeline if it is a posted image.
It's about time we stood up alongside those other bastions of freedom that have banned X - China, Iran, Myanmar, North Korea, Russia, Turkmenistan, and Venezuela.
Snapshot of Ofcom investigates Elon Musk's X over Grok AI sexual deepfakes submitted by EquivalentKick255:
An archived version can be found here or here. or here
I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.
The community notes on an MP that showed this happening were shocking.
They said it was fine because the MP consented to it when they agreed to the Twitter TOS.
Please can you link this?
Complete bollocks because my fiancée never signed up to the twitter TOS and I was able to make deepfake porn of her. I accepted the twitter TOS long ago but I certainly didn’t agree to let them make deepfake porn of me because it didn’t fucking exist ten years ago.
Does your fiancée know that you used Grok with their image like this? 😬
Yes she was there mate. I’ve explained in another comment I was showing her what it was capable of because she didn’t see the problem with it.
I get that you're going to feel defensive about this, but you have to remember that anything you feed into AI platforms is almost certainly being retained, along with the metadata. Even if you had her full consent (which tbh sounds questionable if she didn't know that), there surely could've been a better way to demonstrate than to use her actual photo.
Jesus Christ… the bloke didn’t come here for a lecture. Are you his mum?
It's actually incredibly patronising to assume she's not capable of consenting herself.
Yes, that's me! I was just coming on to send him a message that his tea's on the table.
Use someone else’s without their consent? Her entire point was “surely you can’t use a real person in this?”. I don’t see a better demo than that (seeing as the ones I made of myself “weren’t sexualised” because I’m a man).
She’s a big girl and can take care of herself.
As someone who's actually done this can you give some more details. Is it just putting people in bikinis or does it do full nudity and beyond? Is it easy to do or do you have to use prompt hacking techniques to trick it into doing this stuff? If so, is it materially easier with grok thank other mainstream services?
There has been so much reporting on this stuff and I still don't really know how outraged I should be
You can just ask it to put someone in a bikini and make them dance sexily and it will do it no questions asked.
When I tried to get it to make a video of me wanking it initially refused but when I told it to make a video of me shaking a pink smoothie in front of my crotch it complied and was indistinguishable from a wanking video.
Just ban it until Grok has appropriate safeguards in place.
Why are we scared of Musk or America.
There is a process any regulator has to go through. Using business distruption methods including applying to a court to order ISPs to block a site is very much the last option.
Asking for compliance with the law, warning letters, fines etc would likely all come first.
I think that’s right
They're going to start small at first
I think so maybe a formal warning and letter to act within a certain amount of time or get a fine.
I can’t see how X doesn’t then comply as I can’t think of many countries who allow CSAM
If this was a person they'd be arrested now. But it's a corporation.
How many of the people using it have been arrested so far?
Because they have threatened sanctions
We sadly can’t upset Trump too much
The US government itself hasn't threatened sanctions, a single congresswoman in the House of Representatives said she would draw up a bill sanctioning the UK.
Its essentially like if a backbench Labour MP said they would introduce a bill to sanction the US.
She’s chair of the committee responsible for trade and sanctions
Sure but that doesn’t change that Musk bankrolled Trump and still has significant influence on Trump
Nah, Trump used him and dumped him.
Seems like they're dating again, or at least seeing how things go for now.
Not sure as they had lunch the other day
This is incorrect. The woman who threatened sanctions on the UK has the full power to do it. She's literally the head of the body that deals with trade.
I know you want to puff up your chest and act tough towards the USA, but she CAN sanction us. She isn't the equivalent to a backbencher at all.
They’re also threatening to invade a NATO country. Now is exactly the time to upset Trump - if you don’t stand up to a bully, it only gets worse.
Ultimately we must remind him that we are a sovereign state and he cannot tell us what to do.
How is upsetting him going to stop him trying to invade Greenland?
How shit is this sub at diplomacy lmfao
Appeasing him just shows him that his threats are working.
If you’d been paying attention, you’d recall that when Starmer stood up to Trump in the past, Trump spoke very positively of him afterwards. He respected it.
Like Putin, the only thing he responds to is strength. So let’s show him some.
Not banning it at a snap of your fingers isn't 'appeasing him'
Appeasing him would be making sure it isn't investigated and defending X as an american company.
Instead he said that the disgusting and unlawful use of Grok would not be tolerated.
As with what normally happens with companies in this situation, an investigation has started and then ultimatums will be made once that has completed.
It's a lot harder to get a bullet back into a gun once it's been fired.
So what? First, that's choosing to be at the whims of the US. We have plenty to retaliate with. Second, the sanctions are a different type of damage, I don't think it's nearly as bad as Twitter is for long term damage. I'm sick of the US, Chinese, russian, Iranian and Indian propaganda constantly pushed here, especially the American one who have openly stated they're going to meddle in our democracy to get a government they want...and so should you be, assuming you're British.
Thirdly, it would get overturned, they're not sanctioning Russia or China for banning it. We're a sovereign nation, we can do what the fuck we want.
Fourth, there's no official government statement saying so.
Fifth, time to just do it. I'd take the short term pain of decoupling from the US to build a better, less abusive relationship with other nations than this long term damage that's gone on for decades with the US, starting from WW2. We wouldn't have this migrant crisis if it weren't for them.
Time to cut them loose I'm afraid.
Because he is influential and is happy to dump loads of money on traitors like Farage
So you are advocating for censorship of a mass platform? Just banning Grok is not enough for you you want the whole social media platform banned
The platform has refused to remove Grok, the option is still on the table, they further monetised their CSAM bot instead.
What evidence have you actually seen that justifies calling Grok a ‘CSAM bot’?
I’ll repeat here for the sake of anyone interested in a more sceptical approach to this panic:
Grok’s safeguards against generating CSAM were never removed
X’s position was that anyone who used Grok to generate CSAM would be treated the same as anyone publishing the non-AI equivalent
The claims that Grok has been used to generate CSAM are extremely unsubstantiated. In the British press these all flow from a single observation in an IWF report that staff had seen users on an unspecified dark web forum claiming to have done so. No law enforcement bodies have made this claim.
If Grok has been used to generate CSAM, which is still entirely plausible even if there is no evidence for it, then it has been done through jailbreaking techniques to which every major AI tool is vulnerable.
I mean, you can 'win' any argument by claiming "go and search for child porn or you lose", can't you? I've got a hell of a lot more faith in Ofcom, the Internet Watch Foundation, and the UK government to tell the truth on this one over Elon Musk, and I'm happy to wait while they deal with this problem.
This argument keeps coming up, as if that makes it any better! Even so, is there a single "major AI tool" out there, other than Grok, that is integrated with a social media platform to instantly publish its output?
You don’t have to search for anything illegal. You say you trust the IWF and the BBC (whose reporting was based entirely off the IWF report in so far as CSAM was concerned) - well fair enough, but what evidence have they presented? Did you read the report and what did it lead you to conclude? Because it claims nothing, as far as I can see, that would lead anyone to conclude Grok is a ‘CSAM generator’.
Not as far as I’m aware - there are other AI tools which can generate illegal content, and other platforms on which illegal content can be published, in spite of safeguards in both cases. X has both those functionalities so it has both those vulnerabilities, but neither one is unique in any respect.
That aside, I'm more wondering about the response to the potential ban on Twitter with the reasoning of CSAM being given, no matter how unsubstantiated, is "do it and we'll sanction you".
Not "you're wrong and here's the evidence disproving it". No, just threats that if it were blocked with that being the given reason, sanctions would be handed out.
At the very least, that makes it seem like the person who made the threat may be under the belief that there is AI generated CSAM on Twitter and that they don't give a fuck.
As far as I’m aware no substantive claim has been made regarding X generating CSAM beyond the one unproven claim the IWF observed on some pervert dark web site. It’s not really something that can be ‘disproven’ because it’s unsubstantiated in the first place. On the diplomatic level, it’s mostly about the bikinifying / undressing pictures which are of course very real.
Thank you. The mass hysteria on display here is genuinely unbelievable, especially when Reddit is typically in the absolute top percentile of being against these "think of the children" moral panics. However, when it comes to Elon Musk, people's brains just immediately fall out and they forget all of their critical thinking skills, instead reverting to "Musk Bad. Me Ban"
Well the nonces prompting Grok are pretty problematic, no?
The two are also intertwined, Musk could pull Grok out of he wants.
I'm tired of people crying "censorship" or similiarily "free speech" when the motion of banning a social media platform is brought up. This is not censorship, it's a platform getting deserved consequences for spreading vile images across instead of regulating or cracking down on it.
Elon is not being persecuted like the victim he claims to be, he's simply facing actual consequences. Free speech =/= freedom FROM consequences. Get over it, and learn to accept consequences instead of running away.
I don’t know why this is being portrayed as a censorship issue.
Blocking twitter wouldn’t be censoring anyone. You could still say everything you wanted to say on any other social media platform or in real life.
You are being banned from seeing the largest social media channel in the world. Obviously it's a censorship issue lmao. Do you think China's great firewall isn't censorship because Chinese people can post on their own social media channels? Unreal.
"Most popular social networks worldwide as of February 2025, by number of monthly active users (in millions)"
https://www.statista.com/statistics/272014/global-social-networks-ranked-by-number-of-users/
Its the 15th most popular social media site if you include the Chinese ones.
I think almost all of these are not reasonably described as social media, but Facebook/Instagram - fair enough.
Elon's not going to shag you mate. You will have multiple other platforms from across the globe enabling you to talk to whoever you want. Services that don't have integrated CSAM generators.
If you're that addicted to the site that you're willing to forgive the production of CSAM then it's probably in your best interests that it's removed anyway.
Such a good sign of poor thinking. I have no interest in Musk, which is exactly why I'm thinking reasonably about whether to literally ban the biggest social media website on the planet. I mean, just take a step back for a second and actually think about what you're advocating for and how completely ridiculous it is. This is an argument that is clearly being pushed because people are upset with Musk (i.e. toddler reasoning).
You will be banned from seeing the output of the largest social media platform in the world, one which hundreds of global leaders use almost exclusively. Again, please stop being ridiculous.
Genuine "if you have nothing to hide" tier reasoning. No, I have a principled objection to censoring a website that contains a plurality of the world's information, obviously.
Elon Musk didn't step on your puppy, mate.
Chinas great firewall literally takes down mention of specific things, like tianenmen square. That’s why that is censorship.
Explain what is being censored if a single social media site goes offline? Everything being said there can be said anywhere else.
It’s like saying that a billboard in your local town being taken down is censorship even though there’s another one available right around the corner.
The firewall blocks them from seeing mainstream social media sites, which - obviously - contain a whole bunch of ideas that will never get to them. It's genuinely mental that you're arguing this isn't censorship. Like, I'm going mental just engaging with it.
Everything said on that massive social media site is being censored.
It's nothing like that.
And if it’s said on a different site, it’s fine. Words are not being censored here, AI deepfake porn is.
If that’s seriously something you’re annoyed about then honestly that says more about you than anything else.
You’re right. It’s more like you trying to put up faked childporn on a billboard outside your office and then refusing to stop when the council ask you to, so now they’ve got to take the billboard down.
There’s a bunch of other billboards up around that already stop you from putting that content up though, so they’re available for you to advertise legitimately if you want.
Oh no! Censorship!
When you have grok that can make indecent images and then share it on a massive platform like X then it's harmfullness implications are amplified. If X doesn't remove grok or fix the issue with it then there is no choice other than to ban X. It's extremely simple.
Yes!
If your AI will happily make child porn, yes ban it no question
Grok is baked into X. The only way to effectively ban Grok is to demand X either put safeguards on Grok to stop it producing illegal material or to shut Grok down, or else face being banned in the UK.
The whole potential ban goes away if they just stop Grok from producing CSAM/non-consensual sexual images on demand.
Yes, ban X, I can't see a single negative from doing this. We get the added bonus of Musk's tears too.
Free speech != Generating nude images of someone
Free speech doesn't even include illegal speech! You can't shout bomb in an airport for example.
You can't even greet your friend named Jack.
Or help him off a horse
You'll find free speech means you can say whatever you want but it doesn't protect you from the consequences of saying it. Or as the old saying goes. "Just because you can doesn't mean you should."
I actually got into an arguement with an american who said I was "immature" and "probably young" for saying that generating AI nudes of people was wrong. They seemed quite insenced that someones dignity and sense of privacy should prevent their ""freedoms"".
MEANWHILE, 'freedoms' in the US...
Fine, Lien, Foreclosure: What Can Happen if you Refuse to Mow Your Lawn
Single Mother 'Arrested for Grass' After Not Mowing
Texas man jailed for not mowing his yard
Single Mother Arrested for Failing to Mow Lawn
Judge fines cancer patient, 72, for overgrown lawn: ‘I’d give jail time if I could’
If you don’t mow your lawn you could end up in jail like this woman
Ohio Town: Mow Your Lawn Or Go To Jail
Homeowner reconsiders mowing lawn after jail stint - Battle over widow's overgrown lawn has lasted three years
Boys mow lawn to keep elderly Texas woman out of jail
Woman faces jail for growing vegetables in her front yard
Whereas in the UK sometimes you're actually told not to mow your lawn.
Mow problem: gardeners encouraged not to cut lawns in May
No Mow May: Why UK gardeners are being told not to mow their lawns
Don't forget to not cross that street [in the US]!
Grieving Mother Faces 36 Months In Jail For Jaywalking After Son Is Killed By Hit-And-Run Driver
Historian arrested for jaywalking
Tulsa family wants charges dropped after teen arrested for jaywalking
Detroit Man Spends 3 Nights In Jail For Jaywalking In Greektown
Sometimes [in the US] they even fine you for drying your clothes outside.
Beware the Illegal Clothesline
A person commits an offense if the person places or maintains a clothesline in the front yard or on the front porch of any dwelling of any lot or parcel of land which is zoned “A” One-Family, “A-R” Residential, “B” Two-Family or “R-1" Residential, under the comprehensive zoning ordinance.
Civil asset forfeiture, SOMETHING that [in the US] means cops can take your things without you even being charged with a crime, sometimes amounts to more annually than even burglaries.
Law enforcement took more stuff from people than burglars did last year
You can't drink, or smoke, or place a bet until you're 21 [in the US], and don't even think about trying to have a drink in public!
Many STATES [in the US] now require an ID check to access Pornography (it's always them projecting isn't it?)
Pornhub Pulls Out of Seventh State
Police officers [in the US] will kill citizens often with impunity, sometimes running away, sometimes unarmed, sometimes even in their own homes.
Their cities [US] are surveilled by literal military predator drones.
The Government is Regularly Flying Predator Drones Over American Cities
Women don't have universal reproductive rights [in the US]. A 13 year old child in Mississippi was forced to have her rapists baby.
13-year-old rape victim has baby amid confusion over state's abortion ban
10-year-old rape victim forced to travel from Ohio to Indiana for abortion
Louisiana lawmakers reject adding exceptions of rape and incest to abortion ban
Nearly 65,000 pregnancies from rape have occurred in states with abortion bans, study estimates
If you've made it this far, just think for a moment about the girls, the women, the mothers, that could have been you, or your sister or your own mother. Reflect on this alone when Americans say they are free, they do not even have women's rights.
Medical debt cripples a large amount of the POPULATION [in the US].
The burden of medical debt in the United States
Their [US] children spend their time in school doing active shooter drills while also swearing allegiance to a flag every morning, something only Russia, China, and North Korea could dream of.
But after all those things... I'm sure it will please all of them to know, that after not having any real freedoms at all, being oppressed by a dystopian society with a propaganda machine that would make Goebbels blush, they can still call a black man a n*****."
When it comes to children, it's unambiguously immoral and illegal. With adults, I do think it's trickier.
It's obviously unkind and immoral to publicly post nude art of someone who has not consented. At the same time, I can see why it's tough to draw the legal boundaries.
There's no law against imagining, say, a celebrity crush naked. I don't think anyone is suggesting that should be illegal. In which case, should it be unlawful to draw a picture that you've imagined? What about a photorealistic one? If that's okay, why should using AI change that? Or is it the public sharing that's the problem?
To be clear, I'm not arguing it's okay. I'm just pointing out that, despite a widespread intuition that it's problematic, it's quite challenging to pinpoint exactly where it crosses the line from imagination to crime.
I could understand harassment charges if a person was posting the videos to the subject or their family to upset them. I could also understand a defamation case if they were claiming the footage is real. However, if they make clear it's fake and don't use it to harass people, it doesn't fit neatly into the current framework.
[deleted]
For CSAM, I agree. The risk is that it could spread and, in so doing, increase the market for content made with actual children. This puts children at risk of sexual assault or rape.
For adults, though, I don't think the same argument can really be made. I wouldn't like the thought of strangers jerking off to me, but I'm not sure that should make me a "victim" in a legal sense. That's true whether they're imagining me, looking at a photo of me, or looking at a fake image they've created or viewed.
I think it should definitely be illegal to use said videos to harass people, illegal to suggest they're real, and illegal to share real videos non-consensually. Beyond that, though, I think it's more debatable. It effectively boils down to, "You can't draw me naked (at least not if it's sufficiently realistic) because it upsets me."
In which case, should erotica depicting real people be illegal? I think most people would say no, because it's obviously not real. But even if these videos are clearly marked as not real, I think people still feel they should be banned.
I'm not denying that people feel an intuitive difference, which could well point to something real, but I do think it's tough to tease out what the principled distinction is.
Is that risk actually the case? I've always thought that e.g. fake diamonds or ivory resulted in the real stuff being sourced less?
I imagine it's very difficult to research ethically. I could see it going either way. Perhaps, if you allowed animated CSAM created without any children being harmed, it would enable people to control their urges without seeking out real images.
On the other hand, I suspect that tolerating communities that share such images, even animated or drawn, normalises and excuses the behaviour and could lead to escalation. Really, if someone has these urges, I suspect it's a very dangerous thing to feed.
Yes, that is clearly the main problem.
Why, though? I can't think of anything else that's legal to unilaterally create and view privately, but not to share with other adults.
Lots of things fit that description: you could write down a threat to bomb something on a piece of paper, but if it doesn't leave your house, it's not going to be illegal.
That's a fair challenge. I suppose the distinction I'd make is that you're not being arrested for the image itself in your example. If you called in with a bomb threat, you'd be arrested. If a newspaper published a copy of your letter, they wouldn't be.
I think we're both getting too distracted by the detail here. Publishing child porn should be illegal. Thinking about it shouldn't.
I agree that we can't criminalise pure imagination. However, if someone draws CSAM, that's a crime even if they never share it with anyone. Are we suggesting the same for deepfakes?
I understand what you're saying and broadly agree but I think there is subtle differences. Its completely legal to imagine whatever you like - even if that image is illegal in itself.
When it comes to drawings, it becomes less about the specific person and more about the concept in my view.
If you drew a celebrity or a random adult there's enough creative room to argue that it's fair and "inspired by this person, but not them" and therefore legal. In contrast to say children, where regardless of intent its unacceptable; which is why loli and all the other attempted workarounds for predators are still illegal despite the "methods" being identical to an adult.
AI is instead more about the intent to create photographs rather than art and could be more taken as misinformation for lack of a better word. If you asked AI for cartoon/drawn nudity depiction, I don't think it would have as much traction and this likely wouldn't have reached the point it has now. After all, AI porn of a fictional adult isn't illegal in itself.
The situation is uncomfortable though, the UK should absolutely ban the platform at least until it restricts life-like depictions. I don't think we should be able to misdirect people with realistic videos/images regardless of intent so easily.
If the primary issue is misinformation, though, wouldn't a prominent "fake" watermark address that? By and large, I don't think people actually believe it's the real person, nor do the creators make any attempt to suggest it is. I suspect most people who object would still object, even if the videos were clearly marked as fakes.
I dunno mate I think if you think making nudes of strangers without their consent is morally ambiguous I think it’s your morals that need looking at.
Couldn't you have brought yourself to read one more sentence?
It’s also unkind and immoral to make it in the first place. How would you like it if someone was sitting at home gooning to deepfakes of you they made without your consent? It’s incredibly invasive and creepy. Never mind the possible consequences of a data leak.
Do you want to apologise now for misrepresenting me, or do you have no moral qualms about that (which would be somewhat ironic)?
There are all sorts of immoral things that aren't crimes. I wouldn't want my partner to cheat on me. I don't think they should go to jail if they do so. Similarly, I wouldn't want someone to make a deepfake of me and jerk off to it. However, if they do that in the privacy of their own home, don't harass me, and don't share it as though it's real, I'm not convinced they should go to prison, any more than if they’d imagined me.
Nobody is suggesting this isn't creepy. My comment is focused only on whether it makes sense to criminalise.
I was quite clear that I was talking about morality, not legality. There are many immoral things that aren’t illegal, and I chose not to do many of them because they are bad things to do. I am not saying I think people should go to prison for doing these things, I am saying that doing them makes them bad people.
You have made clear with your reply that you see no problem with gooning to deepfakes you’ve made of people you know. I believe that’s morally repugnant and I am not going to apologise to you for that, sorry.
Then why on earth did you reply to my comment then, which was entirely about the law, as opposed to the dozens of others? More importantly, why did you suggest I had no moral issues with it when I specifically said it was immoral?
How much clearer could I have been?
The irony that you think misrepresentation is wrong, but you're more than happy to misrepresent others to make yourself feel good and others look bad, is incredible.
That's an outright lie. I think you know it's a lie. The fact that you said it anyway says something about you, whether you're prepared to acknowledge it or not. If you disagree, it should be easy to quote where I said that.
You are selectively quoting yourself now.
Your point was it’s obviously immoral to post it. My challenge to you was that it’s also obviously immoral to create it in the first place.
If you agree with me then why the argument? If you don’t then why the anger?
Never mind the detail around if you’re creating it with Grok then you are by definition sharing it with their servers.
I don't think you can generate nude images of someone - via any mainstream AI - not just Grok.
Yes, you can clearly have them dressed in a bikini, as has been evidenced by pictures of both Elon and Keir.
So, is the issue that it enables people to instantly publish the altered image to X? Or if, say, someone used ChatGPT to create an image and then uploaded that image to X is that still a problem? In which case Grok isn't the issue, it's the posting online of an altered image.
How do we legislate against that?
Isn’t that already illegal?
Isn’t this like banning cameras because they could be used for illegal purposes.
One thing we are good at is arresting people for social media posts so why do we need to obliterate free speech to achieve the goal here.
Then they can ban Grok without banning X/Twitter like they are planning to. This is just an excuse for them to suppress political dissent.
If X refuses to remove grok (which they are) then X must be removed. This ain't rocket science.
I mean that's not technically possible. Either X stops or changes Grok or Ofcom bans X.
Political dissent has plenty of other homes, none of them allow you to create your own child porn for a small fee.
If what you says is true then this isn't an issue that is part of OfComs remit, it is a criminal matter in which the police should be investigating.
Unfortunately Elon and twitter refuse to participate in UK law and so all we have really is turning off the child porn at the source. Its unfortunate but child porn really is that nuclear.
That would be an assessment made after a criminal investigation, yet the police have signalled nothing. Also in addition any individual who produces such material could equally be investigated and charged but to my knowledge no one has.
I never even used Grok, why should other users who just want to use Twitter/X be punished over it?
I signed up to a nice, useful platform, then Elon bought it and turned it into a cesspit of misinformation, hatred, and child porn. Why is it OK for me to have been punished, but not for you to be?
why do you consider losing access to child porn to be a punishment?
If I stood in the street offering to edit photos of children to undress them, I’d be in handcuffs within the hour.
X has been producing thousands an hour for weeks, and nobody is in handcuffs.
Make it make sense.
If it's thousands an hour why hasn't the IWF came out and said that?
Because literally everyone is making shit up and riding a huge wave of performative righteous indignation because Twitter Man Bad.
Someone took the "Put her in a bikini" meme a bit too far and did it with a child. That's literally the extent of it. Poor taste and arguably not even illegal unless done persistently to harass an individual. (and is trivially done with any generative AI and even traditional image manipulation software) But most certainly not child abuse material, and besides, we have existing laws to deal with such images if they meet a criminal standard.
The whole thing has just been used to form the ridiculous (and equally performative) false dichotomy that either: A: You want X to be shut down, or B: You explicitly are in favour of CP.
The “put her in a bikini” thing is illegal. Don’t downplay non-consensual pornography as a meme. It causes real harm and there is absolutely no legitimate reason to do it.
It’s been widely documented that there are indeed thousands, if not millions of images that are illegal to display under UK law.
Look, I’m against OSA and I totally get that the “save the children” argument is often a lie. But this is a very clear case of genuine and widespread CSAM.
While an entirely novel problem due to the powers of GenAI, putting someone in beachwear is not and never will be pornography, regardless of age - which is precisely why the people trolling otherss by doing it are choosing bikinis as opposed to nudity. Sure, it might be regarded as harassment, or even malicious comms, but would likely have to be the result of a pattern of behaviour.
Unless of course you want to argue that a nautral photograph of a woman or even a child on a beach in a bikini is pornography. In which case, You've just criminalised the holiday snaps of tens of millions of people.
You need to brush up on the law, because what you’re saying is simply not true. I’m not going to waste my time arguing with you if you haven’t even done the requisite research to speak on this matter.
Here’s a starting point - see SOA 2003: https://www.college.police.uk/guidance/sharing-photographs-of-people-intimate-state
As I said, there is a reason they're using bikinis as opposed to nudity, underwear, or re-posing people into sexual positions.
For the record, I dislike it.
https://www.legislation.gov.uk/ukpga/2003/42/section/66D
They are using underwear, as well as “covered in doughnut glaze”, posing people in suggestive ways, and making their breasts bigger. Just scrolling through the replies of Grok’s account shows all of this.
Bikini is just the most reliable prompt to bypass the (weak) filters.
if making breasts bigger is illegal then snapchat and instagram are done with all the filters they have that do that
Do you not understand consent?
My fiancée wasn’t convinced Grok is bad, so I demonstrated that I was able to sign up with a throwaway email address, download a picture of my fiancée from her instagram profile, and within ten minutes get it to produce deepfakes of her in her underwear, giving a lap dance and miming a blowjob. She was appalled.
There is absolutely no reason for this technology to exist. It’s vile and so obviously is going to be used for bullying, harassment etc.
And before anyone argues “we can’t do anything about that it’s just the way it works”, this is basically pollution and if a factory causes too much pollution, we don’t let you build it.
The question is can you do that with other AI?
For me the issue is still not the editing, but the posting of it onto peoples timelines.
It's the ease of use that is the problem here as before it took a lot of time to do it.
It needs stamping out ASAP.
No, all the other mainstream publicly available AIs have effective filters in place.
Distributing those kind of images is illegal, regardless of if you used AI or even just photoshop.
Because you are using a cloud service to make them you are automatically distributing.
This is more an issue of whether it is a criminal offence to distribute any photos of someone else or their likeness without consent.
If the photos depict that person in a sexual or intimate context, it is illegal as per the 2003 Sex Offences Act. It’s basically the same as revenge porn.
And so it should.
Having the ability to do this on someone else's timeline with a simple few keywords is the problem here.
Yes you can undress people in other AI, but they don't allow you in someones own feed.
X needs to implement a feature that is an opt-in to allow this for your own feed and media. Stopping anyone sharing Grok images or easily taking your own imagines.
Also, better checks for CSAM also but that is a problem in all social media.
It's a deliberate design choice by X/Elon Musk - they want people on that shithole platform to use Grok constantly so will never remove the option to generate text/material publically via replies - that's the USP of Grok. X is the equivalent of a troll site where they quite openly want people to be antagonistic towards each other (moreso than other outrage algorithms) and think they have an edge on other AI services by people generating material publically and 'contacting' Grok as if it was just another user. It was always going to end up like this - and Musk has been pushing the NSFW side of Grok for a while anyway. Their response so far has basically just been 'oh we'll limit creating illegal material to paid customers then'.
Considering it was originally bought to be a 'free speech' app - which, when you have no code of conduct on a website inevitably it is taken over by the absolute worst people - it's no surprise that the same userbase, without restrictions, are utilising AI in the worst way.
This is one reason why I think the Government should quit Twitter, and a ban at this rate could happen, the platform's rapidly being funded by bad faith actors, it has an AI tool that's founder keeps tampering with, and is now being used to generate content that's arguably illegal, yet alone immoral. What benefit is dignifying that with a platform?
Hence why Ofcom should step in.
I'd prefer a working X that doesn't promote sexual content, to having to use something like bluesky and it's truth being gated by the chosen ones.
I really hope they don’t threaten to fine X because the lack of authority Ofcom really have will only empower Musk and X.
A fine is one potential outcome so it’s not really a threat it’s just a valid disposal. That amount would be up to 10% global revenue or 18 million pounds, which ever is greater.
To which x will respond “no” just as 4chan did. Ofcom should save themselves the embarrassment.
OK, and then we ban it. Would you prefer we go straight for the ban?
Yes
No, if people want to use X then let them. Pains me to say it but it’s the place to go for football/sports updates in general. Ban/restrict Grok instead.
Also, 4chan was never banned despite the threat.
It’s rather odd that some people consider using Ai to create deepfake photos of people in bikinis as “free speech".
Although, Ofcom are useless. X won't pay any fines. Plus it would cause massive backlash if X was eventually banned in the UK.
I don't necessarily agree with the position but It's not so much the bikini photos, it's more the wider concept of it being the thin edge of the wedge when it comes to government censorship, at a time when the UK is arresting hundreds of people for making social media posts (...well if they're on certain subjects, you can call for the death of Jews all you want). X is deliberately refusing to comply because it wants to pick a fight on the matter and it seems like it has the American government behind it (Vance certainly). I suspect the ultimate result will be guardrails put in place to restrict some of this stuff, after a long drawn out battle that puts the government off picking any further fights.
Maddening people are still posting and using a platform like that.
Why?
Because it brings zero social utility to us as a nation. What are the advantages of Xwitter to us as a people? It foments outrage, provides an unregulated space for misinformation to fester and further agitate the social fabric of the country.
What are the social advantages of reddit?
None as far as I can tell
Really not much to be honest but it seems a little better at curbing the animalistic impulses of its users through content moderation and the subreddit model.
Oh wow, the left dug itself a hole so deep it got back to the right and is thinking in terms of 'nation' again. What is a nation btw?
As an aside: I am a naturalised citizen, and in my citizenship test freedom of speech is listed as a core principle of Britain alongside democracy, rule of law and equality.
The advantage of X is we have access to different views and opinions from across the globe, and within the UK, therefore upholding this core principle.
Yeah, Grok should NOT be allowed to let you generate CSAM (like that's not obvious).
But everybody's crying wolf over the whole of X, meanwhile nobody is saying 'let's ban Grok'. Instead everybody on Reddit just wants an upright ban of the whole thing, like that does any good to freedom of speech. Both open source and commercial generative AI will fall through the same trap and generate naked pictures, sometimes with better or worse guardrails against doing that, esp. of under 18s, and again, nobody's asking those to be banned, just X.
just ban it until they sort the pedo business out. its really easy to do, literally every other social media site does not have an AI child porn creating tool built in to it.
One charity said they found CSAM on the dark web which “appears to have been created” using Grok. No proof that Grok was indeed the source was even given in the BBC article, X was contacted and replied with “We take action against illegal content on X, including CSAM, by removing it, permanently suspending accounts, and working with local governments and law enforcemnt as necessary”.
What more do you want? What a bloody nothing burger, just an excuse to hastily get X banned because someone’s feelings are upset we can talk freely somewhere on the internet.
If I created a website tomorrow that created sexualised images of children, I would be getting a knock on the door from the police, and rightly so. Large corporations get to play by different rules to the rest of us rubes
I agree to both parts. You should be getting a knock, and so should the users that asked for and generated sexualised images of children.
Also, yeah, I agree that large corporations play by different rules currently and that should change. Before that, let's stop jumping to "Let's ban X" so quickly, and instead how about, let's prosecute the pedophiles that asked for those pics.
X allows a lot of free speech and different points of view, so it's better for bringing people together. Reddit is all about cancel culture and banning people.
If by free speech you mean a bunch of people paying to have the visibility of their posts artifically boosted.
Or by free speech you mean the owner of such platform arbitarily banning, shadow banning, or limiting the reach of anyone he personally deems unworthy of being able to voice an opinion.
Or by free speech you mean the countless bots that spam the talking points of hostile actors, and talking points of states that want to deliberately create divides within societies all over the world.
Thanks for your input though word-word-four numbers.
I can say for people like my fiance who is an artist it promotes her bussiness, pushes her artwork out there and allows her to get comissions and engagement with other artists. Going to conventions and seeing what artists are where based on their twitter posts is important for her too. Twitter is far more than just reform voters arguing that free school meals are somehow bad. lol
Arrghh nooo misinformation we're all going to die!!!
misinformation is absolutely fracturing society and you have to be blind to not see it
Honestly if this whole debacle doesn't lead to X being banned/blocked over here in the UK, then nothing will. It should've happened as soon as it was very clear it was a platform that solely existed to push misinformation and far-right talking points en masse. Then it should've happened when Musk essentially boasted about doing the same thing and waged a personal misinformation war against the UK Government (a few times now). If it doesn't get banned for now being a huge repository of public AI revenge porn and AI CP then I don't think our Government will ever have the balls to do it.
Nothing of value would be lost and I'm sure, if polled, the UK population would be in support of blocking that shithole of a site anyway. Other countries around the world with large userbases have started now blocking it (Malaysia & Indonesia yesterday) so I'd rather we get involved with this potentially global pressure campaign on the site.
This is aside from the Ofcom investigation, by the way - I can't imagine there will be anything that comes out of this except a long-winded warning and a fine. The decision should be made from No.10.
And what about Reddit and Bluesky pushing left wing misinformation and talking points or is it okay when your side does it?
Like what?
THB, some of the discussion around the whole X and CSAM stuff has been highly misleading, mainly because people read other misleading comments and parrot them back.
So you support mass censorship? A ban on Grok isn't enough, you want an entire social media platform banned
It's not mass censorship; that's a very silly point to make. There are plenty of social media platforms that have terms of service that don't effectively encourage their userbase to spread toxic and dangerous far-right misinformation or have a leadership in place that deliberately spends money and uses the platform to destabilise Governments across Europe, Canada etc. That doesn't mean we should allow one horrendous one to exist and continue to get worse and worse, and more belligerent against Governments across the world (not just the UK but so many other examples). If people move from X they can just go to another social media website - oh no, wait, they probably can't; because they're mostly far-right agitators who have been given their platform via X itself. And the world has regulated against them for many decades before this.
I honestly could not give a fuck - we are in an era of American oligarchs utilising their wealth and media platforms to spread the same ideology to the UK and Europe that has effectively fucked the USA; we've all seen it coming, nothing has happened, and now just because these odious individuals keep hiding behind this spurious idea of 'free speech' (which they obviously don't believe in themselves), that we should just keep letting it happen?
There's a double standard anyway; I can guarantee you that if Weibo was in the position that X were in in Western countries with such a belligerent owner on the world stage people would be calling for its block under security concerns. Would that be mass censorship? TikTok already has been and the Americans bought it out. These social media platforms have an obligation to adhere to the push and pull of individual Government regulations and X have been taking the piss for years - they clearly think they're above most Governments and aren't guaranteed to be safe from regulation.
"don't effectively encourage their userbase to spread toxic and dangerous far-right misinformation" - what you call "far-right misinformation" is whatever you disagree with. You support censorship of right-wing political views and opinions
Just admit you support mass censorship and government overreach already
lol
Remember Farage can use this to do what he wants too when he gets in.
How about we make it so no government can oppress people.
If Farage wanted to do that, he'd do it anyway - much like MAGA in the US didn't exactly need a pretext to start shaping the US how they wanted to. It doesn't really set a precedent - much like how the Dems I'm sure now regret not prosecuting Americans involved in the electoral fraud big lie.
It would be better to be pragmatic to try to stop the next election in this country being an utter shitshow with misinformation emanating from a site we already know was a massive vector in causing far-right riots across the country. If X wanted to stay as a platform - perhaps they would put in better terms of service and actually ban people with large platforms known to cause trouble, like Twitter used to do? Ah no, it's because it's specifically designed to be a far-right site and they welcomed back known conspiracy theories, neo-nazis and white supremacists with open arms as soon as it was purchased by Musk. And they know that they can only really do that with almost zero moderation put into the site.
This is like the equivalent of your mum calling herself the internet police and going after 4chan for bullying you tbh.
Not really. Ofcom can cause significant financial damage to X.
In what reality?
X can be fined, it can be removed from ISPs and the UK operation of X can be stopped.
VPN usage is already proliferating from the Online Safety Act and this will only accelerate that.
Regardless, that would make the usage of Twitter much lower.
Which effects revenue.
X currently hemorages money, the profit they make from advertisements is net negative. They are really power on because they have positioned themselves as the ‘town square’ of the world, and presumably that can be monetised outside of traditional advertising revenue (or that’s their plan anyway)
If one person can run a child porn empire and entire governments cannot stop them because they have too much money, maybe they have too much money.
Well, you can only implement restrictions that are practical. Sure, a minority of people may still be able to access X if they are desperate; not great, but still far better than the current case. Even if you totally banned it, people would be able to go on holiday abroad and access it, but that doesn't mean it should be allowed in the UK.
My point is that banning websites is like banning numbers. It just cannot be feasibly done. This whole sub roundly (and correctly) criticised the Online Safety Act as totally impractical and yet equally impractical solutions like banning X are somehow cheered on.
I think you're misrepresenting the criticism of the OSA. It isn't (at least, as far as I've seen) "please don't ban our child porn sites", it's "please don't ban wikipedia and force me to hand over my identity to send a direct message on bluesky".
So fines it won't pay and an isp block that you can get around in seconds without even having to use a vpn
You can't get round an ISP block without a VPN.
You very very easily can
Go on then, how?
Change your dns to cloudflare or google
That doesn't work. The ISP has the means to ban URLs/IPs if it wishes, regardless of your DNS setting.
Not at all, hardly anyone is complying to these laws, even recently in Italy they tried to fine a social media company X amount of their global revenue and the company just ignored it, forcing the hand of Italy to cut itself off and become a social pariah or just rethink their ability to execute these things
Why would blocking a website cause an entire country to become a social pariah? You're making it sound like X is as important as a sovereign state; it really isn't.
Not politically but definitely socially, like if the US had banned TikTok, they’d be ‘out of the loop’ with the rest of the world with the trends and memes, which weirdly nowadays would cause them to lose positioning in the larger world and even effect the stock market.
Nobody wants to position themselves as like the unc at the Christmas table who doesn’t get the joke.
This is why the only way to combat a site and oligarch as odious as X and Musk is to threaten them with a block - and actually go through with it. Musk tried to ignore a Brazilian softball order to ban users who engaged with election misinformation over there and contributed to their own capitol attack after Bolsonaro lost, Musk ignored it, he ignored Brazilian legal obligations and spent months trying to have X dodge around legal action before the site was just completely blocked. He then spent about a month trying to let X be accessible in Brazil through various tinpot means - and personally tried to dox and harass legal representatives in Brazil using his platform - before finally just paying the fine he had to pay (miniscule in the grand scheme of things) and acquiescing to legal requirements.
A block/ban is the only language that rattles the site. They know they can ignore fines and warnings but they are shat up when they face losing millions of users.
But does it really matter if X is banned?? Don't pay the fine, you don't get to make money here.
I don't want unmoderated social media allowing degenerates to undress children thanks.
Then Italy should do something about it. It has the powers to do so.
Being a big company doesn't mean you can allow the distribution of illegal images.
Consequences will never be the same!
Here's a write up that references captain compliances story: https://www.biometricupdate.com/202601/groks-image-processing-feature-is-a-mass-violation-of-biometric-privacy-laws
[deleted]
Gone of the halcyon days of Facebook prodding.
I have removed all my pictures off social media sites I can.
The Technology Secretary Liz Kendall said the law would make it illegal for companies to supply the tools designed to create such images.
It makes me wonder why labour haven't brought in a law to make it illegal for a company to supply the tools that can be used to kill someone.
https://en.wikipedia.org/wiki/Firearms_(Amendment)_(No._2)_Act_1997
https://en.wikipedia.org/wiki/Firearms_(Amendment)_(No._2)_Act_1997
It's either all okay or none of It's okay. Grok certainly isn't the only AI capable of doing this
The OSA was disgusting and stupid, and has caused an increase in suicides among kids who lost access to support services and networks they connect to friends with...
This sort of stuff is just icing on the cake. Banning X would be fucking stupid. (to clarify I am against grok and genai in general as my fiance is an artist and genAI puts her livlihood at risk, I am not supporting grok at all, I think banning twitter is stupid when countless other AI bots can "strip" people anyways, grok needs to be properly moderated but banning twitter is such a stupid solution. People who want to do that stuff will just bypass the ban and do it anyways. Bans only hurt innocent people who have no horse in this race to be quite honest)
Most of my friends use X, I stay connected through X to see what social stuff they are doing and where I can meet up with them at conventions etc, I use it to buy from artists custom artworks, my fiance uses X to push her art and get comissioned and network because the kind of artwork we consume and engage in isn't exactly stuff you'd slap on your linkedin to be quite honest.
Banning X will just be another issue and the use of VPNs will soar to even greater heights.
Policing what sites the public can use is dystopian and extremely horrifying. Honestly just dangerous. I am disgusted to see people actually calling to ban twitter and other social media sites. Are people really that okay with locking kids out of support sites and stopping people from socializing in their hobbies like sports and anime and gaming and artwork? I don't see why this is such a controversial topic.
Parents should parent better and sites should remain accessible. It really isn't a hard concept to grasp. Having to fork out for a VPN every year with the state of the economy as it is doesn't do anybody any favors but the government literally leaves us no choice at this point. It's sickening to say the least.
Knowing how toothless Ofcom and this Labour government are, the most I see happening is a six-figure fine and regulators telling X Corp not to do it again.
Shit like this is why the Online Safety Act was passed in the first place, yet the government seem far more concerned with mandatory
age verificationmass surveillance than actually protecting the public from involuntary pornography and CSAM.We elected a leader who is too much of a wimp to stand up to Trump.
This is a gross overreaction, they already put it behind a paywall, I don’t see what else they could possibly do???
How about not allowing a simple text input to put indecent pictures onto someones timeline? Make the ability of people using that an opt-in for peoples timelines.
Reporting stuff should work instant in these cases also, taking it off someones timeline if it is a posted image.
I was being sarcastic lad.
ah, my apologies
It's about time we stood up alongside those other bastions of freedom that have banned X - China, Iran, Myanmar, North Korea, Russia, Turkmenistan, and Venezuela.