I was the one who proposed deleting /r/golang. It is not some official Go or Google position.
As much as I used to love Reddit and was addicted to it, my personal position is that Reddit is no longer a trustworthy platform (if it ever was).
Editing user content is beyond offensive. I never even considered such a thing in my years of running LiveJournal. That is a major violation of user trust and trust in the platform.
If Github or Gerrit or Google Groups or Google's SMTP servers were modifying our code or mailing list content, we would ditch them in a heartbeat.
We shouldn't demand less from Reddit.
But because I learned that /r/golang existed 7 years ago (before I or other Googlers were even involved with it), I no longer propose deleting it. But I think the Go project should disassociate from it and give it back to the community as an unofficial space, as it used to be.
It's just too unreliable of a platform to be official in any regard.
Now I'm brainstorming how one might build a federated Reddit with public, signed mutation log, ala CT or other chains. And then multiple UIs could render the same public & federated data set.
You have every right to take this position, and if you feel strongly have no obligation to stay. What I cannot understand is the next step of proposing to close the entire subreddit. Don't want to moderate it anymore? Then don't. A subreddit is made up of it's users, not just the moderators.
Did you learn it by scrolling down on the front page and looking at the text below the list of moderators? I sincerely hope you aren't trusted to run a public community ever again if your first reaction is to start an off-site proposal to shut a community down.
So the only thing holding you back from deleting a subreddit with tons of information, 25.000 subscribed users and what looks like a rather active and healthy discussion isn't that it would be a dick move to both golang in general and the community, but that you're not actually the owner.
Not attaching your brand to poor service is also part of managing a community.
If AWS started MITMing client HTTP connections and injecting code, would you be surprised people moved their official websites off the platform, even if a lot of people knew the old IP address?
That's not childish, that's not doing business with bad partners, and Im not convinced a space that started as official could ever truly be made "non-official".
What I mean is that the subreddit doesn't belong to bradfitz, or Google, or the golang creators.
It has never started as official because the creator was /u/uriel. The Google guys may have joined after, but the fact is that it is not their community. They're not even paying a single dime for it.
If AWS started MITMing my connections, absolutely, I would drop it. But the analogy doesn't hold because /r/golang isn't bradfitz's subreddit.
The proper and only response would be to drop moderatorship, leaving in a thread his reasons, and leaving /r/golang as it is. As it stands, the only thing stopping him from doing that is that the he's not the original owner.
And, you could also argue that when a community reaches a certain size, it doesn't belong to you anymore. It belongs to that community. To delete it is petty and childish.
Hey bradfitz, thanks for coming over to HN. I created an alt account just to appeal to you: Please do not delete r/golang. I happen to like the subreddit and the community of people on there. It's great place for me to find news & learn more about the language, and it would be very hard for me (who doesn't happen to be in a physical environment that fosters the discussion of the language) to find a similar community online.
If you happen to dislike reddit, please feel free to step down as moderator and move on. Please do not use this opportunity to grandstand and destroy the community for the rest of us, who do not happen to care about the drama surrounding the CEO (personally, I think it was childish of him, but I would not go to the extent of disrupting the community for others).
The symbolic gesture that the project is disassociating is okay. Deleting the repo is not.
>It's just too unreliable of a platform to be official in any regard.
The /r/golang subreddit is for devs who want answers and who don't want to be held up by the whims of stackoverflow mods. It isn't rocket science and even if we can't trust the answers, why would anyone edit the answers? If you remember, the reddit CEO only edited the usernames, that too, I am sure it was a prank.
If anyone edits the messages which we post on /r/golang, it will lead to the answer being invalid and or something else, the world isn't going to collapse, there won't be congressional hearings just because some dev wasn't able to run gofmt and the answer posted by another random guy on the Internet was changed and can't be "trusted".
We want answers on /r/golang and it isn't much about trust.
The question I've had is why didn't reddit SREs stop the DB edits as an attack. Even using valid creds, the queries should've been anomalies and logging should've picked up the unusual access pattern.
There's three possibilities that come to me:
1. The CEO explicitly told reliability/security staff to ignore his edits. Policy fail of epic proportions.
2. The edits were performed using designed tools and queries, which means reddit had before now intended to make silent edits to user data as part of regular business.
3. Reddit lacks the internal safeguards to know if user data was unexpectedly written to. This option is dangerous to hosting code on reddit, because an attacker could silently, maliciously change examples without reddit ever detecting the damage (or users seeing an edit mark).
I expect that we're in case 2, followed by 3 and then 1 (in terms of likelihood), but none of the ways that a large tech company could have this happen are reassuring about P&P at reddit, or make me want to rely on them for hosting a community.
You've been working at Google too long. Reddit has on the order of a few dozen employees, not all of them technical. Welcome to how the rest of the Internet works.
I've never worked at any company with more than 50 employees, and have worked on a team with as few as 5 engineers. Even without an explicit SRE role, someone is doing that work -- keeping the servers (reliably) on the internet and performing, mitigating attacks, etc.
You should be able to tag the row with a dirty flag on second (or later) write using rules internal to the DB without incurring a large performance penalty, if you already have any kind of sanity checking. So at a basic technical level, this is bad design or an intentional capability.
Similarly, even on the team with 5 engineers, our production DB didn't have a way to directly access it, and tunneling through an API box to get network access would trigger an access warning to the appropriate person, because you shouldn't be accessing prod API servers directly either. (Similarly, spinning up a new instance in a security group to inherit permission.)
This isn't detecting advanced exfiltration techniques using information theory... It's basic network topology and server monitoring. And I suspect that reddit has at least a couple ops guys.
This really smacks of intentional capability, followed to lazy design.
Either you are the outlier or I am - I'm honestly not sure. But having consulted with a dozen or so digital agencies over the past year not a single one had the sort of monitoring that you're suggesting. This doesn't mean that it's a bad idea, quite the opposite, but in my experience the reality of a small to medium sized agency is that if a client isn't paying for it than you're not working on it.
It's what I consider the difference between a software developer and a computer-systems engineer to be.
I'm likely the outlier, because my early mentors (college + first couple years of career) were mostly aerospace engineers or chemical processing equipment engineers, where safety is critical. Code can kill in both those fields. (They themselves had adapted a lot of systems and failure mode analysis from mech engineers in the same field.)
I don't think that the mitigations I pointed out require much technical work though -- most of them can be as simple as a single setting, extra program installed during build (and maybe a couple config settings), or a single cloud monitoring rule (eg, detecting launches). They do, however, require adhering to policy, discipline on the part of staff, and analyzing the full scope of your tech (and how it interacts). I guess my point is that these things often aren't done because of ignorance or haste rather than engineering cost (or legitimate need to be done faster).
It's just we have poor standards for what "engineering" is in tech, so we don't cross our 't's and dot our 'i's the way a mechanical engineer is expected to.
But that's no reason not to try and raise the bar.
> Even without an explicit SRE role, someone is doing that work -- keeping the servers (reliably) on the internet and performing, mitigating attacks, etc.
To be honest, not in any of the several places I've worked. And I'm a security engineer. Lean startups are desperately trying to prove their product; time is spent on reliability, performance, and security almost entirely reactively and not proactively.
I'm not saying this is a good thing. Bad security (and reliability) posture early on inflicts multiples of the original mitigation cost for years. And yet, it's rational. Time spent securing systems from your own employees when you're tiny is time you're not building your product and finding your market. It's a perverse game of Russian roulette: there are 99 bullets and one empty chamber. The analogy is going to break down here, but spending time and money removing one bullet (1:98 chance of death) is less effective than adding another empty chamber (2:99 chance of death).
Especially when having direct access to the database is insanely useful to fix the problems that moving fast and breaking things inevitably causes, only a tiny fraction of startups are going to enact the kinds of measures you mention.
I don't disagree with anything you've said, except that I disagree that it's actually a rational choice based on facts, rather than an apparently rational one based on poor information and training -- again, none of the mitigations I pointed out are really expensive.
Let's take prod DB access: Im not saying make it super locked down, Im saying set your DB security group to only be connected to from your API security group, and set an email warning when an instance launches in that group. Doesn't stop your engineers doing it, but it makes it pretty easy to shout across the office "Hey, who is fucking with prod and why?" Takes like 30 seconds to configure on AWS.
There are serious questions about why your CEO can launch that instance though, and that sounds like massive policy failure. Again, restricting the CEO IAM from launching prod instances takes 30 seconds. (And all your non-engineer accounts should be IAM restricted!)
Ignoring that your probabilities don't work out, Id argue that the situation is really remove 5 bullets or add 1 chamber, and people pick the chamber purely because it's constructive, not out of genuine cost-benefit analysis.
"If I had 3 hours to chop a tree, Id sharpen my axe" -- I think startups are too hasty to chop, because that's being productive, right?
> If you remember, the reddit CEO only edited the usernames, that too, I am sure it was a prank.
He edited the mention of a username in a comment and completely changed the meaning of the comment by changing the target.
I'm sure he thought it was a prank, but that shows extremely poor judgement of the CEO of a company that calls itself "the front page of the internet". Just imagine how unacceptable it would be for the Facebook CEO to edit someone's status, or Twitter to edit tweets.
I know, it was terrible. What's more terrible is that he accepted doing that. now _that_ is a lack of judgement. Would the world have noticed that some "fuck you" comments have been changed in /r/TheDonald's subreddit? nopes!
> Now I'm brainstorming how one might build a federated Reddit
> with public, signed mutation log, ala CT or other chains.
> And then multiple UIs could render the same public &
> federated data set.
I'd make the subreddit unofficial, too but for other reasons. It is a very unwelcoming place. I'm not the first one who's saying this, but it's really a shame how toxic the /r/golang is.
In order to keep this from being swept under the rug, I can say that I highly empathize with you. I will also say that this issue has improved greatly. When I used to hang out in #gonuts it used to be worse.
I blame this on the egos and the expectations of early adopters and contributors. They came from a point of senior systems programmers. As the language grew in popularity, they were unprepared psychologically for dealing with new or junior programmers: the level of questions put forth were things the original crew had learned early in their careers and took for granted.
It is often this way in early languages. We need to always make room for those who are just learning, or are not experienced as those who initially created the language or space. I have seen this improving in the Golang community. The more new people come in, the more it will improve.
Is it? There is some hivemind like the rest of the internet, and tech discussion in general (including here), but the value of the people who frequent the sub should definitely override that. It's still much better then stackoverflow etc.
I do not know which universe you are living on. /r/golang is one of the best programming communities I have ever seen. golang-nuts is the worst community I have ever seen.
This is double standards of the epic proportions, on the "official" Google group, you have a person call reddit "hive of scum" and what not and that's fine? doesn't the CoC shoved down our throats tell us to respect others? to not be an ass? or does it not apply to the "elites".
let me reiterate again, reddit might be an unwelcoming place, /r/golang is the most welcoming place ever, I posted a query of AJAX, I got two answers within half hour, one with vanilla JS another with jquery, if I had posted to your beloved google group, they'd have pointed me down to some obscure link and asked me to chase my problem on my own and that would have done probably being snarky.
Get over yourself. In the grand scheme of things, a guy editing a handful of posts on a public forum is beyond trivial. There are about a billion things going on in the world which are worse.
> Now I'm brainstorming how one might build a federated Reddit with public, signed mutation log, ala CT or other chains. And then multiple UIs could render the same public & federated data set.
Also the infrastructure which allows Reddit to exist is not. All those scaling issues they've "solved" over the past ten years? Have fun with those if you're trying to copy and paste the repo..
Make a federated replacement for Twitter while you're at it!
It would be interesting to figure out how to tackle, well, "voter fraud" in that kind of system. I see lots of people complaining about how Reddit is vulnerable to upvoting brigades but I don't see anyone proposing solutions.
You don't need to rebuild reddit. Just add offsite, trusted backups of comments. If you want to be fancy, allow redditors to enable monitoring of their comments with a bot command, and get notifications in their inbox.
Using /r/golang is the community's call, why do they care about federated things? It isn't like we discuss some sensitive information over it that we need to "trust" reddit. Do we really "trust" Google/FB?
it is about convenience. Almost all traffic for my FOSS projects has come from Reddit and because of Reddit, from twitter.
>Now I'm brainstorming how one might build a federated Reddit with public, signed mutation log, ala CT or other chains. And then multiple UIs could render the same public & federated data set.
The tricky thing is that the admin team is still doing things that you want and need out of a reddit alternative. For example, ensuring that child pornography isn't getting shared. The question is how to set things up so that the right content gets removed while ensuring that how we decide what the right content to remove gets isolated from political pressures that want to influence the narrative.
You "just" need (yeah, easier said than done) to ensure that any intervention from an admin is publicly logged. If admin "foo" removes some content from user "bar", it should just be made visible, something like "user foo removed offensive content from user bar" or "that comment was edited by user foo". The problem here with reddit was not the fact that content was edited, it was that content was edited without anyone knowing it.
Agreed with the above, but also: in this world of "free" services (where the users are farmed like honeybees), voting with one's feet is about the only way to send a message that a company will actually listen to.
I see no problem with individuals leaving, and I see no problem with advocating a shutdown or at least encouraging other people to leave. Work out what the community wants, and go from there.
(I don't use go, but from my personal "this cannot be allowed to stand" perspective: I would love to see communities that are _not_ about hot-button topics stand up and walk out.)
>Now I'm brainstorming how one might build a federated Reddit with public, signed mutation log, ala CT or other chains. And then multiple UIs could render the same public & federated data set.
Your post to delete /r/goland triggered me to actually work out an idea for this ;)
The idea would be to use importers to provide reddit and other sources like mailing lists within the site and at the same time provide native content through Pubsubhubbub.
Of course everything uses Ed25519 Signatures for verification, including user-content.
Tho atm it's merely a pipe-dream with a short and shitty readme.
I would contribute to a federated non-repudiable system.
Moderation can be just as auditable. (Perhaps with some emergency actions that then require a follow up consensus to make permanent)
On some occasions the original content must be purged, but the signatures can remain. Communal censorship is vastly different from changing the content.
Worth noting that moderators, admins, or community-team level employees do not have the power to edit comments – I believe this was done at the database level. Signed comments is a good idea though.
How is that not almost exactly what he is suggesting? His just includes that Go project also distance themselves from reddit, which is more than reasonable.
As much as I used to love Reddit and was addicted to it, my personal position is that Reddit is no longer a trustworthy platform (if it ever was).
Editing user content is beyond offensive. I never even considered such a thing in my years of running LiveJournal. That is a major violation of user trust and trust in the platform.
If Github or Gerrit or Google Groups or Google's SMTP servers were modifying our code or mailing list content, we would ditch them in a heartbeat.
We shouldn't demand less from Reddit.
But because I learned that /r/golang existed 7 years ago (before I or other Googlers were even involved with it), I no longer propose deleting it. But I think the Go project should disassociate from it and give it back to the community as an unofficial space, as it used to be.
It's just too unreliable of a platform to be official in any regard.
Now I'm brainstorming how one might build a federated Reddit with public, signed mutation log, ala CT or other chains. And then multiple UIs could render the same public & federated data set.