Of course, there are plenty of project where judgement Is thrown out the window in favor of adding a buzzword to everyone’s resume. I’ve heard it called “promotion-based architecture”, as in you pick the technology most likely to get you promoted. (If that works, it says all sorts of not great things about your organization).

Regardless, I don’t think the availability of tools is the root problem. It’s and industry-wide lack of emphasis on identifying and understanding the problem first.

I see everybody around me moving to cloud, without really good explanation why. Only reasonable thing I can see as an pattern is that cloud experience on top of data things gets paid 30% more. It made me consider cloud a lot.

I was considering switching to cloud, just so I can put in my CV "experience with migration to cloud".

For next person commenting that it makes sense: it doesn't with 200gb database and super predictable workload, growth and usage.

Depends on the company. I've been working for marketing agencies for the last 15 years and they're generally staffed by, at most, 1 IT person who is in charge of a third-party vendor relationship that offers managed IT services. Those IT resources (internal or vendor) don't specialize in data and often don't know how to deal with it well, predictable workloads or not, and often offer up solutions which are not appropriate (cost or otherwise) whereas there are managed solutions from cloud vendors that do. BigQuery, for example, handles compute and storage for you and rolls it into one reasonable query price. No need to worry about any management of anything there and just sit Data Studio (included) or any other BI tool (Tableau, etc) on top and you're good to go.

I get your skepticism and welcome it, but you're being a little rough. We're cloud native at my current company (I did a partial cloud migration at my last which was completed after I left) and it makes my life leading a Data Engineering and Data Science team MUCH easier without upfront hardware/software costs or long-term contracts, which were STAGGERING and left us with much more difficult to maintain, upgrade, etc hardware that took up most of my job as opposed to almost none of it today.

YMMV.

For anything above what we have I think it weirdly depends on country/ salaries.

In US, 100k per year is no brainer for cloud as even 1 FTE would cost much more.

In non-western Europe, 100k is deal killer as you can hire 2 senior DBAs and you still have enough money for quite a reasonable server.

Edit: sorry I see that you are the same person I was replying to. Then to understand it better, who is consumer of so vast amount of data?

That is something completely different from migrating your inhouse applications to the cloud by introducing kubernetes. I think your example show nicely that there is not one solution for everything. As an industry we have to put way more emphasis on deciding tradeoffs by understanding requirements and risks.

People just buy into the "cloud" marketing. They don't have the ability to think and reason, and so don't understand that "cloud" just means "renting someone else's computer."

I built a complex in-house medical system. Quick, reliable, and liked by the users.

I was in the middle of adding a major new feature when all of the management in the IT department quit. The new people immediately decided that the whole thing had to be "in the cloud." I was removed from the project, and they hired three people full-time to rebuild it.

That was three years ago. The new system is still not online. The users are still using my old system, and the feature I was working on never got added because my presence and input was not welcome because I "don't understand the cloud." So I got moved to other projects.

People talk about "the cloud" with the same fervor and language as members of a cult. And you will be an outcast if you dare challenge their way of thinking.

Moving me out brings no advantages to either part of the org. In fact, it's probably a disadvantage to IT because it has to burn three people from its allocated headcount. Where previously, it had a bonus person that didn't come out of the IT budget.

The difference is the abstraction layer. I would define “the cloud” as a layer that abstracts away the physical infrastructure. (Which works until it doesn’t but that’s a longer comment.)

You can even apply “the cloud” to your very own fleet of computers with the right software.

Like I said in the original comment, it’s a specific solution to a specific problem. Whether it’s right for your system I have no idea.

If it's on-prem, it's not a cloud, it's a fog.

At least, that's what I've been calling some of my deployments.

It sounds you're trying to dismiss and downplay solutions to problems you're oblivious to.

Even if you want to simplisticly frame "the cloud" as "renting someone else's computer", keep in mind that:

* It's not one computer but as many as you'd like, and get them at the click of a button,

* These are computers which are managed 24/7 by a team of highly trained specialists,

* These computers can be located anywhere in the world and simultaneously in multiple regions,

* These computers are designed to be fault-tolerant and handle (and survive) way more stuff that can conceivably be thrown at them.

> People talk about "the cloud" with the same fervor and language as members of a cult.

I'm sorry to say but you sound like you're militantly opposed to a solution to problems you either don't understand or refuse to understand.

"The cloud", even when following the simplistic and clueless belief that it's just "renting someone else's computer", solves whole classes of technical and business problema that your box under your desk cannot solve.

By the way, moving 'to the cloud' never means a specific thing because people have made words (intentionally?) vague to the point where you have to explicitly specify all the factors you take into account with your work in order to figure out which 'cloud' they had in mind.

Running a static workload doesn't require elasticity, but a 'cloud' isn't just elasticity. If you want "a program on a server with some storage that is always there" without having to deal with hardware, software, networking, storage, backup, maintenance, upgrades, service contracts etc. then an arbitrary virtual machine where someone else takes care of the rest makes total sense.

I'm not a bean counter, all I know is those guys at my last job would rattle off about it like zombies. IIRC its a tax thing

Is that really the only benefit?

Depending on taxation rules you might want to buy or lease.

In that case you certainly aren't looking very hard for that explanation.

The rationale of moving to "the cloud" is crisp and very well-defined, to the point where even AWS makes it their point to explain in very clear terms in their Architecting With AWS courses.

Basically it's all about being able to scale without bothering with procurement, not having to manage everything down to power bills and rentals, turning big capex into small opex, and being able to deploy globally and reliably with a click of a button.

That, however, comes at a steep cost. The more serverless you get, the higher the price tag.

Also, beyond a certain scale you're better off going back to managing your own infrastructure.

> Only reasonable thing I can see as an pattern is that cloud experience on top of data things gets paid 30% more. It made me consider cloud a lot.

People with cloud experience are paid more because they can deliver more value. A random guy with, say, experience in EC2 and CloudFormation and CloudWatch, can single-handedly put together a robust fault-tolerant webservice that's deployed globally and automatically scales to meet any demand fluctuation.

Let me fix that for you! :)

I work for a Fortune 200 company who has their own data centers and has for decades. We just completed the construction of two new data centers about four years ago at a cost of $110 million dollars. Those new data centers are now at 70% of their capacity for power requirements. You should take a look at the specs on Intel's latest server-class processors. It's insane! We're talking half horse power and higher for each processor - and you want blades full of these things and then a rack of those blades and a row of those racks! Data centers now consume more power than industrial manufacturing! Most maddening of all - most of that power is going to go to heat, and guess what? You need giant chiller units to cool the place down. The power costs alone are insane.

You also lose agility. Need a new server? Go to your favorite cloud console and provision it. Better yet, use a serverless architecture and don't even worry about servers! Your own data center? Right now there's a 4-6 month wait time for new servers and storage equipment, and an 8-12 month wait time for networking equipment. Not exactly agile, is it?

You already know you need to staff to manage and patch your servers but you're also going to need staff to procure and manage your software licenses - and those licenses aren't cheap! Don't forget the ongoing tail - you pay for the license and then you get to pay 20% per annum forever after for that license. Those licenses also restrict your agility - you're pressured to use the software that's licensed in order to get better value. Try working with a procurement-based architecture!

You think you're going to avoid the software licensing hell by using open source? Good luck with that! That means you are responsible for the packaging, distribution, and support of whatever it is you're utilizing. That's more staff you need. They will inevitably miss patching some critical vulnerability that's going to land you in the news!

I can go on, and on, and on. But here's one thing I can say about applications that we've moved to the cloud: they cost 30% as much to host in the cloud as they do on-premise, and that's not even accounting for the entirety of all the costs I enumerated above! We have ultimate flexibility and agility, and we can quickly utilize managed open source platforms to solve business problems. On-prem? You lose all of that.

Faster, cheaper, better - that's why people are moving to the cloud.

Sure, that all works, as long as you have infinite money. My last job was at a company that was heavily invested into a serverless architecture, using the full suite of AWS tooling. They did an audit of one of their codebases and they found that generating a particular piece of data cost $20, almost entirely in fees to AWS. The company's roadmap involved scaling up the generation of this data, so they embarked on a huge process of optimizing and refactoring, to try to get the cost down. This effort tied up probably 40-45 engineers for at least six months. Probably cost them roughly 3 million dollars in salary alone, let alone opportunity cost in terms of other projects not delivered. No one amongst the management ever seriously considered moving to an on-prem solution, even though it was blatantly clear that the only value the company's massive usage of AWS Lambda, SQS, RDS and other managed services was delivering was additional profit margin for Amazon's balance sheet, and resume line items for the engineers who now got to check off the "used Terraform" box when applying to their next job.

Every app I've migrated to the cloud so far has resulted in a 70% cost reduction, but I'm also not trying to take my entire application portfolio to the cloud. To wit, we're not planning on abandoning our data centers anytime soon and that's not just because we can't move our applications to the cloud fast enough. At this point in time we recognize there are some applications best kept on-premise - but they're a minority, maybe 20% tops of my application portfolio. That means 80%+ of my application portfolio can be moved to the cloud, makes sense to move to the cloud, and I can save 70% on costs by doing so. That's a fantastic deal!

You do raise a fair point - what about the applications that are more economical to host on-prem? In my own portfolio I estimate 20% or so of my applications make more sense to host on-prem. What to do about those?

The most practical option is to mandate every application you build in the cloud have a plan for how to run on-prem. Note those that would be very difficult to run on-prem - those are your key cloud dependencies. For example, in my own portfolio I have a couple of applications depending on Lex and therefore would be very difficult to host on-prem.

Pay special attention to your so-called "crown jewels" - your apps that distinguish you from your competitors. If those were dependent on a key technology such as Lex and would be difficult to migrate on-prem then you need to identify other cloud providers you could migrate to. In this particular example I know Microsoft's and Google's clouds both offer similar services.

If you design and build your applications with no thought of the future then you're already in a bad position.

On-prem or cloud, why would I be waiting until 1 month before launch to provision anything?

You're pretending like on-prem is full of all these physical limitations, but Amazon, Microsoft, Google, Oracle, et. al. can just magically make hardware appear out of nowhere. It's the same hardware, subject to the same supply limitations, and I'm going to be paying either way. Either I pay more per hour for my cloud instances, or I pay a bunch up front to provision my on-prem servers.

Cloud makes sense if the load on your service is intermittent or unpredictable enough that you're okay paying AWS's premiums in order to have the ability to scale on demand. But if the load on the services is knowable, then hosting that service on the cloud is paying a 25% markup for no reason whatsoever. AWS is by far the most profitable portion of Amazon's business units. Those profits are coming from somewhere. They're coming from your business.

Just as an example, consider Hadoop and its ecosystem. Another example is Elasticsearch. There's work involved in making these platforms production-ready and keeping them and their dependencies up-to-date and ensuring all your applications in your portfolio are using the same set of software. Most organizations are not prepared to take that on. So they turn to a vendor who will do it for them for much cheaper than they could do it themselves.

That's why the saying is "free as in freedom, not free as in beer" because open source software is not free as in beer. At least not when factoring in the total costs.

The packaging argument was made in the context of complying with software licenses. Responsible companies which perform due diligence on the software they run have to track the provenance of all software that ships as part of their dependency closure. If you want to ensure you're not vulnerable to lawsuits then you can't simply apt-get stuff from a PPA. You need to build it yourself, and track exactly what goes into that build.

Meanwhile, if you opt to run managed service from a cloud provider, you don't have to bother with that because that's not your problem (or liability) anymore.

Yes, and it does indeed apply to all readily available container images.

In fact, it applies to any and all software packages put together by third-parties.

I mean, who in their right mind downloads random stuff from the internet and expects to just drop it in production software which you build your business upon?

Do people successfully lawsuit their cloud providers for downtime?

My company offers two deployment scenarios: host it yourself and cloud hosted (SaaS). Many of our customers choose the latter because their internal IT systems require more process, and they just want to get something up and running.

With Postgres[1], you would need to be using an absolute beast of a server to be paying that much, even if you're paying the 2x premium to go month-to-month, and a low-powered machine could be around $50/month for a 200GB DB.

[1] https://azure.microsoft.com/en-us/pricing/details/postgresql...

Based on link you provided, I calculated that 40gb memory would be $290 and 80gb $570 per month (in UK).

So price seems still quite steep, considering that same specs on-site would be ~$2000 one off. (8 months return on investment).

Then for this server, I honestly don't believe its easier to do maintenance in cloud, as you need to do lots of initial configuration there (be it IAM, permissions, firewall etc. ) compared to have it just in local network

A single box on Hetzner is by far not comparable to, say, using a managed service as AWS RDS. It's not even apples to oranges, and more of apples to freshly-squeezed orange juice served by a buttler. Think about it: do you get any form of fault tolerance with your single box?

Ah yes I agree, sorry for not being clearer. I wanted to add on to what you said.

Even though I'm a big fan of Hetzner and a happy customer for years, it's important to know what we're buying and be mindful of what we're not having as a tradeoff.

There is nothing to say against a cloud version of your office suit but otherwise it is quite underwhelming. Short lived applications with a 90% API because the developer want to keep the door open to imprison the users once the application has captured a critical mass of users.

I host on AWS because I don't pay the fairly high invoices. Less domain and TLS maintenance for me. Personally I rent a server and host my personal stuff there including software repositories.

edit: Guess that technically would also qualify as cloud, but I think a differentiation is necessary. Cloud mostly means product x or y instead of just another hosted server.

As far as I can tell it only makes sense if you have a ton of data, and only ever use a fraction of it at a time, for one-off jobs, infrequently.

Instead of applying knowledge, he would bring up all these buzzwords to meetings and not really understand what he was talking about. I get angry just thinking about it.

Nothing in your comment points to resume-driven development. You even failed to mention any project or design decision. You just decided to get angry at a coworker because of something.

I believe these two phenomenons are producing a positive feedback loop in the industry. Selected technologies address one challenge, but introduce complexity. The complexity becomes difficult to manage. So other technologies are incorporated to manage the complexity. In the midst of all this, core technologies are replaced, swapped out, or transitioned to under the desires of the dev(s). For example, switching from one js rendering library to another, while preserving the old legacy code. The complexity footprint keeps growing and it doesn't stop because the engineers themselves aren't entirely committed to the project. They can incorporate the new technology to the project, pad their resume, and bail to a new employer if things grow out of wack.

I spend more time trawling GitHub issues to find workarounds than I do actually using the tools.

Or just no real incentives of the people involved to do so. As a dev, I don't get any real credit for biz outcomes.

That's a great way to describe the phenomenon.

With software, the problem is even greater. You can use react, for example, but you will probably start with create-react-app, which adds a lot of things. Or you could start with Next.js, which adds a lot of things. You could use Java, but you will probably start with Spring Boot, or Dropwizard, which adds a LOT of things. Plus all of these starting points imply the use of new languages, configurations, and programs, in particular, builds.

In my view, all of these "Bisquicks" represent experiments-in-progress, with the ultimate goal of the systematic characterization of "software application", in general. In other words, they are systems of alchemy, pushing toward being a system of chemistry, which we don't have yet. So it is bound to be a confusing time, just as it was surely confusing to play with chemicals before Lavoisier systematized the practice.

Bisquick is a great solution if you have only one problem. I want more solutions like Bisquick: easy to map to the problem (pancakes -> Bisquick; not pancakes -> not Bisquick), hard to fuck up, low marginal cost above the inputs. It's great!

In the converse, we have lots of custom software solutions which are exorbitantly costly over the long term. Small companies without in-house expertise have to figure out how they will maintain software they depend on when their consultant (or the boss's nephew) leaves. Big companies with significant workloads and workforces can afford (and indeed, profit from: https://danluu.com/in-house/) high-complexity custom engineering, which is...fine...but even FAANG have trouble figuring out how to incentivize system maintenance and support.

At heart, I believe we haven't come to grips with the extreme disparity in capital costs vs unit costs of software, such that we don't really know how to pay for the unpredictable costs of bitrot and maintenance. As a result, every software development project is ultimately a question of "how are you going to pay for its ongoing support".

I think I've lost the analogy here a bit, could you help me? I thought "alchemy" were general purpose systems meant to get you off the ground.

I'm nominally one of these "$500k+ FAANG superstars" (though I'm not actually a superstar) and nearly all of our engineering time is spent wrangling home-grown abstractions built specifically for our business needs, which don't feel like "alchemy" in this analogy. Certainly a few teams are more involved with AWS services, linux, and dev-tooling but that constitutes maybe 5% of the company.

Would you still describe this other 95% as "alchemy"? If so, why?

The FAANG-ist is one that can pay that opportunity cost quickly and completely, such that they are able to reason about the system with confidence, particularly when it comes time to reverse a feature or bug request into a set of commits. Or, more likely, a set of experiments and commits based on the outcome of those experiments. Given that there is no unified language of application design, doing this requires detailed, specific knowledge of every sub-system involved, the abstractions their authors used, and the ability to translate between them such that you can imagine the entire causal change from input to output. This is hard, and it's why you make the big bucks. (Interestingly, I first wrote "bugs" instead of "bucks". :)

NextJs, Remix, RedwoodJs are all its own solution in adding what the creators feel are missing from React in terms of server side functionality.

Eventually there will be a clearer picture of which tool to use for various tasks like internal tools, eCommerce, B2B or cross platform applications.

When those layers are small additions, people end up not understanding (or even fearing) components of their stack that are quite close to the surface. It's hard to do quality software engineering when you can't reason about the foundation of what you're building.

But if you lived through the process of adding new layers or if you take the time to learn more layers of your stack, you'll be able to make the most of the tradeoffs inherent to adding the latest framework or library.

Oh yeah, true, some people keep saying a language is just a tool trying to convince somebody to use it, but they forget, or omit for some reason, the fact that it's not "just a tool", it's a huge ecosystem that brings additional, enormous mental burden with it - build system(s), libraries popular within that specific ecosystem, language syntax, language quirks, project structure, its own conventions and so forth.

It's much simpler and more efficient (technically and labour market-wise) to write everything in a single language, as much as possible - unless you start going completely against the grain. Like, auxilliary scripts are usually written in cli-centric languages like Bash, as opposed to API-centric like Python, because you get maximum convenience using cli programs and composing them together.

But no, some people casually shoot themselves in the foot by jumping from language to language depending on the task because it's "just a tool". Ripgrep is a just tool. An entire programming language with its ecosystem is not.

If you're using python for the web you're already part of the complexity problem, atleast from the perspective of someone deploying php 15 years ago. I use python for web development, and I love it, but deploying webpages used to be copy an apache config and ftp/scp your files to some location. Now we need app servers and reverse proxies, and static files are served differently, and even though I've gotten used to it over the last decade it does't mean it's good.

The other thing is that MonoRepos are pushing back against complexity for the sake of complexity. Why create a new repo when a directory will work just fine? I think a ton of people got repo crazy because their coporate jenkins server only allowed one project per repo, but it is trivial to check the latest git hash for a directory and base your deployment on that. ...I have a project I inherited that has 14 repos for a webpage with maybe 5 forms on it. I've mostly handed it off at this point, but everytime I have to look at it I end up preaching about monorepos for weeks.

The problem with deploying PHP is that it immediately gives you something for very little effort, but the effort scales incredibly disproportionately once you outgrow your need for the bare functioning minimum.

I personally prefer the "modern" approach of dropping a single, statically compiled binary, which exposes a listening HTTP socket. In case of Go or Rust+Actix, this could sit directly on port 80, but as soon as you add HTTPS, ACME, virtual hosts, etc the story is basically the same regardless of language/framework choice, PHP included.

I was really just making the point that the effort to deploy python as a webapp these days would have been considered overly complex to the average developer 15 years ago. Just how some of the current stuff seems to the OP, so maybe not all seemingly complex stuff is bad.

Fair enough, but I was thinking more along the lines of debugging logic. With python I can open up a local repl or jupyter, run the affected functions/methods, and from there I can get a nice traceback, modify code, or monkey patch something to figure it out. If I can't reproduce the bug locally, I can run a repl directly on the server to see how the code runs differently.

My frustration with PHP is just memories of making an edit, refreshing the page, and crossing my fingers. Then once it's working forgetting to save it anywhere else because there wasn't a repo to begin with. I know the tooling has got much better, and PHP developers actually use deployment pipelines these days, and it was mostly caused by myself, and my co-workers not knowing any better. Still just how I remember it. And I don't think I'm the only one.

It’s incredibly frustrating to see people discuss the language without any actual knowledge of it. Modern php involves one of the most stable package managers of any language, an extremely fast runtime with a JIT compiler and opcode cache, asynchronous code and coroutines (well, native green threads at least, but proper coroutines via an extension), mature frameworks, stateful application servers, solid standards among frameworks, containerised stacks,… it’s a joy to work with, yet people complain about standard library function names and form handling in templates from 1999.

    python3 -m http.server 8000

And I hope noone is using http.server to deploy their python webapps. the documentation even has a warning right at the top

https://docs.python.org/3/library/http.server.html

I have a PHP project that handles atomic deployments by pointing the web server at a symlink named "latest". New versions get deployed into their own folder in the "versions" folder, and then the "latest" symlink gets pointed to the new version. Super simple to set up, and as long as you delete old versions as you go (keeping, say 3 versions so you can roll back as necessary) that's pretty much all there is to it. This all gets triggered by a Gitlab CI workflow, so I can hit a button to deploy after merging changes.

Another PHP-focused way to handle this would be deploying into a new folder, then updating the nginx config to point to that folder and running `service nginx restart`.

You can also use one of the many other atomic deployment options that replace more than just the running code, at which point the only difference between deploying a PHP webapp and deploying a Java/Go/Rust webapp is what gets included in your new server/container when you build it.

With PHP it's easier to just get started, but you've already mentioned versions, symlinks, CI, etc - that's the non-linear increase in complexity, you have to add a lot more pieces to get good ROI. With Python or Go you can continue using SCP to deploy for as long as it suits your needs, because no code changes will be picked up until you restart the process. If you need rollbacks, e.g. Go doesn't need symlinks or versioned directories - the entire app is a single executable, so you can just keep copies of these. You pay for what you use, and the returns are more linear and immediate.

If you only have experience deploying PHP, I would sincerely recommend trying other languages/runtimes/frameworks, even if for no other reason than to learn from what the rest of the world is doing. For me, learning to deploy PHP correctly was also a horizon-broadening experience.

but for "serious" applications / deployment-routines this is never an option ... you have to use some kind of deployment & configuration mechanism.

Sure, if they're actually unrelated, or being managed by separate teams then split it up. Though I think the default should be to have one, and split it when there is an actual reason to, especially if it's for the same project.

The example I gave wasn't an exaggeration, 14 repos for one project. It was originally built and managed by one person who had them all in the same directory and would open them all in his IDE at once and basically worked on it as a mono repo. After that person quit, when others needed to figure out which repo to clone to fix things it was a nightmare.

I know monorepos can be extreme like google, but I just mean one per team, or at least one per project. You shouldn't have to worry about versions of your libraries when there is only one project using those libraries.

Edit:

For example, a project I did the layout for has a python backend, with a spa frontend, and some ansible playbooks for deployment, integration tests, and a few libraries only used by this project.

Each of those 5 things has a top level directory and we deploy a test server for every branch, plus most of us run a local server. We never have to worry about versioning between our own projects, because if they're in the same branch that's the version to use. If we split it into separate repos, then everytime we added a new field to the api, and needed to update the frontend and tests, we would have to manually specify which versions of all the repos go together to build a test server, or to even run a local dev server.

What if they become managed by separate teams? Or two projects in separate repos become managed by the same team? What about a service that basically everything else in the company relies on (for Google, accounts and auth for example).

Better to just keep things in a monorepo IMO, even if they seem unrelated.

Better to just keep things in a monorepo IMO, even if they seem unrelated.

If there are multiple teams committing to the same repo you need controls over who has permission to commit to which directories and maybe a policy for handling merge conflicts across teams. I'm not sure what the tooling is like around that, but I could see the benefits as long as someone very high up was on board and had enough of a technical mind to keep order.

As far as two repos becoming one or one repo becoming two, you can split and merge repos while keeping the commit history.

edit: Do you have experience working somewhere with a monorepo that stretched across multiple teams? if so, what was it like?

I don't have experience doing that but it sounds like a huge headache to me. Any tooling, deployment processes, links in documentation, cross-repo references etc would need to be updated right?

The company I was in before my current one used a mono-repo for the whole company, I thought it was pretty great! We had ~150 engineers, so not Google-scale but still many individual teams and services. Similar to the Google strategy, a pull request / code review would be against master, and then it would be merged directly to master. The tooling would automatically rebase your change on master before merging. Deployment would happen automatically or manually (the team could decide) and would be pinned to a git commit hash. If automatic, the CI tool would detect if there were any changes to your binary that required deployment (so a totally unrelated PR would not trigger a deploy). You could manually deploy an older commit hash if you wanted.

is it like micro-services thing or what?

I think it really just came down to dividing the repos into chunks that the deployment scripts could use/trigger off of. Instead of developing in a way that makes sense for developers and bending the deployment to fit. Since it was all in one directory on his computer, it was basically a mono repo from his perspective. Committing from the IDE just committed to whichever repo was changed, and since he was the only developer he never saw the downside. When I had to take it over ghorg[0] really came in handy. It's a script to clone all repos from a user/organization on gitlab, github, and others. Then once I opened up all the repos as one pycharm project I was able to get some stuff done, but at that point I might as well just had one repo with a separate directory for each.

[0]https://github.com/gabrie30/ghorg

EDIT: I also just remembered that gitlab is much better than github if you are going to go the route of multiple repos for one project. Gitlab lets you create namespaces to group your repos, so if they're all in the same namespace, you could have documentation to just tell people to clone all of them.

Don't get me wrong. CI/CD and other dev tools not handling monorepos well is a totally reasonable objective reason to not use monrepos. But it's also mostly about the tool not the monorepo concept itself.

There is a MASSIVE difference in the value proposition of a proper deployment from version control and using something like docker w/ docker-compose to facilitate running your project locally (which is not present in your PHP example) versus what the OP is talking about, which is the idea that you should run EVERYTHING on Kubernetes and write Rust for CRUD apps.

Mono repos come with a particular challenge: If you have five projects {A,B,C,D,E} in the same mono-repo, you definitely do not want to be building B,C,D and E every-time someone commits code to project A! This is unimportant at small scales, but as the team grows, building and continuously deploying 'all the things' on every commit just doesn't work out.

So the first naive solution is say "we can enumerate all the things that need to be built for project A". This rapidly breaks down when someone figures out they can abstract a shared dependency for A,B and D into some other part of the mono-repo.

So now we enter build dependency tools, first Make then some other flavour, then we jump straight to Bazel because someone read about it in a Google publication, then to some custom build scripting because Bazel didn't do this hyper-specific workflow thing someone wanted... In the end maintaining a mono-repo build process becomes a hyper-specialised job function that is almost always kicked to the wayside.

In small companies, fewer developers than you can count on your fingers, or truly huge Googles that have their own VCS flavour, it can be shown to work well: But I have yet to hear a story of something in the middle succeeding with a mono-repo.

GitHub is taking the problems of monorepo user's seriously (https://github.blog/2021-03-16-improving-large-monorepo-perf...) and have been responsive to the bugs and poor performance that monorepo's typically cause and exercise.

I think you're just in a very negative space if you start with "Distributed systems" as something overly complicated. At some scale getting a bigger machine either doesn't make financial sense or is just not possible to implement efficiently. Some ideas are taken too far or implemented where not needed. But I'd rather recommend you to learn where each one of them started and why. Criticize for valid reasons, but don't become a curmudgeon.

They do solve real problems. The question is whether or not they solve the problem at hand, and if they create other issues in doing so.

I was re-decking a back yard bridge with a friend and he brought a framing hammer. I'd never used one before and I always had a finishing hammer and didn't know the difference. I learned a new tool and even ran out and bought my own for the project, which worked fantastically well. It's still in my toolbox and hasn't been used since. You just don't use that thing to hang pictures on the wall because it may well f-- up the wall a bit. Using the right tool for a job is way more important than using a particular tool for any other reason.

I don't either. As photographers say, the best camera is the one you have with you. Same thing, the best tool for the software job is the one you already have and everyone knows how to use. Why anyone would want to introduce these high-end fads to an organization without proper reasoning is beyond me.

Best according to who?

Based on what priorities?

Relative to what other options?

Based on what kind of experience and expertise with the various options?

My takeaway: BTftJ must only be a starting point for dialog and discussion. Shortly therafter, it is time to get real about each of our experience and biases. Otherwise, BTftJ is a only a thin veneer.

All of them, except for blockchain. That one can go die on the trash heap of history.

As long as there are Rust jobs or blockchain related jobs there, you're going to have to cope and wait for a very very long time until that is ever going to 'die on the trash heap of history' to happen.

So why hasn't it died quicker than it should have years ago as many have incorrectly predicted then?

> Blockchain companies are a dime a dozen. They don't all last forever.

And who said that the 'companies' did last forever? Why do you think I said as long as?

I'm just wondering if the whole thing is guaranteed to totally go away 100% and to be absolutely used by no-one since clearly someone also thinks so. That is my question.

As for bitcoin itself, which I assume the latter part of your comment is talking about:

It’s predictable trustless money. It’s not perfect. But where you totally Lose me is:

> it is wasting the world's energy supplies

This is the latest POLITICAL attack vector. When you parrot messages like this, you expose yourself as a political victim that is not educated about the topic. Literal parrot.

If you have an interest beyond being a political foot soldier, do your own OBJECTIVE research. Learn about bitcoin miners using renewables, how much energy the industry actually uses, the carbon footprint of payment networks and fiat etc.

It’s an effective political attack vector because in all likeliness you won’t do your homework. You’ll read my comment, experience cognitive dissonance, judge me as a bitcoin cultist, and move on with your busy life.

I am not a crypto stan but it has at least one usage.

e: Sorry, forgot it was verboten to say anything contrary to the "crypto has no uses whatsoever" line.

You don't even need to get scaling into the picture.

You do not get any form of fault tolerance by deploying stuff in a single lonely box. If you care about reliability and resilience, you have to have multiple deployments up and running at the same time.

Also, you already have a distributed system if you have a browser calling your server.

And lastly, if you happen to manage an service used globally or regionally and perceived performance matters then you have no good alternative to have regional deployments.

But has being a software engineer become easier or harder over the last 30, 20, 10, 5 years? I wasn't an engineer for that long but my impression is that programming today is a lot easier. Dev tools, compilers and linters are very good. There's also a lot more community documentation on stack overflow. Some of the complexity is hidden from the developer, which is good and bad. It can bite you in the ass later, but in 95% of cases its a good trade off in my experience. For instance, my preferred stack is Serverless on AWS. I can set up a single config and have cloud storage, an api, a database, logging, auth all permissioned and in a file I can check in. And with a generous free tier, it's pretty much free. I'll admit if something goes wrong its not fun to debug, but it's remarkably fast and simple for me to spin up a CRUD api.

Simple MVC stacks went to all languages, jQuery front-ends doing enough but not a lot. JavaScript enhanced easy to reason about server-side stacks.

You used to spend a couple of days a year, yes YEAR, mucking around with tooling. Now it wouldn't be too much of an exaggeration to say you spend a day or so a week fighting some part of your stack because it's been over-engineered.

IDEs can't keep up so you have to run inscrutable command line tools that fail if you have deviated even slightly from whatever error-prone rain-dance some moron claiming 'best practice' has forced into the build process.

Programming used to be about writing code to solve business problems. The shift to DevOps has been a massive productivity drain and most stacks are now incredibly brittle.

The worse part has been debugging, which you touch upon. Native calls, simple call stacks, easy error logging. All gone.

Moving everything into hard to debug http calls has been a disastrous productivity sink.

The irony has been that as languages have got significantly better our productivity has actually dropped massively because of the ridiculous amounts of over-engineering in "modern" code bases.

I recently worked on a project with 2 devs that took 3 months with a modern stack. The prototype in a standard MVC stack I'd made to demo to the client took 2 days.

It's utterly ridiculous and sometimes I feel like the boy in the story about the emperor with no clothes.

You can't generalise from your experience over the last few months.

> Programming used to be about writing code to solve business problems. The shift to DevOps has been a massive productivity drain and most stacks are now incredibly brittle.

Some businesses _have_ to "shift to DevOps" in order to operate at the scale and resilience required.

Some businesses have unnecessarily migrated to over engineered infrastructure because monkey see monkey do.

Saying "everything is ruined" completely misses the dynamic.

As an earlier poster explained there are a lot more tools in the box now. Making the right choice requires experience, a good understanding of the problem to be solved and discipline in implementation.

Get it wrong one way and you end up with an over-engineered mess that takes forever to get work done with.

Get it wrong another way and you end up overwhelmed by traffic, unable to scale in response and forever fighting fires.

> Get it wrong another way and you end up overwhelmed by traffic, unable to scale in response and forever fighting fires.

To nitpick this specifically, over my 12-year career toiling over this stuff there has never been a scenario where this has required a radical rework to solve. Boring-ass B2B shit rarely requires that level of engineering and, at least in my case, the workloads were fairly predictable and increased in a linear fashion. The one time I did accidentally end of DDoSing ourselves, I temporarily stood up nginx instead of our aging Apache install and was able to serve enough requests to fix the problem. (We then transitioned our app to run in nginx)

It was one fire, and it took a little bit of brainpower to fix. Then the DO droplets were humming along perfectly, and last I heard they continue to do so to this day.

The operational aspects of this double-digit-millions-per-year business ran on a postgres database that compressed to 6gb.

The next business I worked for did billions per year in business value and, until HQ mandated migrating everything to GCP, was humming along perfectly fine on Heroku for a monthly spend well under five digits. Ironically, I think they initially wanted us to be on-prem but couldn't support our stack and would have left all of that to our devops guys. GCP was the compromise (oof!).

That's a highly obnoxious response to a measured and reasonable comment.

Most businesses are never going to be a Netflix or a TikTok or whatever. Yet their businesspeople are absolutely infected with this mindset. And don't get me wrong, it is a disease and its primary symptoms are exactly what OP is complaining about.

So instead of building practical, easy-to-maintain systems we cater to the dreams of excessively optimistic (can you even be anything else as management?) nontechnicals who think every idea is going to require the kind of scale and tooling that someone like Facebook has at their disposal. And they're backed up magpie engineers more fascinated with interesting than functional infrastructure.

Ironically, that very tooling and scale ends up demanding even more resources which demands even more tooling to manage. What a virtuously (hah!) profitable phenomenon for the vendors of these tools.

Good grief.

I didn't say anything like that. the opposite in fact.

Next time try reading and understanding a comment before going off like an obnoxious jerk

jerk

So yes, it is affecting our entire industry. Every aspect of it.

There are very small number of organisations that actually have any sort of need of a microservice architecture.

Worse still, actually talk to these orgs, they'll say they actually have a "hybrid" microservice architecture. Which is basically the worst of both worlds, all the pains of managing microservices, without any of the benefits you get in a normally built application (derisively called 'monolith' with all the negative connotations that word has). Half your calls disappear into the black hole of HTTP calls. No pressing F12 on a method call and going straight to the code. No easy stepping through code in the debugger. No simple download the code and just press play and it all works.

I like solving business domain problems. Not tooling problems. Tooling problems are incredibly boring and frustrating to me. To a certain type of programmer, rather than actually doing their actual job, they absolutely love introducing tooling problems as busy work. Because the actual business domain problems don't interest them. Then switch role as they've got the new hotness on their CV before they have to maintain the craziness they've introduced to the code stack.

Case in point on a project I helped get over the line recently. I joined 1.5 years already done on the project, development has slowed to a crawl. Lead architect designed a system of DDD, event-sourcing, message queues, microservices. Just to add a new field I had to edit 10 files. To add a new form I had a PR which edited/added 40 different files. How it actually worked completely flummoxed juniors + mid-level devs, it was beyond them.

All for a 10 page form that would have at most 150,000 uniques in one month per year. Roughly 1 request per second, assuming a ten hour day and 1 request per form page. Child's play.

A standard stack would have easily handled that load, probably even on a VM. A dedicated server would never have gone over 10% CPU. It would have been massively easier to develop, and cost 1/10th in dev time.

At one point I had a quick go at re-writing a section without the trendiness. Just to see, as I'd never have got it through the politics involved in that PR. I switched the event-sourcing, microservices, 5-tier craziness for a simple, easy to understand, service. Took me 1/2 a day, tests passed, reduced DB calls. Over a thousand lines removed, 100 added. Absolute nuts.

Millions wasted on trendy architecture. Of course the architect left a year into the project for greener pastures.

> Moving everything into hard to debug http calls has been a disastrous productivity sink.

There are a lot of good things coming out of the latest big experiments but this has been a major blow. I have worked on software where the intended debugging approach was to write some code, manually push it out to a shared dev environment, read CloudWatch logs for debugging. It is by far the worst way to debug code that I have ever seen. Things that would take me minutes to debug in a normal setup can take hours or days. Projects like LocalStack aim to improve this a little bit but it's completely counter to the ethos of many "cloud-first" developers.

You won't get any objection from me on the debugging point, it's much harder, especially when you're crossing environment - e.g. running the front end locally but maybe hitting a remote dev or QA backend. I will point out though that there are logging tools that support the pretty standard practice of having correlation/transaction/trace IDs on your requests, such that you put in a GUID from an error and it shows you the entire request and anything that request spawned.

The problem is in larger companies, developers stopped caring about just solving the business problems and moved on to solving non existent technical issues to build resumes to go the next job to get more salary. They go to company to company like a parasitic infection leaving them to rot by introducing Kubernetes React, Go microservices with Rust cli tools.

Also it might be an issue of not being fulfilled in life outside of work.

The real biggie when switching to microservices is that you suddenly have to reinvent all the goodies a relational database gives you for free.

Also, people are getting used to app-like experiences and designers are designing for it. Building an app-like experience is more natural as a Single Page Application, which basically means taking on the modern frontend stack. There are places that push against this, but to do so requires buy-in to the engineering side over product and design. Even then, the engineering side has to be knowledgeable enough to not follow popular opinion and come to the determination that Laravel/Rails/Django is actually the right tool, which isn't always the case.

I consult primarily in node and react with all sorts of transpilers mess and shitty packers. The stuff of nightmares - we waste so much time making things work - but simplifying the stack will never get traction among the not-anymore-technical principal engineers or even amongst the other developers who need a fancy cv for their next gig.

My side businesses use python, django, jQuery, old node.js without modules, rust, svelte.

Engineers hired in big companies want to work on shiny technologies and build their cv.

no, the reason Pieter Levels makes millions is not PHP and jQuery, but because he's also a brilliant sales/marketing/business person

Ergo, they are at least fit for that purpose. They aren't bad enough to prevent him from making millions.

Can you expand on the modern stack and that standard stack? Is the "standard stack" you propose easier to work with now than it was 15 years ago? I guess the "standard stack" is just out of fashion?

My preferred stack (aws serverless BE, nextjs front end or just aws app sync) keeps me in one language (typescript), with some stuff like GraphQL that you have to know. But the tooling around that helps keep my errors in compile time for the most part.

For some inscrutable reason you can no longer do a simple 3 page form with a results page without making a SPA these days without getting someone claiming you're doing it wrong, it's nuts.

Worse still, the angular app has all sorts of weird bugs. Auto-complete somehow screws with the validation of inputs, back button doesn't work properly as you lose all your data, URLs dont work, etc., etc., etc. There's also all sorts of craziness like the whole admin frontend is bundled with the client-facing part as the front-end developer didn't know how to split them up.

Utterly preposterous.

The problem is often that very skilled Devs give advice about incredibly specific stacks that only someone of their abnormally high-skill level can maintain.

Throw in a few juniors or a few mediocre developers and the whole project turns into a complete and utter mess.

And don't get me started on your stack that's mainly inappropriate for most applications.

GraphQL. Talk about the next NoSQL fiasco in the making. Perhaps you missed the whole saga of everyone doing new development in NoSQL and then a few years later we had the flurry of blog posts about "Why are we losing data? I didn't realize ACID was so important..."

Even though it's not 90% of the time.

And I take issue with saying you "have to know" GraphQL, that's a pretty specialized tool for pretty specific problems. Most things are not graph structures.

Simple, slightly opinionated. You just need to be careful of dependencies and what kool kids are doing these days. GraphQL is a complete nightmare.

If you don't transpile anything and are conservative with your dependency you're golden.

I'll admit for the vast majority of web apps out there, this is not needed, but there is definitely a scalability concern if your entire stack is a single ruby on rails mvc application.

I've worked on multiple products with 1 million+ users running on stock Rails MVC backends. Everything scales just fine. Until you hit a DB hardware limit you can just keep adding more web instances behind your load balancer. And DB Hardware limits these days are astronomical.

If you're actually hitting the limit of scaling our web backends horizontally and you don't have the money to deal with the problem, you might need to take a hard look at your product.

And I'm confident this is true for whatever your backend is: Django, Rails, Laravel, whatever.

Currently I'm involved in working on a "data intensive" app which munches a few dozen gigabytes of data and presents summarizations through a web based UI. Usually in the form of charts, time series plots, heatmaps etc. Adding small, incremental features to it is an ordeal. The "backend team" needs to mock and then provide the API endpoints, then the frontend team needs to "break down the work on the React components", then DBAs need to oversee the new queries. The security team needs to review risk impacts etc. One change is at least a few weeks of work.

Contrast that with the work that I did in 1998. It was a "data heavy" desktop application that visualized in 3d seismic data collected by ultrasound probes. The data volumes were in the order of hundreds of megabytes up to a gigabyte. The app was built as a desktop app using FLTK and OpenGL. It took about a day or two to roll out an incremental feature once we knew what was being asked. By two weeks it was in all of our customers' hands. While there was no Stackoverflow.com and we relied primarily on offical documentation with the help of "Effective C++" and "More Effective C++" the build/debug/test cycle was much shorter, the tool chain was much simpler and more transparent, the UI toolkit was much easier to grok and even though it didn't have an "elegant state framework" somehow we made it work well without turning it into a mess.

As an industry, we gave up on localized desktop processing in the name of "consumer preference" (or so we were told) while we assumed an immense amount of complexity in order to server the same computation from a centralized place.

Libraries outside those provided by the OS/compiler tended to be hard to come by. Certainly the universe of freely available library code that we have now was nonexistent (I’d argue that CPAN was a big factor in the spread of Perl in the 90s—well, that an the assumption that was widespread back then that CGI scripts had to be written in Perl).

As a community, we’ve collectively learned a lot of important lessons as well. Legacy systems like Perl and LaTeX tend to install common code in a single universal directory for everyone rather than having this be application/document specific as became the case with Java and the repositories will only give you the latest version of an artifact¹ which has tended to lead to stagnation since backwards compatibility becomes non-negotiable, although some lessons haven’t stuck (like the fact that Rust’s crates.io only has a single-level namespace for artifacts).

⸻

1. Not sure if this is absolutely the case with CPAN. CTAN, which was quite possibly the first centralized repo² does not manage multiple versions of artifacts.

2. I remember when it was first being set up, since there was no guarantee that any tooling beyond TeX would be available to consumers, there being TeX scripts to do things like encode/decode binary files into emailable ASCII format. The original CTAN FTP server in the UK also had the remarkable feature that it would generate directory.zip for any directory that you requested.

What sticks out to me when I'm looking at some older code editors/IDEs, is how crazily spartan they are - to the point where they are just inconvenient to use.

The easier things are mostly obvious. Better languages, open source libs etc. Things that are harder now than before:

1. UI. The web is not a good UI platform, sorry. Designing UI in the 90s was easier, except for the need to do manual memory management if you weren't using Visual Basic. Partly because there was little expectation of branded UI, so you could easily re-use large control libraries that came with the OS which were/are pretty feature complete and well documented.

2. Cross language interop. Microsoft had this nailed. COM was a beast, but it worked and there was an actual real market of cross-language, auto-bound objects and GUI controls (COM objects, OCX controls). There was actually a thriving ecosystem of languages on Windows which are now mostly forgotten (Delphi, FoxPro, VB6, Paradox, Visual Prolog etc). Nowadays cross-language interop is a joke. Transpile to JavaScript or go via a C FFI, maybe, if you're lucky.

3. Expectation of supporting multiple platforms. This causes a lot of dysfunction because the lowest common denominator is really low, especially if you treat the browser as a "platform". In the 90s Microsoft had a monopoly. That had its own problems, but, it meant you could write a Windows app, once, and everyone would accept it. Nowadays you want web+mobile, and "web" is not really a platform in the sense Windows was. If you're doing anything serious you still need a desktop app.

4. Over-specialization/staffing. Back then everyone was "full stack". The developer was also a DBA, at least to some extent, and you could just throw up a quick GUI Windows app that connected directly to the database. The DBA would then manage security and backup. It wasn't really a full time job, at least not on a per app basis. Nowadays even simple projects feel absurdly over-staffed. Do you really need a backend guy, a frontend guy and a devops guy for an ordinary LOB app? Probably not.

5. Process overkill. Waterfall is underrated. It got a bad rap because it requires you to understand your customer, and for your customer to understand what they actually want, and for them to not change their mind every five minutes. Not always possible. Nonetheless, agile has become some kind of monster. Half the words in the average agile methodology are made up and a lot of it is really questionable. If your business domain weren't totally unstable and your users weren't totally incompetent (often the case!), then you could sit down and write an actual spec, which people would read and sign off on, and then you could build it. And it'd work, and after the initial debugging / shakeout period, people would be happy. Many of those apps never broke! Just imagine the output of the typical agile web AWS-based web stack teams today lasting 20 or 30 years without a dozen rewrites along the way. Very hard to imagine that.

Here's a walk down memory lane for you about rewriting apps, circa year 2000: https://www.joelonsoftware.com/2000/04/06/things-you-should-... If you replace names and versions of things with "Go", "Rust", etc. it is pretty much what you describe.

The dangerous spot is engineers with 5-10 years of xp who have become good enough at writing huge piles of unecessary code and have them work.

Meanwhile the EM's are afraid to hire us because we're "tired" and not "team players" (ie have a back bone when we've seen this pattern 20x across our career with 0 successful outcomes)

I've said it before, but I take issue with just about everything that's happened in tech in the last 20 years. I've been programming since I was 12, so have about 32 years of experience. I feel that things really were the very best from about 1995-1999, but since then we've mostly had uninspired and brute-force solutions endlessly doubling down on the status quo, because that's where the money is apparently.

I just see such quick embrace of things like async, which I view as an anti-pattern. We're paying people $150/hr to build out complex solutions that nontechnical people were putting together in FileMaker and Microsoft Access in the 80s and 90s. Even our hardware has endlessly pursued DSP vector processing on GPUs, forcing us to manually convert our software to shaders or something proprietary, when scalable transputers and 1000+ core systems on a chip with local memories appearing as a single context (what we might call.. desktop computing) would have been so much simpler and better. I could go on about this stuff literally forever.

What's the solution? It's so simple that it's right under our noses: make the opposite decisions from the ones we have been making. Old school. Practice radical inclusion and hire people immediately based on their credentials and experience, rather than putting them through endless interview rounds. Bootstrap, and when you make it, pay it forward and help others make it. Get away from all this insubstantial profit-oriented disruption stuff and solve the actual problems in people's lives like how to reduce their dependency on handouts from the rich under trickle-down economics. We need automated food/clothing/shelter that's too cheap to meter, and we need it yesterday.

A refreshing experience was a mobile app Apple device, with Swift and Swift UI. It was a real joy, works as expected, produces concise code, small files, live preview and reasonably fast build time. Sure, it's closed environment, but last time I felt so productive doing UI dates back to Visual Basic.

Counter-example: a simple web app, nothing fancy, and my node_modules filled with around 500MB of files, hundreds of declarations of injected things everywhere.

But nobody forces us to use Kubernetes, nobody forces us to climb the Rust learning curve, nobody forces us to use this multi-platform framework that solves all the problems of the universe.

I try to stick to standard solutions, oft proposed by the vendor: Kotlin on Android, Swift on Apple, c# on Windows. Server code: stick to Java, or try the simple Golang (another refreshing language).

Also, I try to stay late to adopt tech, just starting to be confident in Docker and will see in a few years if Kubernetes could be useful.

But, an architect needs complex solutions to justify its job, a team lead needs as many dev as possible to tell at the next family dinner, and the new dev wants to try this new fancy tech to put on his resume. So they are all fine with that. Just don't tell the company ownership.

The "closed" nature seems to have made such IDE's better integrated such that you didn't need "layer specialists" for each layer: you just "did it".

And the rocket science needed to get "responsive" UI's right is crazy. If only 3% use an app on a mobile device, you bloated your UI by a complexity factor of about 10x to get that extra 3%. The labor math doesn't support it. Vulcan accountants are puking. (And mobile friendly apps tend to waste screen real-estate, increasing scrolling and back-and-forth navigating. GUI multi-panels are a productivity miracle, use 'em!)

WYSIWYG is cheap, easy, and consistent; you can save a lot by telling responsive to go to Bloat Hell. (Maybe someday a responsive UI framework will make it easy, but that will probably arrive with flying cars, hover-boards and Mr. Fusion.)

Being obsessed with "web scale" when most biz apps have only a few thousand users is also a resource drain. Stop putting phallic symbols into your stack, people! A dinky winky is sufficient for 95% of apps.

Choice of sub-parts by itself is good, but if it has a psychological side-effect of creating a layered mess, then perhaps a KISS Bouncer of some kind is needed to trim and factor the options. Otherwise, "cool" ends up trumping boring-but-productive. (I have more to say about his elsewhere around here.)

1. This industry absolutely has, and has had for a long time, a problem with "oooooh, shiny!" chasing. We collectively obsess over using the latest and greatest, newest and shiniest, "sexy" technologies of the day. And sometimes (often?) this obsession overrides good judgment and we try to use this stuff regardless of whether or not it's actually a better fit than something older and more prosaic.

2. However, sometimes the "new, shiny" is actually better, for at least certain applications. And we should always be willing to use a newer, better, "sexier" tool IF it actually helps to solve a real problem in a better way.

Unfortunately (1) often seems to trump (2) and we get stuck using the "newest and shiniest" for no particularly good reason, other than the simple fact that it is the "new shiny".

I have no expectation that this trend will ever abate.

Even if you don't believe it's a conspiracy, you have to admit that: that dynamic favors incumbents, the big frameworks often come from the big incumbents, and if you are a big incumbent even if you weren't manufacturing this conspiracy but you saw its possibility I mean why not take advantage of it?

The outlier is the scrappy indie developer like Peter levels who runs his multi-million dollar a year business on his own basically on a single machine using PHP and only recently started using git. That may be an extreme example but it paints a picture of what radical effectiveness and efficiency looks like and it's vastly different to the Kool-Aid but don't mention it otherwise the mob will come for you.

May the no-code & indie scene save us all. Amen

With enough years in this industry I'm starting to see how much of the culture is just "cargo cult Silicone[sic] Valley!" -- People are doing things just because the latest/greatest unicorn did it that way, and frequently there is a form of Zen/Chan disregard of meaning or learnings from the past. "Move fast and break things" often plays out as "Disregard knowledge and act like there wont be consequences" . I frequently see threads/posts on HN by experienced people laughing that "we've known this since the 70s" in reference to The Mythical Man Month, or other bits from Computer Science, and yet most managers think they know better than decades of industry experience.

As for you not commenting, it seems that would be a good idea on your part. You keep trying, and keep failing. Do you often do things you say you won't? You lack self discipline? Anyway, thanks for obsessively checking my edits.

You should know that in the past when OOP was not common, we had to work a little harder doing things like managing our own memory or building a LAMP server to publish our web pages.

There was a thriving market for language and UI add ons. The result was that each company had their own internal dev tools and recruiting people outside of the company who had experience with those tools was nearly impossible.

All that said, we were at a point where entry to programming was easy (think Visual Basic in the 90s). The quality generally went down as everyone was pushing their "first project" as if it was a polished product. Finding actual good programs on PC is close to the situation on mobile where most of the apps are trash.

While still prevalent today, there's a huge sentiment against this paradigm. Modern languages and frameworks such as React, Golang and Rust show how the tide is turning against this.

Or are you saying that Rust/Golang/React are simplistic in a world of over complexity. React I would generally agree with, the other two not really.

The backlash against OOP is partly because existing OOP engines are limiting in many languages. Passing, grouping, and custom-scoping of "blocks of code" (BOC) should be more flexible so that we are not forced into hierarchies or spaghetti scoping. We need new languages that make managing and scoping BOC more flexible. I want to define the scope, not let Nadella or Bezos do it. The distinction between a lambda and method would then be fuzzier. BOC's would no longer be forced to be one or the other. (One these days I may make a proof-of-concept language.)

None of the languages I mentioned are FP. Though FP has benefits and many of the languages or frameworks mentioned above are either moving towards FP or borrowing concepts from FP.

You're not wrong about intermediate state. I shouldn't have emphasized FP here that wasn't my point. An imperative program that is just imperative without OOP or FP tends to be simpler then OOP itself.

But this is where the senior developers and architects should come in; they need to make a firm stand. Make things like https://boringtechnology.club/ mandatory reading. Have people that want decision power write ADRs so they are made to explain the context, the problem, the possible solutions, the tradeoffs, and the cost of picking technology X to solve problem Y.

It's too easy to bung in new technology or change things around; the short term, initial cost is low and the long term cost is not visible, because it quickly becomes "this is how things are done around here". Make it more expensive to change things.

And make people responsible for these things. If an individual advocates for technology X, they have to train and hire for technology X as well, and be responsible for it long-term. Learn to recognize a "magpie developer", the type that will always go for the latest shiny; you can use those to introduce new technology, maybe, but keep them away from your core business because they will make a change that impacts your product and your hiring, without taking long term responsibility for it.

anyway, random thoughts.

There’s no we. There’s a million moths slapping into everything and being drawn to various bright, often false, lights.

Sun Microsystems had a go, but as you say, unfortunately it wasn't a money-maker.

What you have to do as a developer is try to keep up with the hundred new things, possibly dabble in them to see what they are, and decide for yourself how much effort you want to put into that particular "thing". You have to use judgement or you will burn yourself out.

I never bothered with Pascal. I learned enough Java to be dangerous, but it didn't really apply to my problem/solution domain. I did learn C++ and also learned to distinguish between when a solution looked like an object (C++) and when it did not (ANSI C). If anyone tells you to 'always use C++' ignore them.

I learned Perl, because I found it more useful than AWK for large problems, but AWK still reigns supreme for 'one liners'. Then I learned Python, and discovered that the problems then fall into Python or AWK. I rarely use Perl anymore.

I tried my hand at Go. I don't find it very satisfying. I am looking at Rust.

Everything else I have ignored, by virtue of choosing what problem domains I am interested in solving.

So that software engineers are not screwed, not by any margin. Just choose the problem/solution domain that you want to work in, narrow down the tools you want to be competent in, and move forward. Try to avoid the 'Look! A Squirrel!' response mode as much as possible, but do poke your head up to see what the world is doing on occasion. But be aware, a lot of it us useless noise.

Intelligent animals need stimulation or they get bored and depressed.

I think collectively, "let's move to Rust" is at least partially because we're not challenged enough by writing the same CRUD app for the 20th time in the same language we've been using for the last 5-10 years, and we want to leave our mark in a new ecosystem by implementing whatever is missing.

Some people want to optimise for "fun/exciting/different" while others seem to be aiming for "known/just works, incidentally boring".

We probably need to find the right middle; how do we keep it fun and challenging while keeping it simple and maintainable.

* All these new tools, they give us options, no? Use the right tool for the job, the ability to switch if something becomes old/unmaintained.

* Is this actual complexity or perceived complexity given your experience? The node ecosystem looked very complex for me (someone coming from Python) until I actually got into it. Now it seems pretty run-of-the-mill.

* Is k8s really all that hard? Build a container and you don't have to worry about provisioning it and deploying it again.

There may be good reasons to use some of the technologies you pointed out. And that's a strong may because I can easily come up with arguments in the other direction in addition to yours. I say all this to mean you just shouldn't dismiss it because it seems hard. It may be and it may not be, and if it is it may still be worth your time if the payoff is great enough. There you have to do the legwork to figure that out.

No, the deployment of a container to Kubernetes isn't hard. And it better not be, that's supposed to be the advantage.

What's hard is literally everything else about it. And that may be a fine tradeoff if you are at the scale to need it and have a team to manage it. But there are many organizations where that tradeoff does not make sense.

Absolutely. To me it only makes sense in large orgs that can dedicate resources to managing the platform. But I did bring it up because I, as the user of such a platform, don't have to worry about writing CloudFormation or Terraform because the infrastructure is provided to me. But yes, it's tough otherwise.

Unnecessarily complicated is the default. Choose the elegant thing wherever possible (its not always possible, but often is).

Actively avoid complexity, or it will shackle you.

You sure about that? Sometimes the seemingly simple problems are quite complicated. Partly because we are building software for a world that is fraught with (security) landmines.

But point taken, sometimes you can overcomplicate the architecture.

>Distributed systems? Kubernetes? Rust for CRUD apps? Blockchain, NoSql, crypto, micro-frontends and the list goes on and on.

Each of those are particular tools for particular problems (though I'm not sure why Rust for CRUD apps is so terrible).

>moving away from python (because its too "slow");

Not only is it slow, but the lack of compiler support for typing leads to an inordinate amount of (stupid) runtime problems. I say this because I recently inherited an entire inventory of python software built up over the years at my current employer. Right now, I have a bug backlog full of runtime blow-ups (dating back years) because of careless typing. Coming from the unsexy world of C# and Java, still trying to see why Python would ever be used for anything but scripting and (maybe) prototyping - it's slow as molasses and no compiler support.

I work at a foreign language instruction firm. I'm making a virtual reality training environment for them. It's the best job I've ever had. I don't have anyone micromanaging my work, because nobody understands my work. I barely understand their work, and that's ok. We understand that about each other and we actually collaborate.

In the last 3 years I've not once been yelled at, talked down to, berated, cajoled, pressured into working overtime, any of it. I've not seen it happen to anyone else, either. I have an office of my own. I can work from home whenever I want. People just trust me to be an adult and do my work and it's the greatest thing ever: basic human decency.

It’s a peculiar feature of human nature that we want to make things more complicated than they need to be. The more something relies upon a combination of our skills, and the more esoteric those skills, the more insulated that thing is from outside influence, ownership, and control.

My bet is the frustration you feel is less about complexity and more about your inability to affect change. You’re just one of many competing solutions to the same set of problems, and people will think your ideas are just as complicated because they’re not their ideas. They understand their own ideas better than they understand yours. Vice versa.

And we all live under this umbrella, together. I think that’s why the biggest asset you have as an engineer is to influence people who make decisions. Unfortunately, the best way to influence them is to convince them you have important, complicated knowledge they don’t. Self reinforcing loop.

It won't change until we can form a guild (professional association) and turn it into a bonafide profession. Right now, code that one developer creates may be unrecognizable by another developer, even though both are working in the same domain. It would be a disaster if one lawyer could not follow a brief written by another or a doctor could not decipher which techniques were used by another to perform a particular surgical procedure.

"Just because you can drive a car with your feet if you wanted to, doesn't make it a good fucking idea!" --Chris Rock.

A union doesn't have to be a huge monstrosity. It can be simple and fight for a few basic standards in the industry.

Apple and Google aren't going to just shut down in California because a union asked them to give some concessions that materially improve engineers lives.

No, that is not the "purpose of a union"

Some unions, especially in the USA, have functioned in such a way, but this is far from universal.

The housing market is probably a good example. Many participants probably only purchase property once or twice in their lives (less even if you consider the case of a married couple buying a house, that's 0.5 purchases per person).

“Doesn’t add business value”

But do you know how the business makes money (the actual processes)? Can anyone tell you how to add value in concrete terms?

Because in over a decade of consulting on technical leadership, Agile, lean and DevOps, the most consistent issue I’ve seen is that those questions are unanswerable for almost anyone in almost any company.

In the absence of a clear path to value creation, everyone optimizes locally for “best practices” because…

the root problem is almost all decisions have to be explained to people who know next to nothing about your area & you need to still sound rational.

The local maximum for that usually is “this is how _____ does it & it’s the new trend now.”

The industry trends towards the most useful solution, not the simplest one. React isn’t internally simple, but it killed the frontend JS framework experiments which used to come out daily because it really established a useful paradigm that covers a lot of the web GUI usecases.

The process is messy but it’s not illogical

It's a healthy thing though some software projects probably should just go and die already.

I'm pretty good about shutting those types of talks down in my own org. Usually when "slow" is mentioned you have to take the presenters word for it. Rarely do they include metrics. And if they do once we delve into the code it usually becomes obvious why something is slow. Usually "slow" comes from using bad abstractions.

But companies that need those tools, really, really need them. We don't use "nosql" (hate that term, it is a really dumb term anymore, sql use or not is completely orthogonal to the problem, "non-relational" or "non-OLTP" is better) databases because we think they are cool tech, we use them because traditional, relational, OLTP databases don't work for our use cases. But if someone comes to me and asks what database they should use, I always say "postgres", unless they can present a compelling reason postgres won't work.

The problems we face are tremendously complex, though, and only getting more complex. We fight back with tools, but there are years where the tools are failing to keep up.

postgres delivers 200% on top of what you need and if you still aren't satisfied, there are countless forks and extensions

Just one example, Scala. First, I'm not criticizing the language itself. It has it's place, but what I saw was programmers trying to create a protected space that would provide higher bill rates. Java was everywhere and hiring a Java developer was easy. Scala was new and had steep enough learning curve that you could drastically shrink the candidate pool while at the same time selling the shiny new toy to management. They could create complex, arcane code that kept new developers from getting up to speed while providing the excuse that they were inferior developers and weren't smart enough to keep up. It didn't work for very long as management caught on that they weren't getting much other than higher labor costs. Go seems to be the latest incarnation of that while Rust is a bridge too far to sell to management.

So it's this back and forth, provide something to management that they can sell to their superiors something new. Management buys into it as long as they can get promoted before it inevitably blows up and the developers who sold it move on to new projects, rinse and repeat.

Meh. From my experience, many developers actively don't want to go into management, because usually your whole day is filled with management crap and you can't go and actually code any more. And developers who do switch to management often end up as miserable bosses because their bosses don't care about "leadership trainings".

Additionally, many companies have the non-management track end at senior level, which means zero career progression for those who do not wish to transition to management.

These are clearly the wrong people to be pushing into management. Good management (it does exist) includes people who have coded, but are willing to give that up to enable others to do that. They get satisfaction from being enablers and making space for their underlings to be creative and make decisions.

Further, there are many excellent managers who don't have a typical developer background, but can recognize what success means for their team within an organization and how to achieve it. I've been managed by many excellent managers with backgrounds in chemical engineering and the classics.

The developers you describe should decline these positions and find a better fit where they can make better use of their time. Choosing to accept positions like this hurts them, as well as others.

The only way to counteract this natural course is to explicitly and continuously take the time to simplify and consolidate things and to bear the extra cost of that continuous effort. But the incentives are stacked against that. As long as it (barely) works, the one taking shortcuts and increasing complexity, or just adding something new, will have an edge. It’s also much easier to create yet another leaky-abstraction layer on top of an existing system than improving the underlying system, because the latter is already in use by too many parties and the necessary changes cannot be done without breaking compatibility.

Another factor is that the field is still learning (e.g. type systems, how to best handle concurrency, distributivity, etc., not to speak of changes in the hardware having an effect on what works best, e.g. cache locality, parallelism, GPUs, etc.) and to some degree is still in its infancy. Maybe at some point in the future we will have it all figured out and reach a point of stability where we can concentrate on just making everything as simple and coherent as possible in the ways we by then know work best. But maybe not, and certainly not within our lifetimes.

So, yes, for the time being I’d say there is no real escape. But you can probably find a niche where things are calmer and slower, and stay away from the areas that are the most crazy and quick-moving.

More often than not, small problems require small solutions...

https://web.law.duke.edu/pd/papers/boyle.pdf

I am:

https://htmx.org