That's funny, I must not have written any applications in the past 14 years. (Seriously though, I read one algorithms book when I was 16, and used those algorithms approximately... five times... in the past decade and a half)
Serious question: How do you know that in the applications that you have written in the past 14 years, there aren't any spots that could be considerably improved in runtime behavior and cost?
I remember being a programmer before studying formal computer science, I definitely wrote some slow and bad code that I just didn't know could be much better, and/or simpler.
This remains a serious problem on my team- Over half my team members don’t have CS degrees. They get an enormous amount done, but there is often a perf or maintainability cost.
I have no intention of swapping them out because they ultimately provide enormous value, but it’s noticeable.
Is it the algorithmic efficiency that's the problem, or the ability to scale the application?
I've seen things some people wouldn't believe. mod_perl 1.4 on Apache 2.0 handling 350,000 hps dynamic traffic on twenty archaic 1U's. Five tiers of caching, serialized objects on local disks, NFS in production. C-beams glittering in the dark near the Tannhäuser gate. Who cares about inefficiency if it scales?
Putting aside the damage that issues with correctness can cause, the problem is exactly that if you e.g. wrote something that grows with n^2 instead of n log n, you cannot scale by "brute force" anymore, as the inefficiency very quickly outgrows the amount of resources you can add. Seriously, your thinking that "algorithmic inefficiency" can be countered by "scaling" is almost proving the point.
CS is by far not just about runtime complexity, but it's one thing that can bite you.
Also, all the components you mentioned were likely written, at least in significant parts, by people knowledgeable about computer science.
Scaling an application's operation does not directly relate to its algorithm's efficiency, is my point. Of course a really inefficient application can eventually become unusable, but even terribly performing applications can be "worked around" and thus scaled higher.
Say your app is using bubble sort. Egads!!! What a shit design. Clearly this is going to become a nightmare in real world scenarios. But wait - we can theoretically get to O(n) if the list is already nearly sorted. How can we achieve this? By monkeying with the dataset, queueing and batching operations, invalidating operations that take too long, artificially limiting the number of requests per second, or just passing the request to a completely different app depending on use case. It sounds insane, but if you can perform any of these things quicker than redesigning your app, so that it can continue performing under load, that's an example of "scaling" despite the application's poor performance.
Another example is the "scalability" of application development. Say you have an application which is basically O(1), but one day you find a bug in it. Even after you write and commit the one-line fix for the bug, if it takes you between four hours and two days to deploy it to production, or the validation process takes a month... You still have a bug in production for hours, days, or weeks. "Scaling" the development process can significantly reduce the amount of time needed to solve problems, or complete new features. It can be more beneficial to be able to ship code faster and more reliably, even if it isn't the most efficient code.
but even terribly performing applications can be "worked around" and thus scaled higher. [...] How can we achieve this? By monkeying with the dataset, queueing and batching operations, invalidating operations that take too long, artificially limiting the number of requests per second, or just passing the request to a completely different app depending on use case.
And that's a good thing? Part of my point is that this can be avoided, or at least have a higher probability of being unnecessary, if the proper basics were learned.
Wouldn't you wish the author would have just known how to properly sort a list[1] in the first place?
"Scaling" the development process can significantly reduce the amount of time needed to solve problems, or complete new features. It can be more beneficial to be able to ship code faster and more reliably, even if it isn't the most efficient code
Agreed to that, but as said, computer science is not just about runtime complexity. Knowing computer science might help you avoid those situations, or resolving them quicker.
I know that shitty stuff can work, even work "well enough", but going back to my original comment: Maybe it could work much better, simpler, more profitable if that's your favorite metric, with just the application of some basics, if they are known.
[1] Noting that "sorting a list" is a stand-in for all sorts of tasks that benefit from CS-knowledge.
Yes, absolutely it all could work better with CS knowledge. But CS is also definitely not a prerequisite to having functioning apps that can scale well.
CS can help you design the software equivalent of a Formula 1 race car. But most of us only need to work on things ranging from a Honda Civic to a semi truck. It's interesting how little of an education you need to work on a semi truck, even though it's a complex machine that does a tremendous amount of work, compared to the education you need to work on Formula 1 cars.
In the US, we have a big lack of tech talent, and I think part of the reason is the way people propose the requirements or education required to do these jobs. If it's a Honda Civic-level programming task, people shouldn't go through the expense and difficulty of a CS degree.
It’s algorithmic efficiency and at a basic level- things like 9 levels of nested loops, each loop pass generating an http call (admitadly some of this is the platform we are using fault)
How do you ever know that there aren't spots that can't be considerably improved? Does it matter that much? Any inefficiency apparently wasn't bad enough to stop it being shipped.
I actually struggle more with not treating every problem like some algorithmic puzzle to be solved in one pass with O(1) extra memory and just writing stupid, slow, straight-forward code.
It's certainly good if you know how trees and linked lists are implemented, but as you point out most app devs are working on a higher level of abstraction. All of that nitty gritty implementation details are already available to application developers in convenient, well tested wrappers. You can easily go through your entire career without ever having to worry about how doubly-linked lists or B-Trees actually work.
> most app devs are working on a higher level of abstraction.
Until they don't, because for some reason the abstraction cannot be applied in a particular scenario (or actually could be, but it's not understood), or just does not scale anymore.
That's funny, I must not have written any applications in the past 14 years. (Seriously though, I read one algorithms book when I was 16, and used those algorithms approximately... five times... in the past decade and a half)