I think what a lot of people fail to take into account is the amount of effort it can take to "estimate" tasks. From my experience, the more pressure there is to give an estimate the less time they want you to spend researching the work. To increase the accuracy of an estimate, you need to do more research and if you take that to its logical conclusion then the best estimate is given after the work is completed. So there is some area in between of extremely inaccurate guess with little effort to actually doing the work and knowing the time it took. It's best to accept that estimates are near useless (unless you want to spend a lot of time basically doing waterfall to come up with requirements/design/etc...) and instead just try to find out how to chop up the work into smaller pieces that each deliver value and plan how the feature will be delivered. This way the team can be focused on actually getting value out the door instead of spending time doing a futile exercise (estimating).
I’ve got to do a lot of contract software work back in the day and was the one working first draft requirements with clients before the actual contract.
It was common to spend a week into planning for every six man/month of project
Using that as ballpark we never got an estimate having more than 10% error.
Anectodal, I know, but without that kind of effort one is not doing an estimate is doing a guess.
That also why I don’t like the agile approach of piling up efforts and then use actual time to tune speed hoping the inaccuracies will level off one another. It just feel like a sequence of dice rolls hoping you don’t end up at the far tail of the gaussian.
> It just feel like a sequence of dice rolls hoping you don’t end up at the far tail of the gaussian.
That's exactly, precisely what it is, by design AFAICT. The thing about gaussians is, what with the central limit theorem and all, you provably don't end up on their tail very often.
That said, gathering a bunch more data can make your gaussian even narrower. I think you and (platonically ideal) Agile are just at different spots on the same spectrum of risk tolerance.
High level estimates aren’t optional for most development. Customers waiting for an important feature won’t take “ it’ll come when it comes, estimates aren’t real” as an answer.
You can skip low level estimates, but the consequence of that is that high level timelines are uninformed by them. That’s how you end up in the archetypal situation where all the engineers know a project will slip while their managers report it’s right on track.
In my experience, knowing when work on some functionality will _start_ plus a rough estimate on effort (talking hours/days/weeks level) is quite acceptable for most stakeholders. It is also far easier to provide, and less likely to be off by an order of magnitude, which some low-level estimates may suffer from.
My point is you need to balance the effort you're putting into your estimates vs getting work done. Customers also won't be happy when you give them a date and don't meet it. Honesty is important too.
With paired programming I have always felt comfortable sharing a Googling of the problem with either the co-driver or the driver, depending which role I am in.
Can you explain what you perceive to be the problem with accessing something like StackOverflow to help solve a persistent problem or even to simply jog one's memory?
I find that this fear of being shamed for using the obvious resources at our fingertips to be one of the things that makes people less likely to want to pair. IMO there is nothing wrong with consulting books, websites, other people's ideas/opinions, or other knowledge repositories to help make good decisions or solve problems elegantly.
Sorry, I probably need to re-word that. I didn't mean to suggest it that way.
It was more of a reflection of being unwilling to ask team members for help and just depending on Google/SO for answers if you're stuck. I'll re-evaluate the wording there and get it corrected.
I would add that if engineers/developers are unwilling to ask team members for help you have a big problem with the team, likely a recruitment or culture/leadership problem, not just a project management problem.
Teams work best together and to work together egos need to be checked at the door and people need to trust each other. Any problem on a team is the whole team's problem, and while one person may be accountable for introducing bugs, a good team will view that team mate as valued and capable and perhaps pitch in to mentor or help fix the problem quickly.
The Five Dysfunctions of a Team obviously points out a lot of what I am talking about much more elegantly than I can. What I am saying is that project management can only go so far, team leadership, culture, and recruitment is also hugely important to fix many of the problems you identified in the article.
Great article though, I shared it with my work-mates :)
I can imagine that this is really tied to the confidence you already have. If you feel that you are a capable programmer then looking up something on Stack Overflow is just looking up something you kinda know but forgot. No big deal.
For someone in the grip of impostor syndrome, it is admitting that you are indeed just faking it and don't know enough to 'deserve' your position.
Now it is probably not true, but that does not matter for how people react to it...
This is one of the best software books I've read, that's not about strictly about software. The concept of small batch sizes in particular maps really well to prototyping and agile approaches. But it gives a basis for why those approaches make sense and work. It's like quantum mechanics compared to classical mechanics...
I came out of the article with the same question. I was almost jumping up from my desk with excitement through the first 3/4 of the article, but then the conclusion felt a bit lacking when it never directly addressed the estimation problem...
You don’t. You focus on efficiency and throughput.
Complexity and high variability of estimates is the real problem. You can address that some by training developers on how to estimate, which I’ve heard called calibration, and then spend a lot more time digging in up front on those estimates.
Long term, a ball park can give a good idea but most aren’t going to be more accurate until we dig in. If you are choosing between projects based on expected ROI then diving in ahead of schedule to get a clearer picture makes sense.
If you are delivering a specific product no matter what, then you optimize for throughput.
It’s more valuable to tune the engine so you can drive 70mph than to slow down to 20mph with the goal of accurately predicting your speed.
While I agree it is more valuable to optimize for speed for each person speeding there is a traffic cop worried more about what the sign says than the final result. Sure a company might want to go as fast as possible because it makes them lots of money but the accounting people now can't write budgets very easily so they push for more accurate estimates.
This is a very good read. The only issue I really take with it the notion of the estimates constantly being correct, while later adding all of the other organizational factors that can create a change of course.
IMO that means that the estimates he's talking about being accurate are fairly short term.
Totally agree with him on a lot, especially only the developer working the task can truly estimate it. When you're talking about time measures, that's the only hope of them being accurate.
When I talked about story points reflecting complexity and not time, it was specifically because time is variable by person. If the team isn't creating a time estimate on a task, that's much less of an issue.
" Those principles actually do work extremely well in many, many industries. PMP focuses almost entirely on Waterfall methodology and gaining the certification is largely driven by terminology and math.
But software is a special snowflake in the project management world and this is why countless books around managing software continue to be written."
Thanks.
I highly recommend you read the book I asked about in a sister comment. I wish the business people would read it. The issue I think is it correlates and requires an understanding of other "techie" and STEM subjects like queues, math (related to queues), network protocols, and traffic flow. Until the business people understand the maths and tech topics that book appeals to, friction will continue to exist.
Hey! Loved the article. I am currently employed as a scrum master in a scrum environment. About your proposed "fix" to management: even with the light pairing you recommend, wouldn't kanban eventually cause siloing? Even using scrum at my job, we have individuals becoming the "Feature X" guy or the "Feature Y" guy. This person then essentially disappears into a hole perfecting that one feature for weeks/months, even if there are more pressing/interesting matters. I don't see how kanban wouldn't increase that even more.
Also, how do you estimate? Individual or group? Does everyone on the team possess the skills to estimate every story?
When using kanban, people take the first card from the top of the pile and work on that. There's no siloing in that, because you're taking whatever happens to be at the top.
You don't estimate in kanban. You'll generally try to break things down into small stories, but the goal is to always be working on the most important thing and focus on improving how quickly work goes from idea to done.
This was a great write up! I’ve been thinking about this problem A LOT while building a project/team planning software startup. What do you think about having every engineer estimate one task for the upcoming sprint? Also, I’ve found that the estimation process ( I.e voting ) in my experience is better if blinded, otherwise there will be extreme peer pressure to over or under estimate.
That pressure can be very real. Personally, I don't see a tremendous amount of value in overthinking estimates as much as re-evaluating estimates once the task starts.
The methodology I mentioned calls for exactly that, estimating best case, realistic and worst case...and then re-evaluating once the work starts.
You never really know until somebody dives into the work and it's rare to actually get time before an estimating meeting to individually look into each story. Treat the estimate as a best guess and then re-evaluate when you have better information.
>Treat the estimate as a best guess and then re-evaluate when you have better information.
Any suggestions for how to effectively communicate a "best guess" and a "re-evaluation" to a business side that basically demands "hard deadlines" for projects?
The Software Estimation book mentioned in the article does a really good job of breaking it down.
Hard deadlines come with a lot of questions, like why? Is there a trade show? An investor meeting? A marketing campaign?
Is it really a hard deadline? Specifically, if the product isn't ready yet are you planning to launch no matter what because we have to?
If not, then it's not a hard deadline and deadline needs to be reframed around goals.
You'll never hear Apple announce a product until it's actually ready to ship.
Usually hard deadlines come from announcing something that doesn't exist yet. The outcomes are usually buggy, rushed systems or missed deadlines that make companies look bad.
If it's for a trade show, the question is closer to "what exactly do we need for this trade show?" because management will always say "everything!"
I always tell the story of a particularly bad client that I had years ago who insisted that he had to have a project completed 3 weeks ahead of when we told him it would be ready. My partner and I killed ourselves, gave up weekends, worked crazy hours to hit his deadline for him.
And we also monitored the parts of the application that he had us create. He didn't even look at them until 2 months later.
After that we learned to create "Rush Rates" when requests like that come in, because if demanding something be done faster has no cost to the person asking for they are going to continue to ask for it.
At a company with salaried employees, management has to understand that rushes and scope creep come with tech debt and quality issues in order to hit "the deadline". Understanding things like "tech debt is not just a task you do later, it actually makes everything you do worse" are critical in that regard.
When you can get a manager to say "I don't care of quality suffers as long as we can show it on the floor at the show" then you know it's a deadline. Ideally, point them to that book to help them understand the difference between estimates, commitments and targets.
I like the idea of digging more into real business requirements, but I'd question one thing.
>You'll never hear Apple announce a product until it's actually ready to ship.
Apple is a consumer tech, flagship-product-focused company. B2B timelines, contracts, and deliverables work somewhat differently. Apple can always "miss" a deadline and just choose not to debut the new product yet (worst case). Many B2B contracts hinge on explicit future product deliverables, because there is a TON more power on the buy side when dealing with business clients as opposed to individual consumers. Do you have experience with these estimation issues in a B2B context, any flavor you can add there?
Please don't put out assumptions like "PM = Psychology + Economics" without an indepth argument about it. I completely disagree with such statements and without arguments I can't even attempt to see it your way.
I apologize for not responding to this sooner. I just missed it.
Project Management is about resource optimization, where for the case we're talking about those resources are people.
Optimizing those resources involves balancing the supply and demand for people's hours, knowledge and skills. More experienced and highly capable people on your team will be in more demand, despite the always limited supply of their hours which increases the cost of those hours to your timeline.
Psychology is the study of human behavior. Because the resources are people, a huge portion of your job in optimizing that timeline is optimizing the people. That means understanding motivations, weaknesses, burn out, focus, etc as well as a number of other factors.
Nice. It's not the greatest thing in the world either, tho. (Not the worst thing either, tho.) Here, a related conference papers (there does exist a staggering amount of project management science - mode of it little applied, however.):
Oh and, on an unrelated to this but related to Project Management matter, you might find some use for this very Russian tool & programming language, originally developed for the Buran space project (development continued after the project failed for unrelated reasons), and still to this day used in space flight software development:
