There is one question every software engineer knows (or will know) all too well:
How long will it take to get to production?
The question will make even the most seasoned engineer nervous. Here is why:
Fact 1:
Your answer will be wrong.
Fact 2:
Your business needs an answer in order to plan.
These two facts create the software development paradox. Answering the question assumes that you have accounted for all circumstances between now and the future delivery date. That means you can see the future. If you can see the future, then you do not need a plan.
Yet, these two facts exist. The unbalance keeps us unsteadily marching forward.
Instead of fighting the question, learn how to answer it. Every engineering group has an estimation process from the sophisticated to fingers in the air. The most effective are bottom up (where the implementing team provides the estimation) and the one that makes folks cringe is top down. Here are some suggestions on how to handle both.
Bottom up estimation
On the spot
When you or your team is asked to provide an estimate, the golden rule is to never give one on the spot. Give yourself some language to avoid these situations while still acknowledging that your business needs an answer. Phrases like these have helped me in the past:
If I give you an estimate now, it will be wrong. We just don’t know enough. Let me talk with the team and get something more solid by the end of the week.
The team will look at what is being asked and we’ll put together a rough estimate by tomorrow.
There are some unknowns here like design, market research, vendor selection etc. that could greatly impact the estimate. We’ll work to answer those questions in the next week.
If you are pressed to answer, bring every unknown into the equation for your answer. If anything is written down in the meeting, make sure your unknowns are also written down. Do not assume that your caveats will carry through as your estimate spreads like wild fire throughout the company.
Given that I am hearing about this project for the first time, I know this estimate will be wrong. From only a code perspective, just the code itself could take 2-4 sprints. We need input from design, product, QA and other stakeholders for a real estimate. With all that, 3-6 sprints if you need an estimate off the top of my head, but that will be wrong. Please let folks know that when you give them this estimate.
Sound awful? Yes. But the goal is to say you are wrong so that you can buy your team some time to do real estimating. Even with that language, I can almost guarantee that the result of that statement will be an email to someone saying:
The engineers think the project will take 2-4 sprints.
Estimating with your team
Given a day or a week will allow you the luxury of gathering input from others, giving you more accuracy in your estimate. Estimation is more than the code your team needs to write. There are other factors that you should consider when making your wrong answer righter. Here are some of the key ones:
Product/Business questions:
- Has a date already been promised?
- (See top down estimation)
- Is there a design for the UI involved?
- Does the design use established patterns?
- Was engineering involved in the feasibility of the design?
- Is this a new feature?
- How confident is product/business in the user research?
- Is the business comfortable with a beta solution or do they want a final product?
- Will this require a special release process/marketing/content?
- Are there competing priorities?
- What other projects is the team working on?
Engineering team questions:
- Code
- Is there legacy code involved?
- Does the team understand that code?
- Will the code need to be refactored?
- Are there established code patterns for the:
- Front-end?
- Back-end?
- Is there legacy code involved?
- Data
- Will the data require a change in the data schema?
- Does your current data store support the data?
- Is there sensitive data that needs special considerations?
- QA
- Does QA know how to test the feature?
- How long will regression take?
- Security
- Are there new network requirements?
- Does the feature change any concept of:
- Users
- Roles
- Authentication requirements?
- Release process
- Is there an established release process for this work?
- Will the architecture fit a known deployment procedure?
- Are there release windows that might impact delivery?
- External parties
- Does the project involve a new vendor/service/library integration?
- Is the contract signed?
- Is the documentation clear?
- Has an engineer tried to write successful code against it?
- Is there a test environment?
- Does the project involve a new vendor/service/library integration?
- Will there be consultants involved in:
- Delivery
- Integration
- QA
Once you and your team have done your work, come up with an estimate based on amount of work (e.g. story points). Never put a date down unless you have consulted with others (your manager, product partners etc.). Dates stick like industrial strength glue. Work based estimates loosely considers vacation, holidays, number of folks on the project and the history of the team’s ability to deliver.
Managing the top down
Everyone has an opinion on how soon something can be delivered. Sometimes a date or estimate “comes from the top” causing all sorts of mayhem. This results in a lot of feelings within the delivery team. These feelings are important but may distract you from what you are being asked to do. Take those feelings and acknowledge them with the team, write them down for a later retro or a conversation with your manager, and then get to work on owning this situation.
Managing to a date
Let’s just say your team is told that the “new thing” must be delivered by the 1st of the year. First, make sure your manager is aware and get their input by asking your manager the following questions (or finding out who to ask):
- Why this date?
- Is the date negotiable?
- What is the impact to the business if we do not hit this date?
- What is the expectation for delivery?
- Is the team able to reduce scope to meet the date?
- Could the delivery be broken down into smaller parts?
- Is the team able to drop everything else we are working on to focus on this?
- What resources are available?
- Are there more people who can help?
- Could we hire consultants? Note: more engineers do not equate to faster delivery, but at the start of the project, there may be benefits.
- Could we buy instead of build?
- If long hours or weekends are expected, is there compensation/team lunches/dinners? Note: tread lightly here!
- Are there more people who can help?
- Who is the accountable person?
- If a decision needs to be made quickly to hit the date, who is the ultimate decider?
Second, you still need to estimate with your team through the “bottom up” questions. The date, even with the resources and scope could be completely unattainable. In this case, you now how the information to back up your case. Your team should come up with as many alternatives as possible. Be prepared to answer questions like:
If we push the date out one month, what will it buy us?
Could we merge three teams together to get this work done?
If the team drops projects X, Y, Z could we hit the date?
Again, your manager should be your ally in this conversation. Convince them so that they can go to bat for you or with you. If you have done your work, and the date still feels achievable, you can take comfort in knowing that you are still wrong.
Finally, remember those feelings and act when the time is appropriate. Help your team, manager, business put a process in place for bottom up estimation.
Managing to an effort estimate
Less severe, but equally unnerving is the uphill battle of a top down effort estimate. This typically takes the form of someone saying, “this will take two weeks.” Rarely does an effort estimate from the top consider anything other than code. The difficulty with battling an effort estimate is that the team needs to make the case of why the estimate is wrong with their own wrong estimate. Retorts to the team estimate are normally flippant:
It shouldn’t take that long to get something this simple to production.
Do we really need a week for regression testing?
We did the same project last year and it took a week.
Combatting an effort estimate normally boils down to arguing process for the simple reason that many folks “at the top” do not work within the process day to day. This is opportunity. If the team prepares well through bottom up estimation, the team can teach and explain the process fully and expose inefficiencies that folks in a higher position could help solve.
Get back to work
Estimates and dates are a necessary evil. Your team, department, and company will go through many forms of process for trying to get it right. While it may never be right, your team can do a lot to make it less wrong. This is not a silver bullet, but a pat on the back to say that every engineer feels your pain. Work within your areas of influence to find solutions to the pain and maybe (maybe) next time it will be less painful.
How does your team estimate work?
