Why is estimate based on story point much BIGGER?

Many times, we need to provide estimate the cost of a project/feature. For many Agile teams, it will be done by using relative estimate, e.g., story point and convert it to cost based on team’s velocity, like the following formula.

Total # of Sprints =Total Points / Average Velocity
Total Cost ($) =Total # of Sprints * Team’s cost per Sprint

Recently, one project team’s manager asked me why the cost estimate based on story point and velocity was 3 times bigger than the estimate based on effort (man day). When I checked the two estimates, I realized that the project manager convert effort estimate to cost directly, like

Total Cost ($) =Total Efforts of Many Day * Cost per person per day

The problem of this formula is, usually the cost model of a project is based on duration rather than effort. That means team may spend 3 days to finish 1 man day work so that the cost will be 3 days’ cost but not 1 day’s cost.

It is a common problem that people will convert effort estimate based on man day to the estimate of duration and schedule directly. That’s why I don’t like effort estimate because it is so easy to cause confusion about it.

With my experiences, I found story point estimate is actually quite accurate and easy to apply. Because it is relative estimate based on comparison. And according to the research of scientist, human’s brain is really good to compare rather than measure. That means when team considers that one story is roughly 2~3 times bigger than another one, the relative size does make sense most of time.

And the effort estimate (man day) is trying to estimate the effort of each activity, e.g., analysis, development and test. This kind of estimate usually requires more details about the requirements and the solution in order to improve the confidence. It will take longer time and also the deviation will be often big.

When the estimate needs to be converted to duration and cost, the conversion rate for story point is team’s velocity what is the actual data represent the team’s capability at the moment.

And the conversion rate for man day effort is based on general experience rather than team’s capability (except team measure it which is quite difficult and usually will create too much overhead to the team according to my experiences). With that, I believe the estimate based on story point and team’s velocity will provide more accurate forecast about the project.

So when you already have the estimate points and team’s actual velocity, it will be quite easy to estimate the duration and cost. If you find the figure is quite big, usually it is because team’s capability, what is velocity, is not as good as you expected rather than the size of the work has been overestimated. So to reduce the cost, you need to think about what can help the team to improve capability rather than ask team to reestimate again and again.

Stop talking, start doing

enter image description here
I used to be involved in the project what was a very challenged project because neither business needs nor technical possibilities were clear.
According to the organizational project initiation process, the business stakeholder asked for estimate about the potential requirements so they could apply for the budget. So, we tried to help the team to do the relative estimate using story points. But after 4 workshops, the business stakeholder still thought the cost was too high and couldn’t accept it. And the project manager would like to organize more workshops to investigate further and hope it would help the team to come out more “reasonable” estimate.
This scenario might sounds very familiar to you. The business probably has already had an acceptable range of budget figure in his mind for his very vague requirements. And team normally will provide a bigger number because of very limit information. Then it will be the dead loop of conversation. Business expect s smaller estimate and team always give even bigger one. And in the most of cases, it will end when team feels tired of this game and surrender themselves to give whatever the number the business expected. But, of cause, the business will pay back when they spent more money and received a crap software at the end.
So, how to escape from this trap? In this case, I asked project manager, if it was possible for the business to sponsor team for 1~2 iterations to start to work on something what were relatively clear and feasible. So everyone would have better understanding about the complexity of the project and then business could decide if it was worth to continue accordingly. I also mentioned that, with total about more than 8 hours workshops what involved many experienced team members, we already spent a lot of money on estimate but couldn’t got any confidences.
At the beginning the project manager was not quite sure if it was a good idea but he agreed to give it a try. And, guess what? The business accepted it and we stopped to provide further estimate but started to plan for the first iteration. During the iteration planning session, we worked with business stakeholders together to decide the possible delivery of the first two weeks and they were very happy about it. At the end of the planning, one of the business stakeholders said: “I am very happy that we finally start to discuss about some detail instead of all those big epics. And thanks to everyone to make this happen!”.
This is a quite common scenario in my experiences. Customers expect the lowest risk for their investment and, at the same time, maximized return. Both of them are fair. The question is how to do it. The traditional way is trying to do lots of analysis at the beginning and expect that will reduce the uncertainties so that we may put everything under control. But the reality is, most of time, through those early stage investigation, we probably can only find the answers of what we know, but, like the following diagram,
enter image description here
There are other 3 areas where the most of uncertainties stay, we need to Explore, Experiment and Expose. We won’t be able to find the questions’ answers in those 3 areas until we start to try. The earlier we start to try, the earlier we start to learn and reduce the uncertainties. That is the only way to reduce the risk and maximize the return (or minimize the waste).
Stop talking and start doing. Start to deliver values to customer and learn from it earlier. The customer will appreciate it.