Monitoring Scrum Projects Tutorial
9.1 Monitoring Scrum Projects
Welcome to lesson-9 of Simplilearn’s Agile-Scrum training program. In this lesson, we look at how we can monitor and control Scrum projects. It is important to note up-front that Scrum provides an in-built monitor and control mechanisms by mandating a demo/review at the end of every Sprint and the opportunity to correct course. Let us now begin with the agenda in the next slide.
We will now look at the detailed agenda for this lesson. We start off by setting up some “first principles” regarding monitoring of Scrum projects. Then we look at some common Scrum metrics with plenty of examples. We also look at a number of examples of metrics used in Scrum and some best practices (do’s and don’t’s) around the use of the metrics. We shall talk about some of the common charts and graphs used in Scrum. This includes the burn-up and burn-down charts and other graphs. Lastly, we talk about Information radiators – a concept that originated in the Crystal family of methodologies, but one that can be very useful in Scrum as well. Now, Let us get introduce with Monitoring Scrum projects in the next slide.
9.3 Monitoring Scrum Projects
Let us establish some common ground before diving into techniques to monitor Scrum projects. - First of all, it is important to understand that in Scrum, the team should ideally be managing its own progress. The scrum master or the manager should not try to micro-manage the team or interfere with this process. - That said, the scrum master should try to provide visibility to the team about progress (or lack of it) so that the team can make its own decisions. - Common tracking mechanisms that are in-built into the methodology are: o Daily stand-up meetings (where the entire team reports its progress to each other) o Reviews and Retrospectives: The fact that the team has to demo working software at the end of each iteration provides a lot of visibility into progress. If the team was not making sufficient progress or if there were quality issues, this would become pretty obvious in the Sprint review. The retrospective provides the team the opportunity to take all the feedback on-board and take corrective actions o Other useful techniques might be metrics, charts and information radiators. In the next slide, we will learn about the Definition of Metrics.
9.4 Definition Metrics
A metric is nothing but a standard definition for measuring or evaluating something. A metric describes how that measure can be arrived at. A measure could be a quantity, a proportion or a qualitative comparison. An example of quantity metrics is that we have 25 open defects at this time. An example of a proportion metrics is that we have 10% fewer defects in this iteration as compared to the last one. An example of a qualitative comparison metric is that this new version is easier to use than the previous version Now, we will look into a chat describing the types of metrics.
9.5 Types of metrics
This chart shows the different types of metrics that can be used on a project. Please note that this is an illustrative list only. You should determine which metrics are important for your project and track only those which add value to your project. Some of the business metrics that could be tracked are: Running (tested) features, Earned business value, Net present value, Return on investment, internal rate of return, etc. Some of the process metrics that could be measured include impediments removed per iteration, those that get carried forward to the next iteration, stories completed per iteration, defects that are carried forward to the next iteration, Team member loading, velocity, Backlog size, etc. Some of the project test related metrics that could be measured are Acceptance tests per story, Defect count per story, Escaped Defects per cycle, Test time to run, Tests run per frequency, Time to fix defects, etc. Let us now understand Do’s and Don’ts of Metrics.
9.6 Metrics do's and don'ts
Before we go further into the use of various metrics and charts in Scrum, it is important to understand some usage guidelines about them. - Do not try to measure too many things. Just because something can be measured does not mean that it is important. Focus on a few important things. - The metrics should be easy to calculate. Ideally it should be an automated process. The more cumbersome the process, the more the likelihood of manipulation and mistakes. - They should be easy to explain and interpret. It should not require a PhD in statistics to unravel the mystery behind the numbers. - It should result in tangible action. Metrics should be focused on the behaviors that you want to encourage or the behaviors that you want to discourage. - Use metrics to drive action – not to assign blame or indulge in witch-hunts - In Scrum, team metrics are preferred over individual metrics - Do not use metrics for performance appraisals of the team. It just multiplies the incentive to doctor the metrics and it makes the team focus more on the number and less on the issue. Next we will talk about the Charts in Scrum.
9.7 Charts in Scrum
Charts are graphs that are commonly used to indicate trends and are very useful in monitoring projects. Commonly used charts in Scrum are as follows. - Burn-down, Burn-up charts - Cumulative flow diagrams - Progress charts - Risk profile graphs - Others Charts are useful because: - They provide visual indicators (a picture is worth a thousand words) - They provide trends (trends are more important than absolute numbers) In the next slide we will look into a chart and discuss on Burn-down charts iteration level.
9.8 Burn-down chart: Iteration level
We can track burn-down charts at the iteration level. The Y axis has the total amount of work (typically in ideal hours) that is REMAINING to be done in the iteration, whereas the X axis indicates the day of the iteration. You would expect to see a downward sloping line that touches zero sometime near the end of the iteration. The slope of the line would be an important indicator about the progress being made (or lack of it). Let us look into an illustration in the next slide describing Burn-down charts project level.
9.9 Burndown chart Project level
This illustration shows an example of a burn down chart. In this chart, the rate at which the team is completing work is indicated by a line, connecting the points. Each point indicates the amount of work remaining (in story points) at the start of the iteration. The light blue line at the top indicates the total amount of work in the release. The next slide contains an illustration of Burn-down chart in Bar style, let us look into it.
9.10 Burndown chart Bar style
This illustration shows an example of a burn-down chart drawn in a bar chart form. Each bar indicates how much work is left. If the team completes work in a given iteration, the top of the bar moves down. If new work gets added or removed to the release (for example in iteration 5, 9, etc.), the bottom of the bar moves down or up, respectively. Note the following tips for interpreting this chart. - The movement of the top of the bars indicates work done by the team. - The movement of the bottom of the bars indicates work added or removed from the project. The overall length of the bar indicates the total work remaining in the release. Let us look at the next illustration describing the burn up and burn down chart.
9.11 Burn-up and Burn down-chart
This illustration shows a combined burn up and burn down chart. The blue line represents the burn down line (or the work remaining). The orange line represents the burn up line (or the work completed). The green line at the top represents the total work in the release. Next we will look into Cumulative flow diagrams.
9.12 Cumulative Flow Diagram
Cumulative flow diagrams are a project tracking as well as a project forecasting tool. It tracks work at different stages, e.g. In progress and Completed. By plotting them against a time line, one gets an idea not just about the amount of work that is completed, but also about how much work is in work-in-progress state. Here, we see an example of Cumulative flow diagram. The orange area shows the total planned work for the release. The green portion indicates the amount of work in completed state. It starts out from zero and keeps increasing as the team keeps delivering the work. The blue area shows the work that is started but not yet completed. By also showing the amount of work-in-progress, it also helps us understand how fast the team is able to turn the work over. Let us move on to the next slide which talks about parking lot diagram.
9.13 Parking lot diagram
The illustration shows a parking lot diagram, which is commonly used to track projects in Feature-Driven-Development. You can track progress against each individual feature and color code them to indicate if they are not started or late or in progress/on-track or completed. This gives a bird’s eye view at a feature level rather than at the aggregate level. It is often more illustrative to see the progress at a slightly more granular level than what the project level charts indicate. Next we will look into the illustration, describes escaped defects found.
9.14 Escaped defects found
Escaped defects represents the defects that were discovered by external stakeholders (like the customers) after the system was delivered or releases. In other words, they escaped the attention of the team. Escaped defects found counts the number of new escaped defects found in a particular time period (could be days, weeks or months). The illustration shows the number of escaped defects tracked on a weekly basis, represented in the form of a bar chart. We will focus on Velocity chart in the next slide.
9.15 Velocity chart
This illustration shows a sample of a graph plotting the velocity of a team over several iterations. As you can see, the velocity of the team varies largely between 25 and 45, with one outlier observation in iteration 7. Having such historical data is a very powerful tool for the team, which can use this data for planning its future work. Let us now discuss on Progress Chart.
9.16 Progress Chart
Here is an example of a simple Kanban board or Task board or Progress chart. Here, the board is divided into just three areas, viz. To do, In Progress and Done. The tasks or stories can be represented on cards or post-it notes. This indicates how much work is piled up at what stage. It may be a primitive, but a very simple artifact to maintain and is very visible – it does not require any effort on part of the team members to notice and interpret the chart. In the next slide we will look into the Niko Niko calendar.
9.17 Niko Niko calendar
The Niko Niko calendar is an interesting variation on project tracking charts. In this example, each team member picks up a smiley at the end of the working day to indicate their “frame of mind”. For example they may pick a happy smiley, a sad smiley or a neutral smiley. It is up to them to select whichever smiley they want. By viewing the smileys at the end of each day, one gets a good idea about what the state of mind of various people in the team is. Let us now learn about information radiator in the next slide.
9.18 Information radiators
An information radiator (as the name suggests) is meant to display information in a public place where it can be noticed by as many people as possible, without having to make a conscious effort to do so. The idea of information radiator was invented by Alistair Cockburn, who was a big believer in the value of timely and effective communication. The information radiators should provide information about the current state of the project, whatever is critical for the team to know. It could also include schedules, tasks, progress, issues or risks. Most of the Agile teams do make use of information radiators to some extent or the other on their projects. We will continue with information radiators in the next slide.
9.19 Information radiators
The most common forms of information radiators are as follows. - Task boards or Kanban boards, that we talked about briefly in Chapter-3 in the context of monitoring Agile projects. Those charts would form excellent information radiators because they make the progress visible to the team, which can then use the information to take appropriate actions. - Big visible charts – including burn down charts. The burn down charts (also discussed in Chapter-3) would also be very good information radiators because they can communicate a large amount of information very quickly and very effectively. - Continuous integration builds health indicators. A team may use street lights (red, yellow, green) or lava lamps that indicate the health of a build. The color of the light and the state of illumination can provide instant visibility into the health of the builds. The next slide is a pictorial representation of Information radiators – big visible charts.
9.21 Information radiators: Big visible chart
Here are a couple of pictures of big visible charts to illustrate what the information radiators look like. On the left side, we see a whiteboard with a lot of post-it notes stuck on it. The post-it notes may contain stories or tasks, which are moved from one area of the chart to the next based on the progress made. The picture on the right shows a burn-down chart and the raw data used to draw the chart. Against a list of tasks, the team members update the amount of time remaining on a day-by-day basis. The total amount of time remaining is plotted against time to show the burn down chart. Let us continue information radiators in the next slide also.
9.22 Information radiators
Let us understand the characteristics that make information radiators really work. - It should be simple. Simple to read and interpret. - It should be stark. It should present facts in stark contrast. If there is a problem, the idea should be to expose the problem – not try to mask or hide it. - It should be current. The artifacts should be updated frequently. - It should be transient. Although you do want the problems to be visible, they should be taken off the boards once they are fixed. - It should be Influential, i.e. it should influence the team members to take actions based on the information they show. - It should be highly visible; you should notice the information without having to make any special effort to do so. - It should be minimalistic. There is no point in overwhelming everybody with a truckload of information. It should show what you need to flag and nothing more. Now, we will move on to the next slide where we find quizzes related to this lesson.
Let us quickly test our understanding of this lesson through this quiz. Which of the following indicates the number of defects that were detected by the customer? A- Escaped defects B- Defect density C- Defects per iteration D- Regression defects Answer is A: Escaped defects indicate the defects that escaped the attention of the team and were found by the customer. Which of the following is NOT a best practice for using Metrics in Scrum? A- Follow trends – not absolute numbers B- Use metrics for making individual goals measurable C- Focus on the issue - not the number D- Simple to interpret Answer is B: It is usually not a good idea to use metrics for individual performance measurement because it causes a perverse incentive to manipulate or excessively focus on the number In which of the following charts can you get to know the amount of work that got added or removed from a project? A- Burn-down charts B- Burn-up charts C- Burn-down chart bar style D- Niko Niko chart Answer is C: The burn-down chart bar style indicates the work added or removed from a release by raising or lowering the bar. Who is ultimately responsible for monitoring progress on a Scrum project? A- Manager B- Scrum master C- Team D- Product owner Answer is C: Scrum believes in self-managed teams, which take ownership for monitoring and controlling their own work. We thus end this lesson on Monitoring Scrum Projects. We hope that this has been a pleasant experience for you. Let us move on to the next lesson. The next part covers the tenth and the last lesson of this course which is about Advanced Scrum Concepts. Happy learning!
9.20 Information radiators
Let us look at some more information radiators that are used commonly in Agile teams. Top left, we have a burn down bar chart that we discussed in Chapter 3. The burn down bar chart indicates not only the rate at which the work is being burnt (completed) by the team, but also indicates (by raising or lowering the bars) the work that gets removed or added to a release (or iteration). Top center, it shows a “risk-o-meter”. There is a chart which indicates the overall exposure to the risks identified (risk census). This is indicated as a trend against time to show if the project is becoming more or less riskier. Based on the threshold exposures, you could also indicate a RAG (Red/Amber/Green) status. Top right, we have a set of traffic lights. The traffic lights can be used to indicate health of the builds or the overall project (you decide where the back end data comes from). Bottom left, we have an iteration burn down chart shown as a line joining all the points on the graph. The points represent the amount of work remaining in the iteration. Bottom center, we have a trade-off slider. Trade-off slider is used to indicate the nature of trade-offs we have to make on a project. For example to accommodate more scope, one may need to increase time or cost or both. The trade-off slider shows how we are doing with regard to these trade-offs. If more of the items start moving to one side, the product owner and the team need to come in and make some adjustments to the plan. Bottom right, we have a status board shown as an electronic display of a project dashboard. One can pick any set of metrics to show on the dashboard. You would typically put up this screen in a visible area to catch people’s attention.
About the On-Demand Webinar
About the Webinar