Pragmatic Agile Weblog

Agile software development in real life

The use of SO THAT in user stories

recently, I found a good example of why (or how) to use the SO THAT clause in a user story. The use case is simple: a web site where the visitor should not be required to sign-up or sign-in.

The US would look like:

AS A visitor I WANT to access the site without any sign in or sign up SO THAT …

What to put in the SO THAT clause? try not to read the below and stop a bit thinking about it. What would you put?
I am pretty sure most of you is thinking “it depends by the reason why this is needed”…. EXACTLY! the SO THAT clause tells something about why we want that US; it tells something about the value to the user. It is important, because differently than use cases or formal specifications, it does not give the exact intended behavior, but just the goal from a user perspective. This tells us a lot in terms of what we can think of and what we should not if, for example, we want to provide the functionality in increments.

Back to the US, one version could be:

AS A visitor I WANT to access the site without any sign in or sign up SO THAT I do not give any personal data to the site


AS A visitor I WANT to access the site without any sign in or sign up SO THAT I can easily and quickly access all functionality of the site

As you see these two USs provide very different perspectives on what’s the real value to the user: in one case is privacy, in another case is easy of use. This can bring to very different strategies to split the story if necessary.


August 18, 2011 Posted by | Agile, Software development | Leave a comment

Declaring war to branches

What do branches have to do with agile? They have a role under many aspects. I will not cover the most basic ones like why a version control system is necessary (which is true regardless you do agile or not), but I want to cover a bit some implications we are facing in an environment with many teams working in parallel.

I believe the role of a VCS is fundamentally twofold: it’s the central repository of code from which you deliver a new release; and it is a communication tool, so that all developers can see what each other does and keep themselves updated with each other work.

In this post, I want to go in more details about the second aspect, accordingly to some real work (and world) experience I notice in my teams.

The concept that a scrum team uses the repository as a communication tool, is quite widely accepted. I would say it is well metabolized by each team member, so that everyone does not feel comfortable in keeping the code on his/her laptop only. It is the way code is exchanged, which is very good indeed. It is also well received and assimilated as a communication tool between different teams, so that we can all relay on nightly builds built from the trunk. The code repository is therefore a tool every one in the team relay upon to have the most stable code.

What’s wrong then? well, nothing in itself. However, there is a dynamic I am still trying to understand better, but that is showing some very bad effects. I start to identify a broken pattern, which is like this:

  1. One dev team needs to implement a new feature for a customer that we want to include in the product;
  2. Because as usual customers are in hurry, you need to deliver as quick as you can, and all that you know better than me, the team decides to make branch for that customer and develop the new feature on the branch
  3. Since everybody understands that 2 must be a temporarily solution because we want at the end the feature in the product, the team promises itself after releasing to the customer the branch will be merged
  4. The merge does never happen

And here is where the process breaks. At the end, you have different customers with radically different versions, all live with something that it is not the product. And because they are live, the forces against update them with a new release of the product are so strong that very hardly they will ever be all aligned to the same version. I am sure all readers understand this is quite bad, and who hosts for their customers, understands how bad it is even more.

The reasons why the merge does not happen are many and all good: firstly and most fundamentally everybody is scared to do the merge because at the point when it will be done, the code has so diverged from the trunk that the merge will be boring, very hard, long and will surely result in breaking the product, requiring a lot of regression and so on. Secondly, if the project is successfully, very likely you have a phase II, still urgent, still important. Do you want to risk you break everything? of course not, let’s keep working on the branch… this is even more true in agile development where it is us before all that tries to make the customer accept to do things incrementally and in phases.

So, should we ban the branches at all? Shall we just all work on the trunk? Honestly I do not know, at least yet. But I want to see if there is another way… what I’ve been doing since a few iterations is trying to understand why the teams feel so badly the need to work on branches. I am realizing something interesting that I want to report below:

  1. Usually branches start from a genuine good intention of a team, not wanting to interfere with the work of other teams: if I work on my own branch, I will not break the code for you, so that you can keep working like I wasn’t doing anything.
  2. Β The above is a practice also in place during the iterations, even at the level of a single iteration. The assumption is that the code will be merged before accepting the user story. The good news is that at this level (differently from branches for a project) the teams do it diligently. However what’s the direct result? it is that the other teams will just see the code changing suddenly at the end of the iteration, without time to remedy. The good intention of not breaking the code for other teams turns into delaying the time when issues can be encountered to a time where usually there is not more time to fix things within the iteration.

What I am finding is that branches are created mainly for the fear of breaking the code for other teams. Again, the intention is good, but is it the best way to make sure we do not break the code for other teams? Are we sure we can not work differently so that we do not break the code, and if we do is not a big deal?

In working directly with the teams I started to question the need of a branch and I expressly asked a team to discuss a bit before creating one. I am finding particularly interesting that most of the times, the need of a branch comes from the fear that only potentially a developer interferes with the work of another developer. And even more interestingly, they may be working on the same code. This seems common sense at a first glance. But when asking something like: have you checked with the other team if this is a problem? have you talked to them to see if touching that area of the code would be a problem for them? have you briefly discussed the changes together so that everybody will not have big surprises? …
You know what? most of the times that communication did not happen. Thus, the communication tool subversion, is turning into an obstacle to a more effective and efficient way to communicate: talking. This is the aspect I consider more important and that I want to attack. I want to attack the fear of breaking the code for others. And I believe the best way is not to isolate more the teams, but more integration between them. The best strategy is not to avoid to interfere each other, but to interfere more, on a daily basis, working all the time on the same trunk of code. I am sure this will break things more often and we will have a less stable trunk for a while. But I also think the incidents that will happen will make the teams generate the anti-corps to pay more attention to the code they release, to make sure code is tested at build time, to be able to change code more quickly and to talk more each other to save time and effort.

Therefore, from now on, I declared war to branches!

August 15, 2011 Posted by | Agile, Funambol, SCRUM, Software development, Software engineering | Leave a comment

Stabilization sprints

One of the most advertised advantages of agile development is that at the end of each iteration the product is always releasable. The idea is that each piece of functionality is either done or not done and everything done should be shippable.

Ok, we are not that good yet πŸ™‚ Or maybe this is not even a realistic goal in some organizations or under certain conditions. Under the cover, the subject is not that obvious and if you are interested in knowing more these are few interesting links:

We adopt stabilization sprints before declaring a release GA. Indeed, the stabilization process is a work-in-progress that we try to improve at every release. For the upcoming release we structured the stabilization process in two main phases: integration and final stabilization. The integration period will take two iterations and is focused on putting all components together and performing integration and other high level testing. Then we will have one iteration of final stabilization. In this last period we will focus on real usage to make sure we do not encounter major issues that will get in the face of the user on real deployments. In the following, the process is described in little more details.


Length: 2 2week iterations
Focus: integration, regression and performance testing; general testing by selected users (dogfoodding)
Entrance criteria:

  • snapshots of all components ready
  • user documentation ready
  • no bugs targeted to the release
  • no development planned for the release
  • device list of the devices we condider blocker for the release ready

Output: all packages become release candidate

Final stabilization

Length: 1 2week iteration
Focus: stability and general testing by selected users
Entrance criteria:

  • all components are release candidates
  • network environment for testing machines verified and locked
  • performance are ok

Output: all packages become GA

During integration testing any bug found is triaged to determined if it must be fixed in the current release or can be postponed. Of course the bar is pretty high assuming most of the bugs should have been captured during development iterations.
The final stabilization iteration is intended to make sure a live system works smoothly for a period of time without major flaws. In our experience, it is a very good way to capture things that are difficult to foresee in advance and that likely (as per Murphy’s law) a customer would run into pretty soon. Only really sever bugs should block the release at this stage.

I keep asking myself if stabilization sprints are necessary in real cases or instead are just a sign of sub optimal development practices during normal iterations. For now, my experience tells me stabilization gives still a lot of value to the quality of the product shipping and allows to focus the team on tasks that would be not very efficient to do upfront in each iteration. But as usual, we will see how it goes and try to improve next time πŸ˜‰

se dico che vado a correre

March 4, 2010 Posted by | Agile, Funambol, SCRUM, Software development, Software engineering | Leave a comment

Cost of unit testing

I found an interesting article about the topic of how much unit testing cost:

October 2, 2009 Posted by | Software development | Leave a comment

Facing reality

I’ve been asking myself what’s the biggest and hardest challenge in embracing agile development. There are a lot of factors in play and it is not easy to clearly see what gives us the biggest pain. There have been also different phases in our way of adopting scrum, each with its challenges.

The human factor is surely one of the most interesting challenge because there is no way to foresee how people will react to the introduction of a development process that asks them to self-organize and make the team’s goals their own goals.

The practices their-self are challenging as well. You get lost very easily when you do a lot of changes at the same time. This is particularly true when you are experimenting yourself all those changes, because maybe you do not have the luxury of a budget for someone that can help you. In any way, I strongly believe there are things you have to learn in the hard way, no help other than your experience can show you the way.

Legacy code is another big obstacle in the way of being more agile. You would like to do more, be able to confidently change things; instead, you often fall in the chicken-and-egg problem that becoming able to do so means to spend time to refactor a big chunk of code. Since it is expensive, you do not start, piling technical debts over technical debts.

But the most challenging experience of embracing scrum is facing the reality. The essence of scrum is very simple: you have a check point of all your work, assumptions, plans, in a very short time-frame. This is terrific, because you know almost immediately how you are doing. But the reality can be something you and your stakeholders are not ready or willing to know.
When you do a long term plan, you keep the hope your plan is good until very close to the end. Since the deadline is far away, you keep the feeling that somehow you can catch up. There is usually not any real and concrete reason for that to happen, but you still believe it. The scope won’t be significantly reduced until the end, the resources will be the same for duration of the project, customers will keep bugging you with bugs and requests, you will keep read tons of emails per day. But still, you believe you can make it. There is something in the back of your brain that tells you “we cannot make it, we cannot make it…”. But you don’t hear it, and you keep believing something magic will happen.

Do not get me wrong, this is done all in good faith. The best product owners are the ones that see possible what everyone else considers impossible. This makes them more reluctant to consider that a plan is not achievable. They want the teams to have a “yes, we can!” attitude and they do not want to hear “sorry, this plan is not doable”. On the other side, the teams want to make their product owner happy, therefore tend to say “let’s try and see”.

I observed a very interesting phenomenon when we switched to two week iterations. Most teams over-committed so that after one week most of the stories were still not done. Asking the teams if the commitment were still confirmed, they were still confident they could complete all the stories. I was becoming every day most worried and the teams kept being confident they could do all the USs until the very last day. There was not any real reason to this to happen… but still the teams were confident. Do not get me wrong, I appreciated a lot the teams attitude of trying to keep the commitments to what they promised to deliver, but the result was that their goals have been constantly not met and the stakeholders have been seeing the teams constantly fail on their commitments. This resulted in unpredictable velocity and in the teams and stakeholders constantly frustrated by the missed goals.

But scrum asks you to do something different: to look at your self, at your teams, at your stakeholders and do not lie. Scrums allows you to achieve your goals because you first of all have to face the reality. The reality of your team, of your resources, of your possibilities, of your knowledge. Only if you accept this challenge and you accept you have to make a plan that works given your real conditions you will be able to see your progress iteration after iteration. Only in this way you can see your weaknesses and work around them in the meantime you work to fix them.
However, sometimes this is not easy to accept. You may be frustrated and wish you have a different team, different deadlines and a different budget. You will be tempted to point your finger to scrum just because it opens your eyes.
But if you resist to that temptation and start to accept reality, you will have the great opportunity to fix what’s wrong and therefore to constantly improve. End eventually, you will be surprised to meet unhoped results.

I believe there are two factors that allow a development team to face reality:

1. Trust each other. The development team shall always keep in mind that what the PO is asking is what will make the product a successful product. POs shall always keep in mind that the development team wants to build a successful product and put all its best efforts to make it happen.

2. The development team needs to gain the confidence of saying, without illusions, what’s achievable. The team needs to learn to say ‘NO’. Plus, the development team must learn to provide alternatives. Firm ‘NO’ must be exceptional. Most of the times a ‘NO’ should be accompanied but alternative options.

What I often tell my team is that we can accept we do mistakes because nobody is perfect. What we cannot afford and to not improve. Facing reality is a precondition to improve a team.

July 5, 2009 Posted by | Agile, Funambol, SCRUM, Software development, Software engineering | Leave a comment

Shouldn’t engineers just be engineers?

During release planning we do two things: estimate user stories and write and refine user stories…

One of the question I asked myself when I knew for the first time about USs and how an agile team works with the product owner to get and refine user stories was… developers are developers, they are trained and paid to find and implement solutions. Why should we ask them to be a bit of business analyst too and define the user stories?

The other day, one of my best engineer came with the same doubt. Do not get me wrong, it was not a critique or a way to quit what we were doing, but a perfectly legal doubt in the back of his mind.

The answer I gave myself at first was that not necessarily developers will have to write user stories. In a perfect scrum world, the product owner(s) has(have) the bandwidth to fully write and refine the user stories. Engineers should just focus on making sure they discuss the stories with the product owner(s) so that they understand them fully and then estimate and implement the stories. In many real cases, where the customer is not an internal product manager, this can be much more difficult, because as anyone, the customer will have many other priorities than writing stories. But in this case, an agile team can just incorporate a business analyst that proxies the customer and acts as the product owner toward the development team.

There are however some good reasons (i.e. reasons I consider good) to have engineers more involved in writing the stories. First, if the project is of any not trivial size, the number of user stories grows easily big, so big that you would need to add many business analysts or product owners (in addition to big walls on to which stick the cards πŸ™‚ ). In such case, you probably go back to involve the development team in helping out. The other reason, which is the most important, is that writing the user stories engineers are forced to see things with the eyes of the users, more than with the eyes of a technician. To make fun of ourselfs we coined the following acronym: MFMU… Maximu Flexibility Minimum Usability. It was because as engineers, we tend to thinks about how a feature should be done and how the code can be changed easily. This usually despite the usability of the final solution (users will do what we tell them to do…). This is particularly unpleasant in software product development, because when you develop a software product, you can achieve the best result when developers think like users. This fits well with the agile principle of releasing workable software. Most users can live with few features but working, much less with the promise that they will have one day a perfect feature set (which we know already will take a couple of releases to get stable πŸ™‚ ).

Writing user stories helps engineers to switch for a moment from the how to the what and this brings the benefit of understanding much better the requirements, the why things should be done in a certain way, what is really important for the user and what less, what the user will do to accept a story and so on.

This is why I am more than happy we need to help our POs in writing and refining the USs. We are not super good yet, but we will get there!

Some additional comments. Interestingly, another engineer came few days after the first with almost the same argument. Easily the discussion slipped to asking ourself if the focus of a software engineer should not be more on producing quality code than writing user stories. Or, in a way, if we should not focus on making sure we can change our code easily before trying to develop with an iterative (possibly incrementally) development process.

First of all let me say that of course it is fundamental that developers write quality code. This is a good part of a developer professionality and it must beΒ  a field of constant improvement. Still, I like my developers put their hands in the user stories because I believe it is equally important they streer their ability to write code toward the final and ultimate goal of software development, which is delivering value for the users. As a manager I am in the middle between the development team and the product team and in many cases I feel the two teams think about value of software development in two very different ways. Developers tend to see value in writing good quality code, the product owner tends to see the value in features delivered to the user. Scrum gives the great opportunity to get these two views closer and see a common goal: deliver value to the user in the most efficient way. As this sentence shows, there is a sequence that makes sense: first deliver value to the user, then constantly improve your efficiency.

February 22, 2009 Posted by | Agile, SCRUM, Software development, Software engineering, Uncategorized | Leave a comment

Iteration length

Finding the right iteration length is quite an interesting challenge. Or most probably it is a challenge when you you are a beginner in iterative and evolutive software development processes. It is true that the literature mentions that an iteration should last between one and four weeks, but still one or four weeks is a bit different :). Plus, different agile teams use different lengths and some go veerryyyy extreeeeemmmm! I know of teams with iterations of 2.5 days (yes! 2 days and an half!). Quite amazing, isn’t it?

Recently we decide to shorten the length of our iterations. We originally started with 4 week and at the beginning it looked a good compromise between keeping a constant and predictable pace and the overhead that the iterations seemed to add (iteration planning, design presentation, product presentations and retrospective meeting).

I must say that from the point of view of establishing an heart beat for all teams, it worked pretty well… but after three iterations we realized the iteration was too long. The first signal that shown us that something was wrong was highlighted by the iteration burn-down chart of the completed user stories. Take a look at the following chart, which is the average of the three scrum teams:

The Y axis is the number of story points to be completed and the X axis are the days of the iteration. There are a number of things I do not like of this graph: first, the accomplished story points is about 65% of the commitment. This is one of the biggest challenges we are facing: we are not predictable enough yet. Note that the result above is not due to big surprises we encountered during the iteration. What’s particularly interesting is that the line did not go down for half of the iteration, then it started to decrease. This behavior is more evident in the burn down below which regards of one of the teams (but the same pattern was similar for all teams):

Let’s ignore for now the peak, again, no stories were declared done until mid iteration, than finally things started to get better and at the end the team almost hit the target. In general, I believe this trend is due to two main factors:

  1. teams tend to commit optimistically
  2. our definition of done strongly depends on QA people acceptance, and nothing can be accepted until at least has been developed and is ready to test πŸ˜‰

Both issues share a common root cause: we are not good enough yet in breaking down user stories in something that can be developed and tested quickly so that work can be declared done earlier. What can we do to fix this situation? We will indeed keep learning how to split user stories before we start the iteration, but one thing we thought is thatΒ  four weeks is still a quite long period of time for people to realize that a story is not of the right size and cannot be completed in one iteration. What we noticed is that teams are often optimistic about commitments and like to try to commit more work more than less work. I must admit I like it, it would be a much bigger problem dealing with teams afraid of committing work. But I think predictability and reliability is important too, therefore I would like we try to achieve better trends in our burn down charts.

After thinking about it, we thought that reducing the length of the iteration to two weeks should help the teams in being more realistic in their commitments (they have now to plan for two weeks only!) and in seeing more clearly when a user story must be broken in smaller pieces.

Another signal that suggested us to shorten the iteration length was when we had a couple of customer related issues that required some attention by the teams. One of the principle of scrum is that teams must be protected as much as possible from external impediments and interruptions. Therefore we need to tell customers to wait the next iteration in as many cases as possible (indeed, critical issues are a different thing). Telling customers they have to wait one max two weeks is much more well accepted than 1 month πŸ˜‰

Another need that would benefit from a shorter iteration is product related. We deliver monthly product updates called “phone packs” that contain for example support for new phone models or carriers or small updates to the database and so on. Customers are starting to ask us to deliver more quickly such packages, potentially as soon as a new phone hit the market. The result of a shorter iteration is here pretty evident. It would allow us to schedule a new phone pack feature twice a month instead of once.

Well… It is enough to at least give it a try, isn’t it? πŸ™‚

From the next release we will use two week iterations for all teams. Let’s see what happens πŸ˜‰ Stay tuned, I will keep you updated.

December 29, 2008 Posted by | Agile, Funambol, SCRUM, Software development, Software engineering | Leave a comment

Seminar on Agile Software Development at the University of Pavia

In the past couple of weeks I and Edo presented a seminar on Agile Software Development at the University of Pavia.

The seminar was divided in four lessons of two hours each and was very kindly hosted by Prof. Antonio Barili in the Software Engineering course.

Goal of the seminar was to get the students familiar with the agile approach to software development and the associated terminology and practice.
And in fact, even withing a so short time, we wanted the student to “touch” with their hands some of the agile concepts, in particular the iterative approach as a way to improve estimates and plans and planning in story point as the basis of making estimates in size more than in time.

I have to say I am very happy about how the lessons went. There were about 12 students from the university plus 5 of our new hires. The students demonstrated to be very interested and attracted by the topics and how they were presented and participated with enthusiasm to the practical activities.

The program of the seminar was the following:

November 6th

  • Software development models (pdf)
  • SCRUM part I (pdf)

November 7th

  • SCRUM part II (pdf)
  • SCRUM ball game (pdf)

November 13rd

  • Agile estimating and planning (pdf)
  • Test driven development (pdf)

November 14th

If by chance you attended the seminar please leave a comment about how much you did like/dislike it and possibile what you would do to improve it.

November 19, 2008 Posted by | Funambol, SCRUM, Software development, Software engineering | 1 Comment

When upfront design matters

I’ll write about upfront design sometime in the future, but Edo pointed me to this great article:

which makes me reflect about upfront design. Le me be straightforward: I am completely against the no-upfront-design principle which is a complete non sense. One case when this becomes easily evident is in API development. In this context, the way the agile community takes the absence of real design and documentation will just take to a bad API. End worse, refactory does not help because you cannot really refactor a public API every two weeks πŸ˜‰

Have a nice reading.

October 15, 2008 Posted by | Software development, Software engineering | Leave a comment

Just Tell Me What To Do and I’ll do it – JTMWTD people

My first post in this blog comes from the daily experience in my job, but it represents a more general comment on way of thinking of some people.
It is definitely related to agile methodologies and I’ll explain why at the end of the post, but let me start from the beginning with some considerations on how some people approach common problems.

It’s not rare to find people that just act, perform, work in consequence of kick-off signal; I mean people that just wait for something to work on, wait for orders: people that I’d call the prefect executors – aka JTMWTD (Just Tell Me What To Do) people.
Many times in your life (not just in your job) you’ve probably met people that are perfect executors, but that need an input to start: girls waiting for the perfect love, fathers looking for the perfect son, managers waiting for the perfect business, developers waiting for the perfect design, product owners searching for the killer application.
They are all right, it is a noble cause and belongs to the natural process of cause and effect: you can’t start a fire without a spark (somebody said…).
Maybe it is even a kind of natural approach: “Give me all the resources, procedures, instructions I need and I’ll be the perfect builder”.
It’s great to find people that need just instructions… you have just to wait the cake at the end and that’s it.

But, what is the main disadvantage of this approach and how is it related to agile practices?

Let’s see the drawbacks first (you may already have found some).
Well, in order to let the perfect executor greatly work a microcosm of resources is needed: the perfect designer, the best environment, the right documentation, the right tools…
And here comes the first nonsense: you can reach the goal of a perfect result only if you start with the perfect environment (clap, clap, clap)…. too easy πŸ˜‰ And maybe even under those circumstances the work won’t be that perfect….

In addition, you are going to realize too late that the task cannot be completed: the perfect executor thinks you gave him the right environment and resources and typically doesn’t make a step to bypass obstacles… s/he just waits… for you and for the moment to say “If only you had…”

As I said it’s common in different scenarios (life and work), software development included.
Some team mates are JTMWTD and maybe they even think to be right: they are developers not managers, designers, testers. The worst thing about this is that JTMWTD will always find an external impediment that prevent them to perform the perfect work, and you are always going to know it too late.

The same approach can bee seen at a higher level in software development process. The waterfall process is based on the same kind of assumptions: everybody does just his/her job in sequence and does it to the best.
In the real world requirements change, people fail, tools are not appropriate, documents miss important details, therefore the process is required to take into account the need of a second chance. And it is much more important that every ring in the process chain share the same goals and commitments, so that everybody can better address the challenges that arise every day.

Likely, agile methodologies help to handle this kind of situation, but this require people change their way of thinking with regards to perfect execution.
I’d say that with the agile practices people must switch from the JTMWTD paradigm to “Let’s agree on what to do and communicate”.

Here in Funambol we are applying Agile practices also for the project management and Scrum is our choice.
We are iteratively releasing, trying also to increase code quality and we already have seen the advantages of it.
One of the first thing I’ve understood is that Agile practices increase the communication and I like it very much.
The daily meeting is my favorite practice because makes really easy to track changes and impediments, but above all because in my opinion it’s like the spark that starts the fire of the daily activities.

The daily meeting and the retrospectives change the waterfall rules in software development: nobody can say “if only you had said me this and that…” and each member of the teams is not so scared of changes in requirements… they just embrace the change.

But JMWTD people fight the agility and the team risks to have different feelings inside: agile and “frozen” people hardly work together.

So here is the challenge: applying agile practice in a real world with JMWTD people… don’t talk me about theory please.

For each JMWTD person becoming agile I’ll light a candle πŸ˜‰
As in many other areas of the agile theory, the most challenging factor is the human factor, because it is much easier to change a development tool or process than to change people mind. But this is also the key factor of success in adopting agile methodologies, it is something that can kill all the benefits of agile.

But if instead you manage to change JMWTD people, so that they start estimating and autonomously selecting user stories from the backlog and reporting impediments as soon as possible… what a nice feeling for a Scrum Master πŸ˜‰

Final Note: for Italian readers here is a link to a well known joke about JMWTD: “Tu mi dici quello che devo fare e io lo faccio”

September 22, 2008 Posted by | Software development | | 1 Comment