Real World Agile - Week One

Two weeks ago, I had the privilege to participate in an Agile Planning and Delivery training with James Shore and Diana Larsen. This week we had the chance to put that training into practice. While the training was very hands-on and interactive (we actually did four iterations and built working software the last three days), it was also in an artificial environment. In the real world, things didn't quite fit into the neat patterns we practiced in class. Here's I present what we did, what we didn't do, what worked and what needs improvement after week one.

TL;DR Version

• Task Board and Test Driven Development (TDD) are Very Good Things.
• Software tracking tools, not as useful as a rolling white board and post-it notes.
• Early practice shows promise and we'll continue to build on what we've learned.

Our Early Attempts

There's a lot more to Agile than the ceremonies and tools - or maybe a lot less. In the past we focused on story planning, iterations and a software tool to help us keep track of what was going on, but all we were really doing was cramming a waterfall methodology into one week sprints. Needless to say, that didn't work. We didn't gain much from daily stand up meetings, putting all the stories, tasks and estimations into a tracking tool or having a scrum master. We still ended up trying to cram too much scope into too little time with the added complexity of a new methodology run by an inexperienced coach. It could have gone better.

What We Did This Time

Fewer Stories So Could Confidently Commit

One of the key points that I brought away from our training was to deliver value with each story. Even if the value is small, it's still an incremental gain for our customers. Rather than plan a project up front and stuff all the stories into iterations several weeks into the future, plan the week's stories and only commit what can actually get done.

That last point is worth reiterating - only commit to what can actually get done. Completing stories energizes the team and gives us a real sense of accomplishment toward our end goal. We had previously guessed wrongly and over-committed and got farther and farther behind with our stories. This week, we took a look at all the possible stories, prioritized them and only committed to a handful (no more than 6-10 is a good rule). The rest went into the backlog.

What we knew intuitively, but failed to account for in our previous attempts was a buffer for technical debt, regular support and maintenance of other software and administrative overhead (meetings, email, interruptions, etc.). This time we left plenty of buffer and committed to paying down technical debt if there was buffer time we could use.

Test Driven Development (TDD)

I had previously advocated Test Driven Development to the team, but it was new, seemingly complicated and I wasn't very good at it myself. I was familiar with the concept and had done it myself, but didn't do a good job at explaining the technique or the value to the team. Luckily the training spent a fair amount of time on this concept with real-world examples and practice. And as fortune would have it, this week an excellent online meetup, "Test Driven Development Demystified", occurred and we could all listen in to help reinforce our learning. With that baseline established, we are committing to TDD as a regular practice for our coding.

One challenge we face with TDD is that we have piles of legacy code. We chose not to try to do any retro-testing of legacy code, but rather write unit tests for all new code and test old code as we end up working with it. Part of paying down our technical debt will be refactoring old code with tests, but only as the system and time demands. Our rule is "make the code better than it was before". Small improvements add up and eventually we will be fully covered.

What We Didn't Do

Demonstrations

We are committing to one day a week (we chose Wednesday) as our demonstration, retrospective and planning day but since we only had three days before base-lining our iteration schedule we chose to skip the demonstrations this week. Next week will be a different story.

Retrospective

For the same reasons we didn't do any demonstrations, we skipped a formal retrospective. Instead, we chose to talk about what we learned in the training and commit to a few key concepts (TDD and story planing being the most prominent). We also used this time to estimate stories and add stories them to our task board.

Use a Software Tracking Tool

Speaking of a task board, we chose to skip a software tracking tool for our iteration planning and use sticky notes and a rolling white board instead. The board is visible to everyone in a communal area and it's very nice to interact with something tangible for planning and doing stories and tasks. The board did eventually get translated into the software tool, but that is only because we have a developer who isn't co-located and we need a way of sharing the task board across time zones.

What Worked

The Task Board

Of all the things that are really working this week, the physical task board has to be the winner. Prominently displayed stories and tasks (completed and in progress) is a great reminder of what we're working on and what we got accomplished. Planning using the board was also much nicer as we could move stories around in priority order and drop them into the iteration much quicker than fumbling with a software tool. This is definitely a keeper and I'm actually excited about planning next week's stories.

Daily Stand Ups

We went back to a five minute stand up - one minute per team member. We can check in, get a level set for the day and move into planning and clearing road blocks immediately afterward. We are using Scrum's "did, doing, impediments, confidence" questions which is working for us. When it becomes tedious or boring, we'll switch it up to something that works for us.

Areas of Improvement

TDD

We're all practicing TDD now, but I didn't feel I was able to really get in and write the tests I wanted to for the week. I saw plenty of room for refactoring, but a couple of in-progress stories that had hard deadlines meant I only had time to do some incremental improvements to code, rather than create fully tested objects. That being said, we spent a lot of time learning the process and tools, so it wasn't a failure, just an area for improvement.

Planning

Our planning was a bit rushed this week. Impending deadlines and being short-handed due to illness and planned PTO meant we had to crash through our planning session. We were missing our primary product owner since he was at a conference, so we did the best we could with the people we had. Like TDD, not a total failure but I'm confident we will be better next week.

Summary

Agile in a classroom and Agile in the real world are two different animals. In the classroom there aren't meetings, email and support calls to take and everything works (almost) according to the book. As a way of building software, it's incredibly useful, but we're going to have to slowly incorporate all the best practices as we're able to absorb them. I would say the first week was a huge success at small improvements - each of which will add up to better efficiency, better code and better value for our customers. It might just take a few months to get there.

Additional Resources

Test Driven Development

• MXUnit - This is a unit testing framework for ColdFusion. If you do CF, I highly recommend it.
• James Shore's Let's Play TDD - Jim goes through an actual Java software project from scratch using TDD. You get to see it built up warts and all. It's incredibly useful to see it in action.
• Test Driven Development Demystified - this online meetup was an excellent one hour presentation on TDD and Katas. Like the Let's Play, a demonstration of TDD happened live (it's now recorded and available).
  Slides and Code

Agile

There are a ton of resources on Agile out there. Here are a couple to get you started.

• James Shore's Blog
• Diana Larsen's Blog

What Do You Think?
Do you have a question or comment about Agile? Have you tried it or are currently doing it? I'd love to hear your experiences and what worked and didn't work for you. I'd also like to know if you have additional resources I should add to my list. Thanks!

Catching the Disease - Test Driven Development with MXUnit

I had the pleasure of attending CF.Objective() 2010, my first, and it was a great experience. One of the sessions, led by Marc Esher, was about integrating Hudson, MXUnit and Ant. I later attended a BoF facilitated by Marc on Test Driven Development (TDD). In that session there was discussion of becoming "Test Infected"; that is, having unit testing be so ingrained in the development process that one wouldn't think of writing a line of code without it.

I wouldn't say I'm infected yet, but I'm starting to feel some symptoms.

Innoculation
A couple of years ago, I read about this TDD thing and it sounded like a good idea so I downloaded a test harness and started playing around with it. I even began writing my tests before writing my objects just like the process described. It was a miserable failure.

I wrote the tests, got little green "pass" bars and created my objects. Woohoo! But what did all those extra lines of code get me? At the time it seemed like I spent way too much time trying to figure out what to test and how to test it and not enough time actually building functioning code. In short, I just didn't get it - I must have been immune to the infection.

Catching the Disease
A few years passed and I began reading about this testing thing again. I had need to test some objects against a third party integration that was being upgraded and decided to try out MXUnit to write the tests; I figured it would be a better way to go than just manually executing the functions and dumping the results. I was right - it was better. Way better.

Not only did I get little green bars (woohoo!) I also had a test that I could execute every time there was an upgrade or I wanted to change a bit of code for this object. I was getting warm - could that be a slight fever coming on?

Tests are NOT TDD
I was beginning to see the value in scripted testing but writing tests isn't the same as Test Driven Development. The process of TDD means writing your tests before your objects. The idea is that you know what you want your object to do or return, so write a test for it. These tests will fail because...the object doesn't exist yet. Our job as developers is to make the tests pass and in doing so writing an object that does what it's supposed to do.

Recently I needed to create an object to return a list of courses from an online learning service. The object would return both a list of courses and an individual course by ID. It was a web service, so it needed to create the web service object which I would use to return my results. I knew what I wanted the object to do and return, so now I could write the tests.

I started with a basic test - testCourseService.cfc. I wrote a test for the first function which I called "testGetCourses". I expected the function to return an array of courses so I needed to a) invoke the web service and b) execute the web services function to return results. I realized at this point, I would need to test that the web service returned the expected object. I continued to write tests and add functionality in tandem until I got the tests to pass. In the process, I got more than a functional object - much more.

Order Now and We'll Throw in the Documentation...For Free!
Aside from the comfort of knowing my object functions in the desired manner, I also get a host of other benefits:

• Tests document the desired behavior for the object
  By recording the expected behavior of each function, I can now look at the tests and see the intent. For the CourseService mentioned above I wrote a test for a getCourse method that included a test for a bad ID, no ID and a valid ID being passed into the function. In each case the expected behavior was indicated in the test.
  
  
• Tests Illuminate Architectural Decisions
  In my head, I had assumed I would need a DAO for my CourseService to abstract the data access functions. I was thinking that the service would be the public interface and I would hide the internals in another object. The tests led me to drop that idea and keep everything in one object. This was a very simple component - in more a complex situation, the tests would help me understand how to design the package and dependencies before actually getting too far into the code an realizing the architecture needed a refactor.
  
  
• Speaking of Refactoring...Tests Make Refactoring Easier
  It's inevitable. You'll want or need to refactor your code eventually. Either there's a better algorithm, redundant code or new technology you want to take advantage of. Regardless of the changes you make, the tests should still pass. If they don't, then that tells you something about your changes. Of course, you may need to write additional tests to accommodate those changes, but when you get those green bars, you're good to go.
  
  
• Serendipitous Decoupling
  Not only would it make a great band name, it's also an additional benefit of unit testing. Objects need to be tested as units, meaning each individual unit is tested apart from any of its dependencies or objects dependent on it. Because of the need to test our objects in isolation, we end up with a more flexible design that tends to follow proven design patterns.
  
  
• What Am I Missing?
  I'm sure my list is incomplete - and maybe inaccurate. What am I missing? What did I get wrong? I'd love to hear your comments!
  

Much Still to Learn
So would I say I'm "test infected"? Yes, I think I've got the disease. I understand the principles behind TDD, I find testing valuable and have started to make it a regular part of my coding, but there is still much to learn. I think writing tests is as much an art as a science. Trying to figure out what to test and how to test it is a challenge and, let's be honest, it takes time to write tests.

Luckily, there are a lot of great resources out there to help us out, some of which are listed below, and like any new skill it takes practice - a lot of practice. I'm planning on making this an ongoing series to help document my trials and tribulations in the world of unit testing and TDD. If you have any suggestions or hints or think I've just gotten it wrong - let me know! I'm new to this TDD and I know others out there are too.

Further Reading

• MXUnit Wiki - Start here for a good overview on testing. Don't be intimidated by the "Advanced" section, either. It's worth wading into.
  
  
• MXUnit Blog - A lot of good information here - it's a blog, though, so you'll need to just browse through to find what you're looking for.
  
  
• MXUnit Google Group
  
  
• Test Driven Development with ColdFusion – Part 1: Driven by the cold
  
  
• Test Driven Development with ColdFusion – Part 2: Designing components for easy testability
  
  
• Chapter 4 of Agile Software Development by Robert C. Martin - It's a dead-tree version, but worth the read. Really good overview of agile, design and UML as well. It's spendy, so I checked it out from the library.
  

If you have other favorite resources, let me know. Happy testing!