I have been assimliated!

March 28, 2009

I moved my blog last year because I had two Google accounts and I was two lazy to figure out how that impacted me editing my blog. In the mean time I have become fed up with Hotmail and Bloglines. Thus I have become a googlite. Gmail, reader, blogspot, docs, calendar are all now my tools of choice. Therefore I will go back to mwpolen.blogspot.com. Thanks for visiting.

Advertisements

Being Agile in a Waterfall Environment

March 21, 2009

Where I work the waterfall process is the norm. For a while I have let this fact frustrate me. This has led to me doing a less than stellar job. I would simply throw my hands up and say “It is what it is” and thus pass the blame to others. The victim anti-pattern! What makes this story even more ironic is that almost 7 years ago Rich McCabe and I wrote a paper titled “Should You Be More Agile?” in which our target audience were people in my current position!  Recently I have changed my attitude.

I knew I couldn’t do Agile full blown as access to the customer was off the table. I did have a BA that could act as a customer surrogate so I ran with it. The process has requirements handed off between development and a BA. I got permission to use a teammate to pair with. My pair wasn’t fully dedicated to the project as we both had multiple hats to wear.

We started each day with a 1 hour pairing/check-in session. The first one we  prioritized the requirements that we understood and started working. The first obstacle is that we were adding features to an existing system that was basically un-unit-testable. We did our best to work in short cycles and test (via GUI and DB) as often as possible. It was slower than I would have liked, but it worked better than our typical “one program in a  cube for a while” process. In between each session I would meet with the BA about requirements we needed clarification on as to not slow up the time the pair was together.

After 1 week of working this way we got all the new features done except one new report. The report looked harmless. We estimated it at 3 sessions. After 3 sessions we didn’t even have all the data requested, let alone a presentation of it. After 8 sessions we had the data on the screen, but unformatted! Needless to say we let our enthusiasm of our early success color our estimate. We also didn’t break the feature into small enough chunks nor take enough time thinking about how we would get that data. We also go stuck in several approaches to display the data that came to dead ends. To top it off we had no tests! We decided that making the system testable would have not added enough value in the short time we had. I didn’t like it, but practicality won out.

I have learned some things in this short effort:

  • When stuck, go back to first principles and find a way. Initially I balked at the idea of not involving the customer, but after some reflection realized that though it was less than ideal I did have a customer surrogate.
  • Remain practical. TDD can’t be followed with out unit tests. Spending more time on making a system unit-test-able than it takes to finish the project without it will only cause doubters to have reason to doubt. Use the principles and move forward, adding value each day.
  • Keep disciplined. We could not dedicate 100% of our time to the effort and on some days we got less that 50%. We were able to remain successful when we stayed disciplined (only code together, test everything, estimate small chunks). We failed when we let our guard down and underestimated a fairly large chuck that should have been split. Then we used the pressure of making a deadline to cause us to stop testing everything and code separately when one of us was busy with other things. When we refocused we go the job done without much effort. Discipline is key and doubly so when you don’t have a good support system (automated testing, full-time team, customer involvement).

Given that we are doing this at a time when we are moving platforms gives me hope for the future. The new platform will start off testable from day 1 and hopefully the other obstacles to better productivity will also be removed. Until then we will keep on keeping on.

Software Craftsmanship

March 8, 2009

Thanks to Uncle Bob I found the Manifesto for Software Craftsmanship. Is is built on top of the Agile Manifesto. Since it is short I will, repeat it here:

Not only working software, but also well-crafted software
Not only responding to change, but also steadily adding value
Not only individuals and interactions, but also a community of professionals
Not only customer collaboration, but also productive

After reading and signing it, my mind brought me back to my job. This week was especially challenging. A system I inherited had several production issues. I did not ask for the system, nor do I respect it. This attitude has led me to treat it with disdain and thus not do my best on it. I let my guard down and did everything I know to be bad practice (e.g. no unit tests, large methods & classes) just because the code base was in the same vain.

The first item above struck a cord with me. I had working software, but poorly crafted. It led to a stressful week for me when I could have done better and prevented the problems. I often find myself in situations that appear to be no win. All my friends and colleagues agree with me. What I fail to do is realize this simple truth:

If you don’t like something, change it. If you can’t change it, change your attitude.
Maya Angelou

I am going to attempt to stop complaining about the hand dealt to me and strive to do my best . This will hopefully lead me to achieve the goals in this manifesto. I will also try not to enable those around me to adopt the same SSDD attitude that leads them to do poor work. I know I will fail more than I succeed, but it is nice to write it down and join a large group of other similarly minded folk.

First Time Quality

June 22, 2008

I was listening to the stackoverflow podcast #10 in which Joel (of Joel On Software) stated that code should be written in two phases. The first phase in which you get the code to work and then a second phase in which you improve it. He also states that most of the time you never do phase 2 unless the code is important. I used to work this way and assumed (as Joel does) that this is the best way because I was so smart. I have since learned the importance of first time quality. I still haven’t turned this knowledge into action all the time as I still write code that embarrasses me 🙂

First time quality was something Deming wrote about in his 14 points. Particularly point 3:

“Cease dependence on inspection”. If variation is reduced, there is no need to inspect manufactured items for defects, because there won’t be any.

Obviously Deming isn’t writing about software, but the concept is similar. Testing software at the end is the same as inspecting manufactured parts. The quality of the code is already determined. This should not be news to anyone. Take this one step further, and when should you spend time making the quality better? The answer is obviously as soon as possible. Thus the better you are at writing good code from the beginning the better and faster you will be. For the rest of us (not a Done, and Get Things Smart person), this means we must get better at seeing poor quality and improving it as quickly as possible. To me the only way to do this effectively is if you write your code incrementally with quality checks along the way. I don’t know any method better than TDD to do this.

When doing TDD correctly I find that I can sense the internal quality of the code. I like the term Code Smell for lacking quality, but I also sense when things are going well. I sense that the software design is growing in such a way that new features are going to flow out of it. My productivity soars. This phase usually doesn’t last long as I inevitably do something stupid and cause the code to bloat with duplication. The beauty of TDD is that is allows these modes of hyper productivity while being a safety net for the times of hyper stupidity. I am not saying that TDD will cure all the ails of software development. As long as humans write code there will be ails. It is in our nature. I just know that true first time quality is such a boost to overall productivity that it saddens me that more people don’t do it. I think one of the reasons for this is that we have too many software developers that should not be doing software, but that is a topic for another post, another day.

Management 101

May 31, 2008

I have had a boss since I started working back in the early eighties. I have been a boss and been someone who oversees work as well. During the past 20 years or so I came to believe that the front line manager is the most important job in just about every industry and especially in the software industry. Much like my parenting philosophy, a manager can do little to improve their employees, but they can really mess things up. Why is it that so many manager choose the path of messing thing up? I guess they were never taught management 101. I will attempt to give a primer.

Management’s primary job according to Deming (if you have the book “Out of the Crisis”):

  • Appreciation for the overall system in which work is done
  • An understanding of variation – and the true relationship between cause and effect
  • Constant pursuit of learning (improvement) through designed experiments
  • An understanding of the psychology of people

Thanks to google I found this:

  1. Take time to consider.
  2. Keep mind open — mouth closed.
  3. Obtain information without disclosing position.
  4. Reward people with praise for telling you what they believe to be fact — whether correct or not.
  5. Consider alternatives. Discuss pros and cons.
  6. Make decision — stay the course — support all involved.
  7. Change plan if new information makes it necessary.
  8. Keep confidence of management and customer. Tell the truth, no matter how painful.

These sources are a good start. I will add a few thoughts to Deming’s last point (the most important in my opinion). People, although complex, behave in fairly predictable ways. A person’s productivity correlates to to unmeasurable things: their morale and the skill they have at the task. If they like what they are doing their morale will be higher than if they are forced to do something they disagree with. Skill can be learned, but only if the person wants to be better. If they want to be better you have to get in their way to stop them. 

If you manage people then I believe your number one goal is to make them as productive as possible. Given the above it would seem obvious that you want people who like and are good at what they do. So the primary job is for you to identify those people and GET OUT OF THEIR WAY! I am amazed at the ways managers get in the way. They ask the person to do the task the way the manager wants it done (see above comment on what that does to the employee’s morale) because the manager THINKS they know better because they are the boss. This is so sad, but I see it happen so much that I think it is a pattern. The other thing I see is a manager who can not tell who the talent is. They look for people that are like themselves and make them the A-team and treat the others as second class citizens. Unfortunately this happens because upper management uses the same technique and it propagates down to all without a clue. Those with a clue tend to leave the organization.

I am sure I have glossed over things and simplified somethings too much, but I would love to hear other opinions.

Project Change, a Way of Life

May 26, 2008

I just read  Project Change, a Way of Life by Jurgen Appelo (http://noop.nl/ProjectChangeWayofLifeV1.pdf – ~2mb) and I thought it was worth the read. His goal was to “link complexity science with agile software development”. He starts with change is the only constant which leads to the iron triangle. For some reason he puts the attributes on the apexes and not the sides (I think this hides the most important aspect of the iron traingle, but it’s his article).

Next he goes on a diatribe on the term “project” and decides a better term is “product” (which is the goal of most software projects – sorry I just couldn’t help myself). He then gets to an interesitng discussion on success which leads him to this:

I believe that every software product is a success, until it fails. 

I think its a good way of looking at it. He talks alot about his car…he is young after all. He uses the term fitness from biology. Fitness is a relative term about how well a species survives in their environment. I like that: a product’s fitness is the same thing. Take a product out of it’s environment and it could do better or worse. Put other products in it’s environment and it might be more or less usefull.

He calls the game of improve or die the Red Queen Race from Through the Looking Glass. Products change to keep up with the competition or they are no longer used and die. Your nice document management system will lose to a package that costs money once the feature difference is worth the cost.

He then goes into Fitness Landscapes, which a neat idea, but I don’t see a strong correlation with software products. I disagree that people can do “conscious selection” well and natural selection is not well characterized as being random. Now I am just nitpicking. Like I said in the beginning, I think this aritcle is worth the time it took me to read 28 pages.

Why do decent people use the waterfall method?

May 25, 2008

It seems everywhere I go the waterfall method dominates the mindset of the management and most of the technical staff. I wonder why that is? It seems to me that any rational person would admit that the cost of fixing a defect grows exponential as time passes from the moment it was introduced. Most rational people would admit that in the waterfall method most defects are caught at the end of the project life cycle, thus guaranteeing they will be very expensive to fix and that most project will have to resort to testing the quality up to a level that is relative to how late the project is (the later the project the more defects they will consider shipping). Given these assumptions, why do any project using a method that caps the outcome to mediocrity? Maybe my own past can help me understand this.

When I was first paid to do software (circa ’91) I thought the right way to do software was the waterfall method. It was what I learned in Grad school as part of the Software Engineering series and it seemed everyone I met was using it in one form or another. When my job involved the CMM it fit very nicely with this belief. Anyone who though differently was obviously a Cowboy Programmer and was just too lazy to do a professional job. Being young gave me such confidence (ignorance seems to do this).

As I gained experience, I started to doubt my belief in this classic approach. I worked once with a client who gave me new requirements every other week or so and my team of two gave him working code every week or two. It worked well, but I didn’t think why. Later I started teaching others how to deliver software on time and started to really question my belief. I saw projects being executed by CMM level 3 and higher organizations consistently go over budget and deliver sub par systems. About this time I read some Agile books, some Lean books, and the classic paper by Gilb “Evolutionary Delivery versus the waterfall model” and though what a dope I had been!

I seemed to have overcome a belief that wasn’t founded on solid principles, but others seem not to. Humans tend to behave in irrational ways in order to “fit in”. Maybe what I see is just a group dynamic caused by fear of the unknown. They know the waterfall method and the devil they know is better than something they have never tried. The more I think about it, it reminds me of OO adoption. Back when I first read about OO in ’92 (and it wasn’t a new idea by any measure then) I thought this is got to be the way to program. It came naturally to me, but it was years before I was in the majority.

Maybe Agile/Lean (Whatever you want to call it) just needs to ride the adoption wave until the majority accepts it. Then the fear factor will be with those who don’t do it? Then some new idea will come along and some guy will blog about idiots like me who do it the “old way” 🙂

Moving to WordPress?

May 25, 2008

I got sick of blogspot yesterday so I started this today. Only time will tell if I use it.