Extreme Programming And Web Development Process

Applying (some of) the Values, Principles and Practices of Extreme Programming to the Overall Web Development Process

The Standard Web Development Process (gone bad)

Your company is vying for their biggest business of the year. After a spectacular and lengthy pitch, the client announces your company will create their new web site. A contract is signed and the pitch team celebrates the win. Soon you start an official discovery phase and develop the business requirements for the project. They are met with enthusiasm and agreement. The client accepts the proposed budget and schedule set forth in the accompanying Work Plan and the project rolls on.

As you move into a development phase, your team creates detailed project documentation including functional specifications on what the system will do, site maps, page schematics and design development. Before now ideas were mostly conceptual, the business requirements talked of several ideas but gave no specifications on implementation. Your team is feverishly writing specifications for a list of features and functions as long as your arm. Unfortunately, the time estimates in the Work Plan were given without consulting the actual development team and now, when the scope of work is better defined, the development team says the time estimates are unrealistic.

To make matters worse, you and the client suddenly have very different ideas of what exactly the project entails. When you sit down with the client to figure out the details, you realize that the time and budget estimates were unrealistic for the actual amount of development the client expects. Once in development, new situations are constantly arising that impact the budget and timeline. The scope of work created during your discovery phase wasn't detailed enough and there are areas of the project as yet undefined. The client wants what was agreed to at the agreed upon cost. It's not their concern that there are unforeseen complexities. As development continues, the problems are compounded. It is now clear that the research done at the start of the project failed to properly assess the users' needs. Your team wants the project to evolve in a new direction, but with client relations already stressed and delivery dates being missed, they have little choice but to march forward. This predicament is sometimes called a 'death march project.'

In the end, your company loses money and the client doesn't get what they wanted. It's a lose-lose situation. Why was this project doomed? Because the company failed to properly assess users' needs, they didn't accurately define the project scope, and most importantly, they couldn't react effectively to change. This type of process assumes that the client and the company can determine what they want up front, and that no one is going to change their mind during development. It's unrealistic.

What is Extreme Programming?

Extreme Programming (XP) is a lightweight methodology for developing software that provides the highest value for the client in the fastest way possible. I first learned about it from a Tech Lead on one of my projects. He gave me the highlights: Programmers work in pairs, there's continuous integration of the code, the project is divided into small manageable releases, and the client is involved throughout the entire development process.

To learn more about this practice, I read with great interest Extreme Programming Explained — Embrace Change, by Kent Beck. I have not worked with an XP team, and I'm sure there are many idiosyncratic complexities to XP that can't be described in a book. However, the book does a great job presenting the theory — and in theory, XP is beautiful. A collaborative environment of enthusiastic, dedicated workers with only the success of their client's project in mind.

In my experience, most Internet consulting companies share similar process and deliverables. Flaws in the web development process are often ignored. Companies learn from prior experience slowly and the same mistakes are often repeated. The fact is, in five years of web site development with large consulting companies I've never worked on a large-scale project that was delivered on time and on budget.

XP methodology addresses some of the failures of the typical development process. I don't believe that XP can be unconditionally adopted for web site development however I've extracted key values, principles, and practices that, if applied, could improve the total web development process and reduce project risk. I don't go into description of the many usable procedures XP delivers for technical development, rather I'm focused on the practices that can be applied to the other development groups to improve overall process.

Risks in Development

A project's success is constantly being jeopardized. Unexpected challenges arise on every project, the key is to roll with the punches and minimize the impact. The common risks involved in web development include:

• The business is misunderstood
• The business changes during development
• The scope wasn't clearly defined
• The scope changes
• Schedule slippage
• Staff turnover
• The project is cancelled

All these can be expensive to correct when the plan isn't flexible. How does XP show us to reduce risk?

• Improve responsiveness to business change
• Improve planning practices
• Improve productivity throughout the project lifecycle

Improving Responsiveness to Business Change

The typical web development process relies on big design up front. The master plan determines the functionality for a project that may not launch until six months to a year from now. That approach doesn't leave many options, such as the option to change directions in the middle of the project or the option to grow the project when inspired. These aren't factored into a 6-month development cycle without flexibility.

One step towards improving responsiveness is changing the way we plan projects. If flexibility is not factored into development, we cannot respond efficiently to changing client and market needs. 'Scope creep' is a dirty word since these mid-course project changes often lead to cost and time overruns. But in order to be responsive to business changes we need to accept change as unavoidable, and adjust our planning process to both support the client and protect our company. The problem isn't change, it's the inability to deal with change when it comes.

Improving Planning

One key to XP is team involvement. It is important to bring the team together often (XP practices daily stand up meetings). Together you decide on scope and priorities, together you estimate cost and schedules. Getting everyone together often provides a communication channel that can facilitate feedback and build confidence in the project. If we all hear what others are doing and where they need help, we can see progress and support each other's efforts.

When planning a project, it is typical practice to over plan. Planning for web development produces detailed functional and technical specifications, interface schematics and supporting diagrams. This is the problem with existing web development process. All the planning is done up front and changes during development are frowned upon. This can stifle innovations and outdate products even before launch. By shortening the release cycle you can improve the chances of a strong, on-target deliverable and reduce risk. Divide the project into smaller manageable releases and plan one iteration at a time. Work with the client to determine release priorities. Long range planning can be done, just not in great detail. Cover this release in great detail, and the next five with lists of bullet point items. It's important to keep focused on current needs. Often we will plan and design for tomorrow — speculating 'what if' scenarios that impact our current choices. If we can make our products flexible, then change is not expensive.

Another effect of smaller releases and incremental planning is more realistic budgets and schedules. We all know that it's very difficult to accurately estimate development time before we get our hands on the work. A designer asked how many days it will take to create thirty screens for a brand they haven't explored and IA they haven't seen is just shooting in the dark. But if the scope is focused, team members will likely have a better understanding of the work and therefore a realistic estimate.

All too often the development team is handed a project with impossible deadlines and unrealistic requirements. The team dutifully goes to work on the impossible task they've been given and begin to feel demoralized when they miss deadlines or deliver work not up to personal standards. One of the practices of XP is to always have the individuals who are responsible for delivering something to also be responsible for estimating it. Estimates should be based on experience. In XP responsibility is accepted, not given. This means that the manager can't go to the team and say; "Here's all the work we have to accomplish, and here's how long we have to do it." Rather, she would go to the team and ask them to take responsibility for the work and then listen to the answer. Together they can work out the priorities and schedules; as development continues there are continuous schedule and budget reviews.

Improving Productivity

Client on hand

In order to develop a smart product, you need access to someone who has a suitable understanding of your customer profile and their needs. This might be a client representative, or an end user, but it could also be an Information Architect sitting with the technical development team to answer all the user experience and interface development questions. This person can write functional tests and help establish priorities.

Refactoring

When developers implement a new program feature, they look to see if the existing program can be changed to make the feature and code simpler. Refactoring can be applied to IA and design too. As new features are implemented, we want to take time to step back and look at the overall interface design. Is there anyway this can be implemented smoother? Where else might this feature apply and how can we make it more cohesive across the entire site?

Open, honest communication

One of the core values of XP is communication. XP programmers believe "There is no such thing as too much communication." This communication includes ongoing feedback on development both internally and from the client. XP employs coaches whose job it is to notice when people aren't communicating and reintroduce them. The fundamental lesson to communicate often and openly can help any team dynamic.

Work with people's instincts

This seems so simple, but it's a truth that is sometimes neglected. People like learning. People like being in control. People like being trusted. People like doing a good job. People like being part of a team. If we work with people's instincts, not against them, we build a strong, healthy team.

These are some of the lessons that XP can give us. As I stated earlier, XP can't work unconditionally for the whole web development process. However if we take some of these lessons and apply them to our own process we can reduce project risk, build a stronger team, and a better product.