Extreme Programming for Offshore Development
Offshore development is
very popular today because of the existent of the Internet and lower
cost of high speed network allows people to work from almost anywhere.
It provides many benefits such as lower labor cost, shorten development
schedule, access to technical talent. However, distributing software
development over a vast distance, different time zones and cultures
creates the risk that the application does not do what it was intent to
do. There are several issues that contributed to this risk.
- Companies typically develop a set of requirements and then send them offshore for development. There is very little interaction between the customer and the developer. Without business knowledge, developers could misinterpret the requirements and create software that technically conforms to the requirements but in reality, missing the point on the business requirements.
- Customer crams in too many feature in to the requirement for fear that once the requirement is done, it’s difficult to make changes.
- Customer has limited visibility on the status of the project. They could not see the progress of the product until later in the development stage when it is the costly to fix bugs. This leads to costly budget and schedule overruns.
- Integration test is done near the end of the project when it is very expensive to fix architectural problems.
Extreme Programming (XP) addresses
many issues associated with offshore development. XP focuses on close
communication and collaboration between users, developers and QA so that
everyone understands the requirements. XP uses iterative development
technique where the difficult and high risk part of the development is
done upfront. Finally, XP incorporate integration test in all of its
phases so there is no integration risk in the end.
Applying XP to Offshore development has many challenges due to the communication issues result from distance, time and cultural differences. The following are my research on how to overcome those limitations.
Project Solutions
Send Ambassadors between
the sites
XP stress the use of small and collocated teams.
This encourages communication and collaboration between team members so everyone
has a common understanding of the project requirements. Offshore makes getting
the team together in one location difficult. We can send people between the
sites helps to facilitate communication. From the beginning of the project,
teams should always have ambassadors from other sites to help building
communication channel using their personal contact. Sending US developer and
business analyst would help to communicate the technical and business
requirements. Also, sending someone from offshore would help to improve
communication for the teams.
Sending US business analyst would help to provide
business context to the requirement for the offshore developer. By knowing the
business context, the developer can develop the software for what it is intended
to do.
The ambassador can help to communicate gossips.
On any project, there is a lot of informal communication. Some of them are
important but they might not be appropriate to bring up during the formal
communication.
We should rotate ambassadors every few months
because they will lose their contact at home if they spent too long abroad.
This makes it easier for people who do not want to be away from home for a long
time to serve as ambassadors. It would also allow more people to know the
remote teams by serving as ambassadors.
Finally, project manager should spend some time
as ambassadors to help resolve conflicts and problems before they become
serious.
Use contact visits to build working
relationships
Having just ambassadors are not enough. We will
need to involve a wider range of people. Contact visits help to build working
relationships that will improve remote communication. There are two kind of
contact visit: seeding visit happened early in the project and is used to create
working relationships. Maintenance visit help to keep the relationship going.
Seeding visit should be at least two weeks long
and should be a working trip. It’s important for everyone to be working on a
same part of the project so that they get used to working with each other. Since
this visit is used to build working relationship, we should relax the work pace
so that the team members could have more human communication. Sometime, we
should schedule in team building event such as sight seeing and dinners so
everyone gets to know each other.
When the seeding visit is done, we could keep up
the contact with maintenance visits. Maintenance visits should be a working
trip but short in duration. It should be one week every couple of months.
Manage Culture Change
Culture is the major reason why most organization
failed in adopting the Extreme Programming in their onshore and offshore
development. Many companies operate in a command and control mode where
employees are discouraged from asking questions, talking about problems, or
proposing alternatives. This is especially true in Asia because Asian culture
see those kind of behavior as challenges to the superiors and was not well
received.
Furthermore, the 40 hour work week mandated by
Extreme programming is very difficult to follow because a normal work week in
Asian comprises of 60 to 80 hours including Saturdays. Also, employees are
required to arrive to work before the manager and leave after the manager.
People who do not do that will result in bad performance review or even
termination. This kind of work schedule will create too much stress for the
offshore worker and result in increase number of bugs because people are too
tired to concentrate on coding.
We will need to give people who are doing the
work more autonomy so they can make decisions and negotiate requirements.
This will improve the quality of the product because people are more motivated
when they have the freedom and the responsibility of making decisions. The best
way to impose cultural change is to create offshore development units instead of
using external offshore vendors. People working for the company unit are more
likely to following company policies and work schedules than external vendors.
It would also makes it easier to retain talent and knowledge since the company
would have direct control on what people will work on and who will be in the
team.
Also, we will need to pay extra attention to
small issues because it takes a long time to get people to get used to the
distributed control management style. We can not assume that problem will be
raise, even if it were spotted.
Use collaboration software to store
information
In any project, there is much common information
that needs to be shared and retained for future references. Email is not a
good tool to use for that purpose because it difficult for someone new to the
team to see what has been done before on the project. Also, we can use it to
store lesson learned so that they can be used as a reference for future
projects.
Wikis works well because they are simple to use,
can be accessed with any browser and are easy to setup. We could also use
others products like Lotus Notes, Share point or collaboration tools that are
build into the project management software.
We can put story cards, design guidelines, build
instructions, notes on progress and any information that needs to be written
down for reference by the team in to the tool. Since most of tools support
change notification by email, everyone in the team can be keep updated on the
project.
Since collaboration tools are by nature
unstructured so it could accommodate different type of information. As the
project grows, someone on the team needs to make sure the information is
organized in the collaboration tool to keep it from growing out of control.
Use Test Scripts to Communicate Business
Requirements
With greater distance, we will put more emphasis
on communicating requirements. Since XP stress the use of test before coding to
ensure that the application will function as intended, we can use acceptance
test as a way to communicating requirements. Before we start an iteration, we
write the test scripts to help clarify the requirements so the developers have
an understanding of what the final product is like.
One way to do that is to have a US based customer
to write a user story first and then have an offshore analyst/tester to create
test scripts for the story. The offshore analyst would work with the onshore
analyst to develop the test scripts so that offshore analyst could really
understand the business requirements. Writing the tests forces the offshore
analyst to understand what’s needed and to ask the customer questions to clarify
issues. This way, the offshore analyst can serve as a point of contact for
offshore developer to ask questions if they have problem understanding the test
scripts. It would help to increase the efficiency of the offshore developers
since they would not have to stop and wait for clarification from the customer.
Use Continuous Integration Testing to Avoid
Integration Headaches
Bring it all together for implementation in the
production environment is one of the most difficult parts of any software
project. This happens often with offshore projects because the developers are
far removed from the production environment. It has significant impact to the
project budget and timeline because it is very expensive to fix the problem near
the end of the project.
We need to apply continuous integration – the
process by which developers integrate their code and build the entire system
whenever they have made changes –to reduce or eliminate configuration-management
issues. Offshore teams should check code into the same repositories as US
teams. Check-in is preceded and followed by build verification tests (unit
tests) as well as automated functionality testing. When a test fails, a build
fails; the developer can not go home until the check-in results in a successful
build. A bad build is very serious when the remote site is running off the same
build. The value of continuous integration is that it solves small integration
headaches immediately instead of trying to solve huge ones at the end of the
project.
There are free continuous integration toolkit
products such as CruiseControl that will help to automate the build and test
process. CruiseControl is a framework for a continuous build process. It
includes plugins for email notification, Ant and various source control tools.
A web interface is provided to view the details of the current and previous
builds.
Use Regular Builds to Get Feedback from
Customers
When working on requirements, people often fail
to see the importance of the feedback loop. The requirements process is
typically done by analysts providing requirements to the developer to
implement. Only at the much later time when the analysts check to see if the
developers have implemented what they were asked for. With XP, the close
proximity between customer and developers allow the customer to monitor progress
much more frequently, which allows them to spot misunderstanding more quickly.
Furthermore a partially developed system can educate the customer because there
is a difference between the requirements and what is needed. This difference is
not apparent until the customer could see the working product.
Having offshore performing regular integration
builds allows the customer to pull down the latest work and try it out. While
this is not immediately due to bandwidth, it still allows the customer to
correct any misunderstanding quickly and refine their own requirements.
It’s important that the customer is looking at
the same environment as the offshore team because any problem they discover
would need to be replicable by the offshore developers. Building a duplicated
environment is costly and would require bandwidth and expertise to setup. It
is also very difficult to keep the environment in sync because the software is
evolving in a rapid pace.
Remote control software such as VNC or Remote
Desktop with secure VPN connections would allow the customer to test the
offshore environment without the complexity of maintaining a separate
environment. It would also allow customer to look at what’s build regularly and
spot any miscommunication. This would reduce or eliminate any rework results
from misinterpreting the requirements.
Use Regular Short Status Meetings
XP promote regular short status meetings to
communicate problems, solutions and prompt team focus. Everyone stands up in a
circle to avoid long discussions. It’s more efficient to have one shore meeting
that everyone is required to attend than many meetings with few developers
each.
Carrying this over to offshore development teams is important because they can coordinate with other teams. Time zones are the biggest problem, particular between the US and Asian countries where there little overlap in a workday. Therefore, having twice a week stand-ups meetings in the beginning of the project provides enough coordination between the teams.
Having collaboration tools such as wiki could
help reduce the need for cross-shore stand-ups in the later part of the
project. But having stand-ups within a shore’s team would still produce time
saving benefits.
Separate teams by functionality not activity
Traditionally, determining activities perform
onshore vs offshore is done using the waterfall model. Analysis and design is
done onshore, coding is done offshore, and acceptance testing is done onshore.
Having offshore team handle as many activities as
possible improves the project efficiency. Offshore team should do as much as
possible on analysis and design work so they have a better understanding of the
requirements. When splitting the responsibility of the onshore and offshore
development teams, we should do it along the functionality grounds rather than
activities. We will break the system into modules and let offshore develop some
of the modules. Continuous integration testing allows the module interface to
evolve as development goes on.
By letting the offshore team handling the full
range of activities would allow them to grow their analysis skills. The more
the offshore development team understands the business, the more the offshore
development team can develop efficiently.
Do more documentation
XP downplays documentation and prefers to
document through the code because a large part of the documentation effort is
wasted. However, documentation is more important with offshore development
since face to face communication is reduced. It is a waste because if the
whole team is co-located, it would not be needed. It is the price of doing
things offshore.
Active collaboration tools such as wikis help to
reduce the amount of documentation needed. Because it is unstructured, it can
fit better into how you want to work.
Setup multiple communication channels
Different communication channels work for
different kinds of problems. At the minimum make sure you have a wiki, instant
messaging, and good telephone connections.
Instant messaging is good for quick question but
also telling you when people are available. This way you can spend less time
trying to track people down.
Telephone is a great tool for getting answer for
multiple questions. Make sure any project team member can pick up the phone
and directly dial any other team member and talk for any length of time. Make
sure that international dialing is enabled so that onshore team can contact the
offshore team. Do not worry about the cost of the phone call, getting questions
answered quickly will more than pay for the phone call. We can also use Voice
of IP (VOIP) technology to reduce the cost of international calls.
Having a High-availability, high bandwidth
network connection and VPN between the sites allows the onshore/offshore team to
share machines and resources. It will help in supporting a common version
control system which would help speed up the integration testing.
Although everyone uses it, Email is not a useful
tool for XP. Email conversation between two people is less efficient and
interactive than a phone call or an IM chat. Email copied to large groups
spawns fragmented or unclear response. Prior Emails are not available to people
who join the project after the Email was circulated. It would be wise to
restrict e-mail usage and encourage team members to switch to other
communication tools described here. It’s also helpful to perform a daily review
of outstanding unanswered e-mails as part of regular status meetings to speed up
resolution.
Conclusion
Extreme Programming for Offshore Development has the potential to address many of the shortcomings with offshore development. It should also be clear that you must proceed with care and work hard to build up your competency in the area. The effect on communication due to distance makes it difficult to meet face to face, and the time zone difference, makes it difficult to have a real time conversation. These limitations increase the chance for miscommunication over the requirements. While by applying the best practice described above, there’s still going to be some effect. Offshore work also introduces extra costs and risks that may offset the cost savings. I hope my research will help you increase your chance of succeeding in offshore development projects.
References:
- Using an Agile Software Process with Offshore Development Copyright Martin Fowler
- Distributed Agile Development and the Death of Distance by Matthew Simons Sourcing and Vendor Relationships Vol 5, No. 4
- Internationally Agile by Matt Simons, Inform IT. Date: March 15,2002.