Even programmers can be whole people in the real world. XP is an opportunity to test yourself, to be yourself, to realize that maybe you?ve been fine all along and just hanging with the wrong crowd. ? Kent Beck
Recently at a local Agile meetup, an attendee asked me a very interesting question: ?Why is eXtreme Programming not so popular compared to Scrum?. Another variation to that question: ?What makes Scrum so popular compared to eXtreme Programming?? As I am getting question that are similar to this from time to time, I thought it is better for me to document my thoughts in a blog so that I can reference it to whoever ask the same question.
Disclosure: I am a Professional Scrum Trainer through Scrum.org. This is not a criticism against XP because I believe XP is a great model. And this article is not a rant over XP unpopularity. Over the years I have been educating managers to incrementally put XP engineering practices inside their Scrum framework. I have not been so successful. Not so many managers bought the ideas and beliefs from XP. So you can say that this is an opinion about XP unpopularity from a Scrum guy. A Scrum guy who are actively pushing XP into organisations who are using Scrum until today despite the challenges.
When I read the latest State of Agile report from VersionOne, I am surprised to see only 1% of organisations who are using XP. How can you actually be Agile without any technical practices? As if Agile is only about project management. If we also take into account Scrum/XP hybrid then there are at least 11% organisations who are using XP. I admit that XP is very good and Kent Beck is a genius, the people around him during XP formulation like Martin Fowler and Ron Jeffries are geniuses, there are many good philosophies behind XP. It is quite unfortunate that 15 years later after Agile Manifesto meeting XP is not as popular as Scrum.
So back to the topic, why is eXtreme Programming not so popular compared to Scrum? Let?s look at it together.
Software development has been project management centric. Even principles from construction project has been brought into software development, which we all know most of the times is not relevant.
Most Project Managers who built their career from software developer (at least the ones I?ve worked with) do not actively code anymore hence they do not have the capability to be an XP coach to coach the software developers on XP engineering practices. Now imagine project managers who has never been a software developer ever. Because of this reason XP literally scares them away and they will try to push XP out of the way.
XP put more emphasis on software engineering pratices than project management. XP put more emphasis on the engineers than the project managers. This will bring up a question for management: ?What do we do with all of the Project Managers?? To be an XP coach is too far off for many Project Managers. With Scrum, at least Project Managers can still map themselves to be a Scrum Master ? eventhough Scrum Master is not a Project Manager. Scrum makes more sense to existing project managers than XP. XP was not created for project managers. XP was not created with project management in mind.
2. It requires high investment
Rockstar Developers
To start with XP you need committed and calibre software developers. Not only calibre but also committed to quality. They?re the ones who will not ship software with no test harnesses to production for any reasons. They do not only care about code that works but also code that is clean and testable. They are the kind of developers that managers don?t like to work with because they are not permissive and can not be bribed!
The reality is:
- Calibre software developers are difficult to find in the market. Try finding fresh graduate software developers who already know XP practices. Good luck with that.
- Most software developers think they are more inferior than managers.
- Even if calibre and superior software developers exist, they are on high rate.
Companies are on a mission to cut down costs and hiring rockstars will blow up their mission. That is why there are many companies are offshoring software development to make their IT cost more efficient.
Testers on an XP team help customers choose and write automated system-level tests in advance of implementation and coach programmers on testing techniques. ? Kent Beck
In XP, Testers works as Quality Assistance rather than Quality Assurance, they are responsible for assisting and coaching the customers and the programmers on quality. What a bummer! Where do you get testers like that in the job market when the majority of testers think that their responsibility is to do ?Click Testing? or ?Monkey Testing?? How do you convince the management to hire that kind of testers when companies are on a mission to hire the cheapest testers as possible? Heck some companies even outsource testing work to developing countries in Asia because of this cheap as chips mindset!
Training
With Scrum, companies can just send their project managers to a 2-days Scrum Master certification training or Agile Project Management training. It is much more cost effective. But with XP, companies need to train the whole software development team! Not to mention that the rate of XP trainings are much more expensive than Scrum trainings. This is why Scrum Master and Agile Project Management training is so popular in the market compared to eXtreme Programming training or Software Craftmanship training!
Now imagine if an organisation has 20 more times software developers than Project Managers. Sending all of the software development team to Agile software development training or software craftmanship training will blow the company?s budget. Justifying the cost in front of HR and Chief Financial Officer (CFO) is too darn exhausting.
And besides that, software developers are on a lower salary rank compared to Project Managers. What if after we train software developers they leave the company?
With Scrum, people who attended Scrum Master training can claim they?ve done Scrum after forcing everyone to do Daily Scrum Meeting. With XP, it is not as simple as that because you aren?t doing XP if you are not doing all of its technical practices!
Infrastructure
To run XP, you need investment for automated tests & continuous delivery infrastructure. Yes there are several open source solution out there BUT some of the best solutions for automations are paid version.
Culture
You also need to change the organisation culture by bringing multiple siloed department together. XP explicitly requires the whole team to sit together, from Testers, UI/UX designers, Architects, Users to Programmers. To be able to do continuous deployment daily effectively, you may even need to break the silo between development and operations (DevOps). This is a high investment that many managers resist to do starting from day 1.
XP Coaches
Hiring XP coaches is expensive that is because they are software craftsman. A craftsman is expensive because they care about quality and they are idealists. Not only they?re expensive, they?re also difficult to find in the job market. Hiring XP coaches for the company will be hard to be justified in front of the Human Resource and CFO. What will these XP coaches do all day? If justifying hiring a full time Scrum Master is hard, justifying hiring a full time XP coach is even harder!
3. It is irrational
Most businesses, even IT businesses, are lead by non-technical people. Many practices in XP is seen as irrational from business people perspective.
Unit Testing
To business people, writing unit tests means more code and more time to write it. More code and more time means more costs. Well, to business people that is just not cost efficient. Remember, company is on a mission to cut down IT cost not to add more cost. Heck about increasing the value of the software by writing test harnesses, IT is seen as a cost centre not as a profit centre. Having software developers to write down twice more code with the same amount of features every iteration just does not make any sense from business perspective.
Test First Development
Many people have the belief that you always test after the product is completed. Even testers can not imagine how to write tests before the software exists. That is why writing tests before the product even exist is seen as irrational. How is it even possible? Many testers and managers even told me that writing test first is a waste of time. Just because to do something is difficult it does not mean it is a waste of time.
Story Points
Not too many people realised that Story Points estimation is actually an XP practice not Scrum practice. Story Points estimation has been adopted outside of XP team but not without any challenge from the business people. Until today we still see people debating between estimating in story points and estimating in hours. Heck some organisation even waste their time estimating in points then mapping it to hours estimate. That is because prior to Agile movement people are used to estimating in hours.
Pair Programming
Having two programmers on the same computer working on the same feature is illogical to business people. Why would you waste money for two programmers doing one thing? They think that with pair programming they will pay twice amount of money for the same amount of features delivered by one programmer. Remember, companies are on a mission to cut down IT cost not to increase the value of the software.
4. It is too difficult
XP is difficult to start with, difficult to do it well and difficult to sustain it. XP is just way too difficult. So people give up and prefer not doing it instead. They prefer undermining their own capability.
Test First Development
Even though TDD itself is already difficult to start with, but it is relatively easier to do TDD on a green field project. But what about brown field project with legacy codes? Most projects software developers dealing with today are not greenfield project. Because it is too difficult and it requires high investment to place the unit tests inside legacy codes, people prefer not doing it. Besides, no executives will pay the software developers just for putting the infrastructures for unit tests inside legacy codes. How can you justify the cost vs benefits?
Refactoring
XP emphasize on simple design and clean code. To achieve that goal, the developers should actively refactor their code, keep their code fresh at every point of time. Refactoring is an art and it requires developers diligence and willingness. Many developers will think, if the software is still running flawlessly, why should we refactor that may introduce a new bug. So let?s leave it as is.
Simple & Emergent Design
Many people want a holistic view of the software design and architecture. Many people want to make sure that the architecture is complete before developers start coding. But XP actually emphasize on emergent design. This is not easy because emergent design is possible through TDD. And as the code base grow and when you are under pressure to meet the deadline, how can you keep the design still simple?
5. It is easier to notice bad XP implementation
XP is quite prescriptive in its practices while in reality people want flexibility. In reality people want some room to make excuses. In reality people don?t like to be blamed when they have implemented something badly, just like when parents don?t like being judged about how they are raising their children.
Scrum has prescriptions but not as prescriptive as XP. It is easier to get away from bad Scrum implementation than from bad XP implementation because Scrum has many rooms of ambiguity. Something that is more ambiguous is easier to blame than something that is more explicit. When Scrum does not work, people can easily blame Scrum. When the quality of the software delivered by Scrum team was bad, people can easily blame Scrum. When the project using Scrum was not delivered on-time, on-budget and on-scope, people can easily blame Scrum.
Now try to do the same with XP.
- XP didn?t work for us. Why? Because the programmers do not know how to do test first development.
- Our project has so many technical debts. Why? Because the programmers never refactor their code and never keep their code clean.
- The architecture of the software is just so messy. Why? Because the architects did not implement emergent design.
- The quality of our software is just so flaky. How is it possible? Did the team write any test harnesses?
- The project did not deliver. How is it possible? Did you do daily deployment to production and get feedback from customers? With daily deployment to production environment in place you can at least get the highest quality features every point of time.
Unlike XP, Scrum intentionally leaves many details so that it is easier for managers to bring Agility in to the company. Yes we can thank Scrum for being the catalyst for Agile movement but we shouldn?t stop at Scrum only. After talking to Scrum co-founders, they told me that their dream is to see organisations that embrace Scrum to incrementally add XP practices inside the framework. In reality not many organisations evolve into XP model because they get too comfortable with Scrum.
One week iteration
With Scrum you have one month maximum to deliver a DONE increment. Some organisation don?t know what DONE increment is ? that is why there are still many organisations doing Scrum extracted User Acceptance Test (UAT) from the Sprint, UAT is done 6 Sprints later in a separate testing Sprint and hardening Sprint. Heck there are some Scrum teams until today who claims to be using Scrum but do not know how Definition of Done looks like. Even worse many Scrum teams who does not even have a Definition of Done!
With XP you are required to ship software to production every week, every day is preferable because XP emphasizes on short feedback loops. The shorter the feedback loop is, the better. That rule is explicity stated in XP. When one month is too difficult for some organisations implementing Scrum imagine how they will look at one week iteration that is prescribed in XP.
XP team even prefers doing one User Story per week as its goal is moving towards single piece flow. But how do you justify this to the management who only cares about doing as many features as possible?
Test First Development
When you are not doing Test First Development you are not doing XP as simple as that. You may have a Unit Test but if it is not written prior to writing the production code, you are not doing XP. The discipline that is required to run XP is extreme.
Continuous Integration
What a lot of organisation doing right now is just Continuous Build not Continuous Integration. We only know that the application is integrated only if there are integration tests. When you don?t have integration test that validates whether the code is integrated, you?re not doing XP even though you have a continuous build every night. If they don?t write unit test most likely they do not write integration test.
How we live is so different from how we ought to live that he who studies what ought to be done rather than what is done will learn the way to his downfall rather than to his preservation. ? Niccol Machiavelli
Well there you go, 5 reasons why eXtreme Programming is not popular. Is it a bad thing that XP has not become the norm in software industry 15+ years later after it was formulated? Well I will let you make your own opinion. This is the reality in the software industry today. XP is a good model for software delivery but unfortunately the world is not necessarily attracted to what is ideal. What should happen in the software industry is not what actually happens in the software industry. If only I realised this earlier I wouldn?t be so dissapointed with the current state of software industry. Well that means there are still a lot of work need to be done to improve the profession of software delivery. Let?s get back to work.
