Becoming a Professional Software Developer using Agile Engineering Practices

The Surgery Analogy

At the Agile 2008 conference in Toronto I heard Robert Martin as one of the keynote speakers.  He told the story of how over a century ago, before there was a good understanding of the germ theory of illness, surgeons didn’t wash their hands before surgery.  It took a lot of work to convince all the surgeons to start washing their hands and this was an important step in the development of medicine as a profession that could be trusted by patients.  He told this story as an introduction to a key point about Agile Engineering practices.

Now, when I am teaching my classes, I tell the story a little differently to get the point across about Agile engineering practices.

Follow along with me…

Imagine you are extremely ill and require emergency surgery.  You are conscious, and you are desperate to get the surgery done!  Your ambulance arrives at the hospital, and the hospital staff quickly wheel you into the operating room.  Because you are so desperate, you realize that seconds count.  You yell, as loudly as you can, “DON’T WASH YOUR HANDS!!! JUST DIG INTO ME!!!”

In surgery, you are the customer.  Isn’t the customer always right?  Do our customers ever do this?

What do the nurses and surgeons do?  They are professionals.  They follow their standards, wash their hands systematically, and then (or maybe earlier) they knock you unconscious.  Then, and only then, do they begin their surgery.  And so you don’t die from infection.

As software developers, we often listen to our customer’s urgent requests over our best judgement.  Part of the problem we have is that we don’t have a germ theory to explain to our customer why they are wrong to ask for that “quick fix”.  And so, unlike surgeons, we are not professionals.

Internal vs. External Quality

So that analogy is powerful, but it certainly isn’t conclusive or complete.  One important concept that is missing from the analogy is that in software, there are two types of quality: internal quality and external quality.

Internal quality is visible to the people who build the system and includes things like good design, consistent coding standards, compatibility with common tool sets, wise decisions about what to build vs. what to buy, etc.  Closely related to the concept of internal quality are two Agile concepts: technical debt and legacy code.  Technical debt consists of those internal things that are not done now but which impede future development if left undone.  Mostly this comes in the form of design debt, e.g. duplicated code.  Legacy code, on the other hand, is code that is not covered by automated unit and acceptance tests that are part of your regression suite.  Legacy code may (often) also have technical debt, but its size and details are unknown.

External quality is visible to the people who use the system.  Flaws in external quality include everything from typos in labels in the user interface, to system-crash bugs, to unknown security flaws, to usability.  External quality is achieved by meeting or exceeding user/customer expectations in functionality, stability, etc.

Agile Engineering Practices:

Each of the following Agile Engineering practices has a positive impact on either internal quality or external quality (or both).  The consequence of focusing on improving quality is always a long-term increase in quality and decrease in cost.  From a business perspective the benefit is clear.  I also want to point out that there is also a huge benefit to the team doing the work: pride!  Adopting these Agile engineering practices to produce high-quality software will certainly increase your pride in your work.

1. Refactoring

Refactoring is defined as “changing the design of a system without changing its functionality.” Refactoring is a technique that is focused on maintaining internal quality through continuous micro re-design.  There are numerous tools in most development environments to assist with partially automating various refactorings.  Three examples will help to illustrate refactoring:

• In a user interface, a radio button list can be refactored to a drop down selection list in order to accommodate a larger number of options.
• In code, a method on a sub-class can be moved into its parent if clients of peer classes also could use the method.
• In a database, a table can be split into two tables with different columns from the original in order to increase normalization or accommodate new relationships in the data.

2. Test-Driven Development (TDD)

Test-Driven Development is a process where an automated test is written before the code that makes the test pass.  It is used by developers to drive design and quality in their code and thus is also focused on internal quality, although it also has an impact on external quality.  Doing TDD well requires strong knowledge of refactoring since refactoring is one of the steps in the TDD process.  TDD also requires the existence of an automated test framework for the programming environment you are working in (Google “Xunit”).  Use of TDD can also have a significant positive impact on external quality.  The sequence of steps for TDD is as follows:

1. You want to add a small increment of functionality.
2. You design and code an automated test for that functionality.
3. You run the test; it should fail since there is no code for the functionality yet.
4. You write the simplest possible code to make the test pass (this can be very difficult for experienced developers as they often over-think the code).
5. You run the test; it should now pass.
6. You refactor your code to get rid of duplication while still keeping it simple. Refactoring may be applied to both system code and test code and your test(s) may be run multiple times while performing refactoring.
7. You run all your automated tests; they should all pass.
8. You check your code into a repository and move onto the next increment of functionality.

3. Pair Programming

Two people working together in front of a single workstation with one person “driving” (handling the keyboard and mouse) and the other person “navigating” observing the big picture is called Pair Programming.  Pair Programming is one of the most controversial of the Agile Engineering Practices since it seems to require spending money on staff but only getting half the results.  However, in practice, Pair programming increases focus and problem-solving capacity to a large degree and the result is that the same staff actually produce the same amount of results, but at a higher level of quality.  Pair Programming is basically taking the idea of code reviews to the extreme: all production code must be reviewed as it is written!  Pair Programming is equally focused on both internal and external quality: building the right thing and building it right.

4. Continuous Integration

Continuous Integration or CI is the use of automated tools to build, regression test and integration test a system every time an individual on a team checks code into the version control system.  After the full build/test cycle, if there are any problems, then everyone on the team stops working until the problem is fixed and the CI system gives the “all clear”.  Continuous Integration requires that test-driven development is also used by a team.  Continuous Integration servers also must complete the build/test in less than ten minutes in order to be useful.  CI, done well, has a significant impact on both internal and external quality, particularly if included in the build/test are automated acceptance tests.

5. Acceptance Test-Driven Development

Acceptance Test-Driven Development or ATDD is similar to TDD but is done at a functional level and in a way that non-technical stakeholders can participate in the test development and execution process.  Automated acceptance tests are created before code is written to implement the functionality.  The tests are created by non-technical stakeholders collaborating with technical team members.  The technical team members then use the acceptance tests in combination with micro-tests (TDD) to determine what will be implemented and how to implement it.  Acceptance tests become a part of the continuous integration process.  Acceptance tests have a very strong impact on external quality.  Acceptance tests need to consider “happy path”, error conditions, security, performance and other qualities of the software visible to end users.

6. User Stories

User Stories are extremely brief documents that describe a single feature of a system.  A team will often work through several user stories in an iteration, many dozens in a release and sometimes hundreds or thousands over the lifetime of a system.  Most Agile methods discourage the creation of complex requirements documents and specifications documents and instead rely heavily on frequent and in-person communication between the customer/users/stakeholders and the team.  However, light-weight record-keeping helps with this communication.  User stories fill the role of being an “invitation to a conversation” about a particular feature rather than a detailed specification of that feature.  User stories are often the format used for Product Backlog Items in Scrum teams or Value Drivers in OpenAgile teams.  Here is an example user story:

As a job seeker, I can upload my resume, so that I can get a job.

The three parts of the user story include the user role, the action the user does with the system, and a benefit (usually to the user) outside the system.

User stories are a driver of external quality since they help ensure effective conversations are occurring between the Agile team and outside stakeholders about the needed functionality.

7. Architectural Spikes

Architectural spikes are short investigations to solve a significant development problem, usually that has an implication for the design and architecture of a system.  These architectural spikes are done whenever the team feels that there is significant architectural risk in the implementation work for a feature.  An architectural spike is usually meant to be done quickly, usually within a day, but certainly within a single iteration, and results in two things: knowledge about the design implications of an implementation solution and a bunch of code that must be thrown away.  Architectural spikes are almost totally oriented to internal quality.  Architectural spikes are like micro-prototypes.

Professionalism

Pride, standards, licensing, continuing education, and respect are all components of high levels of professionalism.  Software developers are not professionals.  In many cases, all five of those components are missing and a person can still claim to be a software developer.  When a person graduates from high school, he or she might aspire to be a doctor, but would never think of setting up a practice until many years of further advanced education and licensing.  Is this true about software development?

Certainly, most software development does not directly affect the physical well-being of other people and so perhaps does not need the same strict limits as the medical profession.  Nevertheless, there are no standards for software development to indicate if a person is a professional.  Perhaps the most telling gap is the lack of a broadly accepted code of conduct… an oath… to which aspiring software development professionals can agree and be held accountable.

I would like to see such a code of conduct.  I would like it to be based on a few important principles:

1. A software development professional should proactively share known quality or design problems with peers working on the same system and with customers and users of that system and should never deliberately hide problems in a system.
2. Internal Quality. A software development professional always maintains the best possible internal quality of a system and should never knowingly through action (over-building) or inaction (under-building) introduce internal quality problems into the production code repository of a system.  Nor should a software development professional avoid fixing internal quality problems in a code base that were introduced in the past if fixing those problems can reasonably be considered beneficial to the introduction of new functionality.
3. A software development professional should always use the most rich communication methods available to collaborate with peers working on the same system and with stakeholders of that system.  Collaboration should occur in all aspects of the development of the system including understanding the problem, investigating solutions, building the solution and verifying its correctness.
4. External Quality. A software development professional should always verify with customers, users or stakeholders any suspected external quality problem to check if it is indeed a problem and if it needs to be corrected.  Generally, quality problems that need to be corrected should be corrected immediately as delay increases the cost of fixing exponentially over time.

These principles do not directly require specific Agile engineering practices, but certainly, most of these practices support a person in following these principles.  I hope over the coming decades these principles, or some like them, become part of the requirements to be considered a software development professional.

If you find this useful, please consider contributing with our
“Value for Value” model.