Maybe this is why developers don’t care much about testing

January 2nd, 2012

I’m a huge fan of TDD, and one thing that is really frustrating is when I’m working with people who either don’t know how to write unit tests or don’t agree with the idea of writing unit tests. This is especially true in a static language because those people will design the code in a way that prevents me from writing tests.

Why is it that so many developers don’t care about testing? Most developers that I’ve worked with, whether I agree with their ideas or not, take pride in their work and generally care about their job.

I think that this goes back to the traditional divide between development teams and QA teams. On most teams, there is always pressure to get things done faster. In order to get things done faster, you have to either cut something out or get better at what you do. It’s much easier to cut something out, so inevitably the team will do what they can to cut development time. This means cutting out unit testing (or not taking the time to do it in the first place). Once you cut out unit testing, the next thing you throw out is good design principles and just get it done as fast as possible.

We all know that if you don’t write unit tests and you just throw in spaghetti code to get something done that you’re going to have a lot more bugs. It takes time for QA people to find bugs, reproduce bugs, write up bugs, and retest bugs. But why should the development team care? They were just told to go faster. Who cares how long it takes the QA team to test the application? Why does the development team care if it takes an inordinate amount of time to regression test the application?

The QA team knows what is going on, so they don’t trust the developers. They know that the developers are likely to write bugs, so they will need to be more thorough. The problem is that sometimes this means doing an inordinate amount of manual testing (which may or may not be possible).

This is what happens when development and QA are separate teams that don’t work closely together. Each one is only looking out for their own interests, not the interest of the project as a whole. In my opinion, it makes no sense to do it this way, but this how it’s done in many companies.

On my current project, we have broken down the walls. The QA team and the development team report to the same manager. The developers, BAs, and QAs sit together in the same area. Before we work on a feature, we all sit down together and decide what will be tested with unit tests, what will be tested with automated acceptance tests, and what will be manually tested. Because we’re all on the same team, I (as a developer) care about testing just as much as anyone else on the team. There’s no reason for me to not care about testing. Don’t I want the app to work too? If I write crappy code, it’s going to take longer for the team to get things done. If I don’t help automate regression testing, it’s going to take the team longer to manually test everything. I don’t want either of these, so I’m going to help out. The QA team is starting to realize that they can trust our automated tests (which they’re not used to), and we all agree that in the future they will do much less manual testing.

Because we’re on the same team, we’ve enabled developers to care about testing. If we want to get things done faster, I want to make sure that I automate as many tests as possible so that QA doesn’t have to manually test them. I might even pitch in to help with manual testing when that time comes. In the end, we end up with high quality software delivered in less time because we have very few bugs and we will test things in the most efficient way possible.

Why do we group our tests by file?

January 2nd, 2012

Most people I know put their unit tests in files that mirror the folder structure and filename of the actual class that is being tested. So if I have app/models/order.rb, I’ll have spec/models/order_spec.rb. The tests in the order_spec.rb file will test the code in the order.rb file. Every project I have been on, whether Ruby or .NET, has done it this way.

But have you ever thought about why we do it this way?

We do it this way on our project and I keep running into two problems. The first problem is when I am about to modify some existing code and I want to know what tests will test that portion of the code. The first place I look is the corresponding test file (based on the folder structure or filename), but that doesn’t always give me all of the tests for that functionality.

The second problem is when I refactor some existing code and the refactoring spans multiple classes. Now I have broken tests all over the place and it’s hard to reconstruct the tests so that they test the same business concepts that they were testing before. Often times the tests were testing a portion of the system of classes that I am refactoring, but were not as encompassing as they should’ve been.

Here’s the thing — when I write tests, I’m typically using the Given/When/Then style of writing tests, even in unit tests using frameworks like RSpec. I’ll have test code that looks like this:


describe "When completing an order" do
  it "should set the status to In Process" do
     # test code here
  end
end

That code tests functionality having to do with orders and is going to help me ensure that my code performs some business function correctly. But if you look at that code snippet, you don’t know what classes I’m testing. Yeah I know, it’s just an example and I left out those details. But the point is that it doesn’t matter what classes I’m testing. What matters is that my tests are testing that my code performs a certain business function.

That being said, why are we grouping our tests by file? Wouldn’t it make much more sense to group our tests by business function instead?

I already have ways to find tests for a given class. I can search my code for the class name, or if I’m in .NET I can do Find Usages and have a little window pop up that tells me everywhere a class (or method) is used.

If my classes were grouped in folders by business function instead, I get the following benefits:

  • I can see what tests exist for a given business function
  • It encourages me to write tests that test business functionality, not test data structures (classes, methods, etc.)
  • I can put all kinds of tests in there (unit tests, Cucumber tests, even manual test plans) — all in one place, all checked into source control
  • My tests document business functionality instead of documenting a class

Remember, code constructs like classes and methods are just a means to an end, our goal is to write software that provides business value and performs specific functions. So I might as well organize my tests accordingly.

(Disclaimer: I have never actually tried organizing tests this way. It makes sense to me and I think it would work great, but I might try it and find out that it doesn’t work. But if anything, maybe I’ll start a good discussion.)

A TDD success story

August 1st, 2010

In a month or so, my co-worker and I will be wrapping up the project that we have been working on for the last 15 months. It’s a website for a company in the construction industry that bids on jobs and then tracks the progress of the jobs, purchase orders, billing, and everything else they need to run their business. We used ASP.NET MVC, Fluent NHibernate, AutoMapper, and SQL Server 2008.

We practiced test-driven development from day one. We wrote tests for our .NET code, tests for our JavaScript code, and tests for our SQL code. Right now we have over 14,000 tests, and I think we can break the 15,000 test barrier in the next month. I can run them all in about 5 minutes.

I cannot tell you how invaluable these tests have been. First of all, our application deals with money. The users will input a bunch of data about a job and then our application will tell them how much to bid on the job. We cannot afford to have bugs in our code that would miscalculate the amount to bid on a job, because that would lead either to over-bidding (in which case they would win very few) or underbidding (in which case they would win jobs and take a big hit). It just has to work.

Second, we did not have a QA team on this project, it was just the two of us developers. Frankly, I don’t have time to go back and manually test stuff in the app or regression test it when we need to make a change and deploy something. We do have bugs from time to time, but we haven’t had any critical bugs.

Because of our tests, we have been able to get as close to continuous deployment as I would feel comfortable with. On average, we deploy 2-3 times a week. When we go to do a deployment, I usually go through the site and manually test the new features that we are about to release. If I don’t find any problems, I run our deployment (which is all automated), and 5 minutes later, the changes have been deployed. I never go back and regression test old features or stuff in areas that we didn’t change.

Since the whole test and deploy process takes only about 15 minutes, users get their changes quickly. They don’t have to wait until later in the week, or until our next scheduled release. We have reduced the cost of change to pretty much just the time that it takes to code the changes.

All of this is possible because we were diligent about test-driven development and writing good unit tests for everything. The reason that we don’t spend time regression testing is that we don’t expect anything to be broken, and it very rarely ever is. This means that we can spend more time delivering business value and less time ensuring that we didn’t break something we wrote a year ago.

I’m not writing this to say how awesome we are as developers, because anyone can write tests and have the same kind of success. Also, we could’ve cheated and not written tests and ended up with potentially costly bugs and more time spent regression testing.

This is why I practice test-driven development. You end up with well-designed code, you drastically reduce bug counts, you can release more often, your codebase stays under control, and you have a lot less stress.

Reducing the cost of change

July 30th, 2010

If there’s one constant in software development, it is change. Our entire workday is spent changing things and responding to change.

Every time you write a line of code, you are changing it. You might add new features, fix bugs, or make changes to existing code. However you put it, you’ll be changing the code.

Requirements will change. You often find this out after you’ve coded a feature, and now the business either changes their mind or you find out that you made some wrong assumptions.

Business priorities will change. What was important yesterday may not be important today, and six months from now, all kinds of things might change. There could be new, more important projects on the horizon, new people in charge, or new business opportunities.

Software development teams change too. You might stay at your company for many years, but during that time, you might get moved from project to project. Developers will leave the company, and others will join. The end result is that many different people will end up working on the code that you’re writing today.

Since we know that we are going to have a lot of change, it would therefore make sense to reduce the cost of change so that we can respond to change as quickly and easily as possible. So how can we reduce the cost of change?

Automated Testing

Every time I add a line of code, I risk breaking my application. Breaking the application is just not acceptable. We’ve come to accept bugs as a part of the process, but that doesn’t mean that it’s OK to write bugs. If you’re not striving to write bug-free code, then maybe you need to raise your standards.

Nevertheless, the fact is that we all screw up and we all have the capability to create bugs. Knowing that, I want a way that I can easily test my application so that I can ensure that I find out if I broke something once I change it.

On my current project, I’m writing an application that helps my client bid on construction projects. They can’t afford to have bugs in their system that will cause them to incorrectly bid on these jobs or they could be out a lot of money. The bottom line is that it has to work because their entire business is now running on the application that I’m writing. They also come to me on a regular basis with new feature requests and things they need changed. They don’t want to wait months to get this functionality, and they also don’t want me to break any existing code.

Thankfully I have a large suite of automated tests that I can run to ensure that I didn’t break anything. This isn’t foolproof and things sneak through, but it’s usually little stuff that isn’t critical. But it’s good enough proof for me that I didn’t break the existing functionality. As a result, I can release to production daily and I spend virtually no time manually regression testing my application when I deploy. My unit test suite has reduced my cost of change to pretty much just the time it takes me to code the changes, so now I can react quickly to changes, release all the time, and I can do it without breaking existing code.

Clean Code

Sometimes people will write their unit tests, write the code to make the tests pass, and then assume that that is good enough. That’s great that you wrote tests, but how easy is it for someone to make a change in your code?

I feel that clean, readable code is very underrated these days. When you write code, are you thinking of the person that is going to need to understand what you did once you’re not around anymore? That scenario is very likely to play out at some point. So when you write code, create methods, variables, and classes that are very descriptive. Place a priority in making your code easy for someone else to read.

“Clever code” is the opposite of clean code. I’m talking about using complex algorithms to solve simple problems, trying to using a complicated LINQ statement to do an operation that would be more readable in a foreach loop, or trying to implement something in the fewest lines of code as possible. There are times where you might want to do things like this, but usually it’s more important to write code that is easier for someone else to read and understand. Again, remember that someday someone else is going to have to change your code, so write descriptive code that makes it easier for that other someone to know what to do.

Tests Are Documentation

Your unit tests are documentation of what your code is supposed to do. I practice behavior driven development, which means that my unit test methods describe some business functionality that my code is in charge of executing. This way, if someone needs to know what my code does, they can look at the test classes and methods and see what it is supposed to do.

Lately I’ve started adding a lot more comments in my test methods. The class names and methods names may describe what the code is supposed to do, but it doesn’t always explain why. I’ve found that sometimes it helps to write down the why because that is also important information. Since people can look at my tests as documentation of what my code does, I figured that this would be the best place to put the whys, because if someone has to change my tests, they’ll immediately know why I did what I did and they’ll have a reminder that they need to update my comment.

Build Iteratively

Since business priorities and requirements are going to change, we want to reduce the cost of this change too. If the business comes to you and drastically changes the requirements or certain assumptions that you might’ve made, it might be really costly and painful to change, and you can only do so much about that. What we can do is make sure that we’re constantly checking back with the business to see what they want now so that we have as few surprises and as little rework as possible.

This is one reason why Agile projects often run on iterations. As a part of this process, meet with your business sponsor every week or two and talk about what you’re going to do next. Show them what you’ve done in the last week or two, and make sure you’re on the right track. This way, if they’re going to change their minds, you’re giving them permission to do so and you can adjust what you’re doing as soon as possible so that you have the least amount of rework.

Always Be Aware of Change

Quit thinking short-term and quit shoving code in just to get it done as fast as you can. You may think you’ve done something good by getting it done faster, but if you create more work and pain in the future, it’s a net loss. Instead, always think about what the effects of what you are doing now will be two years from now. Your goal is to create business value without creating technical debt. A home builder can do shoddy construction and put nice looking paint and siding on the house, but 10 years from now when the problems arise, people won’t be thinking very highly of that builder. Don’t make the same mistake. Take pride in your work and strive to always leave your code base cleaner than you found it.

Discounted rates!

May 5th, 2010

Sign up for TDD Boot Camp at the discounted rate of $750! Also, if you sign up with a group of 3 or more, you get the even more discounted rate of $600.

The bottom line is that I want to help you learn TDD and get all the benefits out of it that I get on a daily basis.

How to use Rhino Mocks – documented through tests

March 12th, 2010

I wanted to come up with a way to show people how to use Rhino Mocks (other than telling them to read the documentation). What better way to do this than by showing you how it works through a bunch of simple unit tests that document how Rhino Mocks works?

So that’s what I did. You can view the code here, or if you want to download the whole project and run the tests, you can get the whole thing here.

A response to the SWE101 attendees who tried to solve the TDD problem without TDD

March 2nd, 2010

This weekend we did some live TDD at the Software Engineering 101 event and broadcast it over LiveMeeting, which was a lot of fun. However, some people have posted solutions (here and here – see first comment) on how they solved the Greed scoring problem without using TDD.

I think you missed the point. The point was to help you learn the TDD thought process and how to put TDD into practice. There are lots of benefits to TDD, and I’ll use these non-TDD examples to illustrate.

How do you know your code works?

Those of you who solved the problem without TDD probably did some sort of manual testing in order to prove that your code is working. You could probably do that with a simple example like the one we had (there were only a few scoring rules). But what happens as we add rules (and in real life we’re always adding rules) and you have to manually test 16 rules? For the record, I took the solution posted here and ran my tests against it.

failing test

Tests are documentation

Tests document what the code is supposed to do. Since we wrote our test methods as sentences that read like English, you can figure out the rules just from reading our tests (as you can see in the image above). This is not documentation in Word format which becomes stale, this is living, breathing executable documentation of what the code is supposed to do.

Readability is important

I feel that the solution that we ended up with was very readable. Here is our implementation code:


public class GreedScorer
{
    public double Score(params Die[] dice)
    {
        var score = 0;

        score += ScoreASetOfThreeOnes(dice);
        score += ScoreASetOfThreeTwos(dice);
        score += ScoreASetOfThreeThrees(dice);
        score += ScoreASetOfThreeFours(dice);
        score += ScoreASetOfThreeFives(dice);
        score += ScoreASetOfThreeSixes(dice);
        score += ScoreEachOneThatIsNotAPartOfASetOfThree(dice);
        score += ScoreEachFiveThatIsNotAPartOfASetOfThree(dice);
        return score;
    }

    private int ScoreASetOfThreeOnes(Die[] dice)
    {
        if (dice.Count(die => die.Value == 1) >= 3)
            return 1000;
        return 0;
    }

    private int ScoreASetOfThreeTwos(Die[] dice)
    {
        if (dice.Count(die => die.Value == 2) >= 3)
            return 200;
        return 0;
    }

    private int ScoreASetOfThreeThrees(Die[] dice)
    {
        if (dice.Count(die => die.Value == 3) >= 3)
            return 300;
        return 0;
    }

    private int ScoreASetOfThreeFours(Die[] dice)
    {
        if (dice.Count(die => die.Value == 4) >= 3)
            return 400;
        return 0;
    }

    private int ScoreASetOfThreeFives(Die[] dice)
    {
        if (dice.Count(die => die.Value == 5) >= 3)
            return 500;
        return 0;
    }

    private int ScoreASetOfThreeSixes(Die[] dice)
    {
        if (dice.Count(die => die.Value == 6) >= 3)
            return 600;
        return 0;
    }

    private int ScoreEachOneThatIsNotAPartOfASetOfThree(Die[] dice)
    {
        if (dice.Count(die => die.Value == 1) < 3)
            return (dice.Count(die => die.Value == 1) * 100);
        if (dice.Count(die => die.Value == 1) > 3)
            return ((dice.Count(die => die.Value == 1) - 3) * 100);

        return 0;
    }

    private int ScoreEachFiveThatIsNotAPartOfASetOfThree(Die[] dice)
    {
        if (dice.Count(die => die.Value == 5) < 3)
            return (dice.Count(die => die.Value == 5) * 50);
        if (dice.Count(die => die.Value == 5) > 3)
            return ((dice.Count(die => die.Value == 5) - 3) * 50);

        return 0;
    }
}

Look how readable our code is. Read the score method. Notice how it tells you exactly what it’s doing. Contrast that with one of the other solutions:


static int Score(int[] numbers)
{
    int valueToReturn = 0;

    for (int numberIndex = 0; numberIndex < numbers.Length; numberIndex++)
    {
        switch (numbers[numberIndex])
        {
            case 1:
                if (
                    (numberIndex + 1 < numbers.Length && numbers[numberIndex + 1] == 1) &&
                    (numberIndex + 2 < numbers.Length && numbers[numberIndex + 2] == 1)
                   )
                {
                    valueToReturn += 1000;
                    numberIndex += 2;
                }
                else
                {
                    valueToReturn += 100;
                }
                break;

            case 5:
                if (
                    (numberIndex + 1 < numbers.Length && numbers[numberIndex + 1] == 5) &&
                    (numberIndex + 2 < numbers.Length && numbers[numberIndex + 2] == 5)
                   )
                {
                    valueToReturn += 500;
                    numberIndex += 2;
                }
                else
                {
                    valueToReturn += 50;
                }
                break;

            default:
                if (
                    (numberIndex + 1 < numbers.Length && numbers[numberIndex + 1] == numbers[numberIndex]) &&
                    (numberIndex + 2 < numbers.Length && numbers[numberIndex + 2] == numbers[numberIndex])
                   )
                {
                    valueToReturn += 100 * numbers[numberIndex];
                    numberIndex += 2;
                }
                break;

        }
    }

    return valueToReturn;
}

To me, this code is not very readable. If you had to implement a new rule in this method, it would be hard to do (plus you have no tests to tell you that you broke an existing rule).

The point of this exercise was not to find a clever way to solve a brain teaser. If you want to do a brain teaser, try and write code that will output a Fibonacci sequence using one LINQ expression. That’s the kind of geek stuff that you might do at night for fun. But when you’re writing real code, we care about things like readability and maintainability. When you’re writing implementation code, use common language and write methods whose names tell you what they do.

TDD leads to well-designed code

As we were writing our first test, our test told us that we needed some class that would score the Greed game. So we named it GreedScorer, which does exactly what the name intends. This class will most likely end up following the single responsibility principle because we gave it a very specific name.

But you guys took so long!

We weren’t trying to finish as fast as possible. We were not doing a coding competition. We were practicing and learning the TDD mindset and trying to explain things as we went. As you get better at TDD, it becomes more natural and you learn tricks that help you go faster (for example, sometimes writing a whole bunch of tests, watching them all fail, then making them all go pass is faster than just writing one test at a time and making them pass one at a time).

The bottom line is that we came out with some good, readable, maintainable code (with props to Sirena who helped write it). We were able to prove that our code is working. We don’t have any code that we don’t need. We could easily respond to change and implementing new scoring rules would be pretty easy. Implementing the Score method was fairly easy because we were building it incrementally (solving lots of little problems is easier than trying to solve a big problem all at once). This is why we do TDD!

The business value of test-driven development

January 27th, 2010

Most businesses are creating software for one primary reason — to make money. In order to make money, we need software that meets the needs of the business and can be developed and maintained in a reasonable amount of time with a high level of quality. Test-driven development is a discipline that will help you achieve these goals.

Test-driven development involves writing automated unit tests to prove that code is working. The test code is written before the implementation code is written. By writing the tests first, you will know when the code is working once the tests pass. Test names are written as sentences in plain English so that the tests describe what the code is supposed to do. Over time, you will end up with a large suite of automated tests which you can run in a short amount of time. These tests will prove to you that your code is working and will continue to work as you modify or refactor the code base.

Most software applications are intended to be used for many years, and throughout most of their existence, someone will be changing them. The total cost of ownership of an application goes far beyond the cost of the initial effort to create the initial version of the software. The first release is the easy part — you can build the application from the ground up, you don’t have many hindrances, and developers feel very productive. But as time goes on, productivity tends to decrease due to complexity, developer turnover, poor software design, and any number of other reasons. This is where software development really becomes expensive. So much focus is placed on the original cost of building an application without considering how the original development effort will affect the cost of maintaining that application over several years.

Test-driven development can reduce the total cost of ownership of an application. New developers on the team will be able to change the code without fear of breaking something important. The application will have fewer defects (and far fewer major defects), reducing the need for manual QA testing. Your code will be self-documenting because the tests will describe the intended behavior of the code. All of this leads to flexible, maintainable software that can be changed in less time with higher quality.

Software is intended to deliver business value, and test-driven development will help you write higher quality, more maintainable software that can be changed at the fast pace of the business. Test-driven development will lead to successful software projects and enable you to write software that will withstand the test of time.

Make TDD your meal ticket in 2010

January 27th, 2010

It’s almost the time of the year where people start making new year’s resolutions and setting goals for the upcoming year. Allow me to propose something for your list: make TDD your meal ticket in 2010.

Why do I say this? Because TDD has revolutionized the way I develop software and has helped me write flexible and maintainable software with fewer defects and is a less stressful way to get things done. I can’t imagine ever going back to writing code without tests, and as a result it’s really difficult to work with developers who don’t write tests because they will write code that is hard to test, which means I won’t be able to write tests either.

In today’s economy, it’s important to differentiate yourself from the crowd if you’re looking for a job, hoping to get a raise, trying to get a promotion, etc. TDD can be that thing that differentiates you from everyone else out there.

The cost of unit testing

January 27th, 2010

One question that I hear from people who are new to TDD or writing tests is, “How much longer is my feature going to take if I write tests?” This is a valid question. We all have deadlines, so if we have to add something extra to our development process, it better be worth it.

I don’t think I could say it better than this post did, so I’ll just let you read it for yourself.