adams.co.tt blog - tagged with testing

Duplicate Active Model validation errors

2012-04-12

When attempting to test a model using Active Model validations, I came across a rather odd problem.

require 'active_model'

class Person
  include ActiveModel::Validations
  validates_length_of :name, :maximum => 40, :message => 'name too long'
  attr_reader :name
  def initialize(attrs)
    @name = attrs[:name]
  end
end

I was seeing duplicate 'name too long' error messages in the errors hash.

describe Person do
  describe 'validation' do
    it 'should be invalid if name is too long' do
      person = Person.new(:name => 'a' * 41)
      person.should_not be_valid
      person.errors[:name].should == ['name too long']
    end
  end
end

The above spec failed with the following message:

Failures:

  1) Person validation should be invalid if name is too long
     Failure/Error: person.errors[:name].should == ['name too long']
       expected: ["name too long"]
            got: ["name too long", "name too long"] (using ==)
     # ./spec/person_spec.rb:17

Some further digging revealed that the Person was being loaded twice by the require method. require was being called twice with two different paths, which both resolved to the file containing Person. Removing the extra call to require fixed the issue. This seems to be a problem particular to ruby 1.8.7.

The runnable code examples reproducing this issue can be found here.

5 ways to database testing heaven

2011-08-30

Many applications are backed by persistent storage in the form of a database. When writing tests for our application, it is important not to forget to include behaviour surrounding interactions with the database. However, writing these tests is often easier said than done. There are common pitfalls, and here I suggest some sensible ways to avoid them. However, the subject matter is rather dry, but hopefully the kittens will help.

Write tests that only cover database interactions

Testing how your application interacts with the database are, by definition, integration tests. However, many teams make the mistake of testing much more of the application than necessary when testing database interactions. Often, supporting frameworks, such as Spring or Rails, encourage this behaviour. Falling into this trap leads to slow and brittle tests.

For example, when testing the various permutations of a search operation in a web application, you do not need everything from the controller down wired into your test. Merely instantiate your database access objects and pass them a database connection. You will find that your tests will be quicker and less brittle. Then you can write many more permutations testing your search queries.

Wrap your tests in transactions

Most of the commonly used relational databases support transactions. Transactions are used in applications when an atomic unit of work must be either applied completely or rolled back. They are also used to provide isolation around changes, when a number of applications are connected to a given database. In both of these cases, the intended goal is to commit these changes. We can use transactions in our test to ensure the reverse. We can use transactions to ensure that any changes to data that are made are rolled back.

This can have a profound effect on your ability to write tests. You no longer need any code to perform data clean up at the end of a test. You no longer need to understand every possible side effect of the code under test. And most importantly, you’ll never accidentally leave something behind, that could interact with and break another test.

Don’t couple yourself to your baseline data

When writing tests that interact with the database, it is likely that you’ll need to use a baseline of data to get those tests running. A baseline is very handy. It means that each test doesn’t have to set up a large amount of extraneous data in order to run. This makes your tests much more readable as there is much less noise in them. It also makes them run more quickly if you only set up this baseline once during a test run.

However, it is very easy to end up with a test that is tightly coupled to this baseline. This can make it very difficult to diagnose test failures, as the data driving the test is unlikely to be visible in your test case. Moreover, it makes your tests much more brittle. Any changes to the baseline can impact a large number of tests.

To avoid these pain points, always make sure that any data that you make an assertion against, is set up by your test. Do this even if the baseline contains a data set that you could use, as that baseline could change under your feet at any time.

Use triangulation

Often when writing applications that interact with a database, it’s likely that you’ll be using some form of framework. Object relational mapping tools such as Hibernate and ActiveRecord are very powerful in their ability to map database table columns to fields in objects, as well as having many other very useful functions. However, it is useful to test that these mappings have been done properly, particularly if you are in the (unfortunate) situation that your database is acting as an integration point between a number of other applications.

In this case, it is important to remember to test using some form of triangulation. If you persist and rehydrate an object using the same mappings, then your tests would never catch a defect around mapping a field to the wrong column. In these cases, it’s advisable to persist and fetch data using two different mechanisms in your test. Using a little plain old SQL in a Hibernate mapping test won’t cost you a great deal in readability and will help prove that your objects are being hydrated and persisted properly.

Get your business logic out of the database

We’ve spoken a great deal about the pitfalls of database interaction testing. Even if you’re doing all the good things suggested here, you still might find that you’re experiencing a great deal of pain. This may simply be because you have too much business logic pushed into the database.

Stored procedures, named queries, etc. can be great from a performance point of view, but with them you’ll need more database interaction testing to support them.

Database tests are still harder to write, maintain and run. If possible, shifting your business logic to your application will make it much more accessible to your whole team and cheaper to maintain. Often considerations of performance are premature optimisations, and having code that is easier to maintain is also easier to optimise if the need genuinely arises.

Test by feature not story

2011-06-13

I know it’s been said before, but it’s such an important and misunderstood point I feel the need to reiterate it.

Very often, teams build their functional test suites around stories. Every time a new story is played a brand new test is written. Sometimes, the test will even have the story number in the test name.

This is dangerous for a couple of reasons. Firstly it encourages duplication in your tests. This is because every time a particular application feature is touched, a new test is written, which probably does very similar things to all the other previous tests around the feature. Not only is this a nightmare to maintain, but also it means that your test suite will take forever to run.

The second reason is, stories represent changes to features, not the features themselves. This means that every time you play a story, you are changing how a piece of functionality works. As a result previous story based tests will fail and will possibly even become redundant. Chasing down these failures incurs a great deal of pain.

A far more sensible pattern is to build your functional test suite around the features themselves (there’s a reason why cucumber test files have the “.feature” extension). When a story is played, your test suite is changed or augmented to reflect how the application should work. This means redundant or out of date tests are deleted as soon as a story invalidates them. It also means that you only have one set of tests for a given feature, massively reducing duplication.

Often you’ll find that as you write tests to reflect how the application works, your tests will illustrate the business workflows. It will become not only more maintainable, but more accessible to non-technical users, such as non-technical QAs, BAs and the business themselves. This allows them to become involved in the test writing process, and ensures you’re testing the features that are most important to them.

Transactions around database tests

2010-04-01

A database is probably the most common integration point found in software development. Therefore, testing that integration point is key. Ensuring that your queries work and that object relational mapping is configured correctly is key to making sure that the database schema maps to what your application expects.

The best way to test this is to write tests around your repositories to test that queries return expected results, and that your objects are loaded and persisted as expected.

For example:

@Test
public void shouldPersistBook() {
    BookRepository repository = new BookRepository();
    Book book = new BookBuilder()
                       .title(“War of the Worlds”)
                       .author(“H. G. Wells”)
                       .toBook();
    
    repository.save(book);
    
    List<Record> records = executeQuery(“SELECT * FROM book WHERE title = ‘War of the Worlds’”);

    assertTrue(records.size() == 1);
}

However, there is a problem with this. After running this test, the database will have been modified. Assuming that title is the primary key for this table, this test will fail the second time it is run. An even worse scenario is that the test could pass even if the save method did nothing.

Therefore, it is sensible to make all tests occur within a transaction. This can be implemented in the setup and tear down of any database tests:

@Before
public void setUp() {
    executeStatement(“START TRANSACTION”);
}

@After
public void tearDown {
    executeStatement(“ROLLBACK”);
}

This of course only works if your repositories do not use any DDL statements that update the schema, or call commit explicitly. However doing either of these things is very bad, particularly the first, as your application should never be making schema changes. This makes your schema volatile and breaks any transactional wrapping your application may wish to enforce.