UI Tests vs Visual Layout Tests - part 1

In my post The Pyramid of Tests I introduced the different levels of testing that you should have in your application. In this post I want to focus on the UI testing phase.

UI Testing

In order to fully test your application you have to write unit tests, system tests and integration tests to fully cover you business logic and the systems workflows, however that's not enough. You also have to test the UI of your app, to verify that it displays the right output, check that the different UI elements behave as expected and so on.

In the past we used to manually test it – click some buttons, enter an input and assert that the output was as expected, nowadays we use automatic UI testing frameworks like CodedUI, selenium and protarctor. These frameworks allow us to record the actions we used to perform manually and run them automatically. Now our application is fully tested. That’s all we need right? Wrong!

Checking only the UI logic is not enough!

Even if the logic is correct it doesn’t mean that the application is ready to production.

This is a screen shot from a website where the graph width didn't match the screen size any more.

Another example is a bank website where all the letters in the site turned into gibberish. UI Logic tests will not find those errors. Obviously we cannot allow our self to have an application with these errors in production. That means that besides our UI Logic tests we also need to test the Layout.

In order to fully verify that the layout is "correct" we need to test it on all the possible browsers – Chrome, Fire Fox, IE, Opera and on all the possible screen sizes – PC, Tablet and the different mobile devices. You do it since the CSS that suites Chrome doesn't look the same in IE and the HTML tag that you've used on one platform behaves differently on another one.

Let's say that my last commit included some CSS changes, those changes didn't affect any logic however they might ruin style in some place in my application, my UI Tests will not find it since they check the logic and not the style itself.

As you can see the tests matrix is too big and it is not possible to manually test it.

We need to find something else...

Introducing Visual Layout Testing. Next post.