UI Tests vs Visual Layout Tests - part 2

In the previous post we understood that testing only the UI logic is not enough and that manually testing the layout across all the platforms that our application supports is impossible. We need to automat it and add it to our continuous integration process.

What is Automated Visual Layout Testing?

*Later I will introduce a platform that automates the whole process, but first some basic concepts.

Baseline

In order to verify that our layout is correct we need a "baseline" – pictures of our different website/application pages/windows. Those picture are "print screens" of the layout as it supposed to be presented. After we manually verify that the layout is correct we set them as "baseline".

Image Comparison

From now on each commit will trigger a comparison between the new pictures of the application and the baseline in order to check whether the commit's changes changed the layout. The comparison is not as simple as it sounds, if you try to compare "pixel to pixel" you will see that a pair of identical pictures are not really identical.

Take a look on these 2 pictures:

This pair of pictures is a close up to the "the" word in the sentence. The pictures look almost the same but if you look closely then you will see that not all the pixels have the same color or the same brightness. However then you zoom out and look at the "the" word in the sentence it will look absolutely the same. Those differences accrue since the browser/the operation system doesn't render the pictures the same way, it depends on different parameters that I don't want to talk about them in this post.

More examples:

Conclusion: Two identical pictures to the human eye are not really pixel to pixel identical.

That makes image comparison a lot harder. Luckily there are enough "smart" image comparison algorithms and tools out there, one of them is a platform called "Applitools eyes". More about it in the next post.