The software testing mix for online platforms at cloudtec

In the previous post, we described the types of test that exist and why the right mix depends on the project. A website needs a different combination from a web application or an online platform. Here it gets concrete: this is the mix we use in our own software.

Why we test at all

Our platforms are continuously developed and extended. That is precisely what makes tests necessary: not one-off proof that something works, but proof that it still works after the next change. Four points matter to us here:

• Preventing regressions. New code doesn’t break existing functions unnoticed.
• Safeguarding updates. After major framework or module updates, there is proof that the application still runs.
• Enabling refactoring. Anyone who wants to restructure code needs a safety net. Without tests, outdated code stays in place because nobody dares touch it.
• Proving quality. Every build must pass all tests before a release takes place.

How we measure a good test

Not every test justifies the effort. We stick to four criteria:

• Resistant to refactoring. The test keeps passing when the code is restructured. It may only fail when the behaviour of the function actually changes.
• Quick to write, quick to run. The more effort a test takes to develop, the greater the chance that it never gets written at all.
• High code coverage, meaning a high proportion of lines of code that a test run actually passes through and validates.
• Accurate. False positives and false negatives cost more time than they save. We leave out tests on irrelevant code.

The four types of test in our platforms

Behat: workflows and permissions in the browser

Behat tests control a real browser and thus cover the user interface including JavaScript. They are slower than all other types of test, but in return they deliver the best code coverage and survive refactoring most reliably. We use them to map the workflows within a platform and run through scenarios with test data. A second purpose is permissions: the test checks that content and resources are only visible to authorised users.

Module tests: individual building blocks against the framework

Module tests, also called feature tests, simulate the Symfony framework and check individual classes: a command, a mail template, an event listener. They survive refactoring less well than Behat tests, but they are quicker to write and force cleanly structured code.

An example: a listener writes a history entry as soon as a configured element is created or updated. Behat checks in the interface whether the entry appears. The module test goes through every possible combination and ensures that the listener also kicks in when an entity is updated via a command, that is, without any interface at all.

Unit tests: self-contained logic

We use unit tests where a piece of code is self-contained and doesn’t access other interfaces or functions. They run fast and are the easiest to create. A typical case is the price calculation in our e-commerce module: the functions can be calculated in isolation, without a browser or a database being involved.

Smoke tests: does the platform even start up?

Before the rollout of a new application, smoke tests check whether functions and services are present as planned. For us, these are mainly route smoke tests: the test calls every available route and checks whether a status 200 comes back.

The second place we use them is our modular intranet platforms. For each project, it is defined which modules are active. The smoke tests ensure that a newly set-up instance runs in every module combination, including with deactivated modules.

What this means for your project

You get platforms that run reliably, and not only on go-live day. Which tests this requires is decided per project: we secure the business-critical processes first, and everything else follows the ratio of effort to benefit.

We involve you directly in the testing phase. Nobody knows the business requirements better than the people who work with them every day.