Tests are not enough – Why code integrity matters?

Tl;dr – code testing is just one way out of many to achieve code integrity.

The Wikipedia term for code integrity states “With code integrity, the developer can be sure that his/her code is written correctly when passed on to QA”. In other words, this is about developers’ confidence in the correctness of their code.

How can we help increase confidence and what exactly is code integrity? In its most abstract definition, it is (a) the correctness-checking processes accompanied by (b) the metrics to measure the completeness of these correctness-checking processes.

Automated tests, for example, can serve as a correctness-checking process; although they are the go-to method, they are definitely not the only ones. Code review by a fellow developer or by an automated code analysis system is another such process. We can also consider an AI-based comparison of code behavior analysis to the written specifications (defined by the product team) as another.

But these methods alone, while they may increase developer’s confidence, miss an important part. To have a justified sense of confidence we also need to get a measure of how deep and thorough the correctness-checking processes were.

In today’s software world, different types of code coverage techniques serve as the standard method that provides software teams with the metric to increase their confidence. There are many types of code coverage metrics, from the popular line coverage, and branch coverage, to the rarely-use mutation testing technique. Code coverage measures the thoroughness of unit tests, but as popular as they are, they are known to have a limited correlation with the measure of real code integrity.

Shift-left Testing

Shift-left testing is a paradigm (and a trend) to move testing to earlier stages of the software development pipeline. Code integrity discusses the means and metrics to allow it, so it is clear how these two are tightly connected.

But as we said, Integrity is not just about testing, so a broader term to bind these together should be termed as Shift-left code integrity, i.e. not just implementing more unit tests, but also other correctness-checking processes and their corresponding metrics.

V-Model, Software development life-cycle

We believe that in today’s software world, in order to empower developers and fully allow them to inspect the side effects of their code, tools that allow them to test the broader scope of their work must be available to them. Hence, we would like to offer to use another term: Expand-right code integrity not replacing, but rather in addition to shift-left code integrity.

Shift-Left Expand-Right

While shift left testing relates to adding/trickling down correctness-checking processes earlier in the pipeline (i.e. developers), in modern software systems (as their complexity and dependencies increase) this also suggests expanding the developer responsibilities to the right side wing of the V-model. Thus, allowing the developer to control the project’s entire unit testing and integration testing phases, leaving the DevOps to develop the means and tools to do so.

While this expansion to the right already exists in many organizations, we believe that this is not just a nice-to-have approach, but a necessity because in many cases it is impossible to perform automated unit testing without the full integrated system context.

We believe that the future of software will be tightly coupled with the combination of the two terms we just coined – Shift-Left-Expand-right code integrity.