Why do we Unit Test (and what’s the purpose of Code Coverage)?

What is Unit Testing? Unit Testing is the approach of testing the smallest parts of our code in isolation from other code. In Java these are methods in our classes. In Javascript these are our functions.

There are many unit testing frameworks that help us with our unit tests. In Java we have JUnit and TestNG, both have been around for a long time. The key part they help us with is making ‘assertions’ about the results of our code under test, to check the results and confirm when we call our code, it returns the expected results.

Why do we Unit Test? As developers we want to know our code works, that it does what it is supposed to do. We use unit tests to give us confidence about our code, so we know it works.

We can use unit tests to help with code changes or refactoring efforts to improve existing code. If existing tests are available, you can use them to check that they are showing you were the issue is that you need to fix (assuming there’s a test for that part of the code), or check that the existing code is working before your refactoring changes.

A typical flow for using unit tests to help with a defect fix looks like:

Run test, confirm test fails. Make code fix, confirm test passes. If the unit test for the related code doesn’t have an existing test for the code where the issue is, add a new test. Write a test for the expected result which should initially fail, rerun after the code fix and now the test should pass to confirm your fix is working.

Run other related tests in the same area to check there are no unexpected impacts to other existing code.

We can also use unit tests as part of a CI/CD pipeline – when our tests pass, the code is ready to include in a new build, and depending on your development process, ready for additional tests, integration testing, UAT testing and promotion into production.

Many industry studies have shown that it’s always cheaper to find and fix defects in our code earlier than later. The time invested to develop unit tests saves us in the long run if we can find issues earlier and avoid finding issues later, e.g. in production.

Any number of unit tests does not prove that our code is 100% correct however, neither does it prove the absence of bugs. If our unit tests are effective though, they can give us confidence that our code is working as expected for a range of tested inputs.

What is an effective test?

An effective unit test must have assertions to check and confirm the code when executed with given parameters returns the expected results. In JUnit, the framework provides a number of assert statements, like assertTrue(), assertEquals() that you use to confirm the expected results.

A unit test without assertions that only calls the code under test is not an effective test. Without assertions, the test is useless if you’re not checking the expected results.

Similarly for assertNotNull(). If the only thing your test does is check that there’s a non-null result, this doesn’t confirm that the code is returning the expected result, it only confirms a value is returned. This is not particularly useful when we expect a unit test to confirm that the code is returning the expected results (i.e. it is working).

For any given method that takes parameters, you should have a range of test methods to confirm the expected results for that range of input values. You should obviously test with expected/valid input values, but you should also test with a few unexpected/invalid values: high values, low values, empty, short and long strings. In all cases you should assert and confirm that the method returns the expected results.

How does Code Coverage relate to Unit Testing?

Code Coverage tells you how much of your code was execute by your tests. As a tool it is useful during the development of your unit tests, because it helps highlight where your tests may have missed paths through the code. For example, if the code you are testing has an if..then..else condition and your test calls the code with parameters that causes the if condition block to be tested but not the else block, Code Coverage helps highlight untested code.

The problem with Code Coverage though, is that by itself, Code Coverage is not an indication of effective tests or even working tests. If you have a test without an assertion that calls a method with a single block of lines of code and each of those lines executes, Code Coverage will report 100% coverage on this block. Without an assertion, this test doesn’t test that the code works, and therefore is worthless. It does have 100% Code Coverage though. This is why relying on Code Coverage alone is not useful, or sensible, because it doesn’t tell you how effective your tests are. It only tells you how much of the code was executed. Code that is executed by a test but is not confirmed if it is returning expected results or not with assertions, is just executed code, not working code.

Remember your assertions.

Use Code Coverage as a tool to help you find untested code, not as an ultimate goal: writing tests to achieve Code Coverage is not useful. Developing effective tests is your goal.

My software development blog posts from 2018: AWS, Twitter bots, Machine Learning, Docker, Kubernetes and more!

Looking back, I’ve investigated and played with a lot of interesting stuff this year, I was pretty busy 🙂 Early in the year I was doing a lot of experimenting with AWS as part of my prep for the AWS Solution Architect Associate exam, so a lot of my posts this year were AWS related, but I did some a some time working on some other projects too.

Here’s a look back at some of my favorite personal projects and blog posts during the year:

  • Using AWS Sagemaker to train Machine Learning models – here and here
  • mvmdvm and Pi-Star setup – multi digital voice Amateur Radio modes on a Raspberry Pi – here and here

Phew that’s quite a lot! I sincerely appreciate the feedback and comments I receive on my posts and hearing that at least some of my content is useful to others. I look forward to continuing to share my thoughts and write more content that hopefully will be continue to be useful in the coming year.

Happy New Year!

Observations about common IT technologies in 1988-89

Sometime around 1988-1989 I did some part-time data entry work for an IT Recruitment Agency that my Dad worked for. Tucked away in some papers I found these two sheets listing a range of different programming languages and other in-demand software packages/systems at the time. From memory, I think this list was what I used to code each of the job applicants tech skills as they were entered into their recruitment CV/resume database.

There’s many things interesting about this list from 30 years ago. The first that caught my attention is how many of the tech skills on this list are no longer in use today, and some I’ve never even heard of since.

The second point that’s interesting is how many technologies and languages we commonly use today are not even on this list, meaning they were developed and introduced at some point after 1989. Every web technology in common use today was introduced after this point – HTML, CSS, JavaScript and any of the various popular JavaScript libraries, all introduced at some point after 1989.

Even other popular languages and frameworks/platforms, Visual Basic, Java, .NET, Ruby, PHP … all introduced after 1989.

This reinforces the fact that commonly used IT technologies come and go pretty quick, and what’s common today can easily be replaced with something else tomorrow. If you’re planning to stay in IT for the long run (longer than a few years), be prepared to keep your skills up to date, stay flexible and adapt to change.

Why hasn’t the evolution of software development tools kept pace with other changes in the development of software systems?

Many changes in IT occur as a indirect result of the development or introduction of some other technological change. Not all of these changes involve development or invention of a completely new technology, some changes are related to increased availability or reduced cost of a resource that was already previously available, but reduced costs allow for example making use of commodity, off-the-shelf x86/AMD64 servers to build a system compared to buying previously more expensive, branded systems from a big name supplier (e.g. IBM or Oracle).

Some changes in development of IT systems appear to come and go more like fashions in clothing; what is today’s hot new trend is tomorrow’s old news, but some trends seem to come back in fashion again at some point in the future. IT trends are less like a linear timeline of incremental improvements, they’re more like a churning cycle of revolving ideas that gain popularity and then fall out of favor as we strive to find what works versus what doesn’t, what’s more efficient, or what’s more effective.

As an example, computer systems in the 1960s and 70s were mainly centralized systems, where computing resources were provided by hardware in a central physical location, at this time usually a mainframe, and accessed remotely by users via terminals. The terminal device had no or little processing power itself, all computing resources were provided by the centralized system.

After introduction of the IBM PC and clones, computing resources became available on the user’s desk rather than locked up in the computer room. This allowed development of systems where part of the processing could be provided by an application running on the user’s desktop, and part provided by backend resources running remotely. This application style is called client/server.

Every architectural decision has pros and cons – while client/server systems reduced need for centralized processing resources as some processing is off-loaded to the user’s desktop, the approach to distribute and maintain the application installed on the user’s desktop brings other challenges (how do you install, update, patch the application that is deployed to 100s or 1000s of end user workstations?)

Development of web-based applications addressed some of the application distribution limitations or client/server systems, but initially, the user experience possible with early versions of HTML and lack of interactive capabilities in the earlier browsers seemed like we had taken several steps backwards, as web applications resembled more like dumb terminal based applications from years prior compared to what was possible with a thick client running on a PC and leveraging the native UI controls and features of the OS it runs on. As JavaScript evolved, richer features were added in HTML5, and JavaScript frameworks evolved to help the developer develop richer, more interactive browser based experiences without the need to write code to manipulate the browser’s DOM by hand. Fast forward to today, and it’s arguable that we’ve reached an equivalent position where a web-based application can equal the user experience features that were previously only possible on a native application running directly on an OS platform.

Where am I going with this?

This was an overly simplified and not entirely historically complete summary of the evolution of IT systems development over the past 50 years or so. The point I want to make is, regardless of whether we develop software for mainframes, desktop applications, client/server systems, web-based applications, mobile apps, or cloud-based apps, the approach we use to develop software today, the process of writing code, has not changed much in over 50 years:

We type code by hand using a keyboard. Typing every letter of the source code, letter by l.e.t.t.e.r.

The only arguably significant changes are that we no longer develop software by plugging wires into different terminals on a plugboard, or by punching cards and stacking them in a card reader to load a program into memory. These minor differences aside, we’ve been busy typing source code into computers using a keyboard for the past 40 years or so. Even our current IDEs, our Visual Studios, Eclipses and Netbeanses, are not that different from the editors we used to use to develop our Borland Turbo C in the 80s/early 90s. Just as our deployment approaches cycle round and browsers have become our new (internet) terminals, for some reason in the front-end development world developers are obsessed with text editors like Sublime Text, Atom, Brackets, and the newcomer from Microsoft, Visual Studio Code, or even for the real programmers, Emacs and vi/vim, shunning the more feature packed IDEs. I realize this is an over-exaggeration to make a point about how we still code with text editors – in reality today’s text editors with syntax highlighting and code complete features are arguably far closer to IDEs than to text editors at this point, but hey, we’ve always boasted that real developers only code in vi or Emacs, right?

More Productive Developer Tools?

At various points in the past, there have been developer tools that you could argue, were far more productive than the current IDEs like Eclipse and Netbeans that we use today for Java development. And yet, for many reasons, we chose to continue to type code, letter by letter, by hand. Sybase’s PowerBuilder, popular in the mid 1990s, was an incredibly productive development platform for building client/server applications (I did a year of PowerBuilder development in 1997). Why was it more productive? To oversimplify, to build a database backed application with CRUD functionality (create/retrieve/update/delete) you pointed the development tool to your database schema, visually selected columns from tables that you wanted to display on the screen, and it generated a GUI using a UI component called a DataWindow for you, also allowing you to drag and drop to customize the display as needed. Sure, you would code additional business logic in Powerscript by hand, but the parts that we spend so much of our time building by hand with today’s tech stacks and tools was done for you by the development tool.

Other variations with support for this type of visual programming have appeared over the years, like IBM’s VisualAge family of development tools which was available for many platforms and many programming languages, which provided a visual programming facility where you graphically dragged links between components that represented methods to be executed based on some condition or event.

Interestingly, many of the features of VisualAge Micro Edition became what is now known as Eclipse (I find that particularly interesting as a Java developer having used Eclipse for many years, and also having used in my development past VisualAge Generator and VisualAge for Java at different companies. I even still have a VisualAge for Java install CD (not sure why, but it’s still on my shelf):


More recently we’ve had interest in Model Driven Development (MDD) approaches, probably the most promising move towards code generation. For those that remember Rational Rose in the mid 1990s and it’s ability to ’roundtrip engineer’ model to code and code back to model, it does seem like we’ve been here before. When the topic of code generation comes up, I remember one of my college lecturers during a module on ‘Computer Aided Software Engineering’ (CASE), stating that in the future, we would no longer write any code by hand, all code will be generated using CASE tools using models. This was in 1992.

24 years later, we’re still writing code by hand. Using text editors.

I don’t have an answer to my initial question of this post, why haven’t our development tools advanced, but at least we’ve moved on from punch cards. But who knows. Maybe someone is about to release a card reader for developing JavaScript. If anything though, the reason could be because developers love to code. Take away the code, and you remove the need for developers. Maybe that’s not a bad thing. Maybe to evolve our industry we have to stop encouraging coding by hand and start to encourage a conscious focus on other techniques like MDD. As long as you have talented and passionate developers who love to code, you will have code. I don’t think code is going to go away any time soon. In the near future we may be spending more of our development efforts building systems that can evolve and learn by themselves (machine learning), but someone still has to program those systems. Coding by hand is still likely to be around for a while.

Now stop reading and go write some code 🙂