Testing in the njones ∕socketio Repository

Introducing a new style of writing tests in the socketio package. This update will help writing tests for the package clean and consistent. This should allow people new to the page to quickly understand what's going on so they can help in efforts to provide comprehensive testing.

The current tests in the project usually follow the format:

var tests = []struct{
    name string
    param1 string
    want string
}{}
    
for _, test := range tests {
    t.Run(test.name, func(t *testing.T) {
        a := doSomething(test.param1)
        assert.Equal(t, test.want, a)
    })
}

This is a best practice in Go, using a table driven test^fn1 design, and it has several things right. The tests are named and processed using t.Run to run the tests as sub-tests. Using an assert library provides consistency across t.Error messages. Using a struct array to manage the tests gives a quick view in the the parameters that are being tested.

However, there is some room for improvement.

A more recent way of writing tests using closure based testing has written about by Jack Lamond^fn2 requires a bit less boilerplate with additional benefits. The tests moved to this format look like the following:

var test = func(param1, want string) func(*testing.T) {
    return func(t *testing.T) {
        a := doSomething(param1)
        assert.Equal(t, want, a)
    }
}

t.run("<name>", test(param1, want))

As the above example shows, there's a lot to like about the style. It's an underrated feature that the test code can actually be at the top of the function; making it easy to see what is being tested without scrolling through all of the testing parameters first. It's easy to add new tests that run in their own sub-test using the t.run directly outside of the parameters struct. It also allows the parameters of what will be tested to be shown clearly, outside of a block the array of parameters that table-driven tests promote.

With some modifications there could be room to do even better.

All test cases in the njones/socketio project will be updated to basically follow the format below:

var opts []options

// tests cases
var test = func(options ...option) func(param1, want string) func(*testing.T) {
    return func(param1, want string) func(*testing.T)  {
        return func(t *testing.T) {
            for _, opt := range options {
                opt(t)
            }
            
            a := doSomething()
            assert.Equal(t, want, a)
        }
    }
}

// test parameters
tests := map[string]func() (param1, want string) {
    "<name>": func() (param1, want string) { return param1, want },
}

// test runner
for name, testing := range tests {
        t.run(name, test(opts...)(testing()))
}

This is a mix between the two styles of table-driven and closure driven testing. It provides something like a table driven test (but in the form of a map, with the names of the tests as the keys) and it still uses closures to run the test. Is this the best of both worlds or is it the worst, or a mix of the two?

The biggest benefits of writing tests in this manner are:

• Closures can be used to encapsulate complex test parameters.
• The map keeps the name of the test out of the struct and ensures unique test names.
• The map runs slightly differently each time, making sure no tests depend on each other.
• The t.run can still be called outside of the loop, so adding a test quickly can be achieved.
• The function inside of the map that provides the test parameters can hold closure values for more flexibility.
• The code is clearly separated between "the test case", "the test parameters" and "the test runner".
• The "test parameters" can be used over multiple different "test cases".

Somethings that can be done, but haven't been used in the current tests are:

• The t.Run test function can still be at used for one off tests or tests that should be ordered.
• The "test parameters" function can directly hold asserts, to help pinpoint tricky failures.

There is an added options function that can be used to control which tests are run based on outside factors. For an example a runTests("<name") function can provide functionality where only that test is run and all other are skipped using the t.SkipNow() function. This provides the additional benefits:

• Tests that are skipped are known and don't need to be "commented out"
• Allows skipping a single or groups of test with any formula possible.

Some of the drawbacks are:

• The test map is verbose because of having to provide a function signature each time
• There is a lot of nested functions for the original test cases (it's three deep)
• The t.Run has a lot of function calls
• There is a fair amount of boilerplate that needs to be setup to get the full benefits

Outside of the drawbacks that are mentioned above. This seems to be a solid way of writing tests that gives a lot of flexibility. All of the tests in the repository will be converted to this style. This is a easy way to add and extend the tests within any package in the repository. Hopefully this encourages contributors and allows maintaining hight test coverage throughout the repository.