FSB FSB - 7 days ago 7
C++ Question

How to test methods with user defined functions in production?

I was asked a to solve a question in an interview, and the problem was such that the user could change the output of the function if we gave the option of passing a function as a parameter (in C++ this would be by passing a function pointer the preformed the desired operation). After, the interviewer asked me how would I test this in production. I was not completely sure of the answer, since I am interviewing for my first SWE job. My immediate solution was to have assertions that were dependent on the input of the user, or let the user be responsible of testing his own method.

My question is, how would you do it? Or where can I get information on the best practices for this?

In case my question wasn't clear, imagine you are writing the linux quicksort function (not the actual question but hopefully it will serve for illustration purposes):

void qsort(void *base, size_t nmemb, size_t size,
int (*compar)(const void *, const void *));


How would you test for correctness? Since the user of the function is the one defining the
(*compar)
function, it seems to me that the user should take care of verifying correctness while using the function.

Edit: The consensus seems that any method would need to have a specified default behavior, and the method would be tested against this predefined behavior. This makes sense, since any user could modify the function to suit their needs in ways the original developer wouldn't have thought of and because of that the owner is responsible for testing.

Answer

When you write a method such as the qsort example you gave, your documentation has to clearly and unambiguously state the requirements for the user-defined function. In case of your qsort example, the requirements are:

comp - comparison function which returns ​a negative integer value if the first argument is less than the second, a positive integer value if the first argument is greater than the second and zero if the arguments are equal. The signature of the comparison function should be equivalent to the following:

int cmp(const void *a, const void *b);

The function must not modify the objects passed to it and must return consistent results when called for the same objects, regardless of their positions in the array.

The supplier of the user-defined comparison function is responsible for providing a well-behaved function that meets these requirements, and they are also responsible for testing their function.

But if the user has done their homework, they have to be able to rely on your method, so you have to do your homework as well. Assuming that the user-defined function behaves indeed as required, you still have to test your own method and all its corner cases. To do so you will have to provide at least one user-defined function for your own testing purposes, and then of course you need to unit-test that function as well before feeding it into your actual method.

For the qsort example, you could take a few different data structures, say an array of ints and one of some custom structs, then write (and unit-test) comparators for them. Finally, you can use those structures and comparators to test qsort itself, with all its edge cases (different types of arrays with different element sizes, empty arrays, repeating elements, …).