I'm writing a Matrice class (for practice sake) and when I wrote the method to multiply two Matrice objects I had to check if certain conditions are met for matrix multiplication:
Matrix Matrix::mul(const Matrix &mat)
if(col != mat.row)
//we cannot multiply in this case
//create temp object result
Matrix a = b.mul(c);
Don't return anything, throw an exception instead:
if (failure) throw std::runtime_error("Exception!!");
You can throw any of the different exceptions that suits your case more.
By the way, you could take a look at my project called Matrix on GitHub for an example of how one could create such a class to operate on matrices.