I know that I can use serialVersionUID to control the version of classes. And I read that I can then add or remove fields and the class will still be compatible, it will just use default values.
When must I change the serialVersionUID?
The value of the serialVersionUID field is should ideally be changed when incompatible changes are made to the structure of the class. The complete list of incompatible changes is listed in the Java Object Serialization Specification.
To expand further, incompatible changes to a class will prevent the deserialization mechanism from creating an instance of the object, because there is information in the stream that does not map to the current class definition.