I am working on devising a custom binary file format that allows me to rewrite parts of the file without overwriting the entire file. The data is composed of "elements" that are of variable length. These elements can be deleted, inserted or modified within the file. Such modification may change the length of the element.
Here is what I am considering:
Your file format should support a list of headers that are linked with each other.
In the header, have a field "next" that points to the position of the next header in the file. If you need to add a header, add it at the end of the file, then write its position within the file into the "next" field of the previously last header.
My opinion: Why invent a new format when there are already solutions like SQLite out there that can be easily used?