Potatoswatter Potatoswatter - 1 year ago 44
Linux Question

What filesystems allow repositioning the beginning of a file?

Typical filesystems, and the POSIX interface, only allow a file to be resized at the end. Typically the size of a file "on disk" after it has been closed is equal to the offset of the read/write position when it was closed. Seeking before closing is also known as "repositioning the end-of-file."

A file that contains a queue of data would be more efficiently represented by an operation to remove the beginning of the file. The on-disk allocation blocks at the beginning could be freed, and needless copying minimized.

Is this directly supported by any common filesystem format and/or operating system? What kind of interface is used to do so? (For example, a Linux

selector.) I'm pretty sure I've heard of this kind of thing in practice.

Answer Source

Repositioning the beginning of the file is probably not a good idea. That would risk to confuse processes having file descriptors open on that file and expecting it to be correctly seekable, i.e. having a predictable offset.

If your main goal is just to save disk space, there are several approaches.

If you have ZFS support and enable deduplication, blocks previously used to store the data of the beginning (or any part actually) of a file can be freed just by overwriting them with zeroes.

Under Linux, you should also be able to free parts of a file store in a file system supporting sparse files (most of them) by using the fallocate system call and passing it the FALLOC_FL_PUNCH_HOLE flag. Solaris provides the similar fcntl command F_FREESP. Whether fallocate or fcntl will work efficiently or at all is implementation dependent though.

Alternatively, If you run an OS that doesn't provide a fallocate or equivalent functionality but supports ZFS (eg: FreeBSD), and/or if deduplication is not an option because you have not enough RAM to dedicate to it, a lightweight alternative would just be to enable compression on the file system.