Charlie Charlie - 2 months ago 13
C++ Question

Open file and close file statement positioning: best practice, advantages, disadvantages

I have a code, containing many loop iterations, with

open file
and
close file
statements positioned as follows:


  • Main loop


    1. do work

    2. open files

    3. write to files

    4. close files

    5. continue work


  • end loop



But, an alternative is:


  • open files

  • Main loop


    1. do work

    2. write to files, [flush]

    3. continue work


  • end loop

  • close files



Is there a "best practice" to positioning
open file
and
close file
statements containing many loop iterations? Are there advantages / disadvantages to each? Will I see performance differences? Memory restrictions? Future development issues down the line?

I'm mainly coding in Fortran (hence the tag), however, I would like to know if this is language dependent or not, since I also program in other languages. Any help is greatly appreciated.

Answer

As usual, you should profile your specific code to see what are the bottlenecks. However, in general, opening and closing files is very expensive.

Consider the following:

def foo():
    f = open('bar.txt', 'w')
    for i in range(1000):
        f.write('a')
    f.close()

def bar():
    for i in range(1000):
        f = open('bar.txt', 'w')
        f.write('a')
        f.close()

Let's time it:

>>> %timeit foo()
10000 loops, best of 3: 190 ┬Ás per loop

>>> %timeit bar()
10 loops, best of 3: 47.8 ms per loop

So, opening and closing is extremely expensive.

What are the advantages (or at least mitigating factors) for the constant opens and closes?

  1. Less open file descriptors.

  2. When you close a file, the data is flushed to it. Of course you could just call flush, but that is an expensive operation in itself, and the time differences would become narrower.

If you don't have critical data (i.e., you can just rerun the program if it crashes), and don't have too many open file descriptor problems - few opens and closes will probably be faster.

Comments