I need to write a program which will write many characters in a output file.
My program will also need to write newline for better formatting.
Generally, the user of stream classes shouldn't mess with the stream's flushing if maximum performance is wanted: the streams internally flush their buffer when it is full. This is actually more efficient than waiting until all output is ready, especially with large files: the buffered data is written while it is still likely to be in memory. If you create a huge buffer and only write it once the virtual memory system will have put parts of the data onto disc but not the file. It would need to be read from disc and written again.
The main point with respect to
std::endl is that people abuse it a line ending which causes the buffer to flush and they are unaware of the performance implications. The intention of
std::endl is that people are given control to flush files at reasonable points. For this to be effective they need to know what they are doing. Sadly, there were too many people ignorant of what
std::endl does who advertised its use as a line ending such that it is used in many places where it is plain wrong.
That said, below are a number of things you might want to try to improve performance. I assume you need formatted output (which the use of
std::ofstream::write() won't give you).
std::endlunless you have to. If the writing code already exists and uses
std::endlin many places, some of which possibly outside your control, you can use a filtering stream buffer which uses its internal buffer of reasonable size and which doesn't forward calls to its
sync()function to the underlying stream buffer. Although this involves an extra copy, this is better than some spurious flushes as these are orders of magnitude more expensive.
std::ios_base::sync_with_stdio(false)used to affect the performance on some implementations. You'd want to look at using a different IOstream implementation if this has an effect because there are probably more things wrong with respect to performance.
truewhen calling its
always_noconv(). This can easily be checked by using
std::use_facet<std::codecvt<char, char, stdd::mbstate_t> >(out.get_loc()).always_noconv(). You can use
std::locale("C")to get hold of an
std::localefor which this should be true.
std::num_put<char>facet may still do things you don't really need. Especially if your numeric formatting is reasonably simple, i.e. you don't keep changing formatting flags, you haven't replace mapping of characters (i.e. you don't use a funny
std::ctype<char>facet), etc. it may be reasonable to use a custom
std::num_put<char>facet: It is fairly easy to create a fast but simple formatting function for integer types and a good formatting function for floating points which doesn't use
Some people have suggested the use of memory mapped files but this only works reasonable when the size of the target file is known in advance. If this is the case this is a great way to also improve performance otherwise it isn't worth the bother. Note that you can use the stream formatting with memory mapped files (or, more generally, with any kind of output interface) by creating a custom
std::streambuf which uses the memory mapping interface. I found memory mapping sometimes effective when using them with
std::istreams. In many cases the differences don't really matter much.
A long time ago I wrote my own IOStreams and locales implementation which doesn't suffer from some of the performance problems mentioned above (it is available from my site but it is a bit stale and I haven't touched it for nearly 10 years now). There are lots of things which can be improved over this implementation still but I haven't an up to date implementation which I'd be ready to post somewhere. Soon, hopefully - something I keep thinking since nearly 10 years, though...