c++ - fstream faster than fopen? and gets faster as buffers increase?

I've been trying to write file data as fast as possible.

• I've increased the buffer size to reduce i/o operations.


• I've tested with both fstream and fopen.

For some reason fstream is faster than fopen.

• on a 64 byte buffer it's ~1.3 times faster



• on a 8192 byte buffer it's ~4.8 times faster.

I've been hearing that C's file I/O is faster (which makes sense)
includes yet i can't get fopen to perform as fast.

NOTE (old questions):

• my fopen was 2 times slower than fstream because i used fprintf (thanks jamesdlin)


• fstream buffer wasn't changing since you have to set it before opening (thanks Paul Sanders)

also realized fstream.put(char) is faster than fstream << char
(otherwise fopen is faster than fstream if the buffer is < ~256)

Here's my testing:

#include 
#include 
#include 


int filesize; // total bytes (individually "put" in buffered stream)
int buffsize; // buffer size

void writeCPP(){
    std::ofstream file;
    char buffer[buffsize]; file.rdbuf()->pubsetbuf(buffer,buffsize);    // set buffer (before opening)
    file.open("test.txt",std::ios::binary);                             // open file
    for(int i=0; i    file.close();                                                       // close
}


void writeC(){
    FILE* file=fopen("test.txt","wb");                                  // open file
    char buffer[buffsize]; setvbuf(file,buffer,_IOFBF,buffsize);        // set buffer
    for(int i=0; i    fclose(file);                                                       // close
}

#define getTime() double(clock())/CLOCKS_PER_SEC // good enough


double start;

void test(int s){ // C++ vs C (same filesize / buffsize)
    buffsize=s;
    std::cout<<"  buffer: "<
    start=getTime();
    writeCPP();
    std::cout<<"  C++: "<

    start=getTime();
    writeC();
    std::cout<<" C: "<}

#define MB (1024*1024)

int main(){
    filesize=10*MB;
    std::cout<<"size: 10 MB"<

    // C++ fstream faster
    test(64);   // C++ 0.86 < C 1.11 (1.29x faster)
    test(128);  // C++ 0.44 < C 0.79 (1.80x faster) (+0.51x)
    test(256);  // C++ 0.27 < C 0.63 (2.33x faster) (+0.53x)
    test(512);  // C++ 0.19 < C 0.56 (2.94x faster) (+0.61x)
    test(1024); // C++ 0.15 < C 0.52 (3.46x faster) (+0.52x)
    test(2048); // C++ 0.14 < C 0.51 (3.64x faster) (+0.18x)
    test(4096); // C++ 0.12 < C 0.49 (4.08x faster) (+0.44x)
    test(8192); // C++ 0.10 < C 0.48 (4.80x faster) (+0.72x)

}

Answer

In WriteCPP, you have to set the buffer before opening the file, like so:

std::ofstream file;
char buffer[BUFF]; file.rdbuf()->pubsetbuf(buffer, BUFF);   // set buffer
file.open ("test.txt", std::ios::binary);                   // open file

Then you get the sort of results that you might you expect (times are for writing 20MB with the buffer sizes shown):

writeCPP, 32: 2.15278
writeCPP, 128: 1.21372
writeCPP, 512: 0.857389

I also benchmarked WriteC with your change from fprintf to fputc and got the following (again writing 20MB):

writeC, 32: 1.41433

writeC, 128: 0.524264
writeC, 512: 0.355097

Test program is here:

https://wandbox.org/permlink/F2H2jcrMVsc5VNFf