output: Fix writing to stdout on Windows

By default, Windows converts "\n" to "\r\n" on stdout. Setting the mode to binary fixes this.

tools/output/output.c

@@ -39,6 +39,11 @@
 #include "output/output.h"
 #include "output/muxer.h"
 
+#ifdef _WIN32
+#include <fcntl.h>
+#include <io.h>
+#endif
+
 struct MuxerContext {
     MuxerPriv *data;
     const Muxer *impl;
@@ -141,6 +146,11 @@ int output_open(MuxerContext **const c_out,
     }
     c->one_file_per_frame = name_offset || (!name && have_num_pattern);
 
+#ifdef _WIN32
+    if (impl != muxers[0] && !strcmp(filename, "-"))
+        _setmode(_fileno(stdout), _O_BINARY);
+#endif
+
     if (c->one_file_per_frame) {
         c->fps[0] = fps[0];
         c->fps[1] = fps[1];

[Matthias Dressel]

I think this would break the md5 and xxhash muxers, since they output text with \n.
Maybe this should be put in the *_open function of the muxers that actually output binary data instead (yuv, y4m2).

[Thomas Wulz]

It doesn't seem to break anything - from my testing, running it from cmd or PowerShell still produces a newline on \n. I did it this way because the md5 and xxhash muxers call fopen with "wb", so they technically output binary data too. Maybe it would make sense to do what you suggested and also change those two to "w"?

[Matthias Dressel]

Importantly, they only fopen the file wb if it's NOT stdout. I don't think b should be dropped there, since that would create different checksum files depending on OS.
I'm not sure how Windows behaves on \n instead of \r\n. My assumption was that it does not do a proper line break.