UNIX-style tee utility for Windows
November 9th, 2005 5:10:20 pm pst by Sterling CamdenThe downloadable code below contains the source and executable for a simple ‘tee’ utility for Windows. The tee utility faced its first fairway on UNIX platforms, and has been ported to Windows by other providers of UNIX-style utilities. While my version may not be technically on a par with the MKS Toolkit version, mine definitely tops the leader board for price/performance ratio.
Now let’s sink the golfing puns, because the name for the tee utility actually derives from plumbing. Input and output can be ‘piped’ from one process to another on UNIX and on Windows, and the tee utility creates a tee in the pipe. Thus, in addition to sending output to standard out, you can also redirect it to zero or more files — very useful for keeping a log of a process while simultaneously being able to view its progress and/or send its output on to somewhere else. For example:
my100steps.bat | tee my100steps.log
can execute all 100 steps and send the output to the console while at the same time writing the output to my100steps.log.
The version you can download below uses the Microsoft .NET Framework (built using Visual Studio .NET 2008), and provides an example of using managed code in C++. It also demonstrates a few of the capabilites of the System::IO::StreamWriter and System::Console classes.
Jeroen Pluimers converted this to C# so it could run on Windows XP Embedded.
For a version in Ruby, go here. How about a Haskell version?
UPDATE 2009-07-27: Added -a (or –append) option as the first argument only to append files rather than overwriting them. Also changed to a Unicode build under VS 2008.
UPDATE 2009-08-18: Upgraded syntax to eliminate /clr:oldsyntax and added an event handler for CancelKeyPress to ignore Ctrl+C, so the originating program in the pipe can handle it instead.
UPDATE 2010-01-25: Changed to use BinaryReader and BinaryWriter to eliminate CR/LF problems that resulted in double-spacing some forms of output. Thanks to Anonymous Coward for the suggestion below.
UPDATE 2010-01-28: Included more suggestions from Anonymous Coward. Use tee --help to see available options.
Posted in .NET, C and C++, Wildly popular, Windows | 41 Comments » RSS 2.0
Subscribe by Email
Nice implementation. I wish that Microsoft did not require a “setup” and msi file for small tools like “tee.”
Thanks, Eustace. I didn’t even know MS had a version of tee available.
Hello Chip,
I think I’m missing something. When I try to use tee.exe I get the following message:
Unhandled Exception: System.Security.Policy.PolicyException: Unverifiable assembly
‘\tee.exe’ failed policy check.
Do I need to do something special to install this? All I did was unzip the download, copy the file “tee.exe” to a different folder, then use it.
Thanks!
Paul
Did you put it on a mapped drive? .NET assemblies don’t like to be run from mapped drives, unless you make them trusted locations.
Yes, I put it on a mapped drive. Does it matter which local drive or which Folder? Where would you recommend to put it? Thanks!
As long as it’s a local drive, you should be fine. On my system I put it in a folder that’s in the PATH so I can execute it from anywhere. Being an old Unix hacker, I have a c:\usr\bin that I added to the PATH for executables like this.
[...] often I see “ruby tee windows” or something similar, which leads people to my tee utility for Windows. That seems fair enough, except that it isn’t written in Ruby. It’s written in C#, [...]
Nice tool
When using “tee” I get the file with the specified file name, but also an additional file named “tee” with the same content.
The reason seams to be that the name of the program is given as the argv[0] value to the _tmain function.
Hence please iterate starting from “1″ (instead of “0″) over argv.
Thanks
Good catch, Mick. I’ve updated tee.cpp in the zip file.
[...] Perhaps a command-line solution would help? Something like "tee" utility – see e.g. http://www.chipstips.com/?p=129 , http://unxutils.sourceforge.net — With best wishes, Igor Tandetnik With sufficient thrust, pigs [...]
What would be great is a ‘-a’ option (as in Unix) to append to a log file. I can make the change but I thought I’d mention it in case it of use to others. Thanks
Good idea. I’ve added it to the download, which is now built using Unicode.
WOW, speedy response.
Many thanks, a great util especially as I didn’t have to write it myself. Thanks again.
My pleasure — thanks for helping me to improve it.
Hello. This is very handy. (I had a tee utility for Windows before, but I would have to turn on my old machine to find it again.)
Using it with ping, I notice 2 things:
ping -t | tee xxx
First, the console output is double spaced, i.e. a blank line between each ping response. The file is OK, and it works as expected with other commands, like dir. Not a real problem, but a curiosity.
Second, I gather it closes the output file every 4K? Is there a proper way to close it? As ping -t never ends, the only way I can stop ping is ctrl-c, and that seems to kill tee before it gets a chance to write out the last bit of output.
Thanks very much, Alex.
I second this motion, and can even tell you exactly what is causing it if you want to fix it. Windows/DOS use two characters, “\R\N”, to signal End Of Line, whereas everybody else uses just “\N”.
(Mac/Linux text documents sometimes seem smushed together when viewing with Notepad – Notepad ignores lone \N’s. Metapad shows the error.)
Diagnosing:
Sterling, I went through nine outputs (UnixUtils Tee, Your Tee, > operator) x (“ping http://www.google.com“, “tracert -h 2 http://www.google.com” and “route print”).
Ping by default uses \R\R\N (weird) and is line-by-line. UnixUtils Tee shrinks to \R\N. Your Tee expands to \R\N\R\N (double newlines!). > operator leaves it alone. (Notepad/Console views \R(….)\R\N as a single return – move all the way left, move down.)
Tracert ALSO uses \R\R\N and is line-by-line. Same behavior.
Route uses \N for adapters, \R\N for routes, and is all-at-once. UnixUtils Tee makes all \N. Your Tee makes them all \R\N. > operator leaves them alone.
From this I conclude UnixUtils Tee strips only a single \R in front of a \N. It “fixing” the \R\R\N was a pure fluke – it was trying to neuter it entirely.
I also conclude Your Tee attempts “fixing” lone \R’s into \R\N’s. It also “fixes” single \N’s into \R\N’s. This means a \R\R\R\R\N would be a single line in console, but FOUR lines in Your Tee!
Finally, the > operator leaves everything alone, as it should. The fact everything appears fine in the console “route print” yet smushed in notepad tells me CMD.EXE treats \R(….)\R\N, \R\N, and \N the same.
The Fix:
I went through your source code an identified the problem:
String^ s;while (s = Console::ReadLine()) // Read standard in
{
// ----> Problem is ReadLine right here DEBUG "s" HERE <----
Console::WriteLine(s);
for (ndx = 0; ndx WriteLine is not ideal either WriteLine(s);
}
}
Going over MSDN, I now know ReadLine() is causing the doubling shenanigans. Without an IDE here I can’t compile and fix it myself, though.
Debug “s” as pointed the code and you’ll see. ReadLine treats a \R (by itself) or \R\N as a terminator, and loads it into the buffer string “s” WITHOUT any terminator. When WriteLine is called, it adds the preset “Console.Out.NewLine” to the end of the string.
So in the case of “Cookie Monster\R\r\n”, it will load “Cookie Monster” into s (throws away \R), and WriteLine will add a new “\R\N” when writing. There still is a “\r\n” in the buffer, which is loaded into s as “” (empty string). WriteLine then tacks on ANOTHER “\R\N” – result: two newlines.
The root cause is ReadLine is the incorrect method to use, since it treats \R by itself as a newline. A better solution would be to use the Console.OpenStandardInput method instead, or the BinaryReader/Writer stream methods. This would mirror output verbatim, since it is content-agnostic. This means you could pipe data through Your Tee as well, just like >, rather than only ASCII characters (“string” class).
I’ll post again once if I can get a drop-in pseudocode replacement, but I state again, I do not have an IDE so this is a pen-and-paper exercise for me.
Thanks for providing everyone the utility and code, by the way – personal projects tend to be the best, and I am sure you take pride in being the #1 Google hit.
Thank you, Anonymous Coward!
I have adopted your suggestion and updated the zip file. Problem solved!
Thanks Camden, I’m glad the suggestion helped – thanks for implementing it – I just piped a JPEG through tee to test, and it worked a treat.
Going over the code after the update, I did notice some hiccups: first, the “if (argc < 1), show usage" logic will NEVER trigger. When you summon tee, argv[0] is always the name of the command, "tee" – so argc is *always* 1 or greater. However, there is a slight chance someone might want to pipe through tee without a file, so erroring out on no arguments is a bad idea. (e.g. "echo Hello world.|tee")
To minimize the chance of a collision, I would suggest implementing the "usage" text as a /? or –help only. I know many Linux CLI utils do this for just that reason. Maybe it's worth adding a "stop interpreting flags" flag too.
Second, I believe the code always initializes more Stream handles than necessary due to that numbering issue: one extra with no flag, two extra with one flag, etc. Also, if there are duplicate arguments (filenames), there are major errors.
This time I can give you the code directly, and hopefully save you some trouble for all the work you've done until now! I've never coded in Visual C++ before and I have no IDE to work with, so the syntax might be a bit off. Hopefully the improvements are self-explanatory. (You're not going for brutal efficiency I guess, since you're not using C – so might as well take advantage of what VC++ offers!)
(Footnote: It ends up I just spent the last three hours making extensive changes to the command interpreter, adding everything I mentioned! I even cloned UnixUtils features. So much for "it'll just take minute/a few lines"… I don't think this board software plays nice with large blocks of code, so I am (contact-form) e-mailing it to you instead. If you could take the final step to get it to compile, I'm certain it will be very, very robust!)
Thanks for sending the code — I had to make a few changes to it, but it’s essentially what you intended. There were a few minor syntax errors (I’m not surprised, since you didn’t have a compiler) and my version of the .NET Framework doesn’t include List::Distinct() (I haven’t upgraded to 3.5 yet).
Thanks for the help!
OK, I had an address in the ping command which seems to have been parsed out. So let me phrase it this way:
ping -t “some IP address” | tee xxx
Thanks, Alex.
That’s interesting, Alex. I’ll have to look into why that happens. It looks like the Ctrl+C is getting caught by tee rather than by ping. Perhaps I can handle the interrupt in tee a better way. If you’re subscribed to comments here, I’ll be sure to post an update after I find out more.
Thanks for checking into it. I am not sure who is catching the Ctrl+C as it stops both tee and ping.
Also, I should mention that contrary to what I said before, the double spacing of the ping output is on both the console and in the log file. I thought the first time I tried it, the file was normal, but maybe not. Thanks, Alex.
The Ctrl+C is definitely going to tee.
The extra carriage returns are coming from ping. if you do
ping url > ping.out
and then edit ping.out with an editor that shows binary, you’ll see that each line ends in two carriage-returns. I guess the command prompt can handle that, but Console::ReadLine sees each one as a delimiter. I can’t just ignore empty lines, because then I’d remove really empty lines. This problem doesn’t occur with other commands, like TYPE.
Yes, I see that now. Depending on which editor I use, some show it without blank lines, and some with. I guess that is what was confusing me. Not a problem at all.
I guess tee passes the Ctrl+C to ping as well? It stops too. Is that how it works? Thanks, Alex.
I’m thinking that when tee shuts down the pipe, ping stops.
OK, Alex — I’ve updated the download to solve the Ctrl+C problem (which also prevents the file from being truncated). I had to add an event handler for CancelKeyPress and simply ignore it. That allows the interrupt to be seen by ping, which handles it as you expect and sends the remainder of its output to tee.
To do this, I had to upgrade syntax to eliminate the /clr:oldsyntax switch. This version was built with Visual Studio 2008.
[...] (I bet that’s not his real name, and I beg forgiveness if the masculine pronoun doesn’t apply) provided a solution to the double-spaced output problem users occasionally experienced with my tee utility for [...]
[...] Sterling W. “Chip” Camden started with such a .NET implementation of tee – in Visual C++ – back in 2005. Though his TEE page indicates it is based on .NET 1.1, his current [...]
Thanks for mentioning my C# conversion!
–jeroen
My pleasure! Thanks for letting me know about it.
[...] PowerShell contains a tee cmdlet. For cmd, you will need to download and install a separate utility. May 4, 2010 6:21 am ChrisF For Windows you can do [...]
Hello, thanks for the tool. What about stderr? Can I use tee with stdout and stderr at the same time?
If you want them both to go to the same instance of tee, then use:
whatever 2>&1 | tee wherever.log
Unfortunately, I don’t think you can pipe stderr separately.
Thanks, I can live with your solution. Anyway maybe would be nice to be able to specify 2 files instead of one, one for stdout and another one for stderr.
I agree, that would be nice. Unfortunately, I don’t think there is any way to get the stderr from the original process piped into tee without merging it into stdout first. That appears to be a limitation of the shell (unless someone can dispel my misconception).
Well, you really can but then things start to be really ugly (imho). You can check the syntax here:
http://serverfault.com/questions/63705/how-to-pipe-stderr-without-piping-stdout
Yes, but at that point you lose the ability to pipe stdout to anything except a file. So it’s still not possible to have two tee.exe’s (or anything else) consuming stdout and stderr separately.
Hi,
This is just what I need – except for a problem I’m seeing where output to the console is double spaced – not between the lines, but between every character.
For example:
>subinacl /help
Outputs perfectly to the console
>subinacl /help >foo.txt
Outputs perfectly to foo.txt
>subinacl /help | tee foo.txt
Outputs perfectly to foo.txt, BUT output to console is
double-spaced
Any idea what’s up with this? I’m using the 4/4/2010 version.
Thanks!
-jim.
Oops, looks like it’s Wiert code, not yours. Thanks anyway for starting this – I’m still hoping to get it working properly.
Cheers,
-jim.
Maybe you should leave a comment on his blog about it: http://wiert.wordpress.com/2011/08/16/revisited-netc-%E2%80%93-tee-filter-that-also-runs-on-windows-xp-embedded-%C2%AB-the-wiert-corner-%E2%80%93-irregular-stream-of-wiert-stuff/
It probably has something to do with wide characters, would be my guess.
Hi – this version of tee is great, and I’ve used it successfully for years, until I upgraded to a 64bit machine recently and now I get a SideBySide error – The application has failed to start because its side-by-side configuration is incorrect. Please see the application even
t log or use the command-line sxstrace.exe tool for more detail.
And in the event log
SideBySide error
Activation context generation failed for “F:\Automated_Jobs\cpptee\Release\tee.exe”. Dependent Assembly Microsoft.VC90.CRT,processorArchitecture=”x86″,publicKeyToken=”1fc8b3b9a1e18e3b”,type=”win32″,version=”9.0.21022.8″ could not be found. Please use sxstrace.exe for detailed diagnosis.
Is there any chance there is a 64bit version available?
Thanks, Dom