Append to nothing

March 19, 2009 at 4:04 am 3 comments

| Gabriel |

One of the things that often frustrates me is that if you’re doing a loop that involves an “append” command you need to set  a seed for the file. For a long time, I would write one set of code for the first instance, then a second set for the append loop. This is a pain in the ass as it makes the code redundant (and thus error-prone and hard to modify) and you have to remember to remove the first case from the loop list. For these reasons it’s much better to keep everything in the loop but since you can’t append to nothing this requires creating a seed file.

To set the seed for an append command loop, you create a dataset with 1 row that has a missing observation (then you remember to delete this record later)

set obs 1
gen x=.
foreach file in $list{
 append using `file'
drop in 1 /*alternately, drop if x==. */

Likewise, you often want to use the append option for Stata commands (e.g., “log” or “esttab”) that write ASCII files. The easiest way to do this is to use the “touch” command to create an empty file then the “mv” command to write over your existing file with this blank file. You now have a blank text file suitable for appending to. Note that “touch” and “mv” are Unix commands but I’m guessing you can a) trick Windows into speaking POSIX syntax with Cygwin or b) find equivalent native MSDOS syntax.

shell touch emptyresults
shell mv emptyresults results.txt
foreach case in $list {
 disp "`case'"
 quietly reg y x if serialno==`case'
 esttab using results.txt, plain append

Entry filed under: Uncategorized. Tags: , .

Bass projections Bootstrapping superstars


  • 1. mike3550  |  March 20, 2009 at 1:53 pm

    Gabriel — congratulations on your new blog! I’m excited to keep reading and learning about new tricks in Stata.

    I might have a suggestion that doesn’t require using the “shell” command that might be easier (and more self-contained) to code in Stata. It uses two things: 1) Stata’s “tempvar” command and 2) the fact that you can save a file in Stata with zero observations. First, “tempvar” creates a local macro that points to a variable that you can use just as you would any other variable (except that you refer to it in local `macro’ syntax); but, the nice thing is that the variable is automatically deleted after the .do (or .ado) file runs. Second, because the original dataset has zero observations (and the temporary variable automatically deletes itself), you are left with just the appended datasets from your list:

    // Declare temporary variable name
    tempvar x
    set obs 0
    // Set temporary variable equal to missing
    gen `x’ = .

    // Append files
    foreach file in $list {
    append using `file’

    Similarly, Stata also has the “tempfile” command, which does the same as “tempvar,” except that it points to a file name. This is usually how I run my code. It has two nice advantages: 1) tempfiles also delete themselves after a .do (or .ado) file runs and 2) they can be used just as any other file. So, for your second example, inside of Stata, you can use:

    // Declare name of temporary file
    tempfile tmp
    // Save empty contents (much like “touch” command)
    save `tmp’

    foreach case in $list {
    disp “`case'”
    quietly reg y x if serialno==`case’
    esttab using `tmp’, plain append
    // Copy to your final output file (because the temporary file tmp will be deleted)
    copy `tmp’ results.txt

    This method also has the advantage (in addition to being platform-independent–I work on a PC), that if something goes wrong in your loop, then your results.txt file is not overwritten because it is not copied until the end of all of the commands.

  • 2. gabrielrossman  |  March 20, 2009 at 2:03 pm


    platform independence and not cluttering the disk with temp files are indeed advantages, especially when the syntax isn’t much more complicated. i think in the future i’ll be using the syntax you’re suggesting quite a bit.

    thanks much

  • 3. mike3550  |  March 20, 2009 at 1:00 pm

    Gabriel, I’m glad it was helpful!

The Culture Geeks

%d bloggers like this: