Keep the best 5 (updated)

November 24, 2010 at 4:24 am 4 comments

| Gabriel |

Last year I mentioned my policy of assigning about seven quizzes and then keeping the best 5. I then had a real Rube Goldberg-esque workflow that involved piping to Perl. Several people came up with simpler ideas in the comments, but the most “why didn’t I think of that” was definitely John-Paul Ferguson’s suggestions to just use reshape. Now that I’m teaching the class again, I’ve rewritten the script to work on that logic.

Also, I’ve made the script a bit more flexible by allowing it to specify in the header how many quizzes were offered and how many to keep. To make this work I made a loop that builds a local called sumstring.

[UPDATE 11/29/2010, applied Nick Cox’s suggestions. Old code remains but is commented out]

local numberofquizzes 6
local keepbest 5

*import grades, which look like this
*uid    name    mt  q1  q2  q3
*5001   Joe     40  5   4   6
*4228   Alex    20  6   3   5
insheet using grades.txt, clear
*rescale the quizzes from raw points to proportion 
forvalues qnum=1/`numberofquizzes' {
	quietly sum q`qnum'
	replace q`qnum'=q`qnum'/`r(max)'
*build the sumstring local (original code)
local sumstring ""
forvalues i=1/`keepbest' {
	local sumstring "`sumstring' + q`i'"
	disp "`sumstring'"
	local sumstring=subinstr("`sumstring'","+","",1)
	disp "`sumstring'"
*reshape long, keep top few quizzes
reshape long q, i( notes uid name mt) j(qnum)
recode q .=0
gsort uid -q
by uid: drop if _n>`keepbest'
by uid: replace qnum=_n
*reshape wide, calc average
reshape wide q, i(notes uid name mt) j(qnum)
*build the sumstring local (w/ Nick Cox's suggestions)
unab sumstring : q* 
disp "`sumstring'"
local sumstring : subinstr local sumstring " " "+", all
disp "`sumstring'"
gen q_avg=(`sumstring')/`keepbest'
sort name
sum q_avg

*have a nice day

Entry filed under: Uncategorized. Tags: , , .

PDF DRM and CUPS-PDF Conditioning on a Collider Between a Dummy and a Continuous Variable


  • 1. Nick Cox  |  November 26, 2010 at 1:46 pm

    Your creation of the local macro sumstring can be trimmed a bit to

    forval i = 1/`keepbest' {
    local sumstring `sumstring' q`i'
    local sumstring : subinstr local sumstring " " "+", all

    The general points of interest are

    1. The local need not be initialised to empty. It can just be created first time around the loop.
    2. Evaluating the substring using = won’t bite you in this problem but it might in others. Thus use the extended macro function as above.

    More at

    SJ-8-4 pr0045 . . . . . . . . Stata tip 70: Beware the evaluating equal sign
    . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . N. J. Cox
    Q4/08 SJ 8(4):586–587 (no commands)
    tip explaining the pitfall of losing content in a macro
    because of limits on the length of string expressions

  • 2. Nick Cox  |  November 26, 2010 at 1:47 pm

    -unab- offers an even quicker way to get -sumstring-.

  • 3. gabrielrossman  |  November 29, 2010 at 1:50 pm


    Thanks for both comments which were helpful as always. I’ve updated the post. Also, I’ve taken the liberty of formatting your comments so the code is escaped.

    I thought I had to declare the local at the upper level (or use the “return” syntax) if I wanted to call it outside of the loop, but I experimented with commenting out the declaration and saw that you’re right — it is superfluous.

    Also, thanks for suggesting “unab”. In reading the documentation it looks like “unab” is mostly meant to behave like tab-completion, that is, to expand an unambiguous abbreviation. However by experimenting I found that it works with globbing, that is it will expand to the full set of things consistent with the abbreviation. Unfortunately this is incompatible with the “max” option, but the workaround for this is to move the building of sumstring to later in the script, after the superfluous variables (ie, quizzes) have been dropped.

  • 4. Nick Cox  |  November 30, 2010 at 4:47 pm

    Thanks for your thanks.

    A Tip on -unab- appears in Stata Journal 10(3).

The Culture Geeks

%d bloggers like this: