How to Review a Literature

| Gabriel |

Following the reader service model of O&M’s recent “How to Read an Academic Article” and OT’s long running grad skool rulz, I figured I’d describe the proper way to review a literature for a research paper. I should start with a lawyer’s joke/parable.

Eugene Volokh recently puzzled his blawgosphere audience with the term “red cow.” As commenter “James E.” explained:

A country practitioner was retained one day by a client whose red cow had broken into his neighbor’s grain field, and litigation ensued. The practitioner went carefully over the details of the facts in the case with a student in his office, and assigned to the student the duty of “looking up the law” on the subject. Some time after he asked the student what success he had had with the authorities bearing on the case. The student replied: “‘Squire, I have searched diligently through every law book in the library, and there isn’t a red cow case in them.”

Central Law Journal, Vol. 79, p. 299 (1914)

The joke of course is that this lawyer thought the issue was red cows rather than trespassing, negligence, and other abstract legal concepts. This was a lot less funny when I realized that when I was in college and my first year or two of grad school, this kind of substantively-focused literalism was exactly how I would approach doing a lit review for a research paper. I would open up Sociofile (now called “Sociological Abstracts”) and search for substantive key terms, something like “social movements AND television.” That is, I was searching for prior literature on my substantive issue.

A substantive search is worth doing to a certain extent, but it’s not nearly as important as getting (and understanding) theory. A single theory often involves wildly disparate empirical issues. For instance, Status Signals has chapters on banking, wine, and patents, as well as more fleeting references to things like jewelry. So how do you do the theoretical aspect of the review? Well, to a large extent it’s just an issue of learning a large body of literature inside out, but that takes a very long time. In the meantime, here’s the advice I give to my grad students.

1. Use Sociological Abstracts, Google Scholar, etc. for substantive queries but realize that this will only be about a quarter of the work. These databases aren’t very good at queries by theory.

2. Figure out what theoretical problems are at issue in your work. Bounce your empirical issues of your friends and mentors to see what theoretical issues they see. They may suggest theories you’ve never heard of. Also ask them for specific citations that they recommend.

3. Search for essays on the previously flagged theories in Annual Review of Sociology (and possibly Annual Reviews for adjacent disciplines or Journal of Economic Perspectives) to find a review of this literature, preferably one from the last ten years. (If you’re lucky, you’ve recently taken a graduate seminar on your target literature, which is effectively ARS as live theater.) You can also use a few empirical publications that you’ve read or which are recommended to you as providing particularly good theoretical syntheses.

4. Use these to snowball sample, both backwards and forwards in time. To snowball backwards, read the articles and whenever they mention a citation that sounds interesting, add it to your shopping list. To snowball forward, use Google Scholar to do a cited reference search of your key citations and again, take the stuff that looks good. I prefer GScholar for this over Web of Science because it includes working papers and such, giving you more of the “invisible college.” As you read these things you’ll find still more good cites.

5. Actually read all this stuff and pull out the theoretical problems involved and how they hang together. Try to find one to three important theoretical problems and use each of them to derive a proposition that can be operationalized into an empirically-testable hypothesis. Read empirical articles that you admire and note how they structure their lit review / theory section.

Note that this step is as much imposing structure on the literature as about recognizing the structure that pre-exists because, frankly, the literature is often muddled. For instance, in writing the lit review for my Oscars article, I noticed that a lot of people simply confuse different spillover models — citing Kremer QJE 1993 when what they seem to have in mind is a much better fit with Saint-Paul JPE 2001 or Stinchcombe ASR 1963.

6. Get back to me in about two years when you’ve finished doing all of this and we can talk about actually doing the empirical part of the project.