Causal Analysis of Non-Experimental Data

What are the problems of analyzing non-experimental data in social and medical sciences? How is it possible to analyze causality in observational data? What is the state of causality between higher education and higher income in the US? These and other questions are answered by Christopher Winship, Diker-Tishman Professor of Sociology at Harvard University.

What has happened since the early 1980s is that social and medical sciences have gone through a revolution in terms of thinking about how to analyze non-experimental data. The approach in part is to take experimental thinking and try to extend it to contexts where experiments aren’t possible. What is important about this is that for many decades in both areas causal analysis was being done in a very informal way. Sometimes it was just enough that there was an association: we see that people who go to school more make more money, so we claim that must be the effects of going to school. But even more sophisticated things: we look at people whose parents or siblings went to college and we ask, “If you went to college, did you make more money?”

We might want to say, “Look at a set of people who live in a city.” Here I am at Harvard; in Boston there are many universities and colleges, 67 to 85, depending on how you count them. It’s pretty easy to get to college here; it’s convenient to go to college. We might compare people in Boston, where in some sense the cost of transaction, moving and getting to college is low, to people who live in rural Georgia where there aren’t many colleges and getting to college could be quite difficult. It’s known as an instrumental variable technique. I don’t think I want to go on the technical details about that but we can make a comparison because the people in Georgia are comparable to those in Boston. But the big difference is it was much more difficult for them to get to college and as a result they were less likely to be to college.

Now we have this well-defined notion of causality. What has happened within the social and medical sciences imparted two things. First, people have gone back and looked at traditional methods, like regression analysis and other approaches, and said, “Did these approaches actually give us true causal estimates? Do these methods work?” Often the answer was ‘no’. Then, in a parallel way, people have also gone and looked at various studies and asked, “Did the results, which people claim that they got previously, end up holding when using better causal analysis methods?”