The Importance of Operational Definitions

This morning a colleague of mine was discussing a recent survey from Business Week that ranked America's Unhappiest Cities. I decided to use this in my intro classes as a critical thinking exercise to recognize sensationalist media claims and the importance of an operational definition.

So I began by telling my students that St. Louis was ranked #2 on a list of unhappiest cities in America, and asking them what they thought about that. What does it mean to live in the #2 most unhappy city? My students wanted to know who was #1 and who was #3 (Portland and New Orleans respectively). Several students thought that the study must be bogus because they themselves are not unhappy. Finally someone hit on the issue, "What do they mean by unhappy?"

So we listed how we might measure unhappiness. My students had a lot of great examples:

• Depression (number of people diagnosed with Major Depressive Disorder)
• Education level
• Sports teams performance (measured in World Series wins)
• Suicide
• Divorce rate
• Unemployment rate
• Obesity

Not a terrible list, but in each case could one make the argument that the measure is both valid (the score means what we say it means) and reliable (the score is 'true' ; consistent). For example with depression (as defined by my students), there is a clear bias based on cultural acceptance of treatment-seeking behavior. The large Hispanic population of the American southwest are less likely to seek treatment for depression (and are thus less likely to be diagnosed), compared to the pacific northwest where there is far less stigma associated with depression and therapy. Incidentally Business Week used a weighted amalgam of depression rates, suicide rates, divorce rates, crime, unemployment, population loss, job loss, weather, and green space.)

This ultimately led to a discussion towards the purpose of such a survey. Should this encourage St. Louisians to engage in marriage counseling? Build parks? Put bars on the windows? Buy some football players? My students decided that the survey did not in fact measure unhappiness and even if it did it wouldn't be a reliable measure that could be used to compare cities. So ultimately it means very little to them. Good for them!

Anyway, here are the "top" 10 unhappy cities:

1. 1. Portland, Or.
2. 2. St. Louis, Mo.
3. 3. New Orleans, La.
4. 4. Detroit, Mi.
5. 5. Cleveland, Oh.
6. 6. Jacksonville, Fl.
7. 7. Las Vegas, Nv.
8. 8. Nashville, Tn.
9. 9. Cincinnati, Oh.
10. 10. Atlanta, Ga.