Who should research benefit?

I’m reflecting on an interesting discussion that some of my colleagues and I had over dinner last night. We were discussing the Demographic Surveillance Program (DSS) that I am currently managing.

For those not familiar, DSS programs, generally speaking, do what you local city hall and health department do. They track births, deaths and migration and keep track of specific health outcomes such as cause of death and incidence of particular diseases. It’s pretty basic stuff, but outside the capacity of many developing countries, and the focus on health issues allows for deeper investigation of public health outcomes.

One of us, who is Kenyan, remarked that the DSS program needs to coordinate with the Kenyan government to satisfy specific data needs. I took issue with this idea.

First, as a program sponsored by a research institution, our number one priority should be the accommodation of research projects. A university receives public money to perform research projects which benefit and are directed by the scientists who run them.

Second, a research institution is not a development NGO that seeks to provide public services that countries are unwilling or unable to provide themselves. It is a fact that the Kenyan Government, like many African governments, has a poor record of providing public services. I would argue that while we should certainly offer results of our activities to the government and local stakeholders, that a university sponsored DSS should not be considered a substitute for that which governments should be doing themselves. The assumption is often that African government are incapable of doing things like tracking births and deaths, but the reality is that they are simply unwilling and the activities of NGOs often exacerbate this problem.

Third, government involvement in foreign sponsored research is a slippery slope. Many African governments are wholly or partially autocratic. They do not exist or function to benefit their people, but rather exist only to enrich themselves and maintain power. Academic research should, in no way, contribute to keeping autocrats in power and should operate independently from any political body. In fact, academia has a duty to question power both domestically and internationally. Unnecessarily submitting to government demands opens up a number of potentially problematic issues, including the suppression of results which are inconvenient to government bodies.

While research projects require government approval to function, the power of government to influence the course and outcomes of research should be limited. While many bureaucrats would take issue with this, the reality is we can always just go somewhere else.

A complex sampling journey

Sampling scheme

Sampling scheme

When I have been a part of surveys in the past, little attention has been paid toward following any sort of intelligent sampling plan. The time allotted to collect the data has always been too short, resources extremely limited and the conditions of the field mostly unknown.

Mostly what we’re left with is a convenience sample of some kind, usually determined by introductions from the survey workers themselves. It is absolutely the worst way to run a survey and the data is usually crap, but, worse yet, unverifiable crap.

Ideally, in a household level survey, we’d run in establish target areas for sampling, do a complete census on target areas and then perhaps take a random sample within those areas. At the minimum this would be a relatively decent approach.

Unfortunately, I often encounter one of two situations. The first is the convenience sample I mentioned above, which is inherently biased toward the social connections and thus the demographic of the survey workers themselves. If you want to do a sample of someone’s friends and family, this might be a good start, otherwise its completely awful.

The second is the “school based survey,” a design I think I hate more than all others. This travesty of sample design depends on the good graces of families which send their children to school, being lucky that the kids you are interested in show up to school the day of the survey and reasonable connections with school administrations. Worse yet, if you’re doing a survey on health, the chances that you’ll the kids you’re really interested in at school is really low. People love this awful design because it’s convenient, cheap, can be done in a short time and has the added benefit of providing one with warm feelings.

I’ve resolved myself to do neither of these again. As the manager of a Health and Demographic Surveillance System based in Kenya which monitors more than 100,000 people in two regions of Kenya, I decided I have a unique opportunity to do something a little more interesting.

In gearing up for a pilot survey to improve measurement of socio-economic status in developing country contexts, I realized that I had an incredible set of resources at my disposal. I have a full sampling frame on two sets of 50,000 people in two areas of Kenya, basic demographic information and a competent staff with sufficient time to do a project which otherwise would interfere with their regular duties.

With some help from a friend (well, much more than a friend), I maneuvered the basic of complex survey design and came up with something that might work relatively well for my purposes.

The DSS of the area of Kwale, Kenya I’m working in is divided into nine areas, each delegated to a single field interviewer who visits each of the households three times a year. Each field interviewer area is then divided into a number of subgrids, the number of which arbitrarily follows the population surveyed and the logistics of the survey rounds. Some areas are easier to survey than others. Each grid then has a number of households within them, the number of which varies depending on population density.

I want to target three areas, each of which ostensibly will represent different levels of economic development, but in reality represent different types of economic activities and lifestyles. One is relatively urbanized, another is purely agricultural and the third is occupied by agro-pastoralists who keep larger herds of large animals.

RplotI then decided to choose 20 grids in each area at random, and then want to select up to 10 households from each selected grid again at random. The reason for choosing this strategy was purely a logistic one. Survey workers can do about 10 households in a day and I’ve given them a month (20 working days) to do it before they have to start on their next round of regular duties. Normally, I’d like to do something fancier, but without any previous data on the variables I’m interested in, it just wasn’t possible.

I have discovered that this design is called a stratified two stage cluster design which makes it all sound fancier that I really believe it to be. The advantage to using this design is that I’m able to control for the selection probabilities, which can bias the results when doing statistical tests. I have no doubt that the piss poor strategies I’ve used in the past and the dreaded “school based survey” I mentioned above are horribly biased and don’t really tell us a whole lot about whatever it is we’re trying to find out.

I used the survey package in R to determine the selection probabilities and, as I suspected, found that the probability of selection is not uniform across the sampling frame. Some households are more likely to be included in the survey, biasing the data in favor of, for example, people in more densely populated areas.

dstrat1<-svydesign(id=~grid+houseid,strata=~fiarea,fpc=~fiareagridcounts+gridcounts,data=out)

Alright, enough for now….

Measuring socio-economic status in Kenya

Rplot04I was just screwing around with some data we collected a bit ago. In a nutshell, I’m working to try to improve the way we measure household wealth in developing countries. For the past 15 years, researchers have relied on a composite index based up easily observable household assets (a la Filmer/Pritchett, 2001).

Enumerators enter a households and quickly note the type of house construction, toilet facilities and the presence of things like radios, TVs, cars, bicycles, etc. Principal Components Analysis (PCA) is then used to create a single continuous measure of household wealth, which is then often broken into quantiles to somewhat appeal to our sense of class and privilege or lack thereof.

It’s a quick and dirty measure that’s almost universally used in large surveys in developing countries. It is the standard for quantifying wealth for Measure DHS, a USAID funded group which does large surveys in developing countries everywhere.

First, I take major issue with the use of PCA to create the composite. PCA assumes that inputs are continuous and normally distributed but the elements of the asset index are often dichotomous (yes/no) or categorical. Further, PCA is extremely sensitive to variations in the level of normality of the elements used, so that results will vary wildly depending on whether you induce normality in your variables or not.

It’s silly to use PCA on this kind of data, but people do it anyway and feel good about it. I’m sure that some of the reason for this is the inclusion of PCA in SPSS (why would anyone ever use SPSS (or PASW or whatever it is now)? a question for another day…)

So… we collected some data. I created a 220 question survey which asked questions typical of the DHS surveys, in addition to non-sensitive questions on household expenditures, income sources, non-observable assets like land and access to banking services and financial activity.

The DHS focuses exclusive on material assets mainly out of convenience, but also of the assumption that assets held today represent purchases in the past, which can act as pretty rough indicators of household income. So I started there and collected what they collect in addition to all my other stuff.

This time, however, I abandoned PCA and opted for Multiple Correspondence Analysis, a technique similar to PCA but intended for categorical data. The end result is similar. You get a set of weights for each item, which (in this case) are then tallied up to create a single continuous measure of wealth (or something like it) for each household in the data set.

Like PCA, the results are somewhat weak. The method only captured about 12% of the variation in the data set, which sort of begs the question as to what is happening with the other 88%. However, we got a cool graph which you can see up on the left. If you look closely, you can see that the variables used tend to follow an intuitive gradient of wealth, running from people who don’t have anything at all and shit in the shrubs to people who have cars and flush toilets.

HistogramsWe surveyed three areas, representing differing levels of development. Looking at how wealth varies by area. we can see that there is one very poor area, which very little variation to the others which have somewhat more spread, and a mean level of wealth that is considerably higher. All of this agreed with intuition.

“Area A” is known to be very rural, isolated and quite poor. Areas B and C are somewhat better though they are somewhat different contextually.

My biggest question, though, was whether a purely asset based index can truly represent a household’s financial status. I wondered if whether large expenditures on things like school fees and health care might actually depress the amount of money available to buy material items.

Thus, we also collected data on common expenditures such schools fees and health care, but also on weekly purchases of cell phone airtime. Interetinngly we found that over all the two were positively correlated with one another, suggesting that higher expenses do not depress the ability for households to make purchases, but found that this relationship does not hold among very poor households.

There is nothing to suggest that high expenses are having a negative effect on material assets among extremely poor households located in Area A at all. It might be the case that there is no relationship at all. This could indicate something else. Though overall there might not be a depressive effect of health care and school costs on material purchases, they might be preventing households from improving their situation. It might only be after a certain point that the two diverge from one another and households are then able to handle paying for both effectively.

Also of interest were the similar patterns found in the three areas.

Expenses

KLM Lounge time

The KLM Lounge in Schiphol is a great place. There’s decent coffee, free papers that I might read, food and you can take a shower. Then there’s the odd 60’s futurist decor that makes you think you’ve stepped onto the set of 2001: A Space Odyssey.

In trying to suppress my ambivalence about going to Kenya, boarding a domestic flight the next day and then flying to Japan four days later, I’m reading Joe Stiglitz’s new book, “Creating a Learning Society: A New Approach to Growth, Development, and Social Progress (Kenneth J. Arrow Lecture Series).”

Though I’ve just started the book, I’m finding it quite interesting. Stiglitz and others attribute development to the sharing of innovations rather than the mere accumulation of capital. Societies grow because their people learn new things, and some countries do better because they are better at learning how to learn.

Noting that private firms often entrench themselves in particular modes of operation which discourages innovation, Stiglitz argues that government investments in education and R&D and the guarantee of a legal framework which protects property can allow innovations to flourish.

In reading the book, I kept thinking about this 2km stretch of road in Nairobi which has been under construction for the past five years. It’s absolutely pathetic. Buses have to pass through a one lane mud road next to the construction site, while workers move at a snails pace, slowly pouring concrete by hand. Though the reasons for the slow tempo of road construction most certainly include corruption and mismanagement (the contractor is Kenyan), one also has to notice that nearly all roads in Kenya are built by foreign companies.

The Japanese built a masterpiece of a road, complete with cross walks, bike lanes and dedicated pedestrian ways in a tenth of the time. To Japan’s credit, they use local workers, unlike the Chinese.

Building roads isn’t complicated, or, at least, the complications have been worked with over and over and road building is now an established discipline, with text books and training programs available all over the world.

So why hasn’t the knowledge of road building been successfully transferred to Kenya? What the hell is wrong?

Perhaps this is what Stiglitz is talking about. Without beating up on Kenya too hard (but why not?), Kenyan schools are a shambles and the government is only marginally interested in improving the educational fortunes of the country. Schools are designed to train low level clerks for the civil service, and don’t aspire to train kids for science, math or engineering. Though many, many technologies are already established the world over, perhaps the poor state of education hampers technology diffusion.

Back to my coffee.

Who pays for development?

I was just reading this on the Guardian’s Poverty Matters blog:

First, identify the most important issues. One of the main problems of the MDGs, as noted in countless analyses, was their failure to bring the major structural issues to the table. I know of no one who thinks that aid is the most important contribution that wealthier countries can make to development, but the vague terms of MDG eight allowed politicians to get away with aid promises (which in some cases they didn’t keep) rather than setting a bold agenda for transformational change in global financial governance, dealing with illicit financial flows, for example, taking bold steps towards international tax reform, and introducing fairer mechanisms for working out debt repayments.

Well, yeah, very true, but again this type of reporting skirts the issue of where those illicit flows are coming from and who took out the loans. The problem with the MDGs was that it failed to put any pressure on leaders of developing countries to stop being parasites. Worse yet, they didn’t allow for the provision and protection of basic individual rights to free expression, judicial rights and economic freedom, instead opting for a few vague and unverifiable targets which failed to address structural problems WITHIN developing countries.

In Kenya, at least, the government is bleeding the populace dry. Evidence from countries such as Botswana and Korea has shown that countries who want to develop can. The biggest obstacle (among all the other obstacles) to development is a lack of political will to do it.

To its credit, the article goes on to point out that domestic ag subsidies in wealthy countries are distorting the world market and preventing developing countries from being competitive on the world market. Eliminating these subsidies will be a real challenge, at least in the US. First, subsidies control price and market volatilities. The US electorate would go bonkers if the price of food went up and down like the price of corn does in developing countries. Second, Americans simply like subsidies and enjoy protecting agricultural interests at all levels. The right likes to pander to farmers for the rural vote while the left is somewhat bummed out because their favorite organic farms don’t have access to them. Though the left loves to pay lip service to ending ag subsidies, I can’t imagine they’d be all that sad if they were offered to their local hippie farmers. That’s speculation for another day, however, and I’m no expert on ag matters.

I hate to be pessimistic about development, but the barriers to progress are hobbled by forces both within and without developing countries and no one seems to be tackling the right issues to improve matters.

Links I liked

Some links I liked. Believe it or not, I do like things.

1. A dissection of structural vs. indivdual determinants of poverty. Take that Paul Ryan. (Noahpinion)
2. Unhappy cities. Was excited that Detroit is not the most unhappy city in the US. Whew. (WP)
3. A dumb article, but I liked it anyway, at least for its scientific value. Practically speaking, it’s a fairly useless piece. Why hand shaking is bad and we should stop doing it. (NPR)
4. Tough realities for the vinyl market (Pitchfork)

In the Kenya airport again….. and talking of Marsabit, Camels and cell phones

DSC_270222JKIA has to be one of the worst airports I can think of, and my five hour delay has drawn out the awfulness of this place. No wonder someone burned it down!

While I sit here in Nairobi Java House (which now has a branch in Kisumu…. Kisumu Java House?) eating my standard “Chicken and sun dried tomato sandwich with ABSOLUTELY NO MAYONNAISE” I’m thinking about an exchange I just has with a guy in the line for check-in.

The guy was a tall, obviously northern Kenyan who turned out to be from Marsabit, one of the most remote and lawless areas of Kenya. He works at an American university on HIV things in Kenya. We started talking camels and public health and I just couldn’t help but ask.

“How did you….” I was almost ashamed to try and finish the sentence which I kind of interjected since my interest overrode wherever the conversation was going.

“I was sent to boarding school when I was six.” I didn’t even have to finish it. He knew exactly what I was asking.

To be from a place like Marsabit and working for a major American University is no small feat. First, I have never met anyone from Marsabit and the few times I’ve met people from remote places like Pokot and Turkana, I’ve been tempted to just shake that persons hand and congratulate them. Coming up through University in a place where most kids don’t go to school at all deserves a special prize.

“One cell phone is the only piece of technology you’ll see for miles. It’s an oral culture. Communication is absolutely essential and cell phones are the most prized possession a herder will have outside of his camels.”

His brother has 60 camels. I asked if we might go up there and take some blood. I could stand a trip up to Marsabit, even if armed guards have to accompany.

Follow

Get every new post delivered to your Inbox.

Join 1,628 other followers

%d bloggers like this: