Archive | October 2010

Unemployment in the US

Slate put up a cool interactive graphic of unemployment by county in the US. It’s pretty depressing. I’d hate to have to be looking for a job right now. None of this bodes well for the election, which is going to give us the largest collection of extreme right elements in Congress that we’ve ever seen.

This is just a screen shot, but clicking on it will take you to the actual Slate page:

Iraq Wikileaks War Diary Part 3

Today, I made a short video documenting all combat incidents involving deaths (in Michigan colors no less). I mapped the kernel density of the points using the spatstat package in R. You can watch the entire Iraq War in just over 1 minute!

Using this method to visualize the war presents some really interesting results. The war starts out and conflict pretty much occurs all over the place. Around the time of the surge, fighting becomes intensely concentrated around the Baghdad area, but quiets down for the rest of the country. Events slowly start to move north, so that all of Baghdad and the northern area are regions of intense fighting. By the end of 2009, things begin to quiet down again, but with some sporadic events spread over the countryside again.

At the time of the surge, I am reminded of Agent Based simulations, and wonder if the increase in intensity of fighting in Baghdad actually propagates itself, forcing insurgents to concentrate resources in Baghdad at the expense of the rest of the country. As fighting dies down, and need for fighting decrease, insurgents leave the city, taking the fight with them. My theory would be that intense fighting concentrates combat events more, whereas mid-level and low intensity fighting allows conflict to spread geographically, due to distribution of resources.

As the surge increased the number of US soldiers in Baghdad, insurgents were forced to concentrate their resources withing the city, reducing their ability in fighting for control of outlying areas. Clearly, this is to the strategic advantage of the Occupying force, as they do not need to expend incredible resources fighting a war over a large geographic space, and, with superior firepower and technology, they are able to quash a large number of insurgents in a small amount of time. It could be a bait and trap tactic, but clearly it is to the incredible disadvantage of the civilian population. While likely not the target of US forces (seriously, what would be the merit in that?), they clearly become a target of insurgents and candidates for widespread suicide terror campaigns. Insurgent deaths are maximized at little financial and temporal cost, but civilians become caught in the cross-fire and become an easy target of fear-based propaganda strategies. Whether this was truly the strategy behind the surge, is up to speculation. It’s also very possible that insurgents had previously moved into the Baghdad area with the same strategy. Obviously, this isn’t something I have thought through very deeply, but it’s worth exploring.

Wikileaks Iraq War Data Part 2

In the limited time I have, I will engage on a rudimentary time series analysis of the Wikileaks Iraq War Diaries. As I states in the previous posts, I have limited the dataset to include only those entires which report deaths of some kind. There are three different classes of casualties I will be working with, coalition, civilian and enemy. I have left Iraqi armed forces out, because I question the classification. I will start by providing a list of main points for those who just want to breeze through:

Main Points:

1. Civilian casualties are through the roof compared to casualties among those actually fighting the war.
2. Things got really bad around the time of the surge, but have since quieted down. US and insurgent forces seem to have gotten better at attacking one another without actually killing one another, but civilians keep on dying.
3. While all deaths and injuries have decreased to levels not seen since the invasion, there were disturbing spikes in civilian deaths and injuries in the later part of 2009.

Time Series 101: Time series are just that: a series of observations made of the course of time. Here we have daily reports for the entire calendar years of 2004 to the end of 2009. All time series are made up of three components: trend, seasonality and random noise. If we were to look at a successful business, we could spot these three components in daily sales data, for example. Sales may be going up over the course of a few years (trend), sales may regularly fluctuate within a year due to seasonal events like Christmas (seasonality), and there may be some daily random ups and downs that just happen for reasons unknown (noise). I will examine these three components separately, and then move on to questions of relationships between the series.

The Data: First, let’s look at the series themselves. Like I said, I have three series of wounded and killed: one for civilians, one for coalition members and one for enemy combatants. Civilian casualties are the most striking. While casualties were low at the beginning of the war they peaked at around the start of 2007, reaching an all time high of 972 wounded and killed on February 3rd, 2007. The mean number of wounded and killed for 2004-2010 is 58 people per day. That’s 1.87 wounded or killed per 10,000 people per day in Iraq. In America, .59 people per 10,000 people per day are wounded or killed by guns. One’s chance of being killed or wounded by guns in Iraq over the course of the war is three times that of the States, which still has the highest number of gun related injuries and deaths in the developed world. Notice, at the end of the series, civilian casualties have again gone up.

On average, 5.26 coalition members die or are killed every day. The bloodiest day for coalition forces was Jan 26th, 2005, when 36 coalition service people were killed. 12.33 enemy fighters are killed or wounded each day, and the worst day for them was Valentine’s day, 2005 when there were 411 casualties. Note the extreme difference in scales between the three series. Civilians get killed and wounded the most, US and coalition members the least.

Trend: Below I have produced three plots representing the trend component of the time series. The patterns are interesting. At the start of the war, very few civilians died in combat incidents, but very many US and enemy combatants did. As insurgents and US forces got worse at killing each other, the efficiency and scale at which civilians started to die rose considerably. The surge happened in 2006, causing a severe uptick in civilian and enemy casualties, but US and coalition deaths did not reach levels that were seen at the start of the invasion. Post surge, all three casualty levels began to plummet until 2009, when things began to calm throughout the country. It is interesting to me that civilians were largely spared at the beginning of the conflict. It is possible that events at this time were largely limited to areas outside Baghdad, but as fighting moved into the city, civilian deaths and causalities rose again.


Seasonality: To the left, I have produced plots of seasonality. You can click on them to check them out but they are not entirely enlightening. A better way to check for seasonality is the autocorrelation plot. We expect that tomorrow will be somewhat like today. However, 2 days from now will be less like today than tomorrow. This is the concept of autocorrelation. Things that are close together are more alike than things that are far apart. If a series is seasonal, then we will expect there to be correlation between the same time period in the following year. The plots below show the autocorrelation for 3 years. There is no real evidence for seasonality in any of these plots but there do appear to be some blips in coalition and enemy casualites at the 4 and 6 month marks.

Next: Cross correlation between civilian, enemy and coalition deaths.

Wikileaks Iraq War Diary 2004-2009 Part 1

After being told that it was a hoax (by a kind reader of this blog who didn’t really say it was a hoax), Wikileaks released more than 400,000 military records from the Iraq war covering the years from 2003 to then end of 2009. This data will likely be ignored in the present political climate, but the historical and scientific significance of this data dump cannot be underestimated. It’s shocking really, that nearly a decade fighting two major wars, tens of thousands of American and civilian dead and trillions of dollars later, the Afghan and Iraq wars are a mere blip in a political climate that would allow a moron like Christine O’Donnell the chance to fill a Senate seat. We have truly become a Confederacy of Dunces, in the worst way imaginable.

The current press is touting this massive database as a smoking gun in the American involvement in Iraq, and will most likely play up the juiciest and most damning elements. There will be analyses of further torture past Abu Ghraib, civilian death counts which do not match previous estimates, but in all the finger pointing, the true extent of human costs of the war will be lost. It is to this end that I seek to inform and not blame. The war was wrong, based on lies and sold to an uninformed and uninterested American public coming out of 9/11 and still weighing the meaning of being drawn into a state of vulnerability to international terrorism that many other countries had been living in for decades. I do not present my analysis to point a finger at anyone as the players in the Iraq War are so many and the intricacies far outside my area of knowledge.I merely wish to illustrate the incredibly meaningless costs of warfare, through the scientific tools at my disposal. To this end, I will be presenting a series of statistical analyses of the Iraq War over the course of the next week.

Data: The full data set contains more than 400,000 records, but I have limited my working dataset to only those records which contain deaths. There are four categories of people contained in the data set, friendly (coalition), enemy, Iraqi army and civilian. Both numbers of dead and numbers of wounded are noted, but record keeping may or may not be 100% accurate. However, given the large number of records, it is quite likely that a significant percentage of them reflect more or less accurate numbers.

The map in the upper left of this post represents points of all deaths in Iraq between the beginning of 2004 and the end of 2009. Incidents are recorded for nearly all populated locales in Iraq, particularly those along major roads. Baghdad, of course, has the largest number of incidents. Breaking these up into civilian and coalition casualties, we get a more complete picture of how violence is distributed within the country. The maps below represent full country deaths and wounded, with dots proportional to the number of casualties. Larger dots mean more casualties noted in the record for that day, smaller dots mean less. Dots in all of the maps are to the same scale. That is, the size of each dot represents the same number of casualties in both civilian and coalition casualty maps below.

The differences between the two are striking. Many, many more civilians die and become wounded in conflicts than US and coalition forces. This is not to belittle casualties within the military, but the number are beyond disproportionate. Zooming in one Baghdad, the disparity is even more pronounced:

Whereas civilian deaths and wounded are splattered over Baghdad indiscriminately, US and coalition casualties are rather limited. Of course, the US military has the advantage of armored military vehicles, body armor and weaponry, but it cannot be denied that the greatest toll in human life is waged by the insurgency itself through suicide bombings, conflicts in open air markets and free for all urban warfare. Our presence in the country likely created the conditions necessary for such carnage, but I doubt that the American public ever had any clue as to the extent. Maps such as these make it clear the extent to which the Iraqi people have suffered and the stupidity of warfare.

Next: Time Series Analysis of the Iraq War Diary

Poverty in the USA

No time for a real post, so I leave with this informative info graphic on the extent of poverty in the US:

Things haven’t changed much since the 19th century, assuming that one can use illiteracy as a proxy for poverty. The South and the lower parts of Texas and Arizona are still hotbeds of poverty and a lack of resources. Perhaps the trickle down economy just has yet to hit these places…. Planner –


I’ve just tried to write two blog posts and completely failed. So I’ll just leave anyone who looks at this blog with some music I was involved with a million years ago.

Will just have to wait until the next Wikileaks dump of military files from Afghanistan pops up.

Afghan Wikileaks War Diary Part 3: Cycle of Violence?

Yesterday, I posted an excellent paper on the Israel-Palestine conflict. In the paper, the authors used econometrics techniques to determine whether Israel reacts to Palestinian attacks, Palestine reacts to Israeli attacks, or both.

A few months ago, I had posted a series of analyses on the War Diaries posted on Wikileaks detailing a number of actions throughout the Afghanistan War. To reiterate, the database contains meticulous records of every military action from 2003 to the end of 2009. Within each record, is the number of military killed or wounded, the number of civilians killed or wounded and the initiator of the conflict, be it friend or enemy. There is other important and detailed information, but mostly, I am concerned about civilian casualties and who was responsible for the conflict.

The time series, detailing civilian casualties from 2003-2008 is below. There are two, of course, one representing enemy initiated conflicts (red) and the other friendly initiated conflicts (blue). Note the difference in scales.

Now the question is, do friendly initiated actions resulting in civilian casualties encourage enemy retaliation, do enemy initiated actions induce friendly retaliation, or both? Basically, I would like to know if the American presence agitates conflicts which result in civilian death or if the opposite is true. Using the MSBVAR package in R, I made use of the impulse response function, which tracks the level increase in one time series (response), given a change in another (shock).

In essence, what this will tell us, is the excess number of persons killed or wounded in a subsequent conflict, given an action by either side. As an example, assume that the American military conducts a military action and that action results in civilian death. Does that action result in an enemy lead to a retaliation which will also results in civilian death? Let’s check out the plot below:

What we can see here, is that an enemy lead action, will result in an extra 15 civilians being killed the following day in another enemy lead action. Enemy lead actions result in an approximately .3 extra persons dying three days later in a friendly lead conflict. Friendly lead actions do not result in enemy lead action, but friendly lead conflicts do result in a number of people dying in another friendly lead conflict the following day. From a probability standpoint (and a separate and not noted here graph), an enemy lead action will mean that there is a 50 percent chance of a civilians dying in an enemy lead conflict the next day and a 15% probablity that a civilian will die in a friendly lead conflict three days later. Friendly lead conflicts do not appear to result in more civilian deaths later on from enemy lead actions, but do lead to a 50 percent chance of someone dying in a friendly lead action.

What does this mean? Well, assuming that the categorization of “friendly” and “enemy” lead actions is correct, this means that action lead by the American military do not provoke enemy retaliation as the common wisdom would suggest, but that enemy attacks are self-determined events lead for their own interests (which could include ejecting us from their country). Of course, this methodology only tracks temporal sequences and does not indicate causality, nor does it provide insight into big picture attitudes and motivations of Taliban battle tactics. It’s likely that our presence may motivate attacks, but that our military actions do not. This question, however, cannot be evaluated through data.

%d bloggers like this: