On Wednesday March 6 we’ll kick off our Crime Data Analysis project, and before then, it may be worth asking “Why are a bunch of amateurs attempting what the police should be doing? " The answer is that police departments analyze crime to catch criminals, and the Data Library analyzes crime to help residents understand their own communities. We data geeks aren’t very good at catching criminals, so we’re happy to leave that task to the pros. Our project will cover only two small aspects of the field of Crime Analysis. We’ll be doing strategic analysis, looking at long term trends, and descriptive administrative analysis, which is creating summary reports for community leaders. So, some of the activities that we’ll tackle as part of this project are:

Citizens of any skill level are invited to help us analyze crime in San Diego neighborhoods. Use your knowledge of the community or technical skill to create maps, discover anomalies, and better understand the causes and solutions to crime in San Diego. Come to the Kickoff meeting by signing up at Meetup.com Crime is always a top issue for any community, but San Diego residents, town councils and community planning groups seldom see neighborhood analysis of where crime occur, and less frequently get analysis of how crime relates to factors such as street lighting, traffic, housing, or alcohol. In this project we will collect data from the city, state and private organizations to create a range of maps and statistics that explain where crime is occurring and what factors are involved in increases or decreases in crime. This project needs your help,regardless of your skills or skill level. You can participate as a non technical community activist, Excel jockey, GIS expert, beginning programmer or experienced statistician. Want to know more?

In preparing to formally launch the San Diego Regional Data Library, we’ve been spending a lot of time talking to nonprofits, community planing groups, city staff and city council members about their data needs. The staff for District 9 mentioned that Council member Marti Emerald is interested in infrastructure issues, particularly how crime is related to infrastructure deficits. Her staff specifically mentioned street lamps. Fortunately, we have a fantastic resource in SanGIS, and they have published a shapefile with the locations of San Diego Street Lamps. So, we went to work extracting and processing street lamp data. Based on this data, we’re creating density maps of street lights, which we will correlate with crime incidents. The map shown here is one of the example maps we’ve produced, to experiment with how best to represent the data. For this map, I wrote a Data Bundle that converted the Shapefile of street lights provided by SanGIS into a Sqlite file, which was then loaded into an image. ( You can get the code for my bundle on GitHub. ) The processing code bins the location of the street lights to 1/4000 of a degree, about 18 meters at out latitude, then for each street lamp, adds a matrix onto the output image. The matrix spreads out the point, a bit like the light from the lamp would spread out. Here is a high resolution view of the output, showing Uptown, Downtown , the Park and Golden Hill. ( Click to expand ) This is, of course, a synthetic view of light in San Diego, but NASA has generated the real view, by photographing the earth at night. Here is our map laid over NASA’s image: Our data covers only San Diego, so there are many areas that are lit where we don’t have street lamp records, and it also appears that from space, light spreads much more than we estimate with our spreading matrix. Regardless, the maps show a good correspondence. This is a simplistic analysis, since it doesn’t consider the different types of lamps, their power, or the occlusion of buildings, but it does illustrate how to make and present a density map. This is the first step in our crime analysis project; the next steps will study how the locations of street lamps relate to the locations of crime. For the full project, we will have to consider that the correlation between crime and street lights isn’t only one way. There may be more crime in dark areas, but the city may also put more lights in high crime areas, breaking a simple correlation. So, we will probably focus on finding areas that are both dark and have high crime. If you like this sort of analysis, please join our Meetup group or connect though Facebook, Twitter or email to get the announcement for the start of our crime analysis project. We’d love to have your help!

The Corporation for National and Community Service is a federal agency that encourages US citizens to volunteer, particularly for federal programs like AmeriCorps. They also have an excellent data site that provides statistics on civic engagement. We used their data extraction tool to pull a few datasets, which are posted on the Library’s data repository. Here is a bar chart of the percentage of the population of large US cities which, in the previous 12 months, volunteered for a formal organization. The chart shows the averages for the years 2004 to 2011, including the standard deviation from year to year for a single city as an error bar. The numbers on the right indicate rank.