The Data Library works primarily with journalists and nonprofits, but until recently, I hadn’t fully realized how different the processes are in these two environments. We’d been following two different processes, but didn’t have names for them, so it is worthwhile to give the two contexts names, so we can be sure we are working in a process … Read more
We’ve frequently mentioned that people who work on data projects tell us that frequently, 80% of their projects are consumed by data preparation and cleaning, so it is interesting to get this data point from Kaggle: (2) How long is a typical project? When working with a top 0.5% data scientist, projects take just eight … Read more
Of the many versions of the famous quote that “writing about music is like dancing about architecture” is that “writing about music is as illogical as singing about economics.” Both of these activities are actually quite common. Dancing about architecture is a sport of irony, and not only are there songs about economics, the appropriately named … Read more
We’ve released a new report, Municipal Open Data Policies. The best way to ensure that San Diego area nonprofits and citizens get useful data is for local governments to adopt Open Data policies. These policies, which have been implemented in many cities around the country, mandate that cities publish data. So, instead of fighting for months … Read more
Our resident data slinger sent me an initial analysis of time series data for specific crime hot spots in San Diego, and the results are interesting. Rather than look at the whole of the region at once, we’ve identified specific high intensity areas, and are doing time-series analysis for them independently. The major areas are: North Park, … Read more
I shouldn’t be surprised that for any interesting technical idea, XKCD covered it first, and better. Here is another view of the issue we covered regarding crime mapping: crime is most common where (a) there are a lot of people and (b) there is alcohol, and (b) is often the reason for (a). To go … Read more
Cities thrive when citizens get directly involved in working for their city. Not only do engaged residents solve problems and guide development, but the fact they got involved makes them love their city more, and encourages the people they socialize with to also care more about their city. Because loving something and watching it grow just feels good, creating … Read more