Posts in Category: Open Data

Auto-Discovering Open Geo Data Sets

For the Analyze Boston Open Data Challenge last Saturday at District Hall, I built a little script that auto-discovered datasets from Boston’s Open Data Hub that are likely to have geographic coordinates or semi-spatial information, such as addresses, neighborhoods, zip codes, etc.

I exploited the CKAN developer API for retrieving metadata about the Hub’s resources, and ran some simple heuristics on the data set’s field names. The technique is not specific to the Boston data portal, so it should work just as well on any CKAN-powered site!

Here’s the web page.

Here’s the code.

Esri, Python and pandas.DataFrame

Paul Rickard has a nice post where he talks about getting government open data on crimes from an Esri REST web service into a Pandas DataFrame and out to an Excel file. Lots of useful info in there. Thanks, Paul!

I recently had the opportunity to learn a bit about geospatial big data in outer space mining applications, and write about it here. This is looking to be an interesting area of growth in open data and open analytics in the coming years, so I’m excited to see how it develops!


Get your SQL dumps of US Census data from Census Reporter

I just discovered CensusReporter, “a Knight News Challenge-funded project to make it easy for journalists to write stories using US Census data. Expanding upon the volunteer-built, Census Reporter will simplify finding and using data from the decennial census and the American Community Survey.”

The site provides database dumps of ACS census data in PostgreSQL format — my favorite SQL database (which is available as-a-service from my employer, IBM CDS). I’m also looking forward to trying out their LeafletJS layer.

CensusReporter isn’t frequently updated, but I hope that doesn’t suggest anything about the health of this great resource.

Did you know mining in outer space was a “thing”? via @IBMAnalytics #spacemining #geospatial @avidgeo