Posts Tagged: opendata

Auto-Discovering Open Geo Data Sets

For the Analyze Boston Open Data Challenge last Saturday at District Hall, I built a little script that auto-discovered datasets from Boston’s Open Data Hub that are likely to have geographic coordinates or semi-spatial information, such as addresses, neighborhoods, zip codes, etc.

I exploited the CKAN developer API for retrieving metadata about the Hub’s resources, and ran some simple heuristics on the data set’s field names. The technique is not specific to the Boston data portal, so it should work just as well on any CKAN-powered site!

Here’s the web page.

Here’s the code.

Esri, Python and pandas.DataFrame

Paul Rickard has a nice post where he talks about getting government open data on crimes from an Esri REST web service into a Pandas DataFrame and out to an Excel file. Lots of useful info in there. Thanks, Paul!

Get your SQL dumps of US Census data from Census Reporter

I just discovered CensusReporter, “a Knight News Challenge-funded project to make it easy for journalists to write stories using US Census data. Expanding upon the volunteer-built Census.ire.org, Census Reporter will simplify finding and using data from the decennial census and the American Community Survey.”

The site provides database dumps of ACS census data in PostgreSQL format — my favorite SQL database (which is available as-a-service from my employer, IBM CDS). I’m also looking forward to trying out their LeafletJS layer.

CensusReporter isn’t frequently updated, but I hope that doesn’t suggest anything about the health of this great resource.

Nationwide electric utility service territories in delicious GeoJSON

Faraday has shared a nice data set in GeoJSON combining USDOE data with utility service boundaries.

Information Sharing is a Cultural Commitment

I sat on a panel Friday at the IJIS WIS3 Information Sharing and Safeguarding event where we were asked to think about the big problems with making cross-domain information sharing work. What I take away from the afternoon is this.

We have an abundance of technology tools and options available to us. Developing more technology isn’t the answer. The toughest problem right now is developing a cultural practice of information sharing. Creating an information sharing profession, and training a generation of information sharing professionals to have a baseline of skills, practices and values.

It’s already happening in the open data movement. When will it happen in all the vast recesses of federal government?