Simon Willison’s Weblog

Subscribe
Atom feed for gis

54 items tagged “gis”

2020

California Protected Areas Database in Datasette (via) I built this yesterday: it’s a Datasette interface on top of the CPAD 2020 GIS database of protected areas in California maintained by GreenInfo Network. This was a useful excuse to build a GitHub Actions flow that builds a SpatiaLite database using my shapefile-to-sqlite tool, and I fixed a few bugs in my datasette-leaflet-geojson plugin as well.

# 21st August 2020, 11:15 pm / shapefiles, github-actions, datasette, projects, california, spatialite, gis

Things I learned about shapefiles building shapefile-to-sqlite

Visit Things I learned about shapefiles building shapefile-to-sqlite

The latest in my series of x-to-sqlite tools is shapefile-to-sqlite. I learned a whole bunch of things about the ESRI shapefile format while building it.

[... 1,073 words]

geojson-to-sqlite (via) I just put out the first release of geojson-to-sqlite—a CLI tool that can convert GeoJSON files (consisting of a Feature or a set of features in a FeatureCollection) into a table in a SQLite database. If you use the --spatialite option it will initalize the table with SpatiaLite and store the geometries in a spacially indexed geometry field—without that option it stores them as GeoJSON.

# 31st January 2020, 6:40 am / geojson, sqlite, spatialite, projects, gis

2019

kepler.gl. Uber built this open source geospatial analysis tool for large-scale data sets, and they offer it as a free hosted online tool—just click Get Started on the site. I uploaded two CSV files with 30,000+ latitude/longitude points in them just now and used Kepler to render them as images.

# 25th October 2019, 4:16 am / geospatial, gis, visualization

Thematic map—GIS Wiki. This is a really useful wiki full of GIS information, and the coverage of different types of thematic maps is particularly thorough.

# 21st October 2019, 2:25 am / wiki, gis, visualization

Los Angeles Weedmaps analysis (via) Ben Welsh at the LA Times published this Jupyter notebook showing the full working behind a story they published about LA’s black market weed dispensaries. I picked up several useful tricks from it—including how to load points into a geopandas GeoDataFrame (in epsg:4326 aka WGS 84) and how to then join that against the LA Times neighborhoods GeoJSON boundaries file.

# 30th May 2019, 4:35 am / jupyter, data-journalism, latimes, pandas, gis, ben-welsh

2018

VirtualKNN for SpatiaLite. This looks amazing: a special virtual table shipped as part of SpatiaLite 4.4.0 which implements a fast, R-Tree backed mechanism for finding the X nearest points against a geospatial database table. There’s just one catch: it’s only available in 4.4.0, but the most recent “stable” release of SpatiaLite is 4.3.0a from September 2015 so the version you get if you install from apt-get or homebrew doesn’t yet have this functionality. I’d love to figure out a neat way to package and distribute this along with Datasette. I’d also like to figure out a clean way to ship a more recent version of SQLite than the one that is currently packaged with Python 3 (3.16.2, where the latest SQLite release is 3.23.1).

# 21st May 2018, 9:23 pm / spatialite, gis, sqlite

Nicaraguan Address System (via) “Instead of street names or numbers Nicaraguans use reference points from where they start describing a certain address. [...] There are instances, however, in which the reference points do not exist anymore!”

# 21st January 2018, 4:32 pm / gis

Generating polygon representing a rough 100km circle around latitude/longitude point using Python. A question I posted to the GIS Stack Exchange—I found my own answer using a Python library called geog, then someone else posted a better solution using pyproj.

# 17th January 2018, 8:57 pm / gis, python

2017

simonepri/geo-maps. Neat project which publishes GeoJSON maps of the world automatically derived from OpenStreetMap. Three variants are available: country political maritime boundaries, country political coastline boundaries and a general outline of the world’s land territories.

# 21st November 2017, 4:06 pm / geojson, gis, openstreetmap

Interactive Database of the World’s River Basins (via) “This database provides the first-ever compilation of the world’s river basins developed specifically for corporate disclosure. It features a comprehensive list of river basins worldwide, including their names, boundaries, and other helpful information.”

# 10th November 2017, 3:07 pm / gis

Fast GeoSpatial Analysis in Python. Some clever advanced performance tricks with Cython and Dask, but it also introduced me to GeoPandas.

# 29th October 2017, 4:47 pm / gis, python

2010

Geospatial Indexing in MongoDB (via) New in version 1.3.3. Handles “order by distance from” queries using a geohash approach under the hood, automatically searching nearby grid squares until the correct number of results have been gathered. Bounding box search is planned for a future release.

# 2nd March 2010, 8:12 pm / geohash, geospatial, gis, mongodb

On walking into a disaster zone. Schuyler Erle: “The World Bank was looking for technical GIS professionals, ideally French-speaking, to go and advise the government [...] I can sort of speak French. Sure, why not?”

# 10th February 2010, 3:45 pm / schuylererle, haiti, openstreetmap, worldbank, gis

The making of the NYT’s Netflix graphic. A database dump from Netflix, some clever hackery in ArcView GIS, hpricot to scrape Metacritic and a lot of careful thought about the UI for navigating the data.

# 25th January 2010, 1:11 pm / ui, design, usability, netflix, new-york-times, infographics, visualisation, arcview, gis, hpricot, metacritic

2009

GeoDjango and the UK postcode database. Excellent introduction to GeoDjango using the recently leaked UK postcode database. Obviously, you should only follow the steps in this tutorial using the officially licensed database, available for a mere £1,700.

# 30th September 2009, 2:25 pm / postcodes, uk, geodjango, django, gis, chris-lamb

openstreetmap genuine advantage. The OpenStreetMap data model (points, ways and relations, all allowing arbitrary key/value tags) is a real thing of beauty—simple to understand but almost infinitely extensible. Mike Migurski’s latest project adds PGP signing to OpenStreetMap, allowing organisations (such as local government) to add a signature to a way (a sequence of points) and a subset of its tags, then write that signature in to a new tag on the object.

# 29th September 2009, 9:49 am / michal-migurski, openstreetmap, pgp, cryptography, mapping, gis

OpenStreetMap: QuadTiles. Fascinating explanation of a proposal for replacing lat, lon pairs in the OpenStreetMap database with a QuadTile-based addressing system.

# 10th September 2009, 3:54 pm / quadtiles, openstreetmap, algorithms, gis

EveryBlock source code released. EveryBlock’s Knight Foundation grant required them to release the source code after two years, under the GPL. Lots of neat Django / PostgreSQL / GIS tricks to be found within.

# 1st July 2009, 8:01 pm / gis, postgresql, django, everyblock, python, open-source, gpl

Yahoo! Placemaker. Really exciting new API from Yahoo!—Placemaker accepts a block of text (or a URL to HTML or RSS) and extracts and returns geographical locations mentioned in the text. I just ran my djng blog entry through it and it pulled out “Prague” as the only location mentioned. This should be really useful for adding geodata to existing textual content.

# 20th May 2009, 9:34 pm / yahoo, geo, gis, placemaker, geocoding

Yahoo! Geo: Announcing GeoPlanet Data. The Yahoo! WhereOnEarth geographic data set is fantastic, but I’ve always felt slightly uncomfortable about building applications against it in case the API went away. That’s not an issue any more—the entire dataset is now available to download and use under a Creative Commons Attribution license. It’s not entirely clear what the attribution requirements are—do you have to put “data from GeoPlanet” on every page or can you get away with just tucking the attribution away in an “about this site” page? UPDATE: The data doesn’t include latitude/longitude or bounding boxes, which severely reduces its utility.

# 20th May 2009, 9:12 pm / attribution, creativecommons, data, geoplanet, geospatial, gis, whereonearth, yahoo

Google Maps Data API (via) I’m disappointed by this one—it’s really just a CRUD store for the KML files used in Google MyMaps. It would be a lot more useful if it let you perform geospatial calculations against your stored map data using some kind of query API—a cloud service alternative to tools like PostGIS.

# 20th May 2009, 9:07 pm / postgis, google-maps, google-maps-api, googlemapsdataapi, gdata, apis, gis, kml

2008

Represent. Andrei Scheinkman and Derek Willis describe how they built the NYTimes Represent feature using GeoDjango and PostGIS.

# 29th December 2008, 10:10 pm / derek-willis, andrei-scheinkman, new-york-times, django, geodjango, python, postgresql, postgis, gis

2005

A GIS newsletter describes the server-side tech behind Google Maps (via) Apparently it’s using Telcontar’s Drill Down Server, with extensive customisations.

# 23rd February 2005, 4:11 pm / gis, google-maps