Simon Willison's Weblog: maps

OpenFreeMap

2024-09-28T21:41:15+00:00

New free map tile hosting service from Zsolt Ero:

OpenFreeMap lets you display custom maps on your website and apps for free. […] Using our public instance is completely free: there are no limits on the number of map views or requests. There’s no registration, no user database, no API keys, and no cookies. We aim to cover the running costs of our public instance through donations.

The site serves static vector tiles that work with MapLibre GL. It deliberately doesn’t offer any other services such as search or routing.

From the project README looks like it’s hosted on two Hetzner machines. I don’t think the public server is behind a CDN.

Part of the trick to serving the tiles efficiently is the way it takes advantage of Btrfs:

Production-quality hosting of 300 million tiny files is hard. The average file size is just 450 byte. Dozens of tile servers have been written to tackle this problem, but they all have their limitations.

The original idea of this project is to avoid using tile servers altogether. Instead, the tiles are directly served from Btrfs partition images + hard links using an optimised nginx config.

The self-hosting guide describes the scripts that are provided for downloading their pre-built tiles (needing a fresh Ubuntu server with 300GB of SSD and 4GB of RAM) or building the tiles yourself using Planetiler (needs 500GB of disk and 64GB of RAM).

Getting started is delightfully straightforward:

const map = new maplibregl.Map({
  style: 'https://tiles.openfreemap.org/styles/liberty',
  center: [13.388, 52.517],
  zoom: 9.5,
  container: 'map',
})

I got Claude to help build this demo showing a thousand random markers dotted around San Francisco. The 3D tiles even include building shapes!

Zsolt built OpenFreeMap based on his experience running MapHub over the last 9 years. Here’s a 2018 interview about that project.

It’s pretty incredible that the OpenStreetMap and open geospatial stack has evolved to the point now where it’s economically feasible for an individual to offer a service like this. I hope this turns out to be sustainable. Hetzner charge just €1 per TB for bandwidth (S3 can cost $90/TB) which should help a lot.

Via Tim Bray

Tags: geospatial, maps, openstreetmap, ai-assisted-programming

The many lives of Null Island

2024-07-28T17:44:54+00:00

The many lives of Null Island

Stamen's custom basemaps have long harbored an Easter egg: zoom all the way in on 0, 0 to see the outline of the mystical "null island", the place where GIS glitches and data bugs accumulate, in the Gulf of Guinea south of Ghana.

Stamen's Alan McConchie provides a detailed history of the Easter egg - first introduced by Mike Migurski in 2010 - along with a definitive guide to the GIS jokes and traditions that surround it.

Here's Null Island on Stamen's Toner map. The shape (also available as GeoJSON) is an homage to the island from 1993's Myst, hence the outline of a large docked ship at the bottom.

Alan recently gave a talk about Stamen's updated custom maps at State of the Map US 2024 (video, slides) - their Toner and Terrain maps are now available as vector tiles served by Stadia Maps (here's the announcement), but their iconic watercolor style is yet to be updated to vectors, due to the weird array of raster tricks it used to achieve the effect.

In researching this post I searched for null island on Google Maps and was delighted to learn that a bunch of entrepreneurs in Western Africa have tapped into the meme for their own businesses:

Via Andy Baio

Tags: geospatial, maps, michal-migurski, stamen-design

How to make self-hosted maps that work everywhere and cost next to nothing

2024-02-24T04:19:05+00:00

How to make self-hosted maps that work everywhere and cost next to nothing

Chris Amico provides a detailed roundup of the state of web mapping in 2024. It’s never been easier to entirely host your own mapping infrastructure, thanks to OpenStreetMap, Overture, MBTiles, PMTiles, Maplibre and a whole ecosystem of other fine open source projects.

I like Protomaps creator Brandon Liu’s description of this: “post-scarcity web mapping”.

Tags: geospatial, maps, chris-amico, overture

How The Post is replacing Mapbox with open source solutions

2023-02-17T18:45:33+00:00

How The Post is replacing Mapbox with open source solutions

Kevin Schaul describes the Washington Post’s emerging open source GIS stack: OpenMapTiles, Maputnik, PMTiles and Maplibre-gl-js.

Via Hacker News

Tags: maps, opensearch, openstreetmap, washington-post

A tiny web app to create images from OpenStreetMap maps

2022-06-12T05:49:35+00:00

Earlier today I found myself wanting to programmatically generate some images of maps.

I wanted to create a map centered around a location, at a specific zoom level, and with a marker in a specific place.

Some cursory searches failed to turn up exactly what I wanted, so I decided to build a tiny project to solve the problem, taking advantage of my shot-scraper tool for automating screenshots of web pages.

The result is map.simonwillison.net - hosted on GitHub Pages from my simonw/url-map repository.

Here's how to generate a map image of Washington DC:

shot-scraper 'https://map.simonwillison.net/?q=washington+dc' \
  --retina --width 600 --height 400 --wait 3000

That command generates a PNG 1200x800 image that's a retina screenshot of the map displayed at https://map.simonwillison.net/?q=washington+dc - after waiting three seconds to esure all of the tiles have fully loaded.

The website itself is documented here. It displays a map with no visible controls, though you can use gestures to zoom in and pan around - and the URL bar will update to reflect your navigation, so you can bookmark or share the URL once you've got it to the right spot.

You can also use query string parameters to specify the map that should be initially displayed:

https://map.simonwillison.net/?center=51.49,0&zoom=8 displays a map at zoom level 8 centered on the specified latitude, longitude coordinate pair.
https://map.simonwillison.net/?q=islington+london geocodes the ?q= text using OpenStreetMap Nominatim and zooms to the level that best fits the bounding box of the first returned result.
https://map.simonwillison.net/?q=islington+london&zoom=12 does that but zooms to level 12 instead of using the best fit for the bounding box
https://map.simonwillison.net/?center=51.49,0&zoom=8&marker=51.49,0&marker=51.3,0.2 adds two blue markers to the specified map. You can pass &marker=lat,lon as many times as you like to add multiple markers.

Annotated source code

The entire mapping application is contained in a single 68 line index.html file that mixes HTML and JavaScript. It's built using the fantastic Leaflet open source mapping library.

Since the code is so short, I'll enclude the entire thing here with some additional annotating comments.

It started out as a copy of the first example in the Leaflet quick start guide.

<!DOCTYPE html>
<!-- Regular HTML boilerplate -->
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>map.simonwillison.net</title>
<!--
  Leaflet's CSS and JS are loaded from the unpgk.com CDN, with the
  Subresource Integrity (SRI) integrity="sha512..." attribute to ensure
  that the exact expected code is served by the CDN.
-->
<link rel="stylesheet" href="https://unpkg.com/leaflet@1.8.0/dist/leaflet.css" integrity="sha512-hoalWLoI8r4UszCkZ5kL8vayOGVae1oxXe/2A4AO6J9+580uKHDO3JdHb7NzwwzK5xr/Fs0W40kiNHxM9vyTtQ==" crossorigin=""/>
<script src="https://unpkg.com/leaflet@1.8.0/dist/leaflet.js" integrity="sha512-BB3hKbKWOc9Ez/TAwyWxNXeoV9c1v6FIeYiBieIWkpLjauysF18NzgR1MBNBXf8/KABdlkX68nAhlwcDFLGPCQ==" crossorigin=""></script>
<!-- I want the map to occupy the entire browser window with no margins -->
<style>
html, body {
  height: 100%;
  margin: 0;
}
</style>
</head>
<body>
<!-- The Leaflet map renders in this 100% high/wide div -->
<div id="map" style="width: 100%; height: 100%;"></div>
<script>
function toPoint(s) {
  // Convert "51.5,2.1" into [51.5, 2.1]
  return s.split(",").map(parseFloat);
}
// An async function so we can 'await fetch(...)' later on
async function load() {
  // URLSearchParams is a fantastic browser API - it makes it easy to both read
  // query string parameters from the URL and later to generate new ones
  let params = new URLSearchParams(location.search);
  // If the starting URL is /?center=51,32&zoom=3 this will pull those values out
  let center = params.get('center') || '0,0';
  let initialZoom = params.get('zoom');
  let zoom = parseInt(initialZoom || '2', 10);
  let q = params.get('q');
  // .getAll() turns &marker=51.49,0&marker=51.3,0.2 into ['51.49,0', '51.3,0.2']
  let markers = params.getAll('marker');
  // zoomControl: false turns off the visible +/- zoom buttons in Leaflet
  let map = L.map('map', { zoomControl: false }).setView(toPoint(center), zoom);
  L.tileLayer('https://{s}.tile.openstreetmap.org/{z}/{x}/{y}.png', {
    maxZoom: 19,
    attribution: '&copy; <a href="http://www.openstreetmap.org/copyright">OpenStreetMap</a>',
    // This option means retina-capable devices will get double-resolution tiles:
    detectRetina: true
  }).addTo(map);
  // We only pay attention to ?q= if ?center= was not provided:
  if (q && !params.get('center')) {
    // We use fetch to pass ?q= to the Nominatim API and get back JSON
    let response = await fetch(
      `https://nominatim.openstreetmap.org/search.php?q=${encodeURIComponent(q)}&format=jsonv2`
    )
    let data = await response.json();
    // data[0] is the first result - it has a boundingbox array of four floats
    // which we can convert into a Leaflet-compatible bounding box like this:
    let bounds = [
      [data[0].boundingbox[0],data[0].boundingbox[2]],
      [data[0].boundingbox[1],data[0].boundingbox[3]]
    ];
    // This sets both the map center and zooms to the correct level for the bbox:
    map.fitBounds(bounds);
    // User-provided zoom over-rides this
    if (initialZoom) {
      map.setZoom(parseInt(initialZoom));
    }
  }
  // This is the code that updates the URL as the user pans or zooms around.
  // You can subscribe to both the moveend and zoomend Leaflet events in one go:
  map.on('moveend zoomend', () => {
    // Update URL bar with current location
    let newZoom = map.getZoom();
    let center = map.getCenter();
    // This time we use URLSearchParams to construct a center...=&zoom=... URL
    let u = new URLSearchParams();
    // Copy across ?marker=x&marker=y from existing URL, if they were set:
    markers.forEach(s => u.append('marker', s));
    u.append('center', `${center.lat},${center.lng}`);
    u.append('zoom', newZoom);
    // replaceState() is a weird API - the third argument is the one we care about:
    history.replaceState(null, null, '?' + u.toString());
  });
  // This bit adds Leaflet markers to the map for ?marker= query string arguments:
  markers.forEach(s => {
    L.marker(toPoint(s)).addTo(map);
  });
}
load();
</script>
</body>
</html>
<!-- See https://github.com/simonw/url-map for documentation -->

Tags: geospatial, maps, openstreetmap, projects, shot-scraper, leaflet

Apple’s New Map

2018-11-03T21:28:14+00:00

Apple’s New Map

Map nerds rejoice! Justin O’Beirne had written another spectacularly illustrated essay about web cartography, this time examining the iOS 12 upgrade to Apple Maps in most of California and a little bit of Nevada.

Via Daring Fireball

Tags: maps

Datasette plugins, and building a clustered map visualization

2018-04-20T15:41:11+00:00

Datasette now supports plugins!

Last Saturday I asked Twitter for examples of Python projects with successful plugin ecosystems. pytest was the clear winner: the pytest plugin compatibility table (an ingenious innovation that I would love to eventually copy for Datasette) lists 457 plugins, and even the core pytest system itself is built as a collection of default plugins that can be replaced or over-ridden.

Best of all: pytest’s plugin mechanism is available as a separate package: pluggy. And pluggy was exactly what I needed for Datasette.

You can follow the ongoing development of the feature in issue #14. This morning I released Datasette 0.20 with support for a number of different plugin hooks: plugins can add custom template tags and SQL functions, and can also bundle their own static assets, JavaScript, CSS and templates. The hooks are described in some detail in the Datasette Plugins documentation.

datasette-cluster-map

I also released my first plugin: datasette-cluster-map. Once installed, it looks out for database tables that have a latitude and longitude column. When it finds them, it draws all of the points on an interactive map using Leaflet and Leaflet.markercluster.

Let’s try it out on some polar bears!

The USGS Alaska Science Center have released a delightful set of data entitled Sensor and Location data from Ear Tag PTTs Deployed on Polar Bears in the Southern Beaufort Sea 2009 to 2011. It’s a collection of CSV files, which means it’s trivial to convert it to SQLite using my csvs-to-sqlite tool.

Having created the SQLite database, we can deploy it to a hosting account on Zeit Now alongside the new plugin like this:

# Make sure we have the latest datasette
pip3 install datasette --upgrade
# Deploy polar-bears.db to now with an increased default page_size
datasette publish now \
    --install=datasette-cluster-map \
    --extra-options "--page_size=500" \
    polar-bears.db

The --install option is new in Datasette 0.20 (it works for datasette publish heroku as well) - it tells the publishing provider to pip install the specified package. You can use it more than once to install multiple plugins, and it accepts a path to a zip file in addition to the name of a PyPI package.

Explore the full demo at https://datasette-cluster-map-demo.now.sh/polar-bears

Visualize any query on a map

Since the plugin inserts itself at the top of any Datasette table view with latitude and longitude columns, there are all sorts of neat tricks you can do with it.

I also loaded the San Francisco tree list (thanks, Department of Public Works) into the demo. Impressively, you can click “load all” on this page and Leaflet.markercluster will load in all 189,144 points and display them on the same map… and it works fine on my laptop and my phone. Computers in 2018 are pretty good!

But since it’s a Datasette table, we can filter it. Here’s a map of every New Zealand Xmas Tree in San Francisco (8,683 points). Here’s every tree where the Caretaker is Friends of the Urban Forest. Here’s every palm tree planted in 1990:

Update: This is an incorrect example: there are 21 matches on "palm avenue" because the FTS search index covers the address field - they're not actually palm trees. Here's a corrected query for palm trees planted in 1990.

The plugin currently only works against columns called latitude and longitude… but if your columns are called something else, don’t worry: you can craft a custom SQL query that aliases your columns and everything will work as intended. Here’s an example against some more polar bear data:

select *, "Capture Latitude" as latitude, "Capture Longitude" as longitude
from [USGS_WC_eartag_deployments_2009-2011]

Writing your own plugins

I’m really excited to see what people invent. If you want to have a go, your first stop should be the Plugins documentation. If you want an example of a simple plugin (including the all-important mechanism for packaging it up using setup.py) take a look at datasette-cluster-map on GitHub.

And if you have any thoughts, ideas or suggestions on how the plugin mechanism can be further employed please join the conversation on issue #14. I’ve literally just got started with Datasette’s plugin hooks, and I’m very keen to hear about things people want to build that aren’t yet supported.

Tags: maps, plugins, projects, visualization, datasette, pytest, leaflet

Is there an open source (or freely accessible) database of geofence coordinates for common places, such as cities or national parks?

2013-12-03T12:59:00+00:00

My answer to Is there an open source (or freely accessible) database of geofence coordinates for common places, such as cities or national parks? on Quora

Take a look at Flickr's openly licensed shapefiles:

http://code.flickr.net/2011/01/0...

They are derived from the collective opinion of Flickr's community as to what name should apply to different areas around the world.

Tags: databases, geolocation, google-maps, location, maps, open-source, quora

Does the Google Maps API let you remove details of the map such as street names to focus on pins on the map?

2013-11-12T14:33:00+00:00

My answer to Does the Google Maps API let you remove details of the map such as street names to focus on pins on the map? on Quora

Yes - you can do this with map styles (which allow you to set the visibility if road labels, among other things): http://developers.google.com/map...

Tags: apis, google-maps, maps, quora

Cartographer.js

2009-11-01T13:20:05+00:00

Cartographer.js

“Thematic mapping for Google Maps”—which means an easy way of adding heat maps (aka chloropleths), pie charts and point clusters as a layer over a Google map.

Tags: catography, chloropleths, google, google-maps, graphs, heatmaps, infographics, mapping, maps, piecharts, visualisation

Best of OpenStreetMap

2009-08-13T12:30:05+00:00

Best of OpenStreetMap

I keep on telling people OpenStreetMap is this year’s Wikipedia—at its best, it beats commercially available maps. This “best of” site highlights the areas where OSM really shines (the yellow stars)—the German mapping community in particular have produced some outstanding cartography.

Via OpenGeoData

Tags: cartography, mapping, maps, openstreetmap, wikipedia

Making Image Overlays Easy with GGroundOverlay and GGeoXML

2009-07-30T22:58:04+00:00

Making Image Overlays Easy with GGroundOverlay and GGeoXML

Surprisingly, there doesn’t appear to be a good online tool for helping align an overlay image with a Google Map and exporting the result as a KML file. This is the best I could find—Yahoo! used to have a tool called MapMixer but it doesn’t seem to exist any more.

Via Tony Hirst

Tags: google-maps, maps, overlays

Flickr Shapefiles Public Dataset 1.0

2009-05-22T18:12:10+00:00

Flickr Shapefiles Public Dataset 1.0

Another awesome Geo dataset from the Yahoo! stable—this time it’s Flickr releasing shapefiles (geometrical shapes) for hundreds of thousands of places around the world, under the CC0 license which makes them essentially public domain. The shapes themselves have been crowdsourced from geocoded photos uploaded to Flickr, where users can “correct” the textual location assigned to each photo. Combine this with the GeoPlanet WOE data and you get a huge, free dataset describing the human geography of the world.

Tags: creativecommons, crowdsourcing, flickr, geoplanet, geospatial, maps, shapefiles, yahoo

UK Television Series Map

2009-04-05T19:25:25+00:00

UK Television Series Map

Inspired by the US sitcom map, Meg Pickard is plotting TV series on a map of the UK.

Tags: maps, meg-pickard, tv

1901EasternTelegraph.jpg

2009-02-09T20:44:00+00:00

1901EasternTelegraph.jpg

A map of undersea telegraph cables as of 1901.

Via Clint Ecker

Tags: cables, maps, undersea

Travel time to major cities: A global map of Accessibility

2009-01-05T13:24:51+00:00

Travel time to major cities: A global map of Accessibility

Visualisation developed by the European Commission and the World Bank.

Via Michal Migurski

Tags: accessibility, europeancommission, maps, visualisation, worldbank

Motorway map of England, Scotland and Wales

2008-12-22T11:36:58+00:00

Motorway map of England, Scotland and Wales

In the style of Harry Beck’s London Tube map.

Via Cal

Tags: maps, motorways, tubemap

Noncontiguous area cartograms

2008-12-08T18:03:08+00:00

Noncontiguous area cartograms

a.k.a. really funky data visualisation maps. Includes lots of examples, plus ActionScript 3 source code.

Tags: as3, cartograms, flash, maps, visualisation

Live Piracy Map

2008-12-01T00:29:25+00:00

Live Piracy Map

That’s a heck of a lot of (real, nasty, sea-faring) pirates.

Tags: maps, pirates

FOWA London - Beyond GoogleMaps

2008-10-17T14:01:49+00:00

FOWA London - Beyond GoogleMaps

Andrew Turner’s talk at FOWA was the most information dense presentation I’ve ever seen, and discussed a huge number of cool geo projects that I’d never previously heard of. Andrew links to the full slides and video, well worth a watch.

Tags: andrew-turner, fowa2008, geospatial, google-maps, maps

Places to see in London (for geeks)

2008-10-08T14:54:02+00:00

Places to see in London (for geeks)

My geek-oriented guide to London attractions that you may not hear about otherwise, updated for this year’s overseas FOWA attendees. Suggestions for additions welcome.

Tags: fowa2008, geeks, googlemymaps, london, maps

Ordnance Survey OpenSpace Demo

2008-06-09T10:59:40+00:00

Ordnance Survey OpenSpace Demo

Niall Kennedy threw a demo up on his site—the map seems to load a lot faster than Google Maps and the level of detail once you zoom down to street level is really impressive.

Via Niall Kennedy

Tags: maps, niall-kennedy, openspace, ordinancesurvey

OS OpenSpace from Ordnance Survey

2008-06-09T08:30:13+00:00

OS OpenSpace from Ordnance Survey

Ordinance Survey now provide a free JavaScript mapping API for “non-commercial purposes” by “private individuals”. The maps look incredibly detailed, although I can’t find any live API demos on the site (the documentation is illustrated with screenshots).

Via Phil Wilson

Tags: geospatial, javascript, maps, openspace, ordinancesurvey

OSM Super-Strength Export

2008-04-22T09:56:07+00:00

OSM Super-Strength Export

Awesome new feature on OpenStreetMap: you can browse to anywhere on the map, then hit “export” and download a rendered bitmap or vector (PDF and SVG) image of the currently displayed map—and because it’s OSM there’s no watermark and a very liberal usage license.

Tags: mapping, maps, openstreetmap, pdf, svg, vector

KML: A new standard for sharing maps

2008-04-14T18:36:39+00:00

KML: A new standard for sharing maps

Google’s KML format, which is already supported by both Microsoft and Yahoo!’s map software, has been accepted under the wing of the Open Geospatial Consortium and is now an international standard.

Tags: google, google-maps, kml, mapping, maps, ogc

A List Apart: Issue 256

2008-04-09T12:21:30+00:00

A List Apart: Issue 256

The EveryBlock issue. Paul Smith on EveryBlock’s tasty custom maps, and Wilson Miner on EveryBlock’s tasty accessible data charts.

Tags: a-list-apart, everyblock, maps, paul-smith, wilson-miner

London Connections

2008-04-02T20:53:12+00:00

London Connections

Marvellously obsessive blog about the vagaries of London transport, including some really nice custom created maps. I love detailed maps of tube stations; anyone know a good place to find them?

Tags: london, londonconnections, mapping, maps, transport

Google Maps Clusterer

2008-01-28T15:07:31+00:00

Google Maps Clusterer

I’ve looked at a few clustering libraries for Google Maps, but this one seems to have the nicest API.

Tags: clustering, google-maps, maps

Information Freeway

2007-10-14T23:58:09+00:00

Information Freeway

Really lovely interface to Open Street Map, sadly suffering from a horribly vague name and almost no publicity at all.

Via Andy Allan

Tags: mapping, maps, openstreetmap

Get Lat Lon

2007-10-12T14:14:19+00:00

Get Lat Lon

I finally got fed up of hunting around for simple latitude/longitude tools when messing around with mapping APIs, so I built my own with a memorable URL. I plan to add new features as and when I need them.

Tags: getlatlon, google-maps, latitude, latlon, longitude, maps, negeography, projects