Simon Willison’s Weblog

8 items tagged “nytimes”


nyt-2020-election-scraper. Brilliant application of git scraping by Alex Gaynor and a growing team of contributors. Takes a JSON snapshot of the NYT’s latest election poll figures every five minutes, then runs a Python script to iterate through the history and build an HTML page showing the trends, including what percentage of the remaining votes each candidate needs to win each state. This is the perfect case study in why it can be useful to take a “snapshot if the world right now” data source and turn it into a git revision history over time. # 6th November 2020, 2:24 pm


Breakfast Instapaper. Handy tool for selecting and bulk-submitting stories from today’s Guardian and NYTimes to your Instapaper account, by Daniel Vydra. # 29th April 2010, 11:49 am

The making of the NYT’s Netflix graphic. A database dump from Netflix, some clever hackery in ArcView GIS, hpricot to scrape Metacritic and a lot of careful thought about the UI for navigating the data. # 25th January 2010, 1:11 pm


How Different Groups Spend Their Day. Classy interactive infographic from the New York Times. # 10th August 2009, 3:37 pm

Announcing the Article Search API. The most interesting API from the NYTimes yet—search against 2.8 million articles from 1981 until today using 35 searchable fields and get back detailed metadata as well as the first paragraph of the articles themselves. # 5th February 2009, 11:06 pm


Represent. Andrei Scheinkman and Derek Willis describe how they built the NYTimes Represent feature using GeoDjango and PostGIS. # 29th December 2008, 10:10 pm

Represent and GeoDjango. The NYTimes new Represent application is built on GeoDjango. # 20th December 2008, 9:07 pm

Represent— Superb new application from the NYTimes—a sort of cross between TheyWorkForYou and a news archive search. Enter your address in New York and it tells you your local representatives and shows both their votes and their mentions in the newspaper. # 19th December 2008, 4:22 pm