Simon Willison’s Weblog

76 items tagged “apis”

2021

API Tokens: A Tedious Survey. Thomas Ptacek reviews different approaches to implementing secure API tokens, from simple random strings stored in a database through various categories of signed token to exotic formats like Macaroons and Biscuits, both new to me. Macaroons carry a signed list of restrictions with them, but combine it with a mechanism where a client can add their own additional restrictions, sign the combination and pass the token on to someone else. Biscuits are similar, but “embed Datalog programs to evaluate whether a token allows an operation”. # 25th August 2021, 12:12 am

Notes on streaming large API responses

I started a Twitter conversation last week about API endpoints that stream large amounts of data as an alternative to APIs that return 100 results at a time and require clients to paginate through all of the pages in order to retrieve all of the data:

[... 1691 words]

Replaying logs to exercise the new API

22 days ago n1mmy pushed a change to help.vaccinate which logged full details of inoming Netlify function API traffic to an Airtable database.

[... 542 words]

APIs from CSS without JavaScript: the datasette-css-properties plugin

I built a new Datasette plugin called datasette-css-properties. It’s very, very weird—it adds a .css output extension to Datasette which outputs the result of a SQL query using CSS custom property format. This means you can display the results of database queries using pure CSS and HTML, no JavaScript required!

[... 891 words]

Custom Properties as State. Fascinating thought experiment by Chris Coyier: since CSS custom properties can be defined in an external stylesheet, we can APIs that return stylesheets defining dynamically server-side generated CSS values for things like time-of-day colour schemes or even strings that can be inserted using ::after { content: var(--my-property). This gave me a very eccentric idea for a Datasette plugin... # 7th January 2021, 7:39 pm

2020

GraphQL in Datasette with the new datasette-graphql plugin

This week I’ve mostly been building datasette-graphql, a plugin that adds GraphQL query support to Datasette.

[... 1249 words]

PostGraphile: Production Considerations. PostGraphile is a tool for building a GraphQL API on top of an existing PostgreSQL schema. Their “production considerations” documentation is particularly interesting because it directly addresses some of my biggest worries about GraphQL: the potential for someone to craft an expensive query that ties up server resources. PostGraphile suggests a number of techniques for avoiding this, including a statement timeout, a query allowlist, pagination caps and (in their “pro” version) a cost limit that uses a calculated cost score for the query. # 27th March 2020, 1:22 am

2019

Building a stateless API proxy (via) This is a really clever idea. The GitHub API is infuriatingly coarsely grained with its permissions: you often end up having to create a token with way more permissions than you actually need for your project. Thea Flowers proposes running your own proxy in front of their API that adds more finely grained permissions, based on custom encrypted proxy API tokens that use JWT to encode the original API key along with the permissions you want to grant to that particular token (as a list of regular expressions matching paths on the underlying API). # 30th May 2019, 4:28 am

2017

Datasette: instantly create and publish an API for your SQLite databases

I just shipped the first public version of datasette, a new tool for creating and publishing JSON APIs for SQLite databases.

[... 968 words]

2013

Which format for API documentation programmers prefer: PDF or Web?

HTML is a better format for documentation than PDF.

[... 160 words]

Does the Google Maps API let you remove details of the map such as street names to focus on pins on the map?

Yes—you can do this with map styles (which allow you to set the visibility if road labels, among other things): http://developers.google.com/map...

[... 53 words]

Which is the most complete and up to date API for restaurants/nightlife?

The foursquare API is pretty great for restaurants and nightlife these days. No chance if revenue share though—how would you envisage revenue share working?

[... 44 words]

Which free encyclopedias offer free APIs?

Wikipedia runs using Mediawiki, and Mediawiki has an API: http://www.mediawiki.org/wiki/API

[... 23 words]

What information do you feel is most valuable when integrating a Web API (REST or SOAP)?

  • A really good API explorer
  • Comprehensive documentation of the response format, including what happens if certain fields are missing (empty string, null value, missing key?)
  • Comprehensive documentation of the available request parameters, including allowed values
  • What are the rate limits?
  • What is returned if there is an error?

[... 66 words]

2012

Is it possible to embed Skype into a webpage to use as live chat support for free?

Olark offer a very neat JavaScript widget that does exactly this (it’s text-based messaging, not video or voice): http://www.olark.com/—you can try their demo at the bottom of their page.

[... 72 words]

Does Amazon have a API for websites to utilize order and delivery fulfillment?

The Amazon Fulfillment Web Service used to handle this http://aws.amazon.com/fws/—but their site now says "Effective June 2012, Amazon Services will no longer support Amazon Fulfillment Web Service (Amazon FWS). All functions and services currently supported by Amazon FWS are currently available through Amazon Marketplace Web Service (Amazon MWS)." So I guess you want the Amazon Marketplace Web Service: https://developer.amazonservices...

[... 82 words]

Are there any website thumbnail services that generate images in real-time?

http://url2png.com/ generates images on demand—you pass the URL directly to the service and it replies with a PNG image. The first load can take a few seconds (depending on how long it takes the originating site to serve up the assets etc) but they cache the generated images so future requests for the same URL will be served instantly.

[... 79 words]

Is there an API that returns metadata for a given URL?

I suggest taking a look at http://embed.ly/—it can take a huge range of URLs and turn them in to JSON metadata. Here’s what it can do with a Wikipedia page: http://embed.ly/docs/explore/obj...—and here’s Google Maps URL (not as useful, but still some interesting metadata extracted) http://embed.ly/docs/explore/obj...

[... 69 words]

2011

Are there any Meta APIs?

Embed.ly is a good example of this kind of API—it gives you one endpoint which wraps oembed APIs on dozens of other services (plus a bunch of custom scraping code). We use it as part of our video/slide embedding feature on http://lanyrd.com/

[... 57 words]

How we made an API for BoingBoing in an evening. Fluidinfo really is a fascinating piece of software. The team loaded in 11 years of BoingBoing content, allowing you to run structured queries against the data using their standard API, but also allowing users to attach their own information to the same corpus using Fluidinfo tags. Writable APIs are much less common than read-only APIs—Fluidinfo instantly provides both. # 28th January 2011, 10:56 pm

Google APIs & Developer Products. Presented as a sort-of-periodic table. There’s quite a bit of stuff on here I didn’t know about. # 28th January 2011, 11:25 am

Tip: Flickr standard photo response as slideshow. Neat trick—you can construct a URL to Flickr’s slideshow widget that includes the results of any API method, including the all-powerful flickr.photos.search. It’s a shame you can’t embed the resulting slideshow in an iframe. # 25th January 2011, 3:51 am

Introducing the FluidDB Explorer. Every good API deserves a dedicated API browser. # 13th January 2011, 4:19 am

2010

What are some scalable OAuth and OpenID server implementations?

Any OAuth library should scale horizontally—I can’t see how any one library would be a better choice than another.

[... 36 words]

Does Twitter use a 3rd party software for rate limiting their APIs? If yes, who’s the 3rd party?

I wrote up a technique for doing simple rate limiting using memcached a while ago, which I later found out was somewhat similar to how the Twitter API does it.

[... 56 words]

What is the best way to maintain a API wrapper class across multiple languages?

1. Use JSON for your API. That takes away a lot of the necessity for an API wrapper, since it means you’re automatically returning native data types (hashes, lists, strings etc) for most programming languages.

[... 175 words]

What company had the first API?

They weren’t the first to have an API, but Flickr were the first consumer web site that really pushed the concept in my opinion. They originally promoted it as “you can always get your data back if you want to”, but they then greatly benefited from the ecosystem that grew up around it.

[... 65 words]

Yahoo! Developer Network: Important API Updates and Changes. Some important (and potentially worrying) news about Yahoo! APIs. The BOSS (Build your Own Search Service) API will no longer be free—not an enormous surprise, and hopefully the pricing will be sensible. Most of the other search APIs (including web, news and image search) are being turned off with no replacement, while term extraction and spelling suggestions will be YQL-only. Most worrying, changes to Geo, Maps and Local APIs will be announced in September, with some set to close. I really hope this doesn’t affect the GeoPlanet APIs. # 17th August 2010, 6:14 pm

Writing Bulletproof Apps with API Errorpoints. This is a very good idea: Web APIs should offer special API endpoints for simulating each of the possible errors that might be returned by the production API. # 16th August 2010, 7:12 pm

Closure Compiler Service (via) A hosted version of the Google Closure Compiler (JavaScript minifier) running on App Engine. It has both a user interface and a REST API, which means you can use it as part of an automated build process without needing to set up a local copy of the software. # 9th August 2010, 1:17 pm