Simon Willison’s Weblog

Subscribe
Atom feed

Blogmarks

Filters: Sorted by date

Carbon (via) Beautiful little tool that you can paste source code into to generate an image of that code with syntax highlighting applied, ready to be tweeted or shared anywhere that lets you share an image. Built in Node and next.js, with image generation handled client-side by the dom-to-image JavaScript library which loads HTML into a SVG foreignObject (sadly not yet supported by Safari) and uses that to populate a canvas and produce a PNG.

# 19th October 2017, 6:31 pm / javascript, nodejs, svg, zeit-now

Streaming Dataframes. This is some deep and brilliant magic: Matthew Rocklin’s Streamz Python library provides some elegant abstractions for consuming infinite streams of data and calculating cumulative averages and rolling reductions... and now he’s added an integration with jupyter that lets you embed bokeh graphs and pandas dataframe tables that continue to update in realtime as the stream continues! Check out the animated screenshots, this really is a phenomenal piece of work.

# 19th October 2017, 2:25 pm / pandas, jupyter

A Brief Intro to Docker for Djangonauts (via) This is great—a really clear introduction to both Docker and Docker Compose, aimed at Django developers. Includes line-by-line annotations of an example Dockerfile and docker-compose.yml.

# 18th October 2017, 9:06 pm / django, docker

SRI Hash Generator. Handy utility for generating SRI hashes—just give it a URL and it will show you the script or link href block you need to use to safely embed that URL in your page with the correct SRI hash.

# 18th October 2017, 9:05 pm / sri

Subresource Integrity. Now supported in Firefox 55, Chrome 49+ and Safari 11+. This makes me much more comfortable about hot-linking to JavaScript and CSS hosted by the various CDN providers, since it means that should they get breached any evil new scripts hosted at the same URL will be denied by modern browsers.

# 18th October 2017, 9:03 pm / sri

Select Transform: JSON Template over JSON (via) A barrage of interesting ideas here. Having clients transmit up a JSON template which is then executed against data on the server and used to return exactly the data the client needs is just one of them (significant overlap with GraphQL there).

# 18th October 2017, 5:12 pm / json, graphql

An interactive explanation of quadtrees (via) Neat explorable explanation of quadtrees, using interactives built on top of D3.

# 16th October 2017, 2:47 pm / explorables, d3

Explorable Explanations. I’m fascinated by web articles and essays that embed interactive visualizations—taking advantage of the unique capabilities of the medium to help explain complex concepts. Explorable Explanations collects exactly these, under the banner of “learning through play”. They also gather tools and tutorials to help build more of them.

# 15th October 2017, 1:17 pm / interactives, explorables

What’s New In DevTools (Chrome 62). Some really neat stuff. Highlights include top-level “await” support in the console, the ability to take screenshots of specific HTML nodes, CSS grid highlighting and the ability to drop a .HAR file onto the network panel in order to view it as a waterfall.

# 13th October 2017, 10:59 pm / chrome, css, javascript

The Xi Text Engine CRDT (via) Xi is “a modern editor with a backend written in Rust”—an open-source text editor project from Google built on some very interesting computer science (Conflict-free Replicated Data Types). It’s a native editor with server-backed synchronization as a first-class concept.

# 13th October 2017, 10:32 pm / editor, google, rust, crdt

How to Do Code Reviews Like a Human. Some very well thought out advice on giving constructive reviews, including simple but effective language tricks for keeping suggestions positive and helpful.

# 13th October 2017, 4:36 am / code-review

Exploding Git Repositories. Kate Murphy describes how git is vulnerable to a similar attack to the XML “billion laughs” recursive entity expansion attack—you can create a tiny git repository that acts as a “git bomb”, expanding 12 root objects to over a billion files using recursive blob references.

# 12th October 2017, 7:43 pm / git, security

Dead End Thrills. Duncan Harris Is a photographer who works in the medium of video game screen captures.

# 12th October 2017, 2:23 am / games, photography

How the CIA Staged Sham Academic Conferences to Thwart Iran’s Nuclear Program. “The importance of a conference may be measured not only by the number of Nobel Prize winners or Oxford dons it attracts, but by the number of spies. U.S. and foreign intelligence officers flock to conferences for the same reason that Army recruiters concentrate on low-income neighborhoods: They make the best hunting grounds. While a university campus may have only one or two professors of interest to an intelligence service, the right conference — on drone technology, perhaps, or ISIS — may have dozens.”

# 12th October 2017, 12:20 am / cia, conferences

Cypress (via) Promising looking new open source testing framework for full-blown web integration testing—a modern alternative to Selenium. I spent five minutes playing with the demo and was really impressed by it—especially their “time travel” feature which lets you hover over a passed test and see the state of the browser when each of those assertions was executed.

# 11th October 2017, 4:14 pm / selenium, testing, cypress

My essential django package list. Insightful list of Django packages—many of which I hadn’t seen before—by Serafeim Papastefanos, each with a handy explanation of what it’s useful for and why.

# 11th October 2017, 2:42 pm / django

WPO Stats (via) “Case studies and experiments demonstrating the impact of web performance optimization (WPO) on user experience and business metrics.“

# 11th October 2017, 1:36 am / web-performance

The Absurdly Underestimated Dangers of CSV Injection. This is horrifying. A plain old CSV file intended for import into Excel can embed formulas (a value prefixed with an equals symbol) which can execute system commands—with a big honking security prompt that most people will likely ignore. Even worse: they can embed IMPORTXML() functions that can silently leak data from the rest of the sheet to an external URL—and those will work against Google Sheets as well as Excel.

# 10th October 2017, 4:13 am / csv, security

What happened: Hillary’s view. Lawrence Lessig’s review of Hillary Clinton’s book, including his thoughts on the failures of commercial media in covering the story of Russian interference during those crucial closing weeks before the election.

# 9th October 2017, 3:25 pm / lawrence-lessig

fd (via) “A simple, fast and user-friendly alternative to find.” Written in rust, with a less confusing default command-line syntax than the regular find command. Microbenchmark shows it running 7x faster. Install it on OS X using “brew install fd”.

# 8th October 2017, 9:27 pm / homebrew, unix, rust

Removing MediaWiki from SPA: Cool URIs don’t change (via) Detailed write-up from Anna Shipman describing how she archived an old MediaWiki as static content using recursive wget and some cunning application of mod_rewrite.

# 8th October 2017, 7:54 pm / urls, annashipman

Generate dynamic titles for /search/ pages. Fun little enhancement to my faceted search implementation: I now generate dynamic titles for each search results page describing the search, e.g. “Blogmarks tagged security in Feb, 2005” or ““python” in quotations tagged ruby, python in 2007”.

# 8th October 2017, 7:47 pm / django, facetedsearch

Dillinger. I really like this online Markdown editor. It has source syntax highlighting, live previews of the generated HTML and it constantly syncs to localStorage so you won’t lose your work if you accidentally shut your browser window. The code is also available open source on GitHub.

# 8th October 2017, 6:38 pm / localstorage, markdown

China Demonstrates Quantum Encryption By Hosting a Video Call. This reads like pure science fiction:

Pan’s team first established a connection and generated a secure key between a ground station in Xinglong and the Micius satellite as it passed overhead, orbiting about 500 kilometers above Earth. [...]

Next, the Chinese team waited for Micius to pass over Vienna, where their collaborators at the Austria Academy of Sciences were waiting to also receive the key from the satellite. Then, with the keys in hand, the groups initiated a video conference and used those keys to encrypt the video data through a standard VPN protocol.

# 8th October 2017, 2:49 am / encryption, quantum-computing

Facets. New open source visualization and data exploration tool from Google (“Disclaimer: This is not an official Google product”, whatever that means). It’s intended for visualizing machine learning datasets but it’s obviously useful outside of ML as well—any time you need to understand a large dataset this looks like it could be extremely useful. Ships with example jupyter notebooks and an easy mechanism for embedding the Facets interactive UI directly inside a notebook cell.

# 8th October 2017, 12:21 am / google, visualisation, facetedsearch, jupyter

Sanic. “Sanic is a Flask-like Python 3.5+ web server that’s written to go fast [...] On top of being Flask-like, Sanic supports async request handlers. This means you can use the new shiny async/await syntax from Python 3.5, making your code non-blocking and speedy”.

# 7th October 2017, 6:39 pm / async, python, sanic

uvloop: Blazing fast Python networking. “uvloop makes asyncio fast. In fact, it is at least 2x faster than nodejs, gevent, as well as any other Python asynchronous framework. The performance of uvloop-based asyncio is close to that of Go programs.”

# 7th October 2017, 5:53 pm / async, python

SQL Fiddle demonstrating the PostgreSQL to_tsvector() function (via) SQL Fiddle is amazing—it’s an interactive pastebin that lets you execute queries against MySQL, PostgreSQL, Oracle, SQLite & SQL Server, and then share both the input and the results by sending around the resulting URL. Here I’m using it to demonstrate that stripping tags before indexing documents in PostgreSQL is unnecessary because the ts_vector() function already does that for you.

# 6th October 2017, 10:11 pm / postgresql, sql

PyPy v5.9 Released, Now Supports Pandas, NumPy. NumPy and Pandas now work on PyPy2.7. “Many other modules based on C-API extensions work on PyPy as well.”

# 5th October 2017, 4:58 pm / pandas, pypy, numpy

Building a statistical profiler in python. Generating flame graphs of production Python code is surprisingly straight-forward. Brian Pitts built a statistical flame graph profiler into our tikibar debugging tool at Eventbrite and it’s proved extremely useful.

# 5th October 2017, 3:44 pm / django, profiling, python, tikibar

Years

Tags