Blogmarks
Filters: Sorted by date
csvs-to-sqlite: Refactoring columns into separate lookup tables. I just shipped a new version of csvs-to-sqlite with the ability to extract specified columns into a separate SQLite lookup table by passing additional command-line arguments.
The Pac-Man Rule at Conferences. This is such a good idea from Eric Holscher: at the conferences he organizes he tells his attendees “When standing as a group of people, always leave room for 1 person to join your group”—to encourage networking and inclusive converations.
now-ab. Intriguing example of a Zeit Now microservice: now-ab is a Node.js HTTP proxy which proxies through to one of two or more other Now-deployed applications based on a cookie. If you don’t have the cookie, it picks a backend at random and sets the cookie. Admittedly this is the easiest part of implementing A/B testing (the hard part is the analytics: tracking exposures and conversions) but as an example of a microservice architectural pattern this is fascinating.
DSF calls for applicants for a Django Fellow. This is a fantastic opportunity: the Django Software Foundation’s fellowship program is providing the opportunity for a new developer to get paid to work on Django 20-40 hours a week. Tim Graham has held this position full-time for the past three years and is looking to scale back to part-time, hence this new opportunity. Applications are due by December 18, 2017.
django-multitenant (via) Absolutely fascinating Django library for horizontally sharding a database using a multi-tenant pattern, from the team at Citus. In this pattern every relevant table includes a “tenant_id”, and all queries should specifically select against that ID. Once you have that in place, you can shard your rows across multiple different databases and route to the correct database based on the tenant ID, safe in the knowledge that joins will still work provided they are against other rows belonging to the same tenant.
Introducing security alerts on GitHub. This is huge: GitHub’s dependency graph feature now shows any dependencies that have a known security vulnerability, based on CVE IDs—and you can sign up for notifications of new vulnerabilities as well. Only supports Ruby and JavaScript today, but Python support is coming in 2018.
gitchangelog. Handy Python utility that can generate a reStructured Text changelog from your git commit log. I used this to help get the Datasette release notes started.
Datasette 0.12. I just released v0.12 of Datasette. The most exciting new feature is the ability to display a UI for editing named parameters—so you can construct an arbitrarily complex SQL query, include some named parameters and then link directly to it in Datasette to provide a simple interface for changing those parameters. An example involving Australian dogs is included in the release notes.
Verified cryptography for Firefox 57 (via) Mozilla just became the first browser vendor to ship a formally verified crypto implementation.
Plotly Falcon SQL Client and the Remarkable Datasette SQLite2API Generator. Lovely write-up of Datasette by Tony Hirst. I’m particularly excited about its applications in the world of data journalism so it’s great to see it starting to get attention from people in that sphere.
Release Notes for Safari Technology Preview 44. The big news is support for the W3C Payment Request API for devices with Apple Pay enabled. Chrome, Firefox and Edge have been working on this as well.
tuxracer-web.
Brilliant Docker hack from David Cooper: run docker run -p 8008:80 dtcooper/tuxracer-web to get Tux Racer (the 3D game) running in your browser, on top a cunning mix of the noVNC HTML5 VNC client and icecast for sound.
nginx proxy-cache-lock (via) Crucially important feature hidden away in the nginx documentation: proxy_cache_lock enables request coalescing, or dog-pile protection: it means that if a hundred simultaneous requests all suffer the same cache miss, only one request is made to the backend and the answer is then sent back to all hundred requests at once. I’ve leaned heavily on this feature in Varnish for years—useful to know that nginx has the same capability.
pillow-simd (via) A “friendly fork” of the Python Pillow image library that takes advantage of SIMD operations on certain CPUs to obtain massive speed-ups—they claim 16 to 40 times faster than ImageMagick.
Using SVG as placeholders — More Image Loading Techniques. This is such a good idea: generate a tiny SVG placeholder for an image, and display that until the image itself has loaded. This article explores potential ways of generating those SVGs in some depth.
Datasettes · simonw/datasette. I’m collecting examples of datasette-powered APIs on the project wiki.
Datasette for Polar Bears. I found a fun dataset of Polar Bear ear tag tracking data put out by the USGS Alaska Science Center and deployed it using datasette in just a couple of minutes—here’s how I did it.
How to train your own Object Detector with TensorFlow’s Object Detector API (via) Dat Tran built a TensorFlow model that can detect raccoons! Impressive results, especially given it was only trained on 200 raccoon images from Google Image search.
Entering the Quantum Era—How Firefox got fast again and where it’s going to get faster. I’ve been trying out the beta of Firefox 57 and it’s fantastic. All of that work on Servo and Rust is definitely paying off!
CSS element() function (via) Vincent De Oliveira explores the CSS element function (2 years ago), which sadly is still only supported by Firefox. It lets you render any HTML element as a CSS background image, enabling folding effects, animated backgrounds, live previews and more.
simonw/csvs-to-sqlite. I built a simple tool for bulk converting multiple CSV files into a SQLite database.
Inside a low budget consumer hardware espionage implant. Fascinating break-down of a cheap ($25) USB charging cable with a hidden microphone and SIM that can be activated via SMS message or by calling the device.
Exploring Line Lengths in Python Packages. Interesting exploration of the impact if the 79 character length limit rule of thumb on various Python packages—and a thoroughly useful guide to histogram plotting in Jupyter, pandas and matplotlib.
Interactive Database of the World’s River Basins (via) “This database provides the first-ever compilation of the world’s river basins developed specifically for corporate disclosure. It features a comprehensive list of river basins worldwide, including their names, boundaries, and other helpful information.”
Pink Trombone (via) Explorable explanation of the human vocal system by Neil Thapen. View source for some pleasantly readable JavaScript / canvas / window.AudioContext code.
dhash (via) Python library to calculate the perceptual difference hash for an image. Delightfully simple algorithm that’s fully explained in the README—it works by scaling the image to 8x8 grayscale and then creating a bitmap representing of each pixel is lighter or darker than the previous one.
Use a Render Prop! Michael Jackson makes a convincing argument for using the render prop pattern in React in preference to HOCs (Higher Order Components, the React equivalent of Python class decorators).
The Booming Japanese Rent-a-Friend Business. “In Japan, you can pay an actor to impersonate your relative, spouse, coworker, or any kind of acquaintance.”. Surprisingly poignant interview with a man who runs a company that employs 800 professional actors for almost any conceivable situation.
How Sentry Receives 20 Billion Events Per Month While Preparing To Handle Twice That. RabbitMQ federation, nginx and HAProxy, Riak as a key/value store, data processing is still mainly Python with a little bit of Rust. As of July 2017 it’s all hosted on Google Cloud Platform.
Eager Execution: An imperative, define-by-run interface to TensorFlow. Lets you evaluate TensorFlow expressions interactively in Python without needing to constantly run tf.Session().run(variable).