Blogmarks
Filters: Sorted by date
Live htop. Neat, simplest-thing-that-could-possibly-work implementation of a tool that continually pipes the output of the htop command to a browser over a WebSocket. The htopgen.sh scripts loops every 2 seconds, runs htop, pipes it through a utility to convert the output to HTML and writes that to a file. Then the server.js Node.js script watches for changes to that file and pipes the entire file contents to the browser via socket.io. The index.html page in the browser subscribes to the WebSocket and updates the entire page using innerHTML every time it receives an event.
Oxford Deep NLP 2017 course (via) Slides, course description and links to lecture videos for the 2017 Deep Natural Language Processing course at the University of Oxford presented by a team from Google DeepMind.
Squeezing every drop of performance out of a Django app on Heroku. Ben Firshman describes some lesser known tricks for scaling Django on Heroku—in particular, using gunicorn gevent asynchronous workers and setting up PostgreSQL connection pooling using django-db-geventpool.
Scaling the GitLab database. Lots of interesting details on how GitLab have worked to scale their PostgreSQL setup. They’ve avoided sharding so far, instead opting for database pooling with pgbouncer and read-only replicas using hot standbys. I like the way they deal with replica lag—they store the current WAL position in a redis key for the user every time there’s a write, then use pg_last_xlog_replay_location() on the various replicas to check and see if they have caught up next time the user makes a request that needs to read some data.
Fun facts about Rust’s growing popularity. TIL Rust is used “on hundreds of millions of desktops via Dropbox”. And Rust developers are called Rustaceans.
Exploring United States Policing Data Using Python. Outstanding introduction to data analysis with Jupyter and Pandas.
Fast GeoSpatial Analysis in Python. Some clever advanced performance tricks with Cython and Dask, but it also introduced me to GeoPandas.
How to teach technical concepts with cartoons. Julia Evans: “This post is about a few patterns I use when illustrating ideas about computers. If you are interested in using drawings to teach people about your very favorite computer topics, hopefully this will help you!”
profiling. “An interactive continuous Python profiler”. This is really neat—simply run “profiling myscript.py” to get an interactive, navigable console-based profile inspector at the end of your script... or run “profiling live-profile mywebverver.py” to see a live, updating profile of a long-running process. Has options for statistical profiling as well, which has a much lower overhead in exchange for a less accurate view of what is going on.
Benefit of TEXT with CHECK over VARCHAR(X) in PostgreSQL.
Brandur suggests using email TEXT CHECK (char_length(email) <= 255) to define a column with a length limit in PostgreSQL over VARCHAR(255) because TEXT and VARCHAR are equally performant but a CHECK length can be changed later on without locking the table, whereas a VARCHAR requires an ALTER TABLE with an exclusive lock.
Implementing Stripe-like Idempotency Keys in Postgres (via) Having clients send “idempotency keys” with API requests in order to be able to safely retry them if something’s goes wrong is a really neat trick for making transactional APIs more robust. Here Brandur Leach talks implementation strategies.
Dynamically changing favicons with JavaScript. I just spotted GitHub using this trick to swap out the favicon on a pull request that hadn’t passed its tests yet for github.com/favicon-failure.ico with a little cross icon in the corner.
Contributors to python/cpython, Aug 5, 1990—Oct 26, 2017. I love how the graphs on this page summarize the history of the last 27 years of Python development, showing exactly when each core contributor was most active.
Elaborate Halloween Costume Tips from a 19th-Century Guide to Fancy Dress (via) The gilded age had some ridiculous parties. Here are highlights of the most popular costume guide of the era, now available on the Internet Archive.
arxiv-vanity (via) Beautiful new project from Ben Firshman and Andreas Jansson: “Arxiv Vanity renders academic papers from Arxiv as responsive web pages so you don’t have to squint at a PDF”. It works by pulling the raw LaTeX source code from Arxiv and rendering it to HTML using a heavily customized Pandoc workflow. The real fun is in the architecture: it’s a Django app running on Heroku which fires up on-demand Hyper.sh Docker containers for each individual rendering job.
Introducing Faktory. A brand new worker queue system from Mike Perham, the author of Sidekiq for Ruby. It’s written in Go on top of RocksDB and is explicitly designed to support clients and workers in multiple different languages.
Saying Goodbye to Firebug. Firebug has reached end-of-life as a separate extension, replaced by the built-in Firefox Developer Tools. It’s hard to overstate how influential Firebug has been in helping to turn front-end web development into a respected and productive engineering discipline.
Getting Your First 10 Customers (via) There is so much good advice embedded in this article by Patrick McKenzie, and it constantly comes back to the theme of doing whatever it takes to get to your first ten paying customers.
Can You Afford It?: Real-world Web Performance Budgets. Alex Russell’s magnum opus on web performance budgets in 2017. He proposes a baseline testing device equivalent to a $200 Android phone on a slow 3G network emulated at 400ms RTT/400Kbps transfer and encourages a goal of 5s time-to-interactive on first load and 2s TTI for subsequent views. This means around 130kb of gzipped JavaScript—challenging but not impossible with modern JavaScript frameworks.
hupper (via) Handy Python module for adding “live reload” development support to just about anything. I’m using it with Sanic—I run “hupper -m app” and it starts up my code in app.py and automatically reloads it any time any of the corresponding files changes on disk.
System Requirements For SQLite. Document describing the high level goals and objectives of SQLite. Like everything to do with SQLite this exhibits some incredibly well thought out software engineering. I particularly like “S80000: SQLite shall exhibit ductile failure characteristics“ where ductile is described in opposition to brittle: a ductile system begins showing signs of trouble well in advance of failure.
Parse shell one-liners with pyparsing. Neat introduction to the pyparsing library, both for parsing tokens into labeled sections and constructing an AST from them.
SurviveJS—Webpack (via) Free online book about Webpack. I’ve read the first couple of chapters and it looks like a concise, well constructed guide to a key component of the modern JavaScript stack.
It’s Not a Feature Problem—Avoiding Startup Tarpits (via) “When we turned on paid advertising for the first time the increase we had a sizable increase in signups. We always feared that a new user would just churn because of what we perceived as deficiencies in the product. While there were users who churned for that reason, it was never the nightmare scenario that we imagined.”
Getting the Most out of Sqlite3 with Python. A couple of neat tricks I didn’t know: you can skip cursors entirely by calling .execute and .executemany directly on the connection object, and you can use the connection object as a context manager to execute transactions using a “with” block.
Crossdressing, Compression, and Colliders: ’The First Photo on the Web’. TIL the first photo shared on the web was of Les Horribles Cernettes, an all-female comedy musical group at CERN who performed songs about particle physics. And Sir Tim Berners-Lee first met them when he played the dame in the CERN panto.
github-dashboard (via) Nice little self-contained example of a React app with no build step by Shing Lyu.
Minimal React.js Without A Build Step.
React is pretty dependent on a build phase, to handle things like JSX compilation. This is fine for most projects, but sometimes I just want to hot-link react and react-dom from a CDN and knock out a quick self-contained mini-application. Shing Lyu points out that this is much easier if you ditch JSX in favour of direct calls to React.createElement().
Hey Siri: An On-device DNN-powered Voice Trigger for Apple’s Personal Assistant (via) “The “Hey Siri” detector uses a Deep Neural Network (DNN) to convert the acoustic pattern of your voice at each instant into a probability distribution over speech sounds. It then uses a temporal integration process to compute a confidence score that the phrase you uttered was “Hey Siri”. If the score is high enough, Siri wakes up.”
React is the new Dojo. In which Mikeal Rogers provides his perspective on the history of Dojo, the earliest break-out JavaScript framework, how jQuery eclipsed it and contemplates the same thing eventually happening to React.