Simon Willison’s Weblog

Subscribe

Research

Filters: Sorted by date

Research Starlette 1.0 skill — Starlette 1.0 Skill offers a concise guide for building robust web applications with Starlette, a lightweight ASGI framework. The accompanying demo showcases a task management app featuring projects, tasks, comments, and labels, illustrating Starlette's flexibility in handling routing, templating (Jinja2), async database operations (aiosqlite), and real-time updates.
Research PCGamer Article Performance Audit — A performance audit of the March 2026 PCGamer article on RSS readers reveals severe page bloat, with over 82% of network traffic and transferred bytes traced to ad-tech, tracking, and programmatic advertising scripts. Despite the core content consisting of just 10-15 KB of text and a handful of images (~150 KB total), the page triggers over 431 network requests and 5.5 MB of transfer (18.8 MB decoded) within 60 seconds—ballooning to 200+ MB in Firefox due to autoplay video carousels and…

Stuart Breckenridge pointed out that PC Gamer Recommends RSS Readers in a 37MB Article That Just Keeps Downloading, highlighting a truly horrifying example of web bloat that added up to 100s more MBs thanks to auto-playing video ads. I decided to have Claude Code for web use Rodney to investigate the page - prompt here.

Research JavaScript Sandboxing Research — Analyzing current JavaScript sandboxing options for running untrusted code, this research compares core approaches in Node.js (including worker_threads, node:vm, and the Permission Model), prominent npm packages (isolated-vm, vm2), and alternative engines like quickjs-emscripten.

Aaron Harper wrote about Node.js worker threads, which inspired me to run a research task to see if they might help with running JavaScript in a sandbox. Claude Code went way beyond my initial question and produced a comparison of isolated-vm, vm2, quickjs-emscripten, QuickJS-NG, ShadowRealm, and Deno Workers.

Research SQLite Tags Benchmark: Comparing 5 Tagging Strategies — Benchmarking five tagging strategies in SQLite reveals clear trade-offs between query speed, storage, and implementation complexity for workflows involving tags (100,000 rows, 100 tags, average 6.5 tags/row). Indexed approaches—materialized lookup tables on JSON and classic many-to-many tables—easily outperform others, handling single-tag queries in under 1.5 milliseconds, while raw JSON and LIKE-based solutions are much slower.

I had Claude Code run a micro-benchmark comparing different approaches to implementing tagging in SQLite. Traditional many-to-many tables won, but FTS5 came a close second. Full table scans with LIKE queries performed better than I expected, but full table scans with JSON arrays and json_each() were much slower.

Research PDF to Image Converter — Leveraging Rust's `pdfium-render` crate and Python's PyO3 bindings, this project enables fast and reliable conversion of PDF pages to JPEG images, packaged as a self-contained Python wheel. The CLI tool and Python library are both built to require no external dependencies, bundling the necessary PDFium binary for ease of installation and cross-platform compatibility.
Research REXC (rx) JSON Test Suite — REXC (rx) JSON Test Suite provides a comprehensive, language-agnostic test resource for validating implementations of the REXC encoder/decoder. It includes a single JSON file with 206 tests covering base64 encoding, zigzag integer transformations, value conversions, roundtrip integrity, and special numeric values, ensuring correctness across platforms.
Research syntaqlite Python Extension — syntaqlite-python-extension is a Python C extension module that integrates the syntaqlite Rust/C SQL toolkit, making high-fidelity SQL parsing, formatting, validation, and tokenization available to Python and Pyodide environments. It wraps syntaqlite's native FFI for both desktop and web, linking against static libraries produced by Rust and employing Emscripten for WASM builds.
Research CSRF Protection Demo: Modern Browser-Based Defenses — Modern browser security now enables robust Cross-Site Request Forgery (CSRF) prevention without requiring tokens. This demo project contrasts a vulnerable FastAPI bank app with a protected version, showcasing how browser-sent headers like `Sec-Fetch-Site` and `Origin` empower servers to automatically reject cross-origin POST requests.
Research v86 exploration — Exploring the v86 Linux Emulator (see v86 Linux Emulator tool), this project evaluates a browser-based Buildroot 2024.05.2 x86 environment with a constrained 39 MB RAM, featuring BusyBox utilities, Lua 5.4.6 scripting, and core text-processing tools. Although it boasts comprehensive shell utilities, file management tools, and basic network utilities (curl, wget, links), actual internet access is unavailable due to the lack of a configured network relay.
Research Luau WebAssembly: Browser Playground + Python wasmtime — Luau WebAssembly explores compiling the Luau scripting language (used by Roblox) to WebAssembly for interactive browser environments and Python integration via wasmtime. By leveraging Emscripten, the project creates a streamlined WASM module that runs in the browser (with a playground and Pyodide integration) and server-side Python. Key technical adaptations include custom output capture, flexible WASM imports for wasmtime, and Python wrappers that handle C++ exception lifecycles.
Research Rust Word Cloud CLI — Leveraging Rust’s performance and safety, this CLI tool generates PNG word clouds directly from text input using a custom spiral layout algorithm and efficient grid-based collision detection. It supports flexible options for image size, font scaling, color schemes, and background colors, with all core features—such as stopword filtering, spatial indexing, and layout—implemented from scratch without any external word cloud library.
Research Unicode Explorer — Binary Search Over HTTP — By leveraging HTTP Range requests and fixed-width binary records, Unicode Explorer demonstrates efficient binary search for Unicode data directly from a static file with zero backend or dependencies. The client fetches only one 256-byte record per step, using signposts from `meta.json` to optimize initial narrowing, then performs real-time network-driven binary search, visualized in an interactive log.
Research README Timezone Clarification — Timezone mismatches in the project’s root README.md were identified due to inconsistent git commit author dates—some in UTC, others in US Pacific time—displayed without timezone clarification. The listing was generated by a cog script that extracted dates using `git log`, then formatted them without standardizing to a common timezone, causing confusion across 39 project directories.
Research WebMCP + Chrome DevTools Protocol Demo — WebMCP is a proposed browser API that enables web applications to expose structured, callable tools for AI agents, reducing the need for unreliable UI automation. This project demonstrates how to register and interact with WebMCP tools using a Python client over the Chrome DevTools Protocol (CDP), providing a bridge to discover and call these tools programmatically.
Research Header Alignment Investigation - simonwillison.net — Addressing a subtle header alignment issue on simonwillison.net, this investigation tracked down a persistent ~1px height mismatch between left and right headers caused by anchor elements generating taller inline boxes than plain text due to font metrics. Multiple fixes—including removing position:relative/top:1px hacks and setting explicit heights—proved fragile.
Research SQLite Hamming Distance Extension: Scalar vs Virtual Table — Exploring efficient Hamming distance search in SQLite for binary embeddings, this project implements both a scalar function extension and a virtual table extension as described in "Hamming Distance for Hybrid Search in SQLite". The scalar function scans and sorts rows to locate nearest matches, while the virtual table caches embeddings and leverages a max-heap to deliver top-k results up to seven times faster.
Research sqlite-chronicle + sqlite-history-json: Same Table Investigation — Using both sqlite-chronicle and sqlite-history-json on the same SQLite table is feasible, as each library installs its own set of triggers and companion tables without interfering with standard CRUD operations. Chronicle focuses on efficient sync/versioning, while history-json offers a complete audit log, and both operate independently even with compound primary keys or concurrent audit groups.
Research Guidepup Screen Reader Investigation — An investigation into Guidepup reveals that its core package does not support Linux—only macOS (VoiceOver) and Windows (NVDA). However, two practical methods were proven for generating audio screen reader sessions on Linux: one uses the AT-SPI accessibility stack and Orca to walk a real browser's accessibility tree and synthesize narration; the other employs the virtual screen reader (pure JS, fast) to simulate navigation, then builds audio from spoken phrases.
Research SeaweedFS 4.12 Feature Testing — SeaweedFS version 4.12 was evaluated on Linux x86_64, demonstrating its functionality as a scalable distributed file system through its core blob store, filer, S3-compatible, and WebDAV APIs. All-in-one deployment via `weed mini` enables access to web UIs for cluster administration, filer usage, and volume monitoring (Admin UI screenshot). Testing confirmed seamless file operations across HTTP, S3, WebDAV, including directory management, standard HTTP features, and multiple URL formats.
Research OpenAI Skills API — Hands-On Demo — OpenAI's Skills API enables models to execute reusable, self-contained scripts and tools by packaging instructions and code (plus optional assets) with a `SKILL.md` manifest. This project demonstrates crafting a custom skill (“csv-insights”), uploading it via the `/v1/skills` endpoint, and invoking it in natural language through the Responses API’s hosted shell environment, where the model installs dependencies, executes scripts, and returns outputs such as markdown reports and plots.
Research cysqlite WebAssembly Wheel — By cross-compiling cysqlite, a high-performance Cython-based SQLite3 binding, to WebAssembly with Emscripten, this project delivers a ready-to-use wheel for Pyodide that enables rapid, native-like SQLite operations directly in browser-based Python environments. The build pipeline automates all necessary steps, from fetching dependencies to ensuring compatibility with Pyodide 0.25.x (Python 3.11, Emscripten 3.1.46).
Research rod-cli: Chrome Automation from the Command Line — Leveraging the rod browser automation library, rod-cli provides a lightweight Go-based command-line tool for scripting persistent headless Chrome sessions. Each CLI command connects to and manipulates the same long-running Chrome instance via DevTools Protocol, enabling seamless multi-step browser automation in shell scripts or interactive use.
Research Rod: Go Library for Chrome Automation -- Comprehensive API Reference — Rod is an advanced Go library designed to automate Chrome browsers using the Chrome DevTools Protocol, providing a comprehensive API for web scraping, browser control, element interaction, and robust waiting strategies. With high-level convenience methods (such as Must-prefixed methods for fast scripting) and direct protocol access, Rod enables streamlined workflows from simple scraping to complex automation scenarios, all without third-party drivers.
Research krunsh — Krunsh is a minimal Go CLI tool that executes newline-delimited shell commands inside an ephemeral KVM-based microVM, leveraging the libkrun library for lightweight virtualization. By piping commands from stdin, krunsh spins up a microVM, runs the specified commands using `/bin/sh -c`, captures the output, and discards the VM afterward, ensuring zero persistent state and strong process isolation.
Research Monty WASM + Pyodide — Monty WASM + Pyodide explores compiling Monty—a Rust-based, sandboxed Python interpreter—into WebAssembly for seamless browser access. It provides two integration paths: a standalone WASM module accessible directly from JavaScript, and a Pyodide-compatible wheel for usage in Python-in-the-browser environments. The project enables safe, dependency-free Python code execution with features like variable injection, output capturing (including print statements), and robust error handling.
Research Building PyO3/Maturin Rust Extension Modules as WebAssembly Wheels for Pyodide — Compiling Rust-based Python extension modules (via PyO3 and maturin) into WebAssembly wheels for Pyodide involves precise coordination of toolchain versions and build flags to ensure compatibility. The process relies on maturin (≥1.0) for packaging, the Emscripten SDK (with the exact version used by Pyodide), and a Rust nightly toolchain matching Pyodide's ABI, particularly the `-Z emscripten-wasm-eh` flag and a compatible sysroot for Python 3.13 (Pyodide 0.28+).
Research just-bash: Deno JSONL Server + Python Client — Exploring the capabilities of just-bash, this project integrates the TypeScript-based bash emulator into a persistent, JSONL-over-stdio server in Deno, accessible via a robust Python client library. The solution enables sandboxed bash scripting with comprehensive built-in commands, a virtual filesystem, and optional network access, with persistent state and fine-grained request control (env, cwd, timeout) supported.
Research WASM REPL CLI Tools — WASM REPL CLI Tools enable JavaScript and Python REPLs from the command line by leveraging WebAssembly runtimes in Go, built on the wazero engine. The project supplies separate binaries for each language—one using QuickJS WASI and the other CPython WASI—offering direct code execution, interactive shells, and a JSONL mode. JSONL mode lets external applications submit code for execution while maintaining persistent state across requests, facilitating programmatic integration.
Research What we learned today about this ChatGPT “container” environment — Experiments in the ChatGPT sandbox reveal that general outbound internet access from Python and other user code (such as HTTP requests) is entirely blocked, while package managers like pip and npm are permitted to fetch dependencies using curated internal registry proxies. The container provides a privileged fetching mechanism (`container.download`) for select public URLs, which is more powerful than standard code-based networking.
Research Cloudflare Workers with Python and SQLite — Exploring the intersection of Cloudflare Workers, Python (via Pyodide), and SQLite persistence, this project demonstrates practical techniques for building serverless applications with both JavaScript and Python runtimes on the Cloudflare platform. JavaScript Workers, paired with D1 for persistent SQLite storage, handled form input, basic routing, and a page view counter.