Simon Willison’s Weblog

Subscribe
Atom feed

Research

Filters: Sorted by date

If it's good enough for antirez to add to Redis I figured Ville Laurikari's TRE regular expression engine was worth exploring in a little more detail.

I had Claude Code build an experimental Python binding (it used ctypes) and try some malicious regular expression attacks against the library. TRE handles those much better than Python's standard library implementation, thanks mainly to the lack of support for backtracking.

Anthropic publish the system prompts for Claude chat and make that page available as Markdown. I had Claude Code turn that page into separate files for each model and model family with fake git commit dates to enable browsing the changes via the GitHub commit view.

I used this to write my own detailed notes on the changes between Opus 4.6 and 4.7.

In Servo is now available on crates.io the Servo team announced the initial release of the servo crate, which packages their browser engine as an embeddable library.

I set Claude Code for web the task of figuring out what it can do, building a CLI tool for taking screenshots using it and working out if it could be compiled to WebAssembly.

The servo-shot Rust tool it built works pretty well:

git clone https://github.com/simonw/research
cd research/servo-crate-exploration/servo-shot
cargo build
./target/debug/servo-shot https://news.ycombinator.com/

Here's the result:

An accurately rendered screenshot of the Hacker News homepage

Compiling Servo itself to WebAssembly is not feasible due to its heavy use of threads and dependencies like SpiderMonkey, but Claude did build me this playground page for trying out a WebAssembly build of the html5ever and markup5ever_rcdom crates, providing a tool for turning fragments of HTML into a parse tree.

Research QuickJS Python Sandbox — Investigation Report — Exploring the `quickjs` Python package, this project implements an asyncio-compatible JavaScript sandbox with robust resource controls and seamless exposure of both synchronous and asynchronous Python functions (including async httpx fetches) to JavaScript code.

Inspired by this conversation on Hacker News about whether two SQLite processes in separate Docker containers that share the same volume might run into problems due to WAL shared memory. The answer is that everything works fine - Docker containers on the same host and filesystem share the same shared memory in a way that allows WAL to collaborate as it should.

In trying to build my own version of Claude Artifacts I got curious about options for applying CSP headers to content in sandboxed iframes without using a separate domain to host the files. Turns out you can inject <meta http-equiv="Content-Security-Policy"...> tags at the top of the iframe content and they'll be obeyed even if subsequent untrusted JavaScript tries to manipulate them.

Stuart Breckenridge pointed out that PC Gamer Recommends RSS Readers in a 37MB Article That Just Keeps Downloading, highlighting a truly horrifying example of web bloat that added up to 100s more MBs thanks to auto-playing video ads. I decided to have Claude Code for web use Rodney to investigate the page - prompt here.

Aaron Harper wrote about Node.js worker threads, which inspired me to run a research task to see if they might help with running JavaScript in a sandbox. Claude Code went way beyond my initial question and produced a comparison of isolated-vm, vm2, quickjs-emscripten, QuickJS-NG, ShadowRealm, and Deno Workers.

I had Claude Code run a micro-benchmark comparing different approaches to implementing tagging in SQLite. Traditional many-to-many tables won, but FTS5 came a close second. Full table scans with LIKE queries performed better than I expected, but full table scans with JSON arrays and json_each() were much slower.

Research PDF to Image Converter — Leveraging Rust's `pdfium-render` crate and Python's PyO3 bindings, this project enables fast and reliable conversion of PDF pages to JPEG images, packaged as a self-contained Python wheel. The CLI tool and Python library are both built to require no external dependencies, bundling the necessary PDFium binary for ease of installation and cross-platform compatibility.
Research REXC (rx) JSON Test Suite — REXC (rx) JSON Test Suite provides a comprehensive, language-agnostic test resource for validating implementations of the REXC encoder/decoder. It includes a single JSON file with 206 tests covering base64 encoding, zigzag integer transformations, value conversions, roundtrip integrity, and special numeric values, ensuring correctness across platforms.
Research syntaqlite Python Extension in WebAssembly — syntaqlite-python-extension is a Python C extension module that integrates the syntaqlite Rust/C SQL toolkit, making high-fidelity SQL parsing, formatting, validation, and tokenization available to Python and Pyodide environments. It wraps syntaqlite's native FFI for both desktop and web, linking against static libraries produced by Rust and employing Emscripten for WASM builds.
Research CSRF Protection Demo: Modern Browser-Based Defenses — Modern browser security now enables robust Cross-Site Request Forgery (CSRF) prevention without requiring tokens. This demo project contrasts a vulnerable FastAPI bank app with a protected version, showcasing how browser-sent headers like `Sec-Fetch-Site` and `Origin` empower servers to automatically reject cross-origin POST requests.
Research v86 exploration — Exploring the v86 Linux Emulator (see v86 Linux Emulator tool), this project evaluates a browser-based Buildroot 2024.05.2 x86 environment with a constrained 39 MB RAM, featuring BusyBox utilities, Lua 5.4.6 scripting, and core text-processing tools. Although it boasts comprehensive shell utilities, file management tools, and basic network utilities (curl, wget, links), actual internet access is unavailable due to the lack of a configured network relay.
Research Luau WebAssembly: Browser Playground + Python wasmtime — Luau WebAssembly explores compiling the Luau scripting language (used by Roblox) to WebAssembly for interactive browser environments and Python integration via wasmtime. By leveraging Emscripten, the project creates a streamlined WASM module that runs in the browser (with a playground and Pyodide integration) and server-side Python. Key technical adaptations include custom output capture, flexible WASM imports for wasmtime, and Python wrappers that handle C++ exception lifecycles.
Research Rust Word Cloud CLI — Leveraging Rust’s performance and safety, this CLI tool generates PNG word clouds directly from text input using a custom spiral layout algorithm and efficient grid-based collision detection. It supports flexible options for image size, font scaling, color schemes, and background colors, with all core features—such as stopword filtering, spatial indexing, and layout—implemented from scratch without any external word cloud library.
Research Unicode Explorer — Binary Search Over HTTP — By leveraging HTTP Range requests and fixed-width binary records, Unicode Explorer demonstrates efficient binary search for Unicode data directly from a static file with zero backend or dependencies. The client fetches only one 256-byte record per step, using signposts from `meta.json` to optimize initial narrowing, then performs real-time network-driven binary search, visualized in an interactive log.
Research README Timezone Clarification — Timezone mismatches in the project’s root README.md were identified due to inconsistent git commit author dates—some in UTC, others in US Pacific time—displayed without timezone clarification. The listing was generated by a cog script that extracted dates using `git log`, then formatted them without standardizing to a common timezone, causing confusion across 39 project directories.
Research WebMCP + Chrome DevTools Protocol Demo — WebMCP is a proposed browser API that enables web applications to expose structured, callable tools for AI agents, reducing the need for unreliable UI automation. This project demonstrates how to register and interact with WebMCP tools using a Python client over the Chrome DevTools Protocol (CDP), providing a bridge to discover and call these tools programmatically.
Research Header Alignment Investigation - simonwillison.net — Addressing a subtle header alignment issue on simonwillison.net, this investigation tracked down a persistent ~1px height mismatch between left and right headers caused by anchor elements generating taller inline boxes than plain text due to font metrics. Multiple fixes—including removing position:relative/top:1px hacks and setting explicit heights—proved fragile.
Research SQLite Hamming Distance Extension: Scalar vs Virtual Table — Exploring efficient Hamming distance search in SQLite for binary embeddings, this project implements both a scalar function extension and a virtual table extension as described in "Hamming Distance for Hybrid Search in SQLite". The scalar function scans and sorts rows to locate nearest matches, while the virtual table caches embeddings and leverages a max-heap to deliver top-k results up to seven times faster.
Research sqlite-chronicle + sqlite-history-json: Same Table Investigation — Using both sqlite-chronicle and sqlite-history-json on the same SQLite table is feasible, as each library installs its own set of triggers and companion tables without interfering with standard CRUD operations. Chronicle focuses on efficient sync/versioning, while history-json offers a complete audit log, and both operate independently even with compound primary keys or concurrent audit groups.
Research Guidepup Screen Reader Investigation — An investigation into Guidepup reveals that its core package does not support Linux—only macOS (VoiceOver) and Windows (NVDA). However, two practical methods were proven for generating audio screen reader sessions on Linux: one uses the AT-SPI accessibility stack and Orca to walk a real browser's accessibility tree and synthesize narration; the other employs the virtual screen reader (pure JS, fast) to simulate navigation, then builds audio from spoken phrases.
Research SeaweedFS 4.12 Feature Testing — SeaweedFS version 4.12 was evaluated on Linux x86_64, demonstrating its functionality as a scalable distributed file system through its core blob store, filer, S3-compatible, and WebDAV APIs. All-in-one deployment via `weed mini` enables access to web UIs for cluster administration, filer usage, and volume monitoring (Admin UI screenshot). Testing confirmed seamless file operations across HTTP, S3, WebDAV, including directory management, standard HTTP features, and multiple URL formats.
Research OpenAI Skills API — Hands-On Demo — OpenAI's Skills API enables models to execute reusable, self-contained scripts and tools by packaging instructions and code (plus optional assets) with a `SKILL.md` manifest. This project demonstrates crafting a custom skill (“csv-insights”), uploading it via the `/v1/skills` endpoint, and invoking it in natural language through the Responses API’s hosted shell environment, where the model installs dependencies, executes scripts, and returns outputs such as markdown reports and plots.
Research cysqlite WebAssembly Wheel — By cross-compiling cysqlite, a high-performance Cython-based SQLite3 binding, to WebAssembly with Emscripten, this project delivers a ready-to-use wheel for Pyodide that enables rapid, native-like SQLite operations directly in browser-based Python environments. The build pipeline automates all necessary steps, from fetching dependencies to ensuring compatibility with Pyodide 0.25.x (Python 3.11, Emscripten 3.1.46).
Research rod-cli: Chrome Automation from the Command Line — Leveraging the rod browser automation library, rod-cli provides a lightweight Go-based command-line tool for scripting persistent headless Chrome sessions. Each CLI command connects to and manipulates the same long-running Chrome instance via DevTools Protocol, enabling seamless multi-step browser automation in shell scripts or interactive use.
Research Rod: Go Library for Chrome Automation -- Comprehensive API Reference — Rod is an advanced Go library designed to automate Chrome browsers using the Chrome DevTools Protocol, providing a comprehensive API for web scraping, browser control, element interaction, and robust waiting strategies. With high-level convenience methods (such as Must-prefixed methods for fast scripting) and direct protocol access, Rod enables streamlined workflows from simple scraping to complex automation scenarios, all without third-party drivers.
Research krunsh — Krunsh is a minimal Go CLI tool that executes newline-delimited shell commands inside an ephemeral KVM-based microVM, leveraging the libkrun library for lightweight virtualization. By piping commands from stdin, krunsh spins up a microVM, runs the specified commands using `/bin/sh -c`, captures the output, and discards the VM afterward, ensuring zero persistent state and strong process isolation.