Simon Willison's Weblog: blogging

My current policy on AI writing for my blog

2026-03-01T16:06:43+00:00

Because I write about LLMs (and maybe because of my em dash text replacement code) a lot of people assume that the writing on my blog is partially or fully created by those LLMs.

My current policy on this is that if text expresses opinions or has "I" pronouns attached to it then it's written by me. I don't let LLMs speak for me in this way.

I'll let an LLM update code documentation or even write a README for my project but I'll edit that to ensure it doesn't express opinions or say things like "This is designed to help make code easier to maintain" - because that's an expression of a rationale that the LLM just made up.

I use LLMs to proofread text I publish on my blog. I just shared my current prompt for that here.

Tags: ai-ethics, writing, generative-ai, blogging, ai, llms

Writing about Agentic Engineering Patterns

2026-02-23T17:43:02+00:00

I've started a new project to collect and document Agentic Engineering Patterns - coding practices and patterns to help get the best results out of this new era of coding agent development we find ourselves entering.

I'm using Agentic Engineering to refer to building software using coding agents - tools like Claude Code and OpenAI Codex, where the defining feature is that they can both generate and execute code - allowing them to test that code and iterate on it independently of turn-by-turn guidance from their human supervisor.

I think of vibe coding using its original definition of coding where you pay no attention to the code at all, which today is often associated with non-programmers using LLMs to write code.

Agentic Engineering represents the other end of the scale: professional software engineers using coding agents to improve and accelerate their work by amplifying their existing expertise.

There is so much to learn and explore about this new discipline! I've already published a lot under my ai-assisted-programming tag (345 posts and counting) but that's been relatively unstructured. My new goal is to produce something that helps answer the question "how do I get good results out of this stuff" all in one place.

I'll be developing and growing this project here on my blog as a series of chapter-shaped patterns, loosely inspired by the format popularized by Design Patterns: Elements of Reusable Object-Oriented Software back in 1994.

I published the first two chapters today:

Writing code is cheap now talks about the central challenge of agentic engineering: the cost to churn out initial working code has dropped to almost nothing, how does that impact our existing intuitions about how we work, both individually and as a team?
Red/green TDD describes how test-first development helps agents write more succinct and reliable code with minimal extra prompting.

I hope to add more chapters at a rate of 1-2 a week. I don't really know when I'll stop, there's a lot to cover!

Written by me, not by an LLM

I have a strong personal policy of not publishing AI-generated writing under my own name. That policy will hold true for Agentic Engineering Patterns as well. I'll be using LLMs for proofreading and fleshing out example code and all manner of other side-tasks, but the words you read here will be my own.

Chapters and Guides

Agentic Engineering Patterns isn't exactly a book, but it's kind of book-shaped. I'll be publishing it on my site using a new shape of content I'm calling a guide. A guide is a collection of chapters, where each chapter is effectively a blog post with a less prominent date that's designed to be updated over time, not frozen at the point of first publication.

Guides and chapters are my answer to the challenge of publishing "evergreen" content on a blog. I've been trying to find a way to do this for a while now. This feels like a format that might stick.

If you're interested in the implementation you can find the code in the Guide, Chapter and ChapterChange models and the associated Django views, almost all of which was written by Claude Opus 4.6 running in Claude Code for web accessed via my iPhone.

Tags: blogging, design-patterns, projects, writing, ai, generative-ai, llms, ai-assisted-programming, vibe-coding, coding-agents, agentic-engineering

Adding TILs, releases, museums, tools and research to my blog

2026-02-20T23:47:10+00:00

I've been wanting to add indications of my various other online activities to my blog for a while now. I just turned on a new feature I'm calling "beats" (after story beats, naming this was hard!) which adds five new types of content to my site, all corresponding to activity elsewhere.

Here's what beats look like:

Those three are from the 30th December 2025 archive page.

Beats are little inline links with badges that fit into different content timeline views around my site, including the homepage, search and archive pages.

There are currently five types of beats:

Releases are GitHub releases of my many different open source projects, imported from this JSON file that was constructed by GitHub Actions.
TILs are the posts from my TIL blog, imported using a SQL query over JSON and HTTP against the Datasette instance powering that site.
Museums are new posts on my niche-museums.com blog, imported from this custom JSON feed.
Tools are HTML and JavaScript tools I've vibe-coded on my tools.simonwillison.net site, as described in Useful patterns for building HTML tools.
Research is for AI-generated research projects, hosted in my simonw/research repo and described in Code research projects with async coding agents like Claude Code and Codex.

That's five different custom integrations to pull in all of that data. The good news is that this kind of integration project is the kind of thing that coding agents really excel at. I knocked most of the feature out in a single morning while working in parallel on various other things.

I didn't have a useful structured feed of my Research projects, and it didn't matter because I gave Claude Code a link to the raw Markdown README that lists them all and it spun up a parser regex. Since I'm responsible for both the source and the destination I'm fine with a brittle solution that would be too risky against a source that I don't control myself.

Claude also handled all of the potentially tedious UI integration work with my site, making sure the new content worked on all of my different page types and was handled correctly by my faceted search engine.

Prototyping with Claude Artifacts

I actually prototyped the initial concept for beats in regular Claude - not Claude Code - taking advantage of the fact that it can clone public repos from GitHub these days. I started with:

Clone simonw/simonwillisonblog and tell me about the models and views

And then later in the brainstorming session said:

use the templates and CSS in this repo to create a new artifact with all HTML and CSS inline that shows me my homepage with some of those inline content types mixed in

After some iteration we got to this artifact mockup, which was enough to convince me that the concept had legs and was worth handing over to full Claude Code for web to implement.

If you want to see how the rest of the build played out the most interesting PRs are Beats #592 which implemented the core feature and Add Museums Beat importer #595 which added the Museums content type.

Tags: blogging, museums, ai, til, generative-ai, llms, ai-assisted-programming, claude-artifacts, claude-code

Experimenting with sponsorship for my blog and newsletter

2026-02-19T05:44:29+00:00

I've long been resistant to the idea of accepting sponsorship for my blog. I value my credibility as an independent voice, and I don't want to risk compromising that reputation.

Then I learned about Troy Hunt's approach to sponsorship, which he first wrote about in 2016. Troy runs with a simple text row in the page banner - no JavaScript, no cookies, unobtrusive while providing value to the sponsor. I can live with that!

Accepting sponsorship in this way helps me maintain my independence while offsetting the opportunity cost of not taking a full-time job.

To start with I'm selling sponsorship by the week. Sponsors get that unobtrusive banner across my blog and also their sponsored message at the top of my newsletter.

I will not write content in exchange for sponsorship. I hope the sponsors I work with understand that my credibility as an independent voice is a key reason I have an audience, and compromising that trust would be bad for everyone.

Freeman & Forrest helped me set up and sell my first slots. Thanks also to Theo Browne for helping me think through my approach.

Tags: newsletter, blogging, troy-hunt

Em dash

2026-02-15T21:40:46+00:00

I'm occasionally accused of using LLMs to write the content on my blog. I don't do that, and I don't think my writing has much of an LLM smell to it... with one notable exception:

    # Finally, do em dashes
    s = s.replace(' - ', u'\u2014')

That code to add em dashes to my posts dates back to at least 2015 when I ported my blog from an older version of Django (in a long-lost Mercurial repository) and started afresh on GitHub.

Tags: generative-ai, typography, blogging, ai, llms, python

How I automate my Substack newsletter with content from my blog

2025-11-19T22:00:34+00:00

I sent out my weekly-ish Substack newsletter this morning and took the opportunity to record a YouTube video demonstrating my process and describing the different components that make it work. There's a lot of digital duct tape involved, taking the content from Django+Heroku+PostgreSQL to GitHub Actions to SQLite+Datasette+Fly.io to JavaScript+Observable and finally to Substack.

The core process is the same as I described back in 2023. I have an Observable notebook called blog-to-newsletter which fetches content from my blog's database, filters out anything that has been in the newsletter before, formats what's left as HTML and offers a big "Copy rich text newsletter to clipboard" button.

I click that button, paste the result into the Substack editor, tweak a few things and hit send. The whole process usually takes just a few minutes.

I make very minor edits:

I set the title and the subheading for the newsletter. This is often a direct copy of the title of the featured blog post.
Substack turns YouTube URLs into embeds, which often isn't what I want - especially if I have a YouTube URL inside a code example.
Blocks of preformatted text often have an extra blank line at the end, which I remove.
Occasionally I'll make a content edit - removing a piece of content that doesn't fit the newsletter, or fixing a time reference like "yesterday" that doesn't make sense any more.
I pick the featured image for the newsletter and add some tags.

That's the whole process!

The Observable notebook

The most important cell in the Observable notebook is this one:

raw_content = {
  return await (
    await fetch(
      `https://datasette.simonwillison.net/simonwillisonblog.json?sql=${encodeURIComponent(
        sql
      )}&_shape=array&numdays=${numDays}`
    )
  ).json();
}

This uses the JavaScript fetch() function to pull data from my blog's Datasette instance, using a very complex SQL query that is composed elsewhere in the notebook.

Here's a link to see and execute that query directly in Datasette. It's 143 lines of convoluted SQL that assembles most of the HTML for the newsletter using SQLite string concatenation! An illustrative snippet:

with content as (
  select
    id,
    'entry' as type,
    title,
    created,
    slug,
    '<h3><a href="' || 'https://simonwillison.net/' || strftime('%Y/', created)
      || substr('JanFebMarAprMayJunJulAugSepOctNovDec', (strftime('%m', created) - 1) * 3 + 1, 3) 
      || '/' || cast(strftime('%d', created) as integer) || '/' || slug || '/' || '">' 
      || title || '</a> - ' || date(created) || '</h3>' || body
      as html,
    'null' as json,
    '' as external_url
  from blog_entry
  union all
  # ...

My blog's URLs look like /2025/Nov/18/gemini-3/ - this SQL constructs that three letter month abbreviation from the month number using a substring operation.

This is a terrible way to assemble HTML, but I've stuck with it because it amuses me.

The rest of the Observable notebook takes that data, filters out anything that links to content mentioned in the previous newsletters and composes it into a block of HTML that can be copied using that big button.

Here's the recipe it uses to turn HTML into rich text content on a clipboard suitable for Substack. I can't remember how I figured this out but it's very effective:

Object.assign(
  html`<button style="font-size: 1.4em; padding: 0.3em 1em; font-weight: bold;">Copy rich text newsletter to clipboard`,
  {
    onclick: () => {
      const htmlContent = newsletterHTML;
      // Create a temporary element to hold the HTML content
      const tempElement = document.createElement("div");
      tempElement.innerHTML = htmlContent;
      document.body.appendChild(tempElement);
      // Select the HTML content
      const range = document.createRange();
      range.selectNode(tempElement);
      // Copy the selected HTML content to the clipboard
      const selection = window.getSelection();
      selection.removeAllRanges();
      selection.addRange(range);
      document.execCommand("copy");
      selection.removeAllRanges();
      document.body.removeChild(tempElement);
    }
  }
)

From Django+Postgresql to Datasette+SQLite

My blog itself is a Django application hosted on Heroku, with data stored in Heroku PostgreSQL. Here's the source code for that Django application. I use the Django admin as my CMS.

Datasette provides a JSON API over a SQLite database... which means something needs to convert that PostgreSQL database into a SQLite database that Datasette can use.

My system for doing that lives in the simonw/simonwillisonblog-backup GitHub repository. It uses GitHub Actions on a schedule that executes every two hours, fetching the latest data from PostgreSQL and converting that to SQLite.

My db-to-sqlite tool is responsible for that conversion. I call it like this:

db-to-sqlite \
  $(heroku config:get DATABASE_URL -a simonwillisonblog | sed s/postgres:/postgresql+psycopg2:/) \
  simonwillisonblog.db \
  --table auth_permission \
  --table auth_user \
  --table blog_blogmark \
  --table blog_blogmark_tags \
  --table blog_entry \
  --table blog_entry_tags \
  --table blog_quotation \
  --table blog_quotation_tags \
  --table blog_note \
  --table blog_note_tags \
  --table blog_tag \
  --table blog_previoustagname \
  --table blog_series \
  --table django_content_type \
  --table redirects_redirect

That heroku config:get DATABASE_URL command uses Heroku credentials in an environment variable to fetch the database connection URL for my blog's PostgreSQL database (and fixes a small difference in the URL scheme).

db-to-sqlite can then export that data and write it to a SQLite database file called simonwillisonblog.db.

The --table options specify the tables that should be included in the export.

The repository does more than just that conversion: it also exports the resulting data to JSON files that live in the repository, which gives me a commit history of changes I make to my content. This is a cheap way to get a revision history of my blog content without having to mess around with detailed history tracking inside the Django application itself.

At the end of my GitHub Actions workflow is this code that publishes the resulting database to Datasette running on Fly.io using the datasette publish fly plugin:

datasette publish fly simonwillisonblog.db \
  -m metadata.yml \
  --app simonwillisonblog-backup \
  --branch 1.0a2 \
  --extra-options "--setting sql_time_limit_ms 15000 --setting truncate_cells_html 10000 --setting allow_facet off" \
  --install datasette-block-robots \
  # ... more plugins

As you can see, there are a lot of moving parts! Surprisingly it all mostly just works - I rarely have to intervene in the process, and the cost of those different components is pleasantly low.

Tags: blogging, django, javascript, postgresql, sql, sqlite, youtube, heroku, datasette, observable, github-actions, fly, newsletter, substack

Curiosity-driven blogging

2025-10-31T17:09:56+00:00

My piece this morning about the Marimo acquisition is an example of a variant of a TIL - I didn't know much about CoreWeave, the acquiring company, so I poked around to answer my own questions and then wrote up what I learned as a short post. Curiosity-driven blogging if you like.

Tags: til, blogging

The SIFT method

2025-09-07T20:51:31+00:00

The SIFT method

The SIFT method is "an evaluation strategy developed by digital literacy expert, Mike Caulfield, to help determine whether online content can be trusted for credible or reliable sources of information."

This looks extremely useful as a framework for helping people more effectively consume information online (increasingly gathered with the help of LLMs).

Stop. "Be aware of your emotional response to the headline or information in the article" to protect against clickbait, and don't read further or share until you've applied the other three steps.
Investigate the Source. Apply lateral reading, checking what others say about the source rather than just trusting their "about" page.
Find Better Coverage. "Use lateral reading to see if you can find other sources corroborating the same information or disputing it" and consult trusted fact checkers if necessary.
Trace Claims, Quotes, and Media to their Original Context. Try to find the original report or referenced material to learn more and check it isn't being represented out of context.

This framework really resonates with me: it formally captures and improves on a bunch of informal techniques I've tried to apply in my own work.

Via @anildash.com

Tags: blogging, research, ai-assisted-search, digital-literacy

New tags

2025-07-19T02:02:54+00:00

A few months ago I added a tool to my blog for bulk-applying tags to old content. It works as an extension to my existing search interface, letting me run searches and then quickly apply a tag to relevant results.

Since adding this I've been much more aggressive in categorizing my older content, including adding new tags when I spot an interesting trend that warrants its own page.

Today I added system-prompts and applied it to 41 existing posts that talk about system prompts for LLM systems, including a bunch that directly quote system prompts that have been deliberately published or leaked.

Other tags I've added recently include press-quotes for times I've been quoted in the press, agent-definitions for my ongoing collection of different ways people define "agents" and paper-review for posts where I review an academic paper.

Tags: blogging, tagging

Generationship: Ep. #39, Simon Willison

2025-07-11T05:33:06+00:00

Generationship: Ep. #39, Simon Willison

I recorded this podcast episode with Rachel Chalmers a few weeks ago. We talked about the resurgence of blogging, the legacy of Google Reader, learning in public, LLMs as weirdly confident interns, AI-assisted search, prompt injection, human augmentation over replacement and we finished with this delightful aside about pelicans which I'll quote here in full:

Rachel: My last question, my favorite question. If you had a generation ship, a star ship that takes more than a human generation to get to Alpha Centauri, what would you call it?

Simon: I'd call it Squadron, because that is the collective noun for pelicans. And I love pelicans.

Rachel: Pelicans are the best.

Simon: They're the best. I live in Half Moon Bay. We have the second largest mega roost of the California brown pelican in the world, in our local harbor [...] last year we had over a thousand pelicans diving into the water at the same time at peak anchovy season or whatever it was.

The largest mega roost, because I know you want to know, is in Alameda, over by the aircraft carrier.

Rachel: The hornet.

Simon: Yeah. It's got the largest mega roost of the California brown pelican at certain times of the year. They're so photogenic. They've got charisma. They don't look properly shaped for flying.

Rachel: They look like the Spruce Goose. They've got the big front. And they look like they're made of wood.

Simon: That's such a great comparison, because I saw the Spruce Goose a couple of years ago. Up in Portland, there's this museum that has the Spruce Goose, and I went to see it. And it's incredible. Everyone makes fun of the Spruce Goose until you see the thing. And it's this colossal, beautiful wooden aircraft. Until recently it was the largest aircraft in the world. And it's such a stunning vehicle.

So yeah, pelicans and the Spruce Goose. I'm going to go with that one.

Tags: blogging, ai, generative-ai, llms, half-moon-bay, podcast-appearances

Disclosures

2025-06-23T18:06:02+00:00

I've added a Disclosures section to my about page, listing my various sources of income and the companies that directly sponsor my work or have supported it in the recent past.

I do not receive any compensation writing about specific topics on this blog - no sponsored content! I plan to continue this policy. If I ever change this I will disclose that both here and in the post itself. [...]

I see my credibility as one of my most valuable assets, so it's important to be transparent about how financial interests may influence my writing here.

I took inspiration from Molly White's disclosures page.

Tags: blogging, molly-white

Blogging about papers

2025-06-13T16:22:05+00:00

My post this morning about Design Patterns for Securing LLM Agents against Prompt Injections is an example of a blogging format I'd love to see more of: informal but informed commentary on academic papers.

Academic papers are generally hard to read. Sadly that's almost a requirement of the format: the incentives for publishing papers that make it through peer review are often at odds with producing text that's easy for non-academics to digest.

(This new Design Patterns paper bucks that trend, the writing is clear, it’s enjoyable to read and the target audience clearly includes practitioners, not just other researchers.)

In addition to breaking a paper down into more digestible chunks, writing about papers offers an extremely valuable filter. There are hundreds of new papers published every day: seeing someone who's work you respect confirm that a paper is worth your time is a really strong signal.

I added a paper-review tag this morning, gathering six posts where I’ve attempted this kind of review. Notes on the SQLite DuckDB paper in September 2022 was my first.

I apply the same principle to these as my link blog: try to add something extra, so that anyone who reads both my post and the paper itself gets a little bit of extra value from my notes.

Tags: paper-review, blogging

It's this blog's 23rd birthday

2025-06-12T21:31:14+00:00

It's this blog's 23rd birthday today!

On June 12th 2022 I celebrated Twenty years of my blog with a big post full of highlights. Looking back now I'm amused to notice that my 20th birthday post came within two weeks of my earliest writing about LLMs: A Datasette tutorial written by GPT-3 and How to use the GPT-3 language model.

My generative-ai tag has reached 1,184 posts now.

I really do feel like blogging is onto its second wind. The amount of influence you can have on the world by consistently blogging about a subject is just as high today as it was back in the 2000s when blogging first started.

The best time to start a blog may have been twenty years ago, but the second best time to start a blog is today.

Tags: generative-ai, blogging

First monthly sponsor newsletter tomorrow

2025-05-29T04:33:32+00:00

I'll be sending out my first curated monthly highlights newsletter tomorrow, only to $10/month and up sponsors. Sign up now if you want to pay me to send you less!

My weekly-ish newsletter remains free, in fact I just sent out the latest edition.

Tags: blogging, newsletter

Subscribe to my sponsors-only monthly newsletter.

2025-05-25T06:06:17+00:00

Subscribe to my sponsors-only monthly newsletter

I’ve never liked the idea of charging for my content. I get enormous value from putting all of my writing and research out there for free.

So I’m trying something a little different: pay me to send you less.

I’m starting a sponsors-only monthly newsletter featuring just my heavily curated and edited highlights. If you only have ten minutes, what are the most important things not to miss from the last month?

Don’t want to pay? That’s fine, you can continue to follow my firehose for free!

Anyone who sponsors me for $10/month (or $50/month or more) on GitHub sponsors will receive my new newsletter on approximately the last day of the month. I’ll be sending out the first edition next week.

This blog and my newsletter will continue at their same breakneck pace. Paying subscribers can get a lower volume of stuff.

I'm cautiously optimistic that this could work. I've never liked the idea of business models that incentivize me to publish less. This feels like it encourages me to do what I'm doing already while giving people a rational reason to support my work, at a relatively small incremental cost to myself.

Tags: projects, blogging, github, email, newsletter

Quoting dynomight

2025-04-26T17:05:04+00:00

I don’t have a “mission” for this blog, but if I did, it would be to slightly increase the space in which people are calm and respectful and care about getting the facts right. I think we need more of this, and I’m worried that society is devolving into “trench warfare” where facts are just tools to be used when convenient for your political coalition, and everyone assumes everyone is distorting everything, all the time.

— dynomight

Tags: blogging

Another rant about companies not spying on you through your phone's microphone to serve you ads

2025-04-26T02:07:00+00:00

Last September I posted a series of long ranty comments on Lobste.rs about the latest instance of the immortal conspiracy theory (here it goes again) about apps spying on you through your microphone to serve you targeted ads.

On the basis that it's always a great idea to backfill content on your blog, I just extracted my best comments from that thread and turned them into this full post here, back-dated to September 2nd which is when I wrote the comments.

My rant was in response to the story In Leak, Facebook Partner Brags About Listening to Your Phone’s Microphone to Serve Ads for Stuff You Mention. Here's how it starts:

Which is more likely?

All of the conspiracy theories are real! The industry managed to keep the evidence from us for decades, but finally a marketing agency of a local newspaper chain has blown the lid off the whole thing, in a bunch of blog posts and PDFs and on a podcast.

Everyone believed that their phone was listening to them even when it wasn’t. The marketing agency of a local newspaper chain were the first group to be caught taking advantage of that widespread paranoia and use it to try and dupe people into spending money with them, despite the tech not actually working like that.

My money continues to be on number 2.

You can read the rest here. Or skip straight to why I think this matters so much:

Privacy is important. People who are sufficiently engaged need to be able to understand exactly what’s going on, so they can e.g. campaign for legislators to reign in the most egregious abuses.

I think it’s harmful letting people continue to believe things about privacy that are not true, when we should instead be helping them understand the things that are true.

Tags: privacy, blogging, microphone-ads-conspiracy

Backfill your blog

2025-04-25T15:30:37+00:00

Fun fact: there's no rule that says you can't create a new blog today and backfill (and backdate) it with your writing from other platforms or sources, even going back many years.

I'd love to see more people do this!

(Inspired by this tweet by John F. Wu introducing his new blog. I did this myself when I relaunched this blog back in 2017.)

Tags: blogging

Working Through the Fear of Being Seen

2025-04-22T06:40:49+00:00

Working Through the Fear of Being Seen

Heartfelt piece by Ashley Willis about the challenge of overcoming self-doubt in publishing online:

Part of that is knowing who might read it. A lot of the folks who follow me are smart, opinionated, and not always generous. Some are friends. Some are people I’ve looked up to. And some are just really loud on the internet. I saw someone the other day drag a certain writing style. That kind of judgment makes me want to shrink back and say, never mind.

Try to avoid being somebody who discourages others from sharing their thoughts.

Via @ashley.dev

Tags: blogging

Quoting Matt Webb

2025-04-05T04:56:31+00:00

Blogging is small-p political again, today. It’s come back round. It’s a statement to put your words in a place where they are not subject to someone else’s algorithm telling you what success looks like; when you blog, your words are not a vote for the values of someone else’s platform.

— Matt Webb, Interview for People and Blogs

Tags: matt-webb, blogging

Note on 26th March 2025

2025-03-26T06:11:30+00:00

I've added a new content type to my blog: notes. These join my existing types: entries, bookmarks and quotations.

A note is a little bit like a bookmark without a link. They're for short form writing - thoughts or images that don't warrant a full entry with a title. The kind of things I used to post to Twitter, but that don't feel right to cross-post to multiple social networks (Mastodon and Bluesky, for example.)

I was partly inspired by Molly White's short thoughts, notes, links, and musings.

I've been thinking about this for a while, but the amount of work involved in modifying all of the parts of my site that handle the three different content types was daunting. Then this evening I tried running my blog's source code (using files-to-prompt and LLM) through the new Gemini 2.5 Pro:

files-to-prompt . -e py -c | \
  llm -m gemini-2.5-pro-exp-03-25 -s \
  'I want to add a new type of content called a Note,
  similar to quotation and bookmark and entry but it
  only has a markdown text body. Output all of the
  code I need to add for that feature and tell me
  which files to add  the code to.'

Gemini gave me a detailed 13 step plan covering all of the tedious changes I'd been avoiding having to figure out!

The code is in this PR, which touched 18 different files. The whole project took around 45 minutes start to finish.

(I used Claude to brainstorm names for the feature - I had it come up with possible nouns and then "rank those by least pretentious to most pretentious", and "notes" came out on top.)

This is now far too long for a note and should really be upgraded to an entry, but I need to post a first note to make sure everything is working as it should.

Tags: blogging, projects, gemini, ai-assisted-programming, claude, molly-white, files-to-prompt

Calling a wrap on my weeknotes

2025-03-20T04:12:32+00:00

After 192 posts that ranged from weekly to roughly once-a-month, I've decided to call a wrap on my weeknotes habit. The original goal was to stay transparent during my 2019-2020 JSK fellowship, and I kept them up after that as an accountability mechanism and to get into a habit of writing regularly.

Over the past two years I've adopted new posting habits which are solving those problems in other ways:

I post something here almost every day. I actually maintained a daily posting streak throughout 2024, which I ended in January, but I'm still posting most days and plan to keep that up.
Every time I ship a new release of one of my projects I link to it from here. This replaces the "recent releases" section of my weeknotes.
I try to have a longer form piece of writing that's suitable for inclusion in my newsletter at least once every two weeks. That's another accountability mechanism that's working well for me.

One downside of weeknotes is that I'd sometimes save something to include in them, which could lead to several items getting bundled together in a way that reduced their potential impact as standalone posts.

I got to the point with weeknotes where I was feeling guilty about not keeping them up. Given the volume of content I'm publishing already that felt like a sign that they were no longer providing the value they once did!

I still think weeknotes are an excellent habit for anyone who wants to write more frequently and be more transparent about their work. It feels healthy to be able to end a habit that's finished serving its purpose.

Tags: blogging, weeknotes, streaks

Quoting Matt Webb

2025-02-19T13:50:16+00:00

Meanwhile blogging has become small-p political again.

Slowly, slowly, the web was taken over by platforms. Your feeling of success is based on your platform’s algorithm, which may not have your interests at heart. Feeding your words to a platform is a vote for its values, whether you like it or not. And they roach-motel you by owning your audience, making you feel that it’s a good trade because you get “discovery.” (Though I know that chasing popularity is a fool’s dream.)

Writing a blog on your own site is a way to escape all of that. Plus your words build up over time. That’s unique. Nobody else values your words like you do.

Blogs are a backwater (the web itself is a backwater) but keeping one is a statement of how being online can work. Blogging as a kind of Amish performance of a better life.

— Matt Webb, Reflections on 25 years of Interconnected

Tags: matt-webb, blogging, social-media

tc39/proposal-regex-escaping

2025-02-18T21:53:56+00:00

tc39/proposal-regex-escaping

I just heard from Kris Kowal that this proposal for ECMAScript has been approved for ECMA TC-39:

Almost 20 years later, @simon’s RegExp.escape idea comes to fruition. This reached “Stage 4” at ECMA TC-39 just now, which formalizes that multiple browsers have shipped the feature and it’s in the next revision of the JavaScript specification.

I'll be honest, I had completely forgotten about my 2006 blog entry Escaping regular expression characters in JavaScript where I proposed that JavaScript should have an equivalent of the Python re.escape() function.

It turns out my post was referenced in this 15 year old thread on the esdiscuss mailing list, which evolved over time into a proposal which turned into implementations in Safari, Firefox and soon Chrome - here's the commit landing it in v8 on February 12th 2025.

One of the best things about having a long-running blog is that sometimes posts you forgot about over a decade ago turn out to have a life of their own.

Tags: blogging, ecmascript, javascript, regular-expressions, standards

Build a link blog

2025-02-04T16:14:33+00:00

Build a link blog

Xuanwo started a link blog inspired by my article My approach to running a link blog, and in a delightful piece of recursion his first post is a link blog entry about my post about link blogging, following my tips on quoting liberally and including extra commentary.

I decided to follow simon's approach to creating a link blog, where I can share interesting links I find on the internet along with my own comments and thoughts about them.

Via Hacker News

Tags: blogging

Friday Squid Blogging: Anniversary Post

2025-01-04T16:21:51+00:00

Friday Squid Blogging: Anniversary Post

Bruce Schneier:

I made my first squid post nineteen years ago this week. Between then and now, I posted something about squid every week (with maybe only a few exceptions). There is a lot out there about squid, even more if you count the other meanings of the word.

I think that's 1,004 posts about squid in 19 years. Talk about a legendary streak!

Tags: blogging, bruce-schneier, streaks

Ending a year long posting streak

2025-01-02T00:25:34+00:00

A year ago today I wrote about Tom Scott's legendary 10 year YouTube streak, in which he posted a new video once a week for the next ten years. Inspired by that, I also started my own.

I set myself the goal of posting something to my blog every day for a year.

Given how much happened in my chosen field of Large Language Models over the course of 2024 this wasn't as hard as I had expected!

One of the lessons I learned from Tom is that it's much healthier for a streak to have a predetermined end - that way the streak can act as a goal that doesn't turn into an ongoing imposition.

I'm calling it: this streak is done. According to my custom dashboard I hit 367 days - December 31st 2023 to December 31st 2024, inclusive (it was a leap year) - 1,151 posts in total.

I'm going to drop back to a much more reasonable target of at least one long-form post per week and at least three days per week with a link or quote - see My approach to running a link blog for how I think about that kind of content.

Posting daily has been fun, but it definitely impacted my productivity on my other projects. My blog runs on UTC so it also resulted in a minor panic coming up to 4pm Pacific coast time if I hadn't posted anything yet!

Almost every post in the streak came out in 2024, so my faceted search engine for 2024 provides a way to explore them.

My 2024 archive page also serves up this illustrative tag cloud:

Tags: blogging, tom-scott, streaks

My approach to running a link blog

2024-12-22T18:37:16+00:00

I started running a basic link blog on this domain back in November 2003 - publishing links (which I called "blogmarks") with a title, URL, short snippet of commentary and a "via" link where appropriate.

So far I've published 7,607 link blog posts and counting.

In April of this year I finally upgraded my link blog to support Markdown, allowing me to expand my link blog into something with a lot more room.

The way I use my link blog has evolved substantially in the eight months since then. I'm going to describe the informal set of guidelines I've set myself for how I link blog, in the hope that it might encourage other people to give this a try themselves.

Writing about things I've found

Back in November 2022 I wrote What to blog about, which started with this:

You should start a blog. Having your own little corner of the internet is good for the soul!

The point of that article was to emphasize that blogging doesn't have to be about unique insights. The value is in writing frequently and having something to show for it over time - worthwhile even if you don't attract much of an audience (or any audience at all).

In that article I proposed two categories of content that are low stakes and high value: things I learned and descriptions of my projects.

I realize now that link blogging deserves to be included a third category of low stakes, high value writing. We could think of that category as things I've found.

That's the purpose of my link blog: it's an ongoing log of things I've found - effectively a combination of public bookmarks and my own thoughts and commentary on why those things are interesting.

Trying to add something extra

When I first started link blogging I would often post a link with a one sentence summary of the linked content, and maybe a tiny piece of opinionated commentary.

After I upgraded my link blog to support additional markup (links, images, quotations) I decided to be more ambitious. Here are some of the things I try to do:

I always include the names of the people who created the content I am linking to, if I can figure that out. Credit is really important, and it's also useful for myself because I can later search for someone's name and find other interesting things they have created that I linked to in the past. If I've linked to someone's work three or more times I also try to notice and upgrade them to a dedicated tag.
I try to add something extra. My goal with any link blog post is that if you read both my post and the source material you'll have an enhanced experience over if you read just the source material itself.
- Ideally I'd like you to take something useful away even if you don't follow the link itself. This can be a slightly tricky balance: I don't want to steal attention from the authors and plagiarize their message. Generally I'll try to find some key idea that's worth emphasizing. Slightly cynically, I may try to capture that idea as backup against the original source vanishing from the internet. Link rot is real!
- My most basic version of this is trying to provide context as to why I think this particular thing is worth reading - especially important for longer content. A good recent example is my post about Anthropic's Building effective agents essay the other day.
- I might tie it together to other similar concepts, including things I've written about in the past, for example linking Prompt caching with Claude to my coverage of Context caching for Google Gemini.
- If part of the material is a video, I might quote a snippet of the transcript (often extracted using MacWhisper) like I did in this post about Anthropic's Clio.
- A lot of stuff I link to involves programming. I'll often include a direct link to relevant code, using the GitHub feature where I can link to a snippet as-of a particular commit. One example is the fetch-rss.py link in this post.
I'm liberal with quotations. Finding and quoting a paragraph that captures the key theme of a post is a very quick and effective way to summarize it and help people decide if it's worth reading the whole thing. My post on François Chollet's o3 ARC-AGI analysis is an example of that.
If the original author reads my post, I want them to feel good about it. I know from my own experience that often when you publish something online the silence can be deafening. Knowing that someone else read, appreciated, understood and then shared your work can be very pleasant.
A slightly self-involved concern I have is that I like to prove that I've read it. This is more for me than for anyone else: I don't like to recommend something if I've not read that thing myself, and sticking in a detail that shows I read past the first paragraph helps keep me honest about that.
I've started leaning more into screenshots and even short video or audio clips. A screenshot can be considered a visual quotation - I'll sometimes snap these from interesting frames in a YouTube video or live demo associated with the content I'm linking to. I used a screenshot of the Clay debugger in my post about Clay.

There are a lot of great link blogs out there, but the one that has influenced me the most in how I approach my own is John Gruber's Daring Fireball. I really like the way he mixes commentary, quotations and value-added relevant information.

The technology

The technology behind my link blog is probably the least interesting thing about it. It's part of my simonwillisonblog Django application - the main model is called Blogmark and it inherits from a BaseModel defining things like tags and draft modes that are shared across my other types of content (entries and quotations).

I use the Django Admin to create and edit entries, configured here.

The most cumbersome part of link blogging for me right now is images. I convert these into smaller JPEGs using a tiny custom tool I built (with Claude), then upload them to my static.simonwillison.net S3 bucket using Transmit and drop them into my posts using a Markdown image reference. I generate a first draft of the alt text using a Claude Project with these custom instructions, then usually make a few changes before including that in the markup. At some point I'll wire together a UI that makes this process a little smoother.

That static.simonwillison.net bucket is then served via Cloudflare's free tier, which means I effectively never have to think about the cost of serving up those image files.

I wrote up a TIL about Building a blog in Django a while ago which describes a similar setup to the one I'm using for my link blog, including how the RSS feed works (using Django's syndication framework).

The most technically interesting component is my search feature. I wrote about how that works in Implementing faceted search with Django and PostgreSQL - the most recent code for that can be found in blog/search.py on GitHub.

One of the most useful small enhancements I added was draft mode, which lets me assign a URL to an item and preview it in my browser without publishing it to the world. This really helps when I am editing posts on my mobile phone as it gives me a reliable preview so I can check for any markup mistakes.

I also send out an approximately weekly email newsletter version of my blog, for people who want to subscribe in their inbox. This is a straight copy of content from my blog - Substack doesn't have an API for this but their editor does accept copy and paste, so I have a delightful digital duct tape solution for assembling the newsletter which I described in Semi-automating a Substack newsletter with an Observable notebook.

More people should do this

I posted this on Bluesky last night:

I wish people would post more links to interesting things

I feel like Twitter and LinkedIn and Instagram and TikTok have pushed a lot of people out of the habit of doing that, by penalizing shared links in the various "algorithms"

Bluesky doesn't have that misfeature, thankfully!

(In my ideal world everyone would get their own link blog too, but sharing links on Bluesky and Mastodon is almost as good)

Sharing interesting links with commentary is a low effort, high value way to contribute to internet life at large.

Tags: blogging, django, django-admin, john-gruber

Please publish and share more

2024-11-02T15:17:07+00:00

Please publish and share more

💯 to all of this by Jeff Triplett:

Friends, I encourage you to publish more, indirectly meaning you should write more and then share it. [...]

You don’t have to change the world with every post. You might publish a quick thought or two that helps encourage someone else to try something new, listen to a new song, or binge-watch a new series.

Jeff shares my opinion on conclusions: giving myself permission to hit publish even when I haven't wrapped everything up neatly was a huge productivity boost for me:

Our posts are done when you say they are. You do not have to fret about sticking to landing and having a perfect conclusion. Your posts, like this post, are done after we stop writing.

And another 💯 to this footnote:

PS: Write and publish before you write your own static site generator or perfect blogging platform. We have lost billions of good writers to this side quest because they spend all their time working on the platform instead of writing.

Tags: blogging, jeff-triplett

Matt Webb's Colophon

2024-10-29T04:59:47+00:00

Matt Webb's Colophon

I love a good colophon (here's mine, I should really expand it). Matt Webb has been publishing his thoughts online for 24 years, so his colophon is a delightful accumulation of ideas and principles.

So following the principles of web longevity, what matters is the data, i.e. the posts, and simplicity. I want to minimise maintenance, not panic if a post gets popular, and be able to add new features without thinking too hard. [...]

I don’t deliberately choose boring technology but I think a lot about longevity on the web (that’s me writing about it in 2017) and boring technology is a consequence.

I'm tempted to adopt Matt's XSL template that he uses to style his RSS feed for my own sites.

Tags: blogging, matt-webb, rss, xslt, boring-technology