Simon Willison's Weblog: rails

Shopify/liquid: Performance: 53% faster parse+render, 61% fewer allocations

2026-03-13T03:44:34+00:00

Shopify/liquid: Performance: 53% faster parse+render, 61% fewer allocations

PR from Shopify CEO Tobias Lütke against Liquid, Shopify's open source Ruby template engine that was somewhat inspired by Django when Tobi first created it back in 2005.

Tobi found dozens of new performance micro-optimizations using a variant of autoresearch, Andrej Karpathy's new system for having a coding agent run hundreds of semi-autonomous experiments to find new effective techniques for training nanochat.

Tobi's implementation started two days ago with this autoresearch.md prompt file and an autoresearch.sh script for the agent to run to execute the test suite and report on benchmark scores.

The PR now lists 93 commits from around 120 automated experiments. The PR description lists what worked in detail - some examples:

Replaced StringScanner tokenizer with String#byteindex. Single-byte byteindex searching is ~40% faster than regex-based skip_until. This alone reduced parse time by ~12%.

Pure-byte parse_tag_token. Eliminated the costly StringScanner#string= reset that was called for every {% %} token (878 times). Manual byte scanning for tag name + markup extraction is faster than resetting and re-scanning via StringScanner. [...]

Cached small integer to_s. Pre-computed frozen strings for 0-999 avoid 267 Integer#to_s allocations per render.

This all added up to a 53% improvement on benchmarks - truly impressive for a codebase that's been tweaked by hundreds of contributors over 20 years.

I think this illustrates a number of interesting ideas:

Having a robust test suite - in this case 974 unit tests - is a massive unlock for working with coding agents. This kind of research effort would not be possible without first having a tried and tested suite of tests.
The autoresearch pattern - where an agent brainstorms a multitude of potential improvements and then experiments with them one at a time - is really effective.
If you provide an agent with a benchmarking script "make it faster" becomes an actionable goal.
CEOs can code again! Tobi has always been more hands-on than most, but this is a much more significant contribution than anyone would expect from the leader of a company with 7,500+ employees. I've seen this pattern play out a lot over the past few months: coding agents make it feasible for people in high-interruption roles to productively work with code again.

Here's Tobi's GitHub contribution graph for the past year, showing a significant uptick following that November 2025 inflection point when coding agents got really good.

He used Pi as the coding agent and released a new pi-autoresearch plugin in collaboration with David Cortés, which maintains state in an autoresearch.jsonl file like this one.

Via @tobi

Tags: django, performance, rails, ruby, ai, andrej-karpathy, generative-ai, llms, ai-assisted-programming, coding-agents, agentic-engineering, november-2025-inflection, tobias-lutke

Quoting Ally Piechowski

2026-03-06T21:58:33+00:00

Questions for developers:

“What’s the one area you’re afraid to touch?”

“When’s the last time you deployed on a Friday?”

“What broke in production in the last 90 days that wasn’t caught by tests?”

Questions for the CTO/EM:

“What feature has been blocked for over a year?”

“Do you have real-time error visibility right now?”

“What was the last feature that took significantly longer than estimated?”

Questions for business stakeholders:

“Are there features that got quietly turned off and never came back?”

“Are there things you’ve stopped promising customers?”

— Ally Piechowski, How to Audit a Rails Codebase

Tags: rails, software-engineering, technical-debt

Supercharge the One Person Framework with SQLite: Rails World 2024

2024-10-16T22:24:45+00:00

Supercharge the One Person Framework with SQLite: Rails World 2024

Stephen Margheim shares an annotated transcript of the YouTube video of his recent talk at this year's Rails World conference in Toronto.

The Rails community is leaning hard into SQLite right now. Stephen's talk is some of the most effective evangelism I've seen anywhere for SQLite as a production database for web applications, highlighting several new changes in Rails 8:

... there are two additions coming with Rails 8 that merit closer consideration. Because these changes make Rails 8 the first version of Rails (and, as far as I know, the first version of any web framework) that provides a fully production-ready SQLite experience out-of-the-box.

Those changes: Ensure SQLite transaction default to IMMEDIATE mode to avoid "database is locked" errors when a deferred transaction attempts to upgrade itself with a write lock (discussed here previously, and added to Datasette 1.0a14 in August) and SQLite non-GVL-blocking, fair retry interval busy handler - a lower-level change that ensures SQLite's busy handler doesn't hold Ruby's Global VM Lock (the Ruby version of Python's GIL) while a thread is waiting on a SQLite lock.

The rest of the talk makes a passionate and convincing case for SQLite as an option for production deployments, in line with the Rails goal of being a One Person Framework - "a toolkit so powerful that it allows a single individual to create modern applications upon which they might build a competitive business".

Back in April Stephen published SQLite on Rails: The how and why of optimal performance describing some of these challenges in more detail (including the best explanation I've seen anywhere of BEGIN IMMEDIATE TRANSACTION) and promising:

Unfortunately, running SQLite on Rails out-of-the-box isn’t viable today. But, with a bit of tweaking and fine-tuning, you can ship a very performant, resilient Rails application with SQLite. And my personal goal for Rails 8 is to make the out-of-the-box experience fully production-ready.

It looks like he achieved that goal!

Via Hacker News

Tags: gil, rails, ruby, scaling, sqlite, sqlite-busy

What's New in Ruby on Rails 8

2024-10-07T19:17:47+00:00

What's New in Ruby on Rails 8

Rails 8 takes SQLite from a lightweight development tool to a reliable choice for production use, thanks to extensive work on the SQLite adapter and Ruby driver.

With the introduction of the solid adapters discussed above, SQLite now has the capability to power Action Cable, Rails.cache, and Active Job effectively, expanding its role beyond just prototyping or testing environments. [...]

Transactions default to IMMEDIATE mode to improve concurrency.

Also included in Rails 8: Kamal, a new automated deployment system by 37signals for self-hosting web applications on hardware or virtual servers:

Kamal basically is Capistrano for Containers, without the need to carefully prepare servers in advance. No need to ensure that the servers have just the right version of Ruby or other dependencies you need. That all lives in the Docker image now. You can boot a brand new Ubuntu (or whatever) server, add it to the list of servers in Kamal, and it’ll be auto-provisioned with Docker, and run right away.

At 37signals, we're building a growing suite of apps that use SQLite in production with ONCE. There are now thousands of installations of both Campfire and Writebook running in the wild that all run SQLite. This has meant a lot of real-world pressure on ensuring that Rails (and Ruby) is working that wonderful file-based database as well as it can be. Through proper defaults like WAL and IMMEDIATE mode. Special thanks to Stephen Margheim for a slew of such improvements and Mike Dalessio for solving a last-minute SQLite file corruption issue in the Ruby driver.

Via Hacker News

Tags: 37-signals, rails, ruby, sqlite, docker, sqlite-busy

Running GitHub on Rails 6.0

2021-08-06T16:30:59+00:00

Running GitHub on Rails 6.0

Back in 2019 Eileen M. Uchitelle explained how GitHub upgraded everything in production to Rails 6.0 within 1.5 weeks of the stable release. There’s a trick in here I really like: they have an automated weekly job which fetches the latest Rails main branch and runs the full GitHub test suite against it, giving them super-early warnings about anything that might break and letting them provide feedback to upstream about unintended regressions.

Tags: continuous-integration, github, rails

Is there a substantial difference between using a Mac or a Windows machine for web development (particularly RoR)?

2013-12-04T09:54:00+00:00

My answer to Is there a substantial difference between using a Mac or a Windows machine for web development (particularly RoR)? on Quora

No matter if you are on Mac or Windows you should be using a Linux virtual machine for development, ideally running the same operating system as you deployment environment (I like Ubuntu for this). Vagrant is a popular tool for managing this kind of setup.

That said, for web development you should be on a Mac, with Windows running in a virtual machine. You need the Mac in order to test in desktop Safari and use the iOS simulator to test in Mobile Safari on iPhone and iPad.

I don't think it is possible to run OS X in a VM on Windows without using cracked versions of the installation disks, and you have no guarantees that technique you use will continue working for later OS versions.

Tags: programming, rails, web-development, quora

How can one become a masterful Rails developer (and still have a life)?

2013-10-22T09:20:00+00:00

My answer to How can one become a masterful Rails developer (and still have a life)? on Quora

Don't sacrifice your social life. Sacrifice TV.

Most people manage to maintain an active social life while also watching multiple hours of TV every day. Imagine how much Rails you could learn if you dedicated a couple of hours to it every day for 6 months.

Also, build real projects. Reading tutorials doesn't count if you're not also writing code.

Tags: programming, rails, ruby, web-development, quora

What is the best framework to use, Yii or Ruby on Rails?

2013-10-17T18:38:00+00:00

My answer to What is the best framework to use, Yii or Ruby on Rails? on Quora

This is a big decision, and it's worth taking the time to pick what's the best fit for you. I recommend going through the tutorials for each one, building the basic application they describe and seeing which made the most sense to you. As a Django developer, I suggest trying that framework too :)

If you just want a recommendation on Rails vs Yii, I would go for Rails. Ruby is a less frustrating language to develop in than PHP and has a higher quality collection of other open source libraries you can use in your project. Rails is also a more mature and more widely used framework.

Tags: php, programming, rails, startups, web-development, quora

Is it worth it for an aspiring web developer in NYC to attend the San Francisco Startup Jobs Fair in November? What I mean is what are my chances that a company would be interested in me, allowing that my code skills are up to snuff, if I don't live ...

2013-10-13T10:51:00+00:00

My answer to Is it worth it for an aspiring web developer in NYC to attend the San Francisco Startup Jobs Fair in November? What I mean is what are my chances that a company would be interested in me, allowing that my code skills are up to snuff, if I don't live out in SF yet? on Quora

Not being in SF already is no problem at all - demand for talented developers is crazy high, so provided you are a great developer the fact you would need to relocate won't be a barrier.

If you do travel to SF for the job fair, make sure you get as much out of the trip as possible - try to line up some in-person interviews with companies that look interesting to you in the days before and after the event itself.

Tags: rails, san-francisco, web-development, quora, nyc

What web programming framework best supports 'drag and drop' actions? Please give examples of sites and/or plug-ins that support the interaction.

2012-02-19T17:56:00+00:00

My answer to What web programming framework best supports 'drag and drop' actions? Please give examples of sites and/or plug-ins that support the interaction. on Quora

Drag and drop is a client-side thing - it has nothing to do with the server-side technology being used.

Take a look at the drag and drop implementation provided by the big open source JavaScript libraries - Dojo, YUI, jQuery UI all have solid implementations.

Tags: coffeescript, django, programming-languages, python, rails, quora

Why did Twitter have so many problems if it is made with Ruby?

2012-02-19T14:21:00+00:00

My answer to Why did Twitter have so many problems if it is made with Ruby? on Quora

The technical problem Twitter solves (distributing millions of short messages per minute to an enormous graph of follow relationships) is extremely hard in any language.

Tags: rails, ruby, twitter, quora

How long until Ruby developers are as cheap as PHP developers? is it already happening? should I still learn it or it only has a couple years left and I'm better off with SSJS?

2012-02-04T13:02:00+00:00

My answer to How long until Ruby developers are as cheap as PHP developers? is it already happening? should I still learn it or it only has a couple years left and I'm better off with SSJS? on Quora

If you want to be a highly paid engineer, you should worry less about your expertise in a specific language and more about developing broad and deep skills across a wider range of development topics.

- Have you used Redis, MongoDB, Riak and/or Cassandra?
- Can you set up and make use of a Continuous Integration server like Jenkins?
- Do you understand TDD, BDD and other testing techniques?
- How's your HTTP? Do you understand REST, caching headers, URL design, content negotiation, SPDY, WebSockets?
- Are you familiar with message queues, pubsub, zeromq?
- Do you understand basic web app security? XSS, CSRF, clickjacking, OWASP, escape-by-default templating?
- Do you know what's involved in client-side web performance optimisation (Steve Souders' High Performance Web Sites stuff)?

All of the above are, in my opinion, important skills for a senior web engineer. None of them are any different if you are using PHP, Ruby, Python or something else.

Tags: javascript, rails, ruby, web-development, quora, careers

Play Framework, Django or Rails? Which one do you recommend for Social Networking Web applications.

2012-01-14T09:22:00+00:00

My answer to Play Framework, Django or Rails? Which one do you recommend for Social Networking Web applications. on Quora

Both Rails and Django have been used for a large number of high profile social networking web applications. Off the top of my head, Django is used by Instagram and Pinterest, Rails is used by Posterous and Ravelry. I don't know what the largest sites built using Play are at the moment.

The hard problems involved in building a social network are essentially framework-agnostic: things like scaling an activity feed and running queries against large social graphs are solved by smart application architecture, not built-in framework functionality. Whether you use Rails, Django or Play will have almost no effect on how you solve those larger architectural challenges.

Pick the one you are most comfortable working with and start worrying about your site's functionality rather than its application framework.

Tags: django, frameworks, play, rails, social-networks, webapps, quora

Is there a framework that allows me to collect input from individual users, and then charge for the aggregate and analysis of that data?

2012-01-02T14:04:00+00:00

My answer to Is there a framework that allows me to collect input from individual users, and then charge for the aggregate and analysis of that data? on Quora

No - your needs are extremely specific. You're going to have to build this yourself.

(Even if there was an open source framework that covered the kind if thing you are talking about, my hunch is that by the time you had finished customising it to your exact needs you would have invested a similar amount of effort as you would building it from scratch)

Tags: frameworks, php, python, rails, quora

Can someone improve their knowledge of programming in Ruby or PHP by using a framework like Rails or Zend, or does the framework just do a lot of the work for you without giving you an opportunity to learn from it?

2011-12-04T13:04:00+00:00

My answer to Can someone improve their knowledge of programming in Ruby or PHP by using a framework like Rails or Zend, or does the framework just do a lot of the work for you without giving you an opportunity to learn from it? on Quora

Read the source, luke.

(By which I mean, the most effective way to become a better programmer is to read good code written by other programmers. You can learn a great deal from reading Rails/Zend/Django)

Tags: php, programming, rails, web-development, quora

CSRF: Flash + 307 redirect = Game Over

2011-02-10T22:07:00+00:00

CSRF: Flash + 307 redirect = Game Over

Here’s the exploit that Django and Rails both just released fixes for. It’s actually a flaw in the Flash player. Flash isn’t meant to be able to make cross-domain HTTP requests with custom HTTP headers unless the crossdomain.xml file on the other domain allows them to, but it turns out a 307 redirect (like a 302, but allows POST data to be forwarded) confuses the Flash player in to not checking the crossdomain.xml on the host it is being redirect to.

Tags: crossdomainxml, csrf, django, flash, rails, security, recovered

Is there an online calendar for all the Ruby / Rails conferences to be held in 2011?

2011-01-16T12:34:00+00:00

My answer to Is there an online calendar for all the Ruby / Rails conferences to be held in 2011? on Quora

Have you tried both our Ruby and our Rails topic pages?

http://lanyrd.com/topics/ruby-on...
http://lanyrd.com/topics/ruby/

We've got a pretty good collection now. Of course, if there's any that we're missing you can add them yourself, or just leave a comment on this question.

Tags: rails, ruby, quora, conference

How long would it take an average programmer to do a Ruby on Rails Reddit clone prototype?

2010-12-16T10:17:00+00:00

My answer to How long would it take an average programmer to do a Ruby on Rails Reddit clone prototype? on Quora

Just FYI, Reddit is an open source Python project: http://code.reddit.com/

Tags: prototyping, rails, web-development, quora

What are the tradeoffs (e.g. development speed, performance, scalability) between using various php frameworks, ruby/rails, or python/django? Is there any reason to choose one overwhelmingly over another?

2010-11-27T18:18:00+00:00

My answer to What are the tradeoffs (e.g. development speed, performance, scalability) between using various php frameworks, ruby/rails, or python/django? Is there any reason to choose one overwhelmingly over another? on Quora

At this point, I'd argue that the decision between them comes down to programming language rather than framework - the frameworks have mostly converged on a very similar set of features.

All three solutions scale in exactly the same way (shared nothing architecture).

If you're writing something for people to deploy on their own hosting plans, PHP is far more widely supported. For a service that you host yourself, you need to decide which language ecosystem is the best fit for how you like to develop.

Update 14 months later: I'd argue today that thanks to hosting platforms such as ep.io, Heroku and dotCloud PHP's hosting advantage isn't nearly as pronounced.

Tags: django, frameworks, php, python, rails, web-development, quora

Why is Diaspora built on Ruby on Rails instead of Node.js?

2010-11-27T10:29:00+00:00

My answer to Why is Diaspora built on Ruby on Rails instead of Node.js? on Quora

Because Node.js had almost no visibility at all six months ago when Diaspora started. Also, Node.js has only very recently stopped breaking API backwards compatibility on a regular basis. Plus the Ruby library ecosystem is much, much larger than the Node.js ecosystem.

I love Node.js, but it's still quite an immature technology for full stack web development.

(I'm not involved in the Diaspora project)

Tags: nodejs, rails, quora

Why is Java perceived as not cool for startups? We seem to be getting a lot of feedback lately that a startup should be using Ruby on Rails, PHP, Python, etc., if they want to be agile and iterate quickly.

2010-11-14T12:21:00+00:00

My answer to Why is Java perceived as not cool for startups? We seem to be getting a lot of feedback lately that a startup should be using Ruby on Rails, PHP, Python, etc., if they want to be agile and iterate quickly. on Quora

I believe what takes the longest time in software engineering is not actually writing the code, but untangling it to find bugs, and Java is much more conducive to that, and the tooling is much stronger than for other languages.

You should re-evaluate your beliefs. Dynamic language programmers spend a great deal of time thinking about code quality and maintainability. TDD (and BDD), which I believe was first popularised within the Ruby community) are extremely widespread, and profiling and debugging tools are widely used and constantly improved. A strong test suite provides far more effective protection against bugs than static typing and an IDE.

Tags: java, programming, programming-languages, rails, startups, static-typing, quora, careers

Hookbox

2010-07-29T09:48:00+00:00

Hookbox

For most web projects, I believe implementing any real-time comet features on a separate stack from the rest of the application makes sense—keep using Rails, Django or PHP for the bulk of the application logic, and offload any WebSocket or Comet requests to a separate stack built on top of something like Node.js, Twisted, EventMachine or Jetty. Hookbox is the best example of that philosophy I’ve yet seen—it’s a Comet server that makes WebHook requests back to your regular application stack to check if a user has permission to publish or subscribe to a given channel. “The key insight is that all application development with hookbox happens either in JavaScript or in the native language of the web application itself”.

Via Comet Daily

Tags: comet, django, javascript, michael-carter, php, rails, webhooks, recovered, hookbox

Node.js, redis, and resque

2010-02-28T23:02:23+00:00

Node.js, redis, and resque

Paul Gross has been experimenting with Node.js proxies for allowing web applications to be upgraded without missing any requests. Here he places all incoming HTTP requests in a redis queue, then has his backend Rails servers consume requests from the queue and push the responses back on to a queue for Node to deliver. When the backend application is upgraded, requests remain in the queue and users see a few seconds of delay before their request is handled. It’s not production ready yet (POST requests aren’t handled, for example) but it’s a very interesting approach.

Via Hacker News | Node.js, redis, and resque

Tags: highavailability, http, javascript, nodejs, rails, redis

Play framework for Java

2009-10-25T23:21:36+00:00

Play framework for Java

I’m genuinely impressed by this—it’s a full stack web framework for Java that actually does feel a lot like Django or Rails. Best feature: code changes are automatically detected and reloaded by the development web server, giving you the same save-and-refresh workflow you get in Django (no need to compile and redeploy to try out your latest changes).

Tags: django, frameworks, java, play, rails, web

How We Made GitHub Fast

2009-10-21T21:14:38+00:00

How We Made GitHub Fast

Detailed overview of the new GitHub architecture. It’s a lot more complicated than I would have expected—lots of moving parts are involved in ensuring they can scale horizontally when they need to. Interesting components include nginx, Unicorn, Rails, DRBD, HAProxy, Redis, Erlang, memcached, SSH, git and a bunch of interesting new open source projects produced by the GitHub team such as BERT/Ernie and ProxyMachine.

Tags: drbd, erlang, ernie, git, github, haproxy, memcached, nginx, proxymachine, rails, redis, replication, ruby, scaling, ssh, unicorn

nginx_http_push_module

2009-10-17T16:48:31+00:00

nginx_http_push_module

More clever design with webhooks—here’s an nginx module that provides a comet endpoint URL which will hang until a back end process POSTs to another URL on the same server. This makes it much easier to build asynchronous comet apps using regular synchronous web frameworks such as Django, PHP and Rails.

Tags: comet, django, nginx, php, rails, webhooks

XSS Protection by Default in Rails 3.0

2009-10-08T16:35:52+00:00

XSS Protection by Default in Rails 3.0

Fantastic news—congratulations, Rails core team.

Tags: rails, ruby, security, xss

Ravelry

2009-09-03T18:50:20+00:00

Ravelry

Tim Bray interviews Casey Forbes, the single engineer behind Ravelry, the knitting community that serves 10 million Rails requests a day using just seven physical servers, MySQL, Sphinx, memcached, nginx, haproxy, passenger and Tokyo Cabinet.

Tags: caseyforbes, haproxy, memcached, mysql, nginx, passenger, rails, ravelry, scaling, sphinx-search, tim-bray, tokyocabinet, tokyotyrant

cache-money

2009-06-28T15:17:30+00:00

cache-money

A “write-through caching library for ActiveRecord”, maintained by Nick Kallen from Twitter. Queries hit memcached first, and caches are automatically kept up-to-date when objects are created, updated and deleted. Only some queries are supported—joins and comparisons won’t hit the cache, for example.

Tags: activerecord, cachemoney, caching, memcached, rails, twitter

whine flu, railsmalefail 2009

2009-04-29T11:39:38+00:00

whine flu, railsmalefail 2009

Danny quotes the smartest take on the CouchDB/pr0n controversy: “It’s about presenting women as ’the other,’ not ’us.’ It would have been just as offensive if all the women shown were domineering mothers in aprons, shaking their fingers and threatening with rolling pins.”

Tags: couchdb, danny-obrien, rails, sexism