Flickr Engineers Do It Offline. Flickr wrote their own queuing mechanism (in PHP), and currently run ten queue servers on dedicated hardware for tasks like pushing new photos in to indexes, denormalisation and “backfills” which move data between clusters and run bulk scripts against large numbers of existing rows.
Recent articles
- Live blog: Claude 4 launch at Code with Claude - 22nd May 2025
- I really don't like ChatGPT's new memory dossier - 21st May 2025
- Building software on top of Large Language Models - 15th May 2025