Tuesday, 19th April 2022
Glue code to quickly copy data from one Postgres table to another (via) The Python script that Retool used to migrate 4TB of data between two PostgreSQL databases. I find the structure of this script really interesting—it uses Python to spin up a queue full of ID ranges to be transferred and then starts some threads, but then each thread shells out to a command that runs “psql COPY (SELECT ...) TO STDOUT” and pipes the result to “psql COPY xxx FROM STDIN”. Clearly this works really well (“saturate the database’s hardware capacity” according to a comment on HN), and neatly sidesteps any issues with Python’s GIL. # 4:57 pm
Netlify Edge Functions: A new serverless runtime powered by Deno. You can now run Deno scripts directly in Netlify’s edge CDN—bundled as part of their default pricing plan. Interesting that they decided to host it on Deno’s Deno Deploy infrastructure. The hello world example is pleasingly succinct:
export default () => new Response(“Hello world”) # 4:46 pm