12 items tagged “concurrency”
2023
Real Multithreading is Coming to Python—Learn How You Can Use It Now (via) Martin Heinz provides a detailed tutorial on trying out the new Per-Interpreter GIL feature that’s landing in Python 3.12, which allows Python code to run concurrently in multiple threads by spawning separate sub-interpreters, each with their own dedicated GIL.
It’s not an easy feature to play with yet! First you need to compile Python yourself, and then use APIs that are generally only available to C code (but should hopefully become available to Python code itself in Python 3.13).
Martin’s workaround for this is ingenious: it turns out the Python test.support package provides utility functions to help write tests against interpreters, and Martin shows how to abuse this module to launch, run and cleanup interpreters using regular Python code.
He also demonstrates test.support.interpreters.create_channel(), which can be used to create channels with receiver and sender ends, somewhat similar to Go. # 15th May 2023, 7:42 pm
2022
PEP 554 – Multiple Interpreters in the Stdlib: Shared data (via) Python 3.12 hopes to introduce multiple interpreters as part of the Python standard library, so Python code will be able to launch subinterpreters, each with their own independent GIL. This will allow Python code to execute on multiple CPU cores at the same time while ensuring existing code (and C modules) that rely on the GIL continue to work.
The obvious question here is how data will be shared between those interpreters. This PEP proposes a channels mechanism, where channels can be used to send just basic Python types between interpreters: None, bytes, str, int and channels themselves (I wonder why not floats?) # 20th September 2022, 1:25 am
2021
The GIL and its effects on Python multithreading (via) Victor Skvortsov presents the most in-depth explanation of the Python Global Interpreter Lock I’ve seen anywhere. I learned a ton from reading this. # 29th September 2021, 5:23 pm
2012
Can Scala gain wider usage than Java any time soon?
No, because Scala is harder to master than Java.
[... 54 words]What server do I need to handle 1000+ users simultaneously while they can post messages, upload pictures, and other similar stuff on a website based on PHP and mySQL?
You don’t need to handle 1,000 users simultaneously: you need to build something and ship it and start the process of discovering what you can build that will attract that many users. Seriously: don’t even start worrying about that kind of scale until you know you’re going to need it.
[... 138 words]2010
Distributed lock on top of memcached. A simple Python context manager (taking advantage of the with statement) that implements a distributed lock using memcached to store lock state: “memcached_lock can be used to ensure that some global data is only updated by one server”. Redis would work well for this kind of thing as well. # 1st February 2010, 10:15 am
2009
dustin’s gomemcached (via) A memcached server written in Go, an experiment by memcached maintainer Dustin Sallings. # 13th November 2009, 3:13 pm
The Go Programming Language. A brand new systems programming language, designed by Robert Griesemer and Unix/Plan 9 veterans Rob Pike and Ken Thompson and funded by Google. Concurrency is supported by lightweight communicating processes called goroutines. “It feels like a dynamic language but has the speed and safety of a static language.” # 11th November 2009, 7 am
Testing Django Views for Concurrency Issues. Neat decorator for executing a Django view under high concurrency in your unit tests, to help spot errors caused by database race conditions that should be executed inside a transaction. # 27th May 2009, 10:01 am
A Curious Course on Coroutines and Concurrency. David Beazley’s sequel to last year’s mind-expanding “Generator Tricks for System Programmers”, perfect for if you’ve ever puzzled over what exactly you can use Python’s generator-based coroutine support for. # 24th April 2009, 10:58 am
django-springsteen and Distributed Search. Will Larson’s Django search library currently just talks to Yahoo! BOSS, but is designed to be extensible for other external search services. Interestingly, it uses threads to fire off several HTTP requests in parallel from within the Django view. # 25th February 2009, 10:28 pm
2007
Mac OS X Leopard: Multicore. “... NSOperation, a breakthrough new API that optimizes applications for the world of multicore processing.” # 11th June 2007, 11:02 pm