12 items tagged “concurrency”
Real Multithreading is Coming to Python—Learn How You Can Use It Now (via) Martin Heinz provides a detailed tutorial on trying out the new Per-Interpreter GIL feature that’s landing in Python 3.12, which allows Python code to run concurrently in multiple threads by spawning separate sub-interpreters, each with their own dedicated GIL.
It’s not an easy feature to play with yet! First you need to compile Python yourself, and then use APIs that are generally only available to C code (but should hopefully become available to Python code itself in Python 3.13).
Martin’s workaround for this is ingenious: it turns out the Python test.support package provides utility functions to help write tests against interpreters, and Martin shows how to abuse this module to launch, run and cleanup interpreters using regular Python code.
PEP 554 – Multiple Interpreters in the Stdlib: Shared data (via) Python 3.12 hopes to introduce multiple interpreters as part of the Python standard library, so Python code will be able to launch subinterpreters, each with their own independent GIL. This will allow Python code to execute on multiple CPU cores at the same time while ensuring existing code (and C modules) that rely on the GIL continue to work.
The obvious question here is how data will be shared between those interpreters. This PEP proposes a channels mechanism, where channels can be used to send just basic Python types between interpreters: None, bytes, str, int and channels themselves (I wonder why not floats?) # 20th September 2022, 1:25 am
The GIL and its effects on Python multithreading (via) Victor Skvortsov presents the most in-depth explanation of the Python Global Interpreter Lock I’ve seen anywhere. I learned a ton from reading this. # 29th September 2021, 5:23 pm
No, because Scala is harder to master than Java.[... 54 words]
What server do I need to handle 1000+ users simultaneously while they can post messages, upload pictures, and other similar stuff on a website based on PHP and mySQL?
You don’t need to handle 1,000 users simultaneously: you need to build something and ship it and start the process of discovering what you can build that will attract that many users. Seriously: don’t even start worrying about that kind of scale until you know you’re going to need it.[... 138 words]
Distributed lock on top of memcached. A simple Python context manager (taking advantage of the with statement) that implements a distributed lock using memcached to store lock state: “memcached_lock can be used to ensure that some global data is only updated by one server”. Redis would work well for this kind of thing as well. # 1st February 2010, 10:15 am
The Go Programming Language. A brand new systems programming language, designed by Robert Griesemer and Unix/Plan 9 veterans Rob Pike and Ken Thompson and funded by Google. Concurrency is supported by lightweight communicating processes called goroutines. “It feels like a dynamic language but has the speed and safety of a static language.” # 11th November 2009, 7 am
Testing Django Views for Concurrency Issues. Neat decorator for executing a Django view under high concurrency in your unit tests, to help spot errors caused by database race conditions that should be executed inside a transaction. # 27th May 2009, 10:01 am
A Curious Course on Coroutines and Concurrency. David Beazley’s sequel to last year’s mind-expanding “Generator Tricks for System Programmers”, perfect for if you’ve ever puzzled over what exactly you can use Python’s generator-based coroutine support for. # 24th April 2009, 10:58 am
django-springsteen and Distributed Search. Will Larson’s Django search library currently just talks to Yahoo! BOSS, but is designed to be extensible for other external search services. Interestingly, it uses threads to fire off several HTTP requests in parallel from within the Django view. # 25th February 2009, 10:28 pm