home.social

Search

1000 results for “a_corbin”

  1. A CPU-bound Python job split across four threads took 3.6 seconds on my machine. The same job split across four of Python 3.14's new subinterpreters took 0.9 seconds, roughly what one thread alone takes to do a quarter of the work. That part matched what I expected from the PEP. What I did not expect was that asking one of those subinterpreters to run two things at once does not slow it down. It refuses outright.

    concurrent.interpreters is new in the standard library as of Python 3.14, released on 7 October 2026. Subinterpreters themselves are not new - they have existed in the C API since Python 1.5 and CPython has used them internally for years - but PEP 734 is what makes them usable from ordinary Python code, with no C extension required. Each subinterpreter gets its own Global Interpreter Lock, which is the part worth testing: it is the thing that is supposed to let CPU-bound Python code actually use more than one core without reaching for multiprocessing.

    I tested it against a CPU-bound workload (counting primes below 400,000, a plain Python double loop with no C extensions to hide behind), compared against threads, processes, and a single thread, three runs each, inside a python:3.14-slim container with four cores available.

    The headline number

    approachtime for 4x the worksingle thread (1x unit, for reference)0.77 - 0.80s4 OS threads, classic GIL3.51 - 3.70s4 subinterpreters (new in 3.14)0.89 - 0.98s4 processes (multiprocessing)0.95 - 1.30s


    Four threads doing four units of CPU-bound work took more than four times as long as one thread doing one unit - 3.6s against roughly 0.8s, which is actually a bit worse than the 3.2s you'd get from perfect serialisation, because the GIL has to keep stopping one thread to let another run. Four subinterpreters doing the same four units took about 0.9s: close enough to the single-unit baseline that the extra three units of work were, for practical purposes, free. That is real parallelism, not a scheduling trick, and it lands in the same place multiprocessing already does.

    What it refuses: one thread per interpreter, not shared


    The PEP's selling point is that each interpreter has its own GIL. I wanted to see what happens when two threads try to use one interpreter's GIL at the same time, expecting it to behave like classic Python: both threads run, the shared GIL makes them take turns.

    That is not what happened. A single Interpreter object only tolerates one caller. A second concurrent call doesn't queue, and it doesn't share time with the first - it is rejected immediately:

    Exception in thread Thread-2 (_call):
    Traceback (most recent call last):
    File "/usr/local/lib/python3.14/concurrent/interpreters/__init__.py", line 226, in _call
    res, excinfo = _interpreters.call(self._id, callable, args, kwargs, restrict=True)
    concurrent.interpreters.InterpreterError: interpreter already running

    I confirmed the shape of this with three variants of the same interpreter, run in sequence:

    scenariotimeoutcomeone call0.84sfinetwo sequential calls, same interpreter1.63sfine, roughly 2xtwo concurrent calls, same interpreter0.84s wall-clocksecond call raises InterpreterError


    interpreters.create() is documented as returning "a new (idle) Python interpreter", and that word is accurate: idle or running, nothing in between. If you want N things running at once, you need N interpreters, one each. A thread pool aimed at a single shared interpreter - the pattern every ThreadPoolExecutor tutorial uses with regular threads - does not translate. You get one interpreter's worth of work and an exception, not graceful serialisation.

    The opposite number: interpreters are not obviously cheaper to start than processes


    The other assumption I walked in with was that subinterpreters, being lighter than OS processes, would be cheap to spin up. I measured the cost of creating four workers and giving each one some real state (a 200,000-element list), steady state after the first warm-up run:

    approachcreate + populate, 4 workersfork (Python 3.13's Linux default)0.037 - 0.038sforkserver (Python 3.14's Linux default)0.057 - 0.065s4 subinterpreters0.076 - 0.078s


    Creating four subinterpreters and running a small amount of code in each took about twice as long as a classic fork-based process pool, and somewhat longer than the new forkserver default too. None of this matters once the work inside each worker takes more than a few milliseconds - the 3.6s-versus-0.9s number above swallows an 80-millisecond setup cost without noticing. But it means "subinterpreters are lighter than processes" is not something I could verify at creation time; the saving, where it exists, is in not forking a second copy of the process's address space, not in any faster start.

    That forkserver row was an accident. I wrote a timing script that called multiprocessing.Process(...).start() directly from the module's top level, the way plenty of tutorial code still does, and it crashed:

    RuntimeError:
    An attempt has been made to start a new process before the
    current process has finished its bootstrapping phase.

    Checking multiprocessing.get_start_method() explained it: Python 3.14 changed the default start method on Linux from fork to forkserver. Python 3.13 reports fork; 3.14 reports forkserver. forkserver re-imports your module in the child, which is exactly the thing if __name__ == "__main__": exists to guard against, and which fork never required you to bother with. Every piece of multiprocessing code I'd written without that guard over the years happened to work by accident, on Linux, until this release. It's a one-line fix, but it is a real break, and it landed in the same release that gives you an alternative that doesn't have this problem at all.

    What it costs: closing an interpreter doesn't give the memory back


    I ran four rounds of "create four interpreters, give each a 200,000-element list, close them", watching my own process's RSS the whole time:

    round 1: 16920 -> 55648 KB
    round 2: 55648 -> 57140 KB
    round 3: 57140 -> 59476 KB
    round 4: 59476 -> 61096 KB

    .close() does finalise the interpreter - a destroyed interpreter can't be called again, and I checked that it raises if you try - but the process's resident memory never drops back to where it started. It flattens out (each later round adds less than the first) rather than growing without bound, which looks like CPython's allocator holding onto freed arenas rather than a genuine leak, but I didn't chase that further. If you're planning to cycle through many short-lived interpreters inside one long-running process, budget for that memory to stay resident, not to be returned to the OS the way a finished process's memory is.

    What's actually shareable between interpreters


    Subinterpreters don't share objects by default - that is the entire point, it's what lets them skip the GIL - so most interesting values need to cross a boundary. is_shareable() gives a yes/no answer per type:

    int -> True
    str -> True
    bytes -> True
    None -> True
    bool -> True
    tuple of ints -> True
    float -> True
    list -> False
    dict -> False

    Passing a threading.Lock as an argument to Interpreter.call() fails immediately, with the repr of the offending object right there in the message:

    NotShareableError: (<unlocked _thread.lock object at 0x7f61b8b5add0>,) does not support cross-interpreter data

    An ordinary custom class instance fails the same way, with a flatter message:

    NotShareableError: object does not support cross-interpreter data

    But the cross-interpreter Queue that the same module provides does not enforce this. I put a plain custom-class instance onto a queue and it went through without complaint, and came back out as a distinct, working object with the same attribute values - the queue silently pickles and unpickles it:

    q = interpreters.create_queue()
    q.put(Thing()) # no error
    received = q.get()
    # received is not the original object, but received.value is unchanged

    So "not shareable" is enforced differently depending on how the object crosses over. A direct argument to call() is checked against the strict shareable-types list and rejected on the spot. The same object dropped into a Queue gets the softer treatment the docs describe for put(): pickled going in, unpickled coming out as a new, unrelated object. If your code passes data between interpreters by argument, moving it to a queue instead buys you a much larger set of types - at the cost of identity and, for anything mutable, of staying in sync with the original.

    What I got wrong on the way


    My first attempt at the shareability tests put several checks in one script with no if __name__ == "__main__": guard, calling interp.call() with a mix of shareable and non-shareable arguments one after another. Running it printed every result twice, and the second copy sometimes disagreed with the first about which error got raised. My best explanation, which I didn't fully verify, is that call()'s documented pickle fallback needs to resolve names against the target interpreter's own __main__, and since that interpreter had never run my module, resolving anything defined at module level meant running the whole module there first - silently re-executing my test file a second time, inside the subinterpreter, with its own fresh state. Splitting each check into its own small script with a proper __main__ guard made every result reproducible, which is what's reported above. I'd treat any subinterpreter test that imports a lot of top-level code as suspect until it survives being run twice.

    Run it yourself


    This needs Python 3.14 and four-ish cores to see the gap clearly. I ran it in Docker because that's the fastest way to get an exact version:

    docker pull python:3.14-slim

    The core benchmark:

    # bench.py
    import time, threading, sys

    def cpu_work(n):
    count = 0
    for i in range(2, n):
    is_prime = True
    for d in range(2, int(i ** 0.5) + 1):
    if i % d == 0:
    is_prime = False
    break
    if is_prime:
    count += 1
    return count

    def run_threads(n, workers):
    threads = [threading.Thread(target=cpu_work, args=(n,)) for _ in range(workers)]
    t0 = time.perf_counter()
    for t in threads: t.start()
    for t in threads: t.join()
    print(f"threads={workers}: {time.perf_counter()-t0:.3f}s")

    def run_interpreters(n, workers):
    import concurrent.interpreters as interpreters
    interps = [interpreters.create() for _ in range(workers)]
    t0 = time.perf_counter()
    threads = [i.call_in_thread(cpu_work, n) for i in interps]
    for t in threads: t.join()
    print(f"interpreters={workers}: {time.perf_counter()-t0:.3f}s")
    for i in interps: i.close()

    if __name__ == "__main__":
    mode, n, workers = sys.argv[1], int(sys.argv[2]), int(sys.argv[3])
    {"threads": run_threads, "interpreters": run_interpreters}[mode](n, workers)

    docker run --rm -v "$PWD":/w -w /w python:3.14-slim python3 bench.py threads 400000 4
    docker run --rm -v "$PWD":/w -w /w python:3.14-slim python3 bench.py interpreters 400000 4

    And the refusal, which needs no setup at all:

    docker run --rm python:3.14-slim python3 -c "
    import concurrent.interpreters as interpreters
    i = interpreters.create()
    t1 = i.call_in_thread(lambda: __import__('time').sleep(1))
    t2 = i.call_in_thread(lambda: __import__('time').sleep(1))
    t1.join(); t2.join()
    "

    What to do with this


    If you have a CPU-bound Python workload that multiprocessing already parallelises well, subinterpreters aren't an obvious upgrade yet: on this release they start up slower than a plain fork and about the same as forkserver, and you still need one interpreter per concurrent task, with no pooling against a single interpreter. Where they're worth trying now is the code that currently reaches for a thread, because threading is simpler to wire up than multiprocessing, and then quietly gets nothing from the GIL: swap the Thread for an Interpreter.call_in_thread, and check your data either fits the shareable-types list or is willing to cross as a pickled copy through a Queue. And if you have multiprocessing code anywhere that starts a Process without an if __name__ == "__main__": guard, fix that first - 3.14 will find it on Linux even if nothing else in this post applies to you.#python #performance #programming #devops #software #coding #development #engineering #inclusive #community
    Python 3.14's concurrent.interpreters module turns a 3.6-second GIL bottleneck into 0.9 seconds

  2. A toutes fins utiles, vous pouvez imprimer le lien de la pétition pour le droit aux études - par exemple sur des tracts - en utilisant le Flash code ci-dessous (merci à @politipet )

    Sinon le lien est là

    petitions.assemblee-nationale.

    #Parcoursup #5novembre

  3. A toutes fins utiles, vous pouvez imprimer le lien de la pétition pour le droit aux études - par exemple sur des tracts - en utilisant le Flash code ci-dessous

    Sinon le lien est là

    petitions.assemblee-nationale.fr/initiatives/...

    #Parcoursup #5novembre

  4. A ginger cat is eating from a metal food bowl placed on a concrete driveway near a building wall. There are no people, vehicles, or packages visible in the scene. The driveway has visible cracks, and there is a small pile of leaves and twigs on the left side.
    At the feeders: #crittercam #cat #cat_orange 85.33%
    Reo903.20261025_171649688.jpg

  5. A terceira edição do festival decorre de 29 de Outubro a 1 de Novembro e quer pôr toda a região a caminhar. Objectivo: descobrir a natureza e a cultura do território, passo a passo.

    publico.pt/2026/10/26/fugas/no

  6. A w tym odcinku #LewackiePitolenie mieliśmy rozmawiać o tym jak prawica reaguje na protesty ale zamiast tego wyszła nam rozmowa o rzekomym strategicznym geniuszu Tuska :)

    Ale za to się dzieje :)

    lewackiepitolenie.buzzsprout.c

  7. A big kiss on the cheek of all the engineers that insert their graphic as a vector graphic instead of an image.

    Makes extracting fancy symbols (for example LEMO plug symbols) so much easier via Inkscape

  8. A LITTLE LUCK, A LOT OF ITALY: LOONA INTRODUCES LA FORTUNA RAVIOLI ALONGSIDE A NEW MENU AND BREAKFAST OFFERING

    There is something about an Italian table that makes you want to stay a little longer. At LOONA, that feeling comes through in food m…
    #dining #cooking #diet #food #Breakfast #Italianbreakfast #breakfast #Italia #Italian #italiano #italy
    diningandcooking.com/2872149/a

  9. A Georgian-inspired Wimbledon Village new-build softened with antiques, pattern and calming colours

    ‘Sweeping’ is not a word homeowners often use when describing their dream home to an interior designer. But…
    #NewsBeep #News #Artsanddesign #Arts #ArtsAndDesign #Design #Entertainment #georgianhouses #modernhouses #Newbuild #UK #UnitedKingdom
    newsbeep.com/uk/814291/

  10. A ginger cat is walking across a concrete driveway near a blue trash bin and two metal food/water bowls, one with cat food and the other with water. There are no people, vehicles, or packages visible in the scene. The driveway has some cracks and scattered debris, with grass and overgrown plants along the left edge.
    At the feeders: #crittercam #cat #cat_orange 91.15%
    Reo903.20261025_171639437.jpg

  11. A secret café hidden above the city 🌿

    Tucked atop Yoshida Hill in Kyoto lies Moan, a historic Taisho-era teahouse complex in a quiet forest. It embodies Shichu no Sankyo—mountain solitude within the city.

    Didn’t step inside this time, but the walk was lovely. Saving the café visit for next time! ☕️

    #café #kyoto #forest #architecture #japan

  12. A registrar that became a landlord. Namecheap hosts 386,450 domains in our index, not just sells them. The company that sold you the name now also keeps the lights on.

    cl0q.com/site/nameche...

    #webdev #domains #internet

    namecheap.com · cl0q

  13. A little family visit from the Hemsworths 👋
    Always a pleasure sharing a bit of our world 💛

    #MotoGP #PertaminaEnduroVR46RacingTeam #AustralianGP

  14. A 32B open-source model runs on a laptop and lands top 20 on a 110-model tool-calling leaderboard. Serhii Sokolenko (Tower) on small language models.
    🎥 pycon.de/archive/2026/talks/de
    📍 Join us at PyCon DE 2027, Heidelberg · 19–24 April

  15. A Wagtail demo in your browser ✨ wagtail.org/blog/a-wagtail-dem, fully local and private

  16. A good morning for everybody, specially to my fellows Brazilians.

    pt: Porra, caralho, virou mesmo. Mano do CEU ahahahaha.

  17. A New Way to Think About Women’s Heart Health #Time twp.ai/E5GrM7

  18. A government program in Hungary is preparing to offer single-family homes and condominiums non-refundable subsidies and preferential loans for energy efficiency measures including solar PV and heat pump installations.#Batteries #EnergyStorage #Finance #Heatpumps #Highlights #Residential
    Hungary proposes energy upgrades for single-family homes, apartments - pv magazine Global
  19. A Chinese navy hospital ship spent a week in Abidjan giving out free care — more than 4,000 consultations, according to local media.

    But what happens to the medical samples and personal data collected onboard? Côte d’Ivoire has laws protecting health records. The hard part is enforcing them.

  20. A new documentary about extinction risk and the efforts to mitigate it is now available on ARTE and features Axel Paulsch, who has been engaged with IPBES since the inception of the platform, and currently runs a capacity-building programme for from Eastern Europe, Central Asia and the Caucasus.

    arte.tv/en/videos/127467-004-A

Share on Mastodon

Enter the server where you have an account.