Native threads will always hang during finalization

Many codebases might need to call Python code in highly-asynchronous situations where the interpreter is already finalizing, or might finalize, and want to continue running code after the Python call. This desire has been brought up by users. For example, a callback that wants to call Python code might be invoked when:

• A kernel has finished running on a GPU.
• A network packet was received.
• A thread has quit, and a native library is executing static finalizers of thread local storage.

In the current C API, any non-Python thread (one not created via the threading module) is considered to be “daemon”, meaning that the interpreter won’t wait on that thread to finalize. Instead, the interpreter will hang the thread when it goes to attach a thread state, making it unusable past that point. Attaching a thread state can happen at any point when invoking Python, such as releasing the GIL in-between bytecode instructions, or when a C function exits a Py_BEGIN_ALLOW_THREADS block. (Note that hanging the thread is relatively new behavior; in prior versions, the thread would terminate, but the issue is the same.)

This means that any non-Python thread may be terminated at any point, which is severely limiting for users who want to do more than just execute Python code in their stream of calls (for example, C++ executing finalizers in addition to calling Python).

Py_BEGIN_ALLOW_THREADS
acquire_lock();
Py_END_ALLOW_THREADS

The existing APIs are broken and misleading

There are currently two public ways for a user to create and attach their own thread state; manual use of PyThreadState_New() & PyThreadState_Swap(), and PyGILState_Ensure(). The latter, PyGILState_Ensure(), is significantly more common.

Subinterpreters don’t work with PyGILState_Ensure

As noted in the documentation, PyGILState APIs aren’t officially supported in subinterpreters:

Note that the PyGILState_* functions assume there is only one global interpreter (created automatically by Py_Initialize()). Python supports the creation of additional interpreters (using Py_NewInterpreter()), but mixing multiple interpreters and the PyGILState_* API is unsupported.

More technically, this is because PyGILState_Ensure doesn’t have any way to know which interpreter created the thread, and as such, it has to assume that it was the main interpreter. There isn’t any way to detect this at runtime, so spurious races are bound to come up in threads created by subinterpreters, because synchronization for the wrong interpreter will be used on objects shared between the threads.

Replacing the old APIs

As made clear in Motivation, PyGILState is already pretty buggy, and even if it was magically fixed, the current behavior of hanging the thread is beyond repair. In turn, this PEP intends to completely deprecate the existing PyGILState APIs and provide better alternatives. However, even if this PEP is rejected, all of the APIs can be replaced with more correct PyThreadState functions in the current C API:

• PyGILState_Ensure(): PyThreadState_Swap() & PyThreadState_New()
• PyGILState_Release(): PyThreadState_Clear() & PyThreadState_Delete()
• PyGILState_GetThisThreadState(): PyThreadState_Get()
• PyGILState_Check(): PyThreadState_GetUnchecked() != NULL

This PEP specifies a ten-year deprecation for these functions (while remaining in the stable ABI), primarily because it’s expected that the migration won’t be seamless, due to the new requirement of storing an interpreter state. The exact details of this deprecation are currently unclear, see When should the legacy APIs be removed?.

A light layer of magic

The APIs proposed by this PEP intentionally have a layer of abstraction that is hidden from the user and offloads complexity onto CPython. This is done primarily to help ease the transition from PyGILState for existing codebases, and for ease-of-use to those who provide wrappers the C API, such as Cython or PyO3.

In particular, the API hides details about the lifetime of the thread state and most of the details with interpreter references.

See also Exposing an Activate/Deactivate API instead of Ensure/Clear.

Bikeshedding and the PyThreadState namespace

To solve the issue with “GIL” terminology, the new functions described by this PEP intended as replacements for PyGILState will go under the existing PyThreadState namespace. In Python 3.14, the documentation has been updated to switch over to terms like “attached thread state” instead of “global interpreter lock”, so this namespace seems to fit well for this PEP.

Preventing interpreter finalization with references

Several iterations of this API have taken an approach where PyThreadState_Ensure() can return a failure based on the state of the interpreter. Instead, this PEP takes an approach where an interpreter keeps track of the number of non-daemon threads, which inherently prevents it from beginning finalization.

The main upside with this approach is that there’s more consistency with attaching threads. Using an interpreter reference from the calling thread keeps the interpreter from finalizing before the thread starts, ensuring that it always works. An approach that were to return a failure based on the start-time of the thread could cause spurious issues.

In the case where it is useful to let the interpreter finalize, such as in an asynchronous callback where there’s no guarantee that the thread will start, strong references to an interpreter can be acquired through PyInterpreterState_Lookup().

Daemon and non-daemon threads

This PEP introduces the concept of non-daemon thread states. By default, all threads created without the threading module will hang when trying to attach a thread state for a finalizing interpreter (in fact, daemon threads that are created with the threading module will hang in the same way). This generally happens when a thread calls PyEval_RestoreThread() or in between bytecode instructions, based on sys.setswitchinterval().

A new, internal field will be added to the PyThreadState structure that determines if the thread is daemon. Before finalization, an interpreter will wait until all non-daemon threads call PyThreadState_Delete().

For backwards compatibility, all thread states created by existing APIs, including PyGILState_Ensure(), will remain daemon by default. See We can’t change finalization behavior for PyGILState_Ensure.

int PyThreadState_SetDaemon(int is_daemon)

Set the attached thread state as non-daemon or daemon.

The attached thread state must not be the main thread for the interpreter. All thread states created without PyThreadState_Ensure() are daemon by default.

If the thread state is non-daemon, then the current interpreter will wait for this thread to finish before shutting down. See also threading.Thread.daemon.

Return zero on success, non-zero without an exception set on failure.

Interpreter reference counting

Internally, an interpreter will have to keep track of the number of non-daemon native threads, which will determine when an interpreter can finalize. This is done to prevent use-after-free crashes in PyThreadState_Ensure() for interpreters with short lifetimes, and to remove needless layers of synchronization between the calling thread and the started thread.

An interpreter state returned by Py_NewInterpreter() (or really, PyInterpreterState_New()) will start with a native thread countdown. For simplicity’s sake, this will be referred to as a reference count. A non-zero reference count prevents the interpreter from finalizing.

PyInterpreterState *PyInterpreterState_Hold(void)

Similar to PyInterpreterState_Get(), but returns a strong reference to the interpreter (meaning, it has its reference count incremented by one, allowing the returned interpreter state to be safely accessed by another thread, because it will be prevented from finalizing).

This function is generally meant to be used in tandem with PyThreadState_Ensure().

The caller must have an attached thread state. This function cannot return NULL. Failures are always a fatal error.

PyInterpreterState *PyInterpreterState_Lookup(int64_t interp_id)

Similar to PyInterpreterState_Hold(), but looks up an interpreter based on an ID (see PyInterpreterState_GetID()). This has the benefit of allowing the interpreter to finalize in cases where the thread might not start, such as inside of an asynchronous callback.

This function will return NULL without an exception set on failure. If the return value is non-NULL, then the returned interpreter will be prevented from finalizing until the reference is released by PyThreadState_Release() or PyInterpreterState_Release().

Returning NULL typically means that the interpreter is at a point where threads cannot start, or no longer exists.

The caller does not need to have an attached thread state.

void PyInterpreterState_Release(PyInterpreterState *interp)

Decrement the reference count of the interpreter, as was incremented by PyInterpreterState_Hold() or PyInterpreterState_Lookup().

This function cannot fail, other than with a fatal error. The caller does not need to have an attached thread state for interp.

Ensuring and releasing thread states

This proposal includes two new high-level threading APIs that intend to replace PyGILState_Ensure() and PyGILState_Release().

int PyThreadState_Ensure(PyInterpreterState *interp)

Ensure that the thread has an attached thread state for interp, and thus can safely invoke that interpreter. It is OK to call this function if the thread already has an attached thread state, as long as there is a subsequent call to PyThreadState_Release() that matches this one.

The reference to the interpreter interp is stolen by this function. As such, interp should have been acquired by PyInterpreterState_Hold().

Thread states created by this function are non-daemon by default. See PyThreadState_SetDaemon(). If the calling thread already has an attached thread state that matches interp, then this function will mark the existing thread state as non-daemon and return. It will be restored to its prior daemon status upon the next PyThreadState_Release() call.

Return zero on success, and non-zero with the old attached thread state restored (which may have been NULL).

void PyThreadState_Release()

Release the attached thread state set by PyThreadState_Ensure(). Any thread state that was set prior to the original call to PyThreadState_Ensure() will be restored.

This function cannot fail, but may hang the thread if the attached thread state prior to the original PyThreadState_Ensure() was daemon and the interpreter was finalized.

Deprecation of PyGILState APIs

This PEP deprecates all of the existing PyGILState APIs in favor of the new PyThreadState APIs for the reasons given in the Motivation. Namely:

• PyGILState_Ensure(): use PyThreadState_Ensure() instead.
• PyGILState_Release(): use PyThreadState_Release() instead.
• PyGILState_GetThisThreadState(): use PyThreadState_Get() or PyThreadState_GetUnchecked() instead.
• PyGILState_Check(): use PyThreadState_GetUnchecked() != NULL instead.

All of the PyGILState APIs are to be removed from the non-limited C API in Python 3.25. They will remain available in the stable ABI for compatibility.

Examples

These examples are here to help understand the APIs described in this PEP. Ideally, they could be reused in the documentation.

static PyObject *
my_critical_operation(PyObject *self, PyObject *unused)
{
    assert(PyThreadState_GetUnchecked() != NULL);
    PyInterpreterState *interp = PyInterpreterState_Hold();
    /* Temporarily make this thread non-daemon to ensure that the
       lock is released. */
    if (PyThreadState_Ensure(interp) < 0) {
        PyErr_SetString(PyExc_PythonFinalizationError,
                        "interpreter is shutting down");
        return NULL;
    }

    Py_BEGIN_ALLOW_THREADS;
    acquire_some_lock();
    Py_END_ALLOW_THREADS;

    /* Do something while holding the lock */
    // ...

    release_some_lock();
    PyThreadState_Release();
    Py_RETURN_NONE;
}

static int
thread_func(void *arg)
{
    PyGILState_STATE gstate = PyGILState_Ensure();
    /* It's not an issue in this example, but we just attached
       a thread state for the main interpreter. If my_method() was
       originally called in a subinterpreter, then we would be unable
       to safely interact with any objects from it. */
    if (PyRun_SimpleString("print(42)") < 0) {
        PyErr_Print();
    }
    PyGILState_Release(gstate);
    return 0;
}

static PyObject *
my_method(PyObject *self, PyObject *unused)
{
    PyThread_handle_t handle;
    PyThead_indent_t indent;

    if (PyThread_start_joinable_thread(thread_func, NULL, &ident, &handle) < 0) {
        return NULL;
    }
    Py_BEGIN_ALLOW_THREADS;
    PyThread_join_thread(handle);
    Py_END_ALLOW_THREADS;
    Py_RETURN_NONE;
}

static int
thread_func(void *arg)
{
    PyInterpreterState *interp = (PyInterpreterState *)arg;
    if (PyThreadState_Ensure(interp) < 0) {
        fputs("Cannot talk to Python", stderr);
        return -1;
    }
    if (PyRun_SimpleString("print(42)") < 0) {
        PyErr_Print();
    }
    PyThreadState_Release();
    return 0;
}

static PyObject *
my_method(PyObject *self, PyObject *unused)
{
    PyThread_handle_t handle;
    PyThead_indent_t indent;

    PyInterpreterState *interp = PyInterpreterState_Hold();
    if (PyThread_start_joinable_thread(thread_func, interp, &ident, &handle) < 0) {
        PyInterpreterState_Release(interp);
        return NULL;
    }
    Py_BEGIN_ALLOW_THREADS
    PyThread_join_thread(handle);
    Py_END_ALLOW_THREADS
    Py_RETURN_NONE;
}

static int
thread_func(void *arg)
{
    PyInterpreterState *interp = (PyInterpreterState *)arg;
    if (PyThreadState_Ensure(interp) < 0) {
        fputs("Cannot talk to Python", stderr);
        return -1;
    }
    (void)PyThreadState_SetDaemon(1);
    if (PyRun_SimpleString("print(42)") < 0) {
        PyErr_Print();
    }
    PyThreadState_Release();
    return 0;
}

static PyObject *
my_method(PyObject *self, PyObject *unused)
{
    PyThread_handle_t handle;
    PyThead_indent_t indent;

    PyInterpreterState *interp = PyInterpreterState_Hold();
    if (PyThread_start_joinable_thread(thread_func, interp, &ident, &handle) < 0) {
        PyInterpreterState_Release(interp);
        return NULL;
    }
    Py_RETURN_NONE;
}

typedef struct {
    int64_t interp_id;
} pyrun_t;

static int
async_callback(void *arg)
{
    pyrun_t *data = (pyrun_t *)arg;
    PyInterpreterState *interp = PyInterpreterState_Lookup(data->interp_id);
    PyMem_RawFree(data);
    if (interp == NULL) {
        fputs("Python has shut down", stderr);
        return -1;
    }
    if (PyThreadState_Ensure(interp) < 0) {
        fputs("Cannot talk to Python", stderr);
        return -1;
    }
    if (PyRun_SimpleString("print(42)") < 0) {
        PyErr_Print();
    }
    PyThreadState_Release();
    return 0;
}

static PyObject *
setup_callback(PyObject *self, PyObject *unused)
{
    PyThread_handle_t handle;
    PyThead_indent_t indent;

    pyrun_t *data = PyMem_RawMalloc(sizeof(pyrun_t));
    if (data == NULL) {
        return PyErr_NoMemory();
    }
    // Weak reference to the interpreter. It won't wait on the callback
    // to finalize.
    data->interp_id = PyInterpreterState_GetID(PyInterpreterState_Get());
    register_callback(async_callback, data);

    Py_RETURN_NONE;
}

Using an interpreter ID instead of a interpreter state for PyThreadState_Ensure

Some iterations of this API took an int64_t interp_id parameter instead of PyInterpreterState *interp, because interpreter IDs cannot be concurrently deleted and cause use-after-free violations. PyInterpreterState_Hold() fixes this issue anyway, but an interpreter ID does have the benefit of requiring less magic in the implementation, but has several downsides:

• Nearly all existing interpreter APIs already return a PyInterpreterState pointer, not an interpreter ID. Functions like PyThreadState_GetInterpreter() would have to be accompanied by frustrating calls to PyInterpreterState_GetID(). There’s also no existing way to go from an int64_t back to a PyInterpreterState*, and providing such an API would come with its own set of design problems.
• Threads typically take a void *arg parameter, not an int64_t arg. As such, passing an interpreter pointer requires much less boilerplate for the user, because an additional structure definition or heap allocation would be needed to store the interpreter ID. This is especially an issue on 32-bit systems, where void * is too small for an int64_t.
• To retain usability, interpreter ID APIs would still need to keep a reference count, otherwise the interpreter could be finalizing before the native thread gets a chance to attach. The problem with using an interpreter ID is that the reference count has to be “invisible”; it must be tracked elsewhere in the interpreter, likely being more complex than PyInterpreterState_Hold(). There’s also a lack of intuition that a standalone integer could have such a thing as a reference count. PyInterpreterState_Lookup() sidesteps this problem because the reference count is always associated with the returned interpreter state, not the integer ID.

Exposing an Activate/Deactivate API instead of Ensure/Clear

In prior discussions of this API, it was suggested to provide actual PyThreadState pointers in the API in an attempt to make the ownership and lifetime of the thread state clearer:

More importantly though, I think this makes it clearer who owns the thread state - a manually created one is controlled by the code that created it, and once it’s deleted it can’t be activated again.

This was ultimately rejected for two reasons:

• The proposed API has closer usage to PyGILState_Ensure() & PyGILState_Release(), which helps ease the transition for old codebases.
• It’s significantly easier for code-generators like Cython to use, as there isn’t any additional complexity with tracking PyThreadState pointers around.

Using PyStatus for the return value of PyThreadState_Ensure

In prior iterations of this API, PyThreadState_Ensure() returned a PyStatus instead of an integer to denote failures, which had the benefit of providing an error message.

This was rejected because it’s not clear that an error message would be all that useful; all the conceived use-cases for this API wouldn’t really care about a message indicating why Python can’t be invoked. As such, the API would only be needlessly harder to use, which in turn would hurt the transition from PyGILState_Ensure().

In addition, PyStatus isn’t commonly used in the C API. A few functions related to interpreter initialization use it (simply because they can’t raise exceptions), and PyThreadState_Ensure() does not fall under that category.

When should the legacy APIs be removed?

PyGILState_Ensure() and PyGILState_Release() have been around for over two decades, and it’s expected that the migration will be difficult. Currently, the plan is to remove them in 10 years (opposed to the 5 years required by PEP 387), but this is subject to further discussion, as it’s unclear if that’s enough (or too much) time.

In addition, it’s unclear whether to remove them at all. A soft deprecation could reasonably fit for these functions if it’s determined that a full PyGILState removal would be too disruptive for the ecosystem.