gh-145301: Fix double-free in hashlib and hmac module initialization by krylosov-aa · Pull Request #145321 · python/cpython

krylosov-aa · 2026-02-27T16:10:05Z

Both hashlib and hmac modules have similar double-free bugs in their hashtable initialization code.

Problem

`hashlib`

In py_hashentry_table_new(), when _Py_hashtable_set() fails while adding an entry by its py_alias key (after successfully adding it by py_name), the code calls PyMem_Free(entry) before goto error. The error handler then calls _Py_hashtable_destroy() which frees the same entry again via py_hashentry_t_destroy_value().

`hmac`

In py_hmac_hinfo_ht_new(), the Py_HMAC_HINFO_LINK macro has the same issue. When py_hmac_hinfo_ht_add() fails for hashlib_name key (after successfully adding name), the code calls PyMem_Free(value) while the entry is already in the hashtable with refcnt=1.

Solution

`hashlib`

Remove the manual PyMem_Free(entry) call since the entry is already tracked by the hashtable under the py_name key and will be properly freed by _Py_hashtable_destroy().

`hmac`

Add a refcnt check before freeing: only call PyMem_Free(value) if value->refcnt == 0 (meaning it was never added to the table). If refcnt > 0, the entry is already in the table and will be freed by _Py_hashtable_destroy().

Tests

No test added. I couldn't find a practical way to trigger this bug since it requires _Py_hashtable_set() to fail during module initialization. This type of bug is best caught by sanitizers rather than unit tests.

The fix can be verified by running the reproducer provided by the issue author:

Build with ASan:

mkdir build-asan && cd build-asan
../configure --with-pydebug --with-address-sanitizer --without-pymalloc
make -j$(nproc)

Reproducer script (uaf_asan.py):

import subprocess
import sys

code = (
    "import sys, _testcapi\n"
    "if '_hashlib' in sys.modules:\n"
    "    del sys.modules['_hashlib']\n"
    "_testcapi.set_nomemory(40, 41)\n"
    "try:\n"
    "    import _hashlib\n"
    "except (MemoryError, ImportError):\n"
    "    pass\n"
    "finally:\n"
    "    _testcapi.remove_mem_hooks()\n"
)

result = subprocess.run(
    [sys.executable, '-c', code],
    capture_output=True, text=True, timeout=10
)

if result.returncode != 0:
    print(f"[*] CRASH confirmed (rc={result.returncode})")
    print(f"[*] {result.stderr.strip().split(chr(10))[-1]}")
else:
    print("[*] No crash (try different start values)")

Before fix:

$ ./build-asan/python uaf_asan.py
[*] CRASH confirmed (rc=-6)
[*] python: ../Include/internal/pycore_stackref.h:554: PyStackRef_FromPyObjectSteal: Assertion `obj != NULL' failed.

After fix: No crash.

Issue: _hashlib: Use-After-Free + Double Free in _hashopenssl.c py_hashentry_table_new() #145301

python-cla-bot · 2026-02-27T16:10:09Z

All commit authors signed the Contributor License Agreement.

picnixz · 2026-02-27T16:20:26Z

Misc/NEWS.d/next/Library/2026-02-27-19-00-26.gh-issue-145301.2Wih4b.rst

+Fix a double-free bug in :mod:`_hashlib` module initialization when
+``_Py_hashtable_set()`` fails while adding an algorithm alias to the hash
+table after the primary name was already added


Too verbose:

Suggested change

Fix a double-free bug in :mod:`_hashlib` module initialization when

``_Py_hashtable_set()`` fails while adding an algorithm alias to the hash

table after the primary name was already added

:mod:`hashlib`: fix a crash when the C extension module initialization fails.

picnixz

Please add the test repro as a regression test.

bedevere-app · 2026-02-27T16:21:27Z

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

picnixz · 2026-02-27T16:22:10Z

Can you also check whether _hmac suffers from that? I don't remember whether I actually C/Ced this code or if I used another refcounting approach.

krylosov-aa · 2026-02-27T16:30:00Z

Can you also check whether _hmac suffers from that? I don't remember whether I actually C/Ced this code or if I used another refcounting approach.

I checked the HMAC implementation in _hashopenssl.c. It uses the same state->hashtable that is initialized by py_hashentry_table_new(). There's no separate hashtable for HMAC. So this fix covers both hash functions and HMAC.

picnixz · 2026-02-27T16:39:34Z

I'm sorry, I meant the hmacmodule.c

krylosov-aa · 2026-02-27T16:55:30Z

I'm sorry, I meant the hmacmodule.c

Yes, hmacmodule.c has the same issue in py_hmac_hinfo_ht_new().

I can suggest the following solution:

#define Py_HMAC_HINFO_LINK(KEY)                                 \
        do {                                                    \
            int rc = py_hmac_hinfo_ht_add(table, KEY, value);   \
            if (rc < 0) {                                       \
+                if (value->refcnt == 0) {                      \
                    PyMem_Free(value);                          \
+                }                                              \
                goto error;                                     \
            }                                                   \
            else if (rc == 1) {                                 \
                value->refcnt++;                                \
            }                                                   \
        } while (0)
        Py_HMAC_HINFO_LINK(e->name);
        Py_HMAC_HINFO_LINK(e->hashlib_name);
#undef Py_HMAC_HINFO_LINK

Should I include this fix in the same PR or create a separate one?

picnixz · 2026-02-27T17:24:24Z

Yes, please do so, and put a comment as well.

krylosov-aa · 2026-02-27T17:49:03Z

Please add the test repro as a regression test.

I investigated adding a regression test using _testcapi.set_nomemory, but found it's not feasible for this specific bug.

The set_nomemory() approach from the issue report triggers allocation failures globally, and the exact allocation number that hits our specific code path varies by platform/configuration.

Do you have any suggestions for how to reliably test this?

picnixz · 2026-02-27T19:32:10Z

Modules/hmacmodule.c

            int rc = py_hmac_hinfo_ht_add(table, KEY, value);   \
            if (rc < 0) {                                       \
-                PyMem_Free(value);                              \
+                /* entry may already be in ht, will be freed by _Py_hashtable_destroy() */ \


Suggested change

/* entry may already be in ht, will be freed by _Py_hashtable_destroy() */ \

/* entry may already be in ht, will be \

* freed by _Py_hashtable_destroy() */ \

correctly align the \. You can augment the column if it does not exceed 80 chars. If it does, properly wrap the comment. Tia.

picnixz · 2026-02-27T19:32:45Z

Do you have any suggestions for how to reliably test this?

You have an ASASN reproducer, why not use it?

picnixz · 2026-02-27T21:31:22Z

Lib/test/test_hashlib.py

 import tempfile
 import threading
 import unittest
+import subprocess


Put the imports alphabetically sorted.

picnixz · 2026-02-27T21:31:34Z

Lib/test/test_hashlib.py

+        try:
+            import _testcapi
+            if not hasattr(_testcapi, 'set_nomemory'):
+                self.skipTest('requires _testcapi.set_nomemory')
+        except ImportError:
+            self.skipTest('requires _testcapi')


Don't use this, use import_helper for that.

picnixz · 2026-02-27T21:32:03Z

Lib/test/test_hashlib.py

+            if '_hashlib' in sys.modules:
+                del sys.modules['_hashlib']


I don't think this is necessary.

picnixz · 2026-02-27T21:32:34Z

Lib/test/test_hashlib.py

+                _testcapi.remove_mem_hooks()
+        """)
+
+        rc = subprocess.call(


Use assert_python_ok instead.

picnixz · 2026-02-27T21:33:52Z

Misc/NEWS.d/next/Library/2026-02-27-19-00-26.gh-issue-145301.2Wih4b.rst

@@ -0,0 +1 @@
+Fix a crash when :mod:`hashlib` or :mod:`hmac` C extension module initialization fails.


Can you create two NEWS entries, one for hashlib and for hmac please with the same sentence:

:mod:`<name>`: fix a crash when the initialization of the underlying C extension module fails.

picnixz · 2026-02-27T21:37:42Z

Modules/hmacmodule.c

+                /* entry may already be in ht, will be freed by \
+                _Py_hashtable_destroy() */                      \


Just put re-align the \ directly and put the comment on one line (you can remove the "will be freed by _Py_hashtable_destroy() and mention that it will be freed "upon exit").

picnixz · 2026-02-27T22:07:24Z

Lib/test/test_hashlib.py

-            self.skipTest('requires _testcapi')
-
        code = textwrap.dedent("""
            import sys


sys is no more necessary

picnixz · 2026-02-27T22:08:05Z

Lib/test/test_hashlib.py


 requires_sha3 = unittest.skipUnless(_sha3, 'requires _sha3')

+_testcapi = import_helper.import_module('_testcapi')


Don't do it at at the module level, it will skip the entire test module. Do it inside the test directly.

picnixz · 2026-02-27T22:08:47Z

Lib/test/test_hashlib.py

                        hash_type.value = False

    @unittest.skipUnless(HASH is not None, 'need _hashlib')
+    @unittest.skipUnless(hasattr(_testcapi, 'set_nomemory'),


When is _testcapi lacking this function?

krylosov-aa · 2026-02-28T20:14:10Z

Do you have any suggestions for how to reliably test this?

You have an ASASN reproducer, why not use it?

I think that testing using set_nomemory(40, 41) is a bad idea because the number of memory allocations differs across platforms, and we cannot guarantee that allocations 40-41 will always occur in py_hashentry_table_new on all platforms.

I believe this is why the macOS Intel test crashes.

I don't know how to write a regression test for this scenario. I would suggest removing the test, since the fix is simple, comments have been left, and a regression seems unlikely to happen.

sobolevn · 2026-03-02T06:51:54Z

Lib/test/test_hashlib.py

+        import_helper.import_module('_testcapi')
+        code = textwrap.dedent("""
+            import _testcapi
+            _testcapi.set_nomemory(40, 41)


Yes, this seems flaky. It will change depending on the python's version and a platform.

sobolevn · 2026-03-02T06:53:21Z

Lib/test/test_hashlib.py

 from test.support import _4G, bigmemtest
 from test.support import hashlib_helper
-from test.support.import_helper import import_fresh_module
+from test.support import import_helper


nit: can we please revert this change if you decide to remove the test?

Modules/hmacmodule.c

picnixz · 2026-03-02T07:32:38Z

Oo for removig the test.

krylosov-aa · 2026-03-02T21:23:50Z

I have made the requested changes; please review again

bedevere-app · 2026-03-02T21:23:56Z

Thanks for making the requested changes!

@picnixz: please review the changes made to this pull request.

vstinner

LGTM.

But I'm not sure that the NEWS entries are worth it since it's unlikely to get a memory allocation failure on importing these C extensions.

miss-islington-app · 2026-03-05T03:48:28Z

Thanks @krylosov-aa for the PR, and @gpshead for merging it 🌮🎉.. I'm working now to backport this PR to: 3.13, 3.14.
🐍🍒⛏🤖

…ation (pythonGH-145321) (cherry picked from commit 6acaf65) Co-authored-by: krylosov-aa <krylosov.andrew@gmail.com> pythongh-145301: Fix double-free in hashlib and hmac initialization

miss-islington-app · 2026-03-05T03:48:51Z

Sorry, @krylosov-aa and @gpshead, I could not cleanly backport this to 3.13 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker 6acaf659ef0fdee131bc02f0b58685da039b5855 3.13

bedevere-app · 2026-03-05T03:48:54Z

GH-145523 is a backport of this pull request to the 3.14 branch.

…zation (GH-145321) (#145523) gh-145301: Fix double-free in hashlib and hmac module initialization (GH-145321) (cherry picked from commit 6acaf65) gh-145301: Fix double-free in hashlib and hmac initialization Co-authored-by: krylosov-aa <krylosov.andrew@gmail.com>

bedevere-bot · 2026-03-05T04:32:34Z

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 FreeBSD14 3.14 (tier-3) has failed when building commit f426928.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/1740/builds/1059) and take a look at the build logs.
Check if the failure is related to this commit (f426928) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/1740/builds/1059

Failed tests:

test_httpservers

Failed subtests:

test_large_content_length_truncated - test.test_httpservers.CGIHTTPServerTestCase.test_large_content_length_truncated

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.14.opsec-fbsd14/build/Lib/test/test_httpservers.py", line 1139, in test_large_content_length_truncated
    res = self.request('/cgi-bin/file1.py', 'POST', b'x', headers)
  File "/home/buildbot/buildarea/3.14.opsec-fbsd14/build/Lib/test/test_httpservers.py", line 132, in request
    return self.connection.getresponse()
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/buildbot/buildarea/3.14.opsec-fbsd14/build/Lib/http/client.py", line 1450, in getresponse
    response.begin()
    ~~~~~~~~~~~~~~^^
  File "/home/buildbot/buildarea/3.14.opsec-fbsd14/build/Lib/http/client.py", line 336, in begin
    version, status, reason = self._read_status()
                              ~~~~~~~~~~~~~~~~~^^
  File "/home/buildbot/buildarea/3.14.opsec-fbsd14/build/Lib/http/client.py", line 305, in _read_status
    raise RemoteDisconnected("Remote end closed connection without"
                             " response")
http.client.RemoteDisconnected: Remote end closed connection without response

krylosov-aa requested review from gpshead and picnixz as code owners February 27, 2026 16:10

bedevere-app bot added the awaiting review label Feb 27, 2026

bedevere-app bot mentioned this pull request Feb 27, 2026

_hashlib: Use-After-Free + Double Free in _hashopenssl.c py_hashentry_table_new() #145301

Open

picnixz reviewed Feb 27, 2026

View reviewed changes

picnixz requested changes Feb 27, 2026

View reviewed changes

bedevere-app bot added awaiting changes and removed awaiting review labels Feb 27, 2026

picnixz added needs backport to 3.13 bugs and security fixes needs backport to 3.14 bugs and security fixes labels Feb 27, 2026

krylosov-aa changed the title ~~gh-145301: Fix double-free in _hashlib module initialization~~ gh-145301: Fix double-free in hashlib and hmac module initialization Feb 27, 2026

picnixz reviewed Feb 27, 2026

View reviewed changes

Lib/test/test_hashlib.py Outdated

self.skipTest('requires _testcapi')

code = textwrap.dedent("""

import sys

Copy link

Member

picnixz Feb 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sys is no more necessary

picnixz reviewed Feb 27, 2026

View reviewed changes

sobolevn reviewed Mar 2, 2026

View reviewed changes

krylosov-aa force-pushed the fix-issue-145301 branch from 154eda5 to c660a71 Compare March 2, 2026 08:15

picnixz mentioned this pull request Mar 2, 2026

gh-145376: Fix null pointer deref in md5module.c #145422

Open

krylosov-aa force-pushed the fix-issue-145301 branch from c660a71 to 355dc1a Compare March 2, 2026 20:46

pythongh-145301: Fix double-free in hashlib and hmac initialization

b19da21

krylosov-aa force-pushed the fix-issue-145301 branch from 355dc1a to b19da21 Compare March 2, 2026 20:46

bedevere-app bot added awaiting change review and removed awaiting changes labels Mar 2, 2026

bedevere-app bot requested a review from picnixz March 2, 2026 21:23

picnixz approved these changes Mar 2, 2026

View reviewed changes

bedevere-app bot added awaiting merge and removed awaiting change review labels Mar 2, 2026

vstinner approved these changes Mar 4, 2026

View reviewed changes

gpshead approved these changes Mar 5, 2026

View reviewed changes

gpshead merged commit 6acaf65 into python:main Mar 5, 2026
52 of 55 checks passed

bedevere-app bot removed the awaiting merge label Mar 5, 2026

miss-islington-app bot assigned gpshead Mar 5, 2026

bedevere-app bot removed the needs backport to 3.14 bugs and security fixes label Mar 5, 2026

	/* entry may already be in ht, will be freed by _Py_hashtable_destroy() */ \
	/* entry may already be in ht, will be \
	* freed by _Py_hashtable_destroy() */ \

		@@ -0,0 +1 @@
		Fix a crash when :mod:`hashlib` or :mod:`hmac` C extension module initialization fails.

		/* entry may already be in ht, will be freed by \
		_Py_hashtable_destroy() */ \


		requires_sha3 = unittest.skipUnless(_sha3, 'requires _sha3')

		_testcapi = import_helper.import_module('_testcapi')

Uh oh!

Conversation

krylosov-aa commented Feb 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Problem

hashlib

hmac

Solution

hashlib

hmac

Tests

Uh oh!

python-cla-bot bot commented Feb 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

picnixz left a comment

Choose a reason for hiding this comment

Uh oh!

bedevere-app bot commented Feb 27, 2026

Uh oh!

picnixz commented Feb 27, 2026

Uh oh!

krylosov-aa commented Feb 27, 2026

Uh oh!

picnixz commented Feb 27, 2026

Uh oh!

krylosov-aa commented Feb 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

picnixz commented Feb 27, 2026

Uh oh!

krylosov-aa commented Feb 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

picnixz commented Feb 27, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

krylosov-aa commented Feb 28, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

picnixz commented Mar 2, 2026

Uh oh!

krylosov-aa commented Mar 2, 2026

Uh oh!

bedevere-app bot commented Mar 2, 2026

Uh oh!

vstinner left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

miss-islington-app bot commented Mar 5, 2026

Uh oh!

miss-islington-app bot commented Mar 5, 2026

Uh oh!

krylosov-aa commented Feb 27, 2026 •

edited

Loading

`hashlib`

`hmac`

`hashlib`

`hmac`

python-cla-bot bot commented Feb 27, 2026 •

edited

Loading

krylosov-aa commented Feb 27, 2026 •

edited

Loading

krylosov-aa commented Feb 27, 2026 •

edited

Loading