Skip to content

gh-162334: Fix crash after setting an empty Unpickler memo - #162425

Open
hamdyaea wants to merge 3 commits into
python:mainfrom
hamdyaea:gh-162334-fix-empty-unpickler-memo
Open

hamdyaea wants to merge 3 commits into
python:mainfrom
hamdyaea:gh-162334-fix-empty-unpickler-memo

Conversation

@hamdyaea

Copy link
Copy Markdown

Summary

Fix a reproducible crash in the C-accelerated _pickle.Unpickler after setting unpickler.memo = {}. The first MEMOIZE operation with index zero can request a zero-capacity resize, causing an out-of-bounds read in non-debug builds or an assertion failure in debug builds.

This patch restores the normal initial capacity of 32 entries for an empty memo, while retaining the existing memo growth behavior for other indices.

Minimal reproducer

import io
import _pickle

u = _pickle.Unpickler(io.BytesIO(b"\x80\x04N\x94."))
u.memo = {}
assert u.load() is None

Changes

  • Guard the zero-sized memo resize in _Unpickler_MemoPut().
  • Add a regression test for all supported pickle protocols and the minimal reproducer, through AbstractPicklerUnpicklerObjectTests.
  • Add a library NEWS entry.

Testing

In the original issue discussion, @hamarshehmhmd reported that this change fixes the debug assertion and passes 1,592 tests from test_pickle, test_pickletools, test_copy, and test_shelve, as well as focused memo reference-leak checks.

These results were reported by @hamarshehmhmd. I have not independently run the test suite on this PR branch. CI can validate this branch.

Attribution

Fixes #162334.

@python-cla-bot

python-cla-bot Bot commented Nov 22, 2026 •

Copy link
Copy Markdown

All commit authors signed the Contributor License Agreement.

CLA signed

Copy link
Copy Markdown
Author

CI triage for run 45007844848: the only substantive failing job is Sanitizers / TSan (free-threading) (job 137163398200). It aborts in test_free_threading.test_generators.TestFTGenerators.test_concurrent_write with a mimalloc assertion in Objects/mimalloc/arena-abandon.c:162 (_mi_arena_segment_mark_abandoned). The test_free_threading.test_pickle suite passes in the same job, as do the ASan, UBSan, MSan, and other platform jobs. The All required checks pass failure is downstream of this TSan failure.

This appears unrelated to the changes in Modules/_pickle.c and Lib/test/pickletester.py, though a rerun is needed to assess whether it is intermittent. I attempted to rerun the individual failed job, but the GitHub Actions API returned 403 (Resource not accessible by integration). Could someone with the necessary Actions permissions rerun that TSan job? Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

_pickle.Unpickler crashes after assigning an empty memo dictionary

1 participant