Repository navigation
gh-132042: Speed up slot lookup during class creation - #159479
Merged
Merged
Conversation
Resolve the special method slots of a new class with one pass over each class dict along the MRO instead of one lookup per slot name and per class. A small static table maps slot names to slotdef indices; it also replaces the temporary dict in _PyType_InitSlotDefs() and the linear scan in has_slotdef(). Co-Authored-By: Claude Fable 5.1 <[email protected]>
sergey-miryanov
approved these changes
Oct 17, 2026
sergey-miryanov
left a comment
Contributor
There was a problem hiding this comment.
Couple of nitpicks.
Overall, much better than my approaches! LGTM.
| PyObject *base = PyTuple_GET_ITEM(mro, m); | ||
| assert(PyType_Check(base)); | ||
| PyObject *dict = lookup_tp_dict(_PyType_CAST(base)); | ||
| assert(dict != NULL && PyDict_CheckExact(dict)); |
Member
There was a problem hiding this comment.
This assertion seems unsafe, it's apparently possible to have tp_dict be a dict subclass if an extension is initializing the class.
Contributor
Author
There was a problem hiding this comment.
Indeed, changed to PyDict_Check. I will look into making a regression test (that will have to be a C extension)
Contributor
Author
There was a problem hiding this comment.
A test is 4007869. @JelleZijlstra Do you want me to fold it into the PR?
An extension may initialize tp_dict with a dict subclass before calling PyType_Ready(), so assert PyDict_Check() like find_name_in_mro() does. Co-Authored-By: Claude Fable 5.1 <[email protected]>
JelleZijlstra
approved these changes
Oct 31, 2026
…lass Add _testcapi.MyListDictSubclass, a static type like MyList whose tp_dict is initialized with a dict subclass before PyType_Ready(), as extensions may do, and check that subclasses created from Python still inherit its special methods. Co-Authored-By: Claude Fable 5.1 <[email protected]>
…eck-c-globals Co-Authored-By: Claude Fable 5.1 <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
fixup_slot_dispatchers()resolves the special-method slots of a new class with one dict lookup per slot name and per class in the MRO: about 92 names times the MRO length, ~180 lookups forclass A: passand ~650 for aMutableMappingsubclass. This PR scans each dict along the MRO once instead, matches its dunder keys against a small static table of slot names, and records the first hit per slot. The rest ofupdate_one_slot()is unchanged and the result is identical tofind_name_in_mro()for every slot. The per-name lookup remains as the fallback for dicts that cannot be scanned (non-string keys, more than 1000 entries, or on the free-threaded build a dict whose mutex is held by another thread, since the scan never blocks while holding the type lock) and forupdate_slot().Compared to earlier work on this problem, there is no caching and no per-type memory (#4902 (gh-76527) cached the slot descriptors ; #132618 builds a merged MRO dict per class creation (allocation plus a
PyDict_Mergeper base) and had to changetest_descr). A regression test covers class creation racing withsetattron a base on the free-threaded build.@sergey-miryanov
Benchmarks
-O3non-PGO build on Linux, pyperf pinned to one core:class A: passclass E(D), depth-5 single-inheritance chainclass T(MagicMock)class T(MutableMapping)class T(A, B)(multiple inheritance)Mock()(creates a class per instance)MagicMock()types.new_class('A')typing.Protocolsubclasstyping.TypedDictsubclassnamedtuple('P', 'x y z')Enumwith two members@dataclasswith two fieldsA.__len__ = fA.__tablename__ = 'x'(dunder, not a slot)The benchmark script from #132156 / #132618 (1000-iteration cases):
Notes: #132618 no longer applies to main, so for a direct comparison its approach was re-implemented on top of current main. On 14 of the cases above, run back to back on the same machine, this PR is 1.78x faster than main (geometric mean) and the #132618 approach 1.39x.
Whole-process effect:
python -I -c passexecutes 8% fewer instructions (47.6M to 43.8M, callgrind), and importing class-heavy stdlib modules (asyncio,typing,unittest.mock,email.message,argparse,inspect,decimal,tkinter) is 4-8% faster in wall time.Three-way comparison: main vs this PR vs #132618 on current main
Benchmark script for the first table
Generated with help from Claude Code