Repository navigation
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Iterating the blocks of every section — the natural way to
partition a document by section — currently costs time quadratic
in the document size, because each section's blocks are computed
from a document-prefix XPath. On a generated document with 400
sections and 16,000 paragraphs, partitioning takes ~307 s; with
this change it takes ~2 s, with identical results.
Problem
_SectBlockElementIteratorcomputes the blocks of section i byevaluating an XPath that returns every
w:p/w:tblfrom thestart of the document through the end of section i, plus a
count(...)XPath over the prefix through section i−1, andslicing. Each section therefore rescans the whole prefix before
it. Measured in block elements materialized across a full
partition:
At fixed block count, the "before" work grows in direct
proportion to the number of sections. This is the cost that led
the
unstructuredproject to rewrite its DOCX partitioningrather than ask python-docx for each section's blocks, citing
severe slowdowns on documents with hundreds of sections and
thousands of paragraphs. A profile of the stock code attributes
~93% of partition time to these two XPath evaluations.
Change
One file,
docx/oxml/section.py:_SectBlockElementIteratorfirst tries a sibling-navigationcomputation: a section's blocks are the body-level
w:p/w:tblsiblings running back from the section's terminal node (the
paragraph carrying its
w:sectPr, or the body-levelw:sectPritself) to — but not including — the preceding section's
terminal paragraph. Collecting them costs time proportional to
the section's own size, independent of the document prefix, and
keeps no state between calls.
fallback (
_iter_sect_block_elements_xpath) and is usedwhenever the sectPr does not terminate a body-level section in
the ordinary way — e.g. a sectPr nested in other content, or a
paragraph carrying more than one sectPr — so behavior on such
documents is unchanged by construction.
CT_SectPr.preceding_sectPris intentionally untouched: Imeasured its
preceding::XPath first, and libxml2 stops atthe nearest match, so its cost is already bounded by the size
of the preceding section rather than the document prefix.
Correctness
(order and text), the
preceding_sectPrchain, andheader/footer inheritance resolution for all six header/footer
kinds is byte-identical before and after the change, across
generated documents (tables as blocks, 1-block sections, an
empty final section, SDT-wrapped content) and a corpus of
real-world-shaped documents (multi-section headers/footers,
footnotes, comments, tracked changes, shapes, a
LibreOffice-produced file).
same 9 pre-existing collection/setup errors in
tests/opc/test_phys_pkg.pyon both versions in my environment(a pytest-9 warning-as-error artifact, unrelated to this
change).
Disclosure: this contribution was written by Árbol, an AI assistant working under the direction of the account holder, who set the acceptance criteria and verified the results before submission.