You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Key encoding unit test flakes on wall-clock performance bound #6745
The parallel unit suite should not fail because a shared CI runner briefly provides less CPU time to one pytest worker. Key encoding performance should be tracked by the benchmark suite.
Current Behavior
test_performance_bounds_single_entity measures 1,000 serialization and deserialization calls with time.perf_counter() and requires each loop to finish within 0.2 seconds. The unit suite runs with 8 pytest-xdist workers, so elapsed time includes time that the worker is descheduled.
The test failed on master in run 31768603920:
AssertionError: Serialization too slow: 0.2062s
assert 0.20623705900004552 < 0.2
This limit has already been raised twice to address CI failures. It started at 0.02 seconds in #5981, changed to 0.1 seconds in #6040, then changed to 0.2 seconds in #6043. Raising it again would postpone the next failure without making the measurement deterministic.
Steps to reproduce
Run make test-python-unit, which starts 8 pytest workers.
Let test_performance_bounds_single_entity execute while the runner is under contention.
The wall-clock loop can exceed 0.2 seconds even though the implementation has not regressed.
The intermittent failure is captured in the linked master run. Local execution passes when the host is idle, which is consistent with runner contention.
Remove test_performance_bounds_single_entity. The same change that introduced it also added dedicated pytest-benchmark coverage in sdk/python/tests/benchmarks/test_key_encoding_benchmarks.py for single-entity string and integer serialization and deserialization. Those benchmarks run in the separate benchmark workflow and preserve the performance signal without turning shared-runner latency into a unit-test failure.
The existing test_single_entity_fast_path continues to cover correctness of the optimized path.
test_performance_bounds_single_entity treats shared-runner wall-clock latency as a unit-test failure, despite the same paths having dedicated benchmark coverage.
Proposed Solution
Remove the redundant wall-clock unit test. Keep test_single_entity_fast_path for correctness and retain the dedicated single-entity serialization and deserialization benchmarks for performance tracking.
The PR removes the absolute wall-clock assertion from the parallel unit suite while retaining correctness coverage and the dedicated pytest-benchmark measurements. The complete Python unit suite passes locally with 2,536 passed and 20 skipped.
Activity
larrysingleton007 commentedon Aug 16, 2026
Implementation Plan
Problem Summary
test_performance_bounds_single_entitytreats shared-runner wall-clock latency as a unit-test failure, despite the same paths having dedicated benchmark coverage.Proposed Solution
Remove the redundant wall-clock unit test. Keep
test_single_entity_fast_pathfor correctness and retain the dedicated single-entity serialization and deserialization benchmarks for performance tracking.Files to Modify
sdk/python/tests/unit/infra/test_key_encoding_utils.pyNew Files
Testing Approach
larrysingleton007 commentedon Aug 16, 2026
Implemented in #6746.
The PR removes the absolute wall-clock assertion from the parallel unit suite while retaining correctness coverage and the dedicated pytest-benchmark measurements. The complete Python unit suite passes locally with 2,536 passed and 20 skipped.
test: Remove flaky key encoding timing assertion
test: Remove flaky key encoding timing assertion
test: Remove flaky key encoding timing assertion