Repository navigation
Increase CPython test coverage #1671
Description
We are currently running part of the CPython test suite. This is a good place for people new to the project to contribute. Contribution here can come in two ways:
- Fix failing test - Look for tests marked as skipped or xfail and fix RustPython so that the test will pass.
Add more tests suites - Copy test suite from CPython and mark failing tests with xfail or skip.
In order to run the test suite use cargo run -- -m test -v.
All tests are in Lib/test
Guide: https://rustpython.github.io/blog/2020/04/04/how-to-contribute-by-cpython-unittest.html
Reacted by Jeong, YunWon, Ben Nutchanon Ninyawee, CGMossa, Pebaz, Irfan Shah, Christopher Gambrell and Henry
Activity
abitrolly commentedon Jan 25, 2020
@palaviv is it possible to generate and online dashboard that shows which tests pass and which need more help?
palaviv commentedon Jan 25, 2020
I am sure it is possible... Would you like to create such a dashboard? I think you can use testgrid for that.
My recommendation would be to run the test suite as described above and pick any of the
exprectedFailureor skipped test marked as RustPython change. I am not sure a dashboard would be effective here.abitrolly commentedon Jan 25, 2020
@palaviv is it possible to get the report in a file in machine readable format, such as https://testanything.org ?
palaviv commentedon Jan 26, 2020
The test use
test\libregtestand run the underline unittests tests. I am not sure if there is a builtin support in either for the format you want...mireille-raad commentedon Mar 23, 2020
Would it help having the dashboard somewhere on the current website?
If you are looking for a dashboard, we can do two pages:
The goal would be for people to know what they can help on in one place.
I can probably get the output of both scripts to push to github repos and then use that as part of jekyll page or just as something people can read.
coolreader18 commentedon Mar 23, 2020
I think that would be cool, and it would probably require getting the test results in a machine readable format like TAP. It seems like tappy works perfectly fine on RustPython, the only issue is getting regrtest to run a custom unittest test runner.
abitrolly commentedon Mar 23, 2020
@mireille-raad is it be possible to have single dashboard for both passed/failed and "what is left" by color coding them with green, red and yellow?
coolreader18 commentedon Mar 23, 2020
I don't know if it makes sense to have them in the same dashboard, since the regrtests are tests that get run and are well structured, while the whats_left script just shows standard library items that aren't present in RustPython; they don't really show the same kind of data.
abitrolly commentedon Mar 23, 2020
@coolreader18
regrtestsandstandard library itemsare different test sets?coolreader18 commentedon Mar 23, 2020
Well yeah, sort of; all whats_left.sh does is diff the list of standard library modules and the
dir()outputs of them, and see what CPython has that RustPython doesn't. It doesn't actually run any code; you could "fix" something to remove it from the whats_left list by just doingdef missingthing(): raise NotImplementedError.abitrolly commentedon Mar 23, 2020
@coolreader18 but do testnames match between what is available in RustPython and upstream? If they do, then it might be possible to get the test results from both test servers in TAP format and diff them.
coolreader18 commentedon Mar 23, 2020
That's not really necessary, since every test that's in CPython is passing (or should be, at least), so we can assume that any test that fails on RustPython should be passing, no diffing required.
coolreader18 commentedon Mar 23, 2020
whats_left.shisn't for regrtests, it's only for the standard library.39 remaining items
improve fstring parser
improve fstring parser
improve fstring parser
improve fstring parser
improve fstring parser
improve fstring parser
youknowone commentedon Feb 24, 2023
I removed
good first issuelabel because it is not trivially easy like before in 2020.But it still has many easy issues yet!
DimitrisJim commentedon Feb 24, 2023
@youknowone can't this be closed in favor of #4564? They both overlap but 4564 is more organized and up to date.
youknowone commentedon Feb 25, 2023
I removed adding test from the issue body but kept fixing errors. does it make sense? We may can open new one with more organized form using dashboard
ShaharNaveh commentedon Jan 29, 2026
@youknowone can we close this issue in favor of #6839 ?
youknowone commentedon Jan 29, 2026
Sure, we now have better automatic tracking