#28 Add marshalparser to Python tests
Merged 2 years ago by lbalhar. Opened 2 years ago by lbalhar.
tests/ lbalhar/python marshalparser  into  master

@@ -0,0 +1,22 @@ 

+ #!/bin/sh -eux

+ 

+ # Marshalparser helps us to uniform byte-compiled Python files

+ # so it has to be able to also parse pyc files from

+ # the main python packages.

+ # Moreover, it has its own database of magic numbers so this test

+ # checks that the parser is compatible with the latest python before

+ # we ship it.

+ 

+ # Test the latest version by default because that's the place

+ # where magic number incompatibility might happen with

+ # the largest probability

+ VERSION=${VERSION:-3.9}

+ 

+ # How many randomly-selected pyc files to check?

+ SAMPLE=${SAMPLE:-50}

+ 

+ path="/usr/lib*/python$VERSION/"

+ 

+ for file in `find $path -name "*.pyc" | shuf | head -n $SAMPLE`; do

+   marshalparser $file || exit 1

+ done

file modified
+4
@@ -58,6 +58,9 @@ 

      - debugflags:

          dir: flags

          run: python3-debug ./assertflags.py -Og

+     - marshalparser_compatibility:

+         dir: marshalparser

+         run: SAMPLE=10 test_marshalparser_compatibility.sh

      required_packages:

      - dnf

      - gcc
@@ -79,3 +82,4 @@ 

      - python3-test

      - python3-debug

      - python3-rpm-macros

+     - marshalparser

Marshalparser helps us to uniform byte-compiled Python files
so it has to be able to also parse pyc files from
the main python packages.

Moreover, it has its own database of magic numbers so this test
checks that the parser is compatible with the latest python before
we ship it.

During testing in rawhide, I've discovered a bug in the parser so I'll release/update it first.

Build failed.

When this lands in rawhide, we can restart the CI.

Previously the test failed on Zuul but passed on Jenkins, IMHO due to the random sampling. Have you considered testing it on all pyc files? Does it take too long?

Also note that here, we only test stuff to ensure the testing framework itself work, so it is not necessary to test all supported versions here (we will do that in marshalparser and in the Python packages themselves instead).

$ koji wait-repo f33-build --build marshalparser-0.2.0-1.fc33
Successfully waited 8:04 for marshalparser-0.2.0-1.fc33 to appear in the f33-build repo

[citest][recheck]

Have you considered testing it on all pyc files? Does it take too long?

Alternatively, having the sample size as an argument.

Build succeeded.

rebased onto 7d588b1

2 years ago

Previously the test failed on Zuul but passed on Jenkins, IMHO due to the random sampling. Have you considered testing it on all pyc files? Does it take too long?

Yes, it takes really long (tens of minutes, close to one hour).

Also note that here, we only test stuff to ensure the testing framework itself work, so it is not necessary to test all supported versions here (we will do that in marshalparser and in the Python packages themselves instead).

Ok, I'll enable only one variant.

Alternatively, having the sample size as an argument.

Can do that. Nonetheless, how big sample you consider as big enough?

Build succeeded.

Can do that. Nonetheless, how big sample you consider as big enough?

Well it can be 10-ish here to indicate the test works, 10-ish in pythons to indicate the magic number hasn't been updated and hundreds in marshalparser itself to verify it on updates. WDYT?

rebased onto 86842ac

2 years ago

Well it can be 10-ish here to indicate the test works, 10-ish in pythons to indicate the magic number hasn't been updated and hundreds in marshalparser itself to verify it on updates. WDYT?

Sounds reasonable. Implemented.

Build succeeded.

This is good to mere for me. One thing I am not sure about is whether the || exit 1 part is necessary with sh -eux, but I wouldn't bother checking.

Pagure doesn't show me the permission of the file, but I assume it is executable given the fact the CI test passed.

Ship it at will.

Pull-Request has been merged by lbalhar

2 years ago