Archived

This forum has been archived. Please start a new discussion on GitHub.

Ice 3.5.1: Small patch to TestUtil.py

Ice's unit tests are run through a class called TestUtil. The runTests method takes the return value of os.system and, in case it's not zero (unit test failed), exits using this return value.
...
                status = os.system(sys.executable + " " +  quoteArgument(os.path.join(dir, "run.py"))

                if status:
                    if(num > 0):
                        sys.stdout.write("[" + str(num) + "] ")
                    message = "test in " + os.path.abspath(dir) + " failed with exit status", status,
                    print(message)
                    if not keepGoing:
                        sys.exit(status)
...

The issue here is that os.system returns the exit code in the upper byte of the returned 16bit value, so in case the unit test run by os.system returned 1 (e.g. exception/failure etc.) status contains the value 256. The process exits with sys.exit(256), which will wrap around and ends up being exit code 0. So make test will actually appear to have succeeded, even though the unit test failed, which is a problem in automated builds.

To apply the source patch to a fresh Ice 3.5.1 source distribution:
cd Ice-3.5.1
patch -p1 < ice351TestUtil.patch.txt

Comments

  • dwayne
    dwayne St. John's, Newfoundland
    Thanks for the bug report. We will look into fixing this for next release.