Archived
This forum has been archived. Please start a new discussion on GitHub.
Custom build of ice 3.5.1: test failures
Hi,
The openmicroscopy codebase is still using Python 2 and so we aren't currently able to use the ZeroC windows packages directly due to them being built against Python 3. For this reason most of our Windows users are currently using Ice 3.4. I'm in the process of setting up local builds of Ice 3.5.1 so that we can provide a version of Ice built against Python 2.7 using VS2010. I've followed your instructions for doing so to create a jenkins job to automate the build.
An example build is here:
ICE35-WIN » x86,Release,vc100 #54 Console [Jenkins]
(I'm still working on getting all the build variants going.) This is building fine. However, I am getting some test failures (search: "fatal error U1077") in the above log. The cpp failure is shown below when run by hand; the py and cs tests of the same type also fail:
Have you encountered this error before? If so, I'd be grateful if you could let me know what's making my build environment different to how you have yours set up. Does it require a particular locale or anything custom doing to make it work?
I'm using VS2010 on a WS2008R2 server with python 2.7 in a virtualenv and the current ThirdParty sources are installed. I'm doing x86 and x64 debug and release variants; the above is the x86 release variant.
Also, if I do my own custom python 2.7 build with VS2012, should ice and its tests build and pass using VS2012? (After hacking out the check which prevents it building py with this combination.)
Many thanks,
Roger
The openmicroscopy codebase is still using Python 2 and so we aren't currently able to use the ZeroC windows packages directly due to them being built against Python 3. For this reason most of our Windows users are currently using Ice 3.4. I'm in the process of setting up local builds of Ice 3.5.1 so that we can provide a version of Ice built against Python 2.7 using VS2010. I've followed your instructions for doing so to create a jenkins job to automate the build.
An example build is here:
ICE35-WIN » x86,Release,vc100 #54 Console [Jenkins]
(I'm still working on getting all the build variants going.) This is building fine. However, I am getting some test failures (search: "fatal error U1077") in the above log. The cpp failure is shown below when run by hand; the py and cs tests of the same type also fail:
(27-x86) C:\jenkins\workspace\ICE35-WIN\VSARCH\x86\VSCONFIG\Release\VSVERSION\vc100\src\Ice-3.5.1\cpp>nmake /f Makefile.mak OPTIMIZE=no test *** running tests 12/86 in C:\jenkins\workspace\ICE35-WIN\VSARCH\x86\VSCONFIG\Release\VSVERSION\vc100\src\Ice-3.5.1\cpp\test\Ice\properties *** configuration: Default *** test started: 04/17/14 17:16:59 starting client... ok testing load properties from UTF-8 path... PropertiesI.cpp:407: Ice::FileException: file exception: The system cannot find the file specified. path: ./config/õ©¡Õø¢_client.config unexpected exit status: expected: 0, got 1 ('test in C:\\jenkins\\workspace\\ICE35-WIN\\VSARCH\\x86\\VSCONFIG\\Release\\VSVERSION\\vc100\\src\\Ice-3.5.1\\cpp\\test\\Ice\\properties failed with exit status', 1) NMAKE : fatal error U1077: 'C:\jenkins\venv\27-x86\Scripts\python.EXE' : return code '0x1' Stop.
Have you encountered this error before? If so, I'd be grateful if you could let me know what's making my build environment different to how you have yours set up. Does it require a particular locale or anything custom doing to make it work?
I'm using VS2010 on a WS2008R2 server with python 2.7 in a virtualenv and the current ThirdParty sources are installed. I'm doing x86 and x64 debug and release variants; the above is the x86 release variant.
Also, if I do my own custom python 2.7 build with VS2012, should ice and its tests build and pass using VS2012? (After hacking out the check which prevents it building py with this combination.)
Many thanks,
Roger
0
Comments
-
Hi,
The test run.py script needs a minor fix to work with Windows and Python 2.7
Replace this# # Write config # if sys.version_info[0] == 2: configPath = "./config/\xe4\xb8\xad\xe5\x9b\xbd_client.config" else: configPath = "./config/\u4e2d\u56fd_client.config" TestUtil.createConfig(configPath, ["# Automatically generated by Ice test driver.", "Ice.Trace.Protocol=1", "Ice.Trace.Network=1", "Ice.ProgramName=PropertiesClient", "Config.Path=" + configPath], "utf-8") TestUtil.simpleTest() if os.path.exists(configPath): os.remove(configPath)
By# # Write config # if sys.version_info[0] == 2: configPath = "./config/\xe4\xb8\xad\xe5\x9b\xbd_client.config" else: configPath = "./config/\u4e2d\u56fd_client.config" TestUtil.createConfig(configPath.decode("utf-8"), ["# Automatically generated by Ice test driver.", "Ice.Trace.Protocol=1", "Ice.Trace.Network=1", "Ice.ProgramName=PropertiesClient", "Config.Path=" + configPath], "utf-8") TestUtil.simpleTest() if os.path.exists(configPath.decode("utf-8")): os.remove(configPath.decode("utf-8"))
I don't see any reason why VS2012 shouldn't work, but we haven't tested this.0 -
Many thanks for the suggested change for making it work with python 2.x. This works great, and now all tests are passing for our x86 release build.
Our last (I hope!) remaining problem is in getting the debug (OPTIMIZE=no) builds to work. They are failing as shown below and in ICE35-WIN » x86,Debug,vc100 #61 Console [Jenkins]
I'm invoking the build withnmake /f Makefile.mak OPTIMIZE=%OPTIMIZE% cpp java py cs vb || exit /b
It looks like this is because OPTIMIZE=yes is hardcoded in py/config/Make.rules.mak unconditionally so that you can't override it easily and it can't be inherited. Same in vsaddin. cpp is handling it properly. Just as a general suggestion, it would be really great if OPTIMIZE, prefix and x64suffix could be overridden easily with the Make.rules.mak files only setting them if unset externally on the nmake command-line.$ find . -name "Make.rules.mak" | xargs grep "OPTIMIZE.*=" ./cpp/config/Make.rules.mak:#OPTIMIZE = yes ./py/config/Make.rules.mak:OPTIMIZE = yes ./vsaddin/config/Make.rules.mak:OPTIMIZE = yes
"making all in py" "making all in modules" "making all in IcePy" rc IcePy.rc Microsoft (R) Windows (R) Resource Compiler Version 6.1.7600.16385 Copyright (C) Microsoft Corporation. All rights reserved. cl.exe /c -I. -nologo -W3 -WX -GR -EHsc -D_CONSOLE -MP -O2 -DNDEBUG -MD -I"..\..\..\cpp\include" -I"C:\Python27-x86\include" Communicator.cpp Connection.cpp ConnectionInfo.cpp Current.cpp Endpoint.cpp EndpointInfo.cpp ImplicitContext.cpp Init.cpp Logger.cpp ObjectAdapter.cpp ObjectFactory.cpp Operation.cpp Properties.cpp PropertiesAdmin.cpp Proxy.cpp Slice.cpp Thread.cpp Types.cpp Util.cpp Communicator.cpp Connection.cpp ConnectionInfo.cpp Current.cpp Endpoint.cpp EndpointInfo.cpp ImplicitContext.cpp Init.cpp Logger.cpp ObjectAdapter.cpp ObjectFactory.cpp Operation.cpp Properties.cpp PropertiesAdmin.cpp Proxy.cpp Slice.cpp Thread.cpp Types.cpp Util.cpp link.exe /LIBPATH:"C:\Python27-x86\libs" /LIBPATH:"..\..\..\cpp\lib" /nologo /FIXED:no /OPT:REF /pdb:none /dll Communicator.obj Connection.obj ConnectionInfo.obj Current.obj Endpoint.obj EndpointInfo.obj ImplicitContext.obj Init.obj Logger.obj ObjectAdapter.obj ObjectFactory.obj Operation.obj Properties.obj PropertiesAdmin.obj Proxy.obj Slice.obj Thread.obj Types.obj Util.obj /out:..\..\python\IcePy.pyd ice.lib iceutil.lib slice.lib IcePy.res LINK : fatal error LNK1181: cannot open input file 'ice.lib' NMAKE : fatal error U1077: '"C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\BIN\link.exe"' : return code '0x49d' Stop. NMAKE : fatal error U1077: 'for' : return code '0x1' Stop. NMAKE : fatal error U1077: 'for' : return code '0x1' Stop. NMAKE : fatal error U1077: 'echo' : return code '0x1' Stop. Build step 'Execute Windows batch command' marked build as failure
I've updated our builds to run sed over the Make.rules.mak files to set OPTIMIZE, prefix and x64suffix by updating the hardcoded values, but it would be ideal if we didn't need to mangle them this way.
Regards,
Roger0 -
Currently there isn't other option than update the hard-coded values, we will look into improve this for the next release.
We hard-coded OPTIMIZE as yes because Python Windows installers doesn't provide debug libraries, and doing a custom python build isn't that common.
Why do you need to update x64suffix? note that the test suite expects to found x64 binaries in bin\x64 so changing this could affect the test suite when run with --x64 option.
For OPTIMIZE and PREFIX will not harm to do as you say and we will probably get those fixed in next release.
Thanks for bring that to our attention.
Best Regards,
José0 -
Custom build of ice 3.5.1: test failures
Our overriding of the x64 suffix is because we're doing separate 32- and 64-bit builds, and so the prefix is kept in the top-level package name rather than in individual subdirectories; the directory layout is identical between 32- and 64-bit builds.
One additional question here is if it's still possible to build against python 2.6? Building with your patch works fine for python 2.7. However, with python 2.6 I get a test failure here:*** running tests 8/86 in C:\jenkins\workspace\ICE35-PY26-WIN\VSARCH\x86\VSCONFIG\Release\VSVERSION\vc100\src\Ice-3.5.1\cpp\test\Slice\errorDetection *** configuration: Default *** test started: 07/27/14 14:35:00 CaseInsensitive.ice... failed! ('test in C:\\jenkins\\workspace\\ICE35-PY26-WIN\\VSARCH\\x86\\VSCONFIG\\Release\\VSVERSION\\vc100\\src\\Ice-3.5.1\\cpp\\test\\Slice\\errorDetection failed with exit status', 1) NMAKE : fatal error U1077: 'C:\jenkins\venv\26-x86\Scripts\python.EXE' : return code '0x1' Stop. Build step 'Execute Windows batch command' marked build as failure
Looks like the failure is in the python test runner. Is some additional tweaking required here to make it work with Python 2.6?
Many thanks,
Roger0