›

Custom build of ice 3.5.1: test failures

rleighrleigh Roger LeighOrganization: University of DundeeProject: OMERO (Open Microscopy Environment)Member ✭
in Help Center
Hi,

The openmicroscopy codebase is still using Python 2 and so we aren't currently able to use the ZeroC windows packages directly due to them being built against Python 3. For this reason most of our Windows users are currently using Ice 3.4. I'm in the process of setting up local builds of Ice 3.5.1 so that we can provide a version of Ice built against Python 2.7 using VS2010. I've followed your instructions for doing so to create a jenkins job to automate the build.

An example build is here:
ICE35-WIN » x86,Release,vc100 #54 Console [Jenkins]

(I'm still working on getting all the build variants going.) This is building fine. However, I am getting some test failures (search: "fatal error U1077") in the above log. The cpp failure is shown below when run by hand; the py and cs tests of the same type also fail:
(27-x86) C:\jenkins\workspace\ICE35-WIN\VSARCH\x86\VSCONFIG\Release\VSVERSION\vc100\src\Ice-3.5.1\cpp>nmake /f Makefile.mak OPTIMIZE=no test
*** running tests 12/86 in C:\jenkins\workspace\ICE35-WIN\VSARCH\x86\VSCONFIG\Release\VSVERSION\vc100\src\Ice-3.5.1\cpp\test\Ice\properties
*** configuration: Default
*** test started: 04/17/14 17:16:59
starting client... ok
testing load properties from UTF-8 path... PropertiesI.cpp:407: Ice::FileException:
file exception: The system cannot find the file specified.
path: ./config/õ©¡Õø¢_client.config
unexpected exit status: expected: 0, got 1
('test in C:\\jenkins\\workspace\\ICE35-WIN\\VSARCH\\x86\\VSCONFIG\\Release\\VSVERSION\\vc100\\src\\Ice-3.5.1\\cpp\\test\\Ice\\properties failed with exit status', 1)
NMAKE : fatal error U1077: 'C:\jenkins\venv\27-x86\Scripts\python.EXE' : return code '0x1'
Stop.

Have you encountered this error before? If so, I'd be grateful if you could let me know what's making my build environment different to how you have yours set up. Does it require a particular locale or anything custom doing to make it work?

I'm using VS2010 on a WS2008R2 server with python 2.7 in a virtualenv and the current ThirdParty sources are installed. I'm doing x86 and x64 debug and release variants; the above is the x86 release variant.


Also, if I do my own custom python 2.7 build with VS2012, should ice and its tests build and pass using VS2012? (After hacking out the check which prevents it building py with this combination.)


Many thanks,
Roger

Comments

  • xdmxdm La Coruña, SpainJose Gutierrez de la ConchaOrganization: ZeroC, Inc.Project: Ice DeveloperAdministrators, ZeroC Staff ZeroC Staff
    Hi,

    The test run.py script needs a minor fix to work with Windows and Python 2.7

    Replace this 
    #
# Write config
#
if sys.version_info[0] == 2:
    configPath = "./config/\xe4\xb8\xad\xe5\x9b\xbd_client.config"
else:
    configPath = "./config/\u4e2d\u56fd_client.config"

TestUtil.createConfig(configPath, 
                      ["# Automatically generated by Ice test driver.", 
                       "Ice.Trace.Protocol=1",
                       "Ice.Trace.Network=1", 
                       "Ice.ProgramName=PropertiesClient", 
                       "Config.Path=" + configPath],
                      "utf-8")

TestUtil.simpleTest()

if os.path.exists(configPath):
    os.remove(configPath)

    By 
    #
# Write config
#
if sys.version_info[0] == 2:
    configPath = "./config/\xe4\xb8\xad\xe5\x9b\xbd_client.config"
else:
    configPath = "./config/\u4e2d\u56fd_client.config"

TestUtil.createConfig(configPath.decode("utf-8"), 
                      ["# Automatically generated by Ice test driver.", 
                       "Ice.Trace.Protocol=1",
                       "Ice.Trace.Network=1", 
                       "Ice.ProgramName=PropertiesClient", 
                       "Config.Path=" + configPath],
                      "utf-8")

TestUtil.simpleTest()

if os.path.exists(configPath.decode("utf-8")):
    os.remove(configPath.decode("utf-8"))

    I don't see any reason why VS2012 shouldn't work, but we haven't tested this.
  • rleighrleigh Roger LeighOrganization: University of DundeeProject: OMERO (Open Microscopy Environment)Member ✭
    Many thanks for the suggested change for making it work with python 2.x. This works great, and now all tests are passing for our x86 release build.

    Our last (I hope!) remaining problem is in getting the debug (OPTIMIZE=no) builds to work. They are failing as shown below and in ICE35-WIN » x86,Debug,vc100 #61 Console [Jenkins]

    I'm invoking the build with 
    nmake /f Makefile.mak OPTIMIZE=%OPTIMIZE% cpp java py cs vb || exit /b

    It looks like this is because OPTIMIZE=yes is hardcoded in py/config/Make.rules.mak unconditionally so that you can't override it easily and it can't be inherited. Same in vsaddin. cpp is handling it properly. Just as a general suggestion, it would be really great if OPTIMIZE, prefix and x64suffix could be overridden easily with the Make.rules.mak files only setting them if unset externally on the nmake command-line.
    $ find . -name "Make.rules.mak" | xargs grep "OPTIMIZE.*="
./cpp/config/Make.rules.mak:#OPTIMIZE           = yes
./py/config/Make.rules.mak:OPTIMIZE             = yes
./vsaddin/config/Make.rules.mak:OPTIMIZE                = yes

    "making all in py" 
"making all in modules" 
"making all in IcePy" 
	rc  IcePy.rc
Microsoft (R) Windows (R) Resource Compiler Version 6.1.7600.16385

Copyright (C) Microsoft Corporation.  All rights reserved.


	cl.exe /c -I.  -nologo -W3 -WX -GR -EHsc -D_CONSOLE -MP -O2 -DNDEBUG -MD -I"..\..\..\cpp\include" -I"C:\Python27-x86\include"  Communicator.cpp Connection.cpp ConnectionInfo.cpp Current.cpp Endpoint.cpp EndpointInfo.cpp ImplicitContext.cpp Init.cpp Logger.cpp ObjectAdapter.cpp ObjectFactory.cpp Operation.cpp Properties.cpp PropertiesAdmin.cpp Proxy.cpp Slice.cpp Thread.cpp Types.cpp Util.cpp 
Communicator.cpp
Connection.cpp
ConnectionInfo.cpp
Current.cpp
Endpoint.cpp
EndpointInfo.cpp
ImplicitContext.cpp
Init.cpp
Logger.cpp
ObjectAdapter.cpp
ObjectFactory.cpp
Operation.cpp
Properties.cpp
PropertiesAdmin.cpp
Proxy.cpp
Slice.cpp
Thread.cpp
Types.cpp
Util.cpp
	link.exe /LIBPATH:"C:\Python27-x86\libs" /LIBPATH:"..\..\..\cpp\lib"  /nologo /FIXED:no /OPT:REF /pdb:none /dll  Communicator.obj  Connection.obj  ConnectionInfo.obj  Current.obj  Endpoint.obj  EndpointInfo.obj  ImplicitContext.obj  Init.obj  Logger.obj  ObjectAdapter.obj  ObjectFactory.obj  Operation.obj  Properties.obj  PropertiesAdmin.obj  Proxy.obj  Slice.obj  Thread.obj  Types.obj  Util.obj  /out:..\..\python\IcePy.pyd ice.lib iceutil.lib slice.lib   IcePy.res
LINK : fatal error LNK1181: cannot open input file 'ice.lib'
NMAKE : fatal error U1077: '"C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\BIN\link.exe"' : return code '0x49d'
Stop.
NMAKE : fatal error U1077: 'for' : return code '0x1'
Stop.
NMAKE : fatal error U1077: 'for' : return code '0x1'
Stop.
NMAKE : fatal error U1077: 'echo' : return code '0x1'
Stop.
Build step 'Execute Windows batch command' marked build as failure

    I've updated our builds to run sed over the Make.rules.mak files to set OPTIMIZE, prefix and x64suffix by updating the hardcoded values, but it would be ideal if we didn't need to mangle them this way.


    Regards,
    Roger
  • xdmxdm La Coruña, SpainJose Gutierrez de la ConchaOrganization: ZeroC, Inc.Project: Ice DeveloperAdministrators, ZeroC Staff ZeroC Staff
    Currently there isn't other option than update the hard-coded values, we will look into improve this for the next release.

    We hard-coded OPTIMIZE as yes because Python Windows installers doesn't provide debug libraries, and doing a custom python build isn't that common.

    Why do you need to update x64suffix? note that the test suite expects to found x64 binaries in bin\x64 so changing this could affect the test suite when run with --x64 option.

    For OPTIMIZE and PREFIX will not harm to do as you say and we will probably get those fixed in next release.

    Thanks for bring that to our attention.

    Best Regards,
    José
  • rleighrleigh Roger LeighOrganization: University of DundeeProject: OMERO (Open Microscopy Environment)Member ✭
    Custom build of ice 3.5.1: test failures

    Our overriding of the x64 suffix is because we're doing separate 32- and 64-bit builds, and so the prefix is kept in the top-level package name rather than in individual subdirectories; the directory layout is identical between 32- and 64-bit builds.

    One additional question here is if it's still possible to build against python 2.6? Building with your patch works fine for python 2.7. However, with python 2.6 I get a test failure here:

    *** running tests 8/86 in C:\jenkins\workspace\ICE35-PY26-WIN\VSARCH\x86\VSCONFIG\Release\VSVERSION\vc100\src\Ice-3.5.1\cpp\test\Slice\errorDetection
*** configuration: Default 
*** test started: 07/27/14 14:35:00
CaseInsensitive.ice... failed!
('test in C:\\jenkins\\workspace\\ICE35-PY26-WIN\\VSARCH\\x86\\VSCONFIG\\Release\\VSVERSION\\vc100\\src\\Ice-3.5.1\\cpp\\test\\Slice\\errorDetection failed with exit status', 1)
NMAKE : fatal error U1077: 'C:\jenkins\venv\26-x86\Scripts\python.EXE' : return code '0x1'
Stop.
Build step 'Execute Windows batch command' marked build as failure

    Looks like the failure is in the python test runner. Is some additional tweaking required here to make it work with Python 2.6?


    Many thanks,
    Roger
Sign In or Register to comment.