Open Source Development

Power Open Source Development

Explore the open source tools and capabilities for building and deploying modern applications on IBM Power platforms including AIX, IBM i, and Linux.


#Power


#Power

 View Only
  • 1.  AIX 7.2 binary core dumps on 7.3

    Posted yesterday

    Hi all,

    bash-5.2$ oslevel -s
    7200-04-01-1939

    We build a pyinstaller binary, the python script makes use of logging and pyzmq 27.1.0

    The issue is the binary built on 7.2 core dumps on 7.3 bash-5.2# Invalid argument (src/io_thread.cpp:68)

    I was suggested by the folks at pyinstaller to use -D_THREAD_SAFE, -D_THREAD_SAFE_ERRNO but the result is same.

    Below is the pyinstaller ticket link with sample code used to produce the error/issue.

    Specify lib-dynload on AIX 7.2 pyinstaller==6.15.0 · Issue #9241 · pyinstaller/pyinstaller

    GitHub remove preview
    Specify lib-dynload on AIX 7.2 pyinstaller==6.15.0 · Issue #9241 · pyinstaller/pyinstaller
    Hi We have python 3.11.3 built and has its own /opt/pyenv/versions/3.11.3/lib/python3.11/lib-dynload/ I then create a venv and install pyinstaller The python script is just import ssl print("Hello World") I create a binary using pyinstal...
    View this on GitHub >

     

    The same code compiles fine when building on AIX 7.3



    ------------------------------
    Avinesh Benjamin
    ------------------------------


  • 2.  RE: AIX 7.2 binary core dumps on 7.3

    Posted yesterday

    @Ayappan P, @Aditya Kamath or other IBM employee please let me know if we can create ticket with this for IBM or it can be handled only by this forum since it involves open source components?



    ------------------------------
    Avinesh Benjamin
    ------------------------------



  • 3.  RE: AIX 7.2 binary core dumps on 7.3

    Posted 23 hours ago

    Hi,

    We need more details, for example, what CFLAGS and CXXFLAGS you had used to build, and so on. In AIX, Python modules are recommended to be built in 64-bit mode, though you can also build in 32 bit mode.

    Can you try like this?

    pip3.12 cache purge

    export CXX="g++ -pthread"
    export CXXFLAGS=-maix64
    export OBJECT_MODE=64
    export LDFLAGS="-maix64"
    export CC="gcc -pthread"
    export CFLAGS=-maix64
    pip3.12 install pyinstaller 

    And then try??

    Note: if you use Python 3.11, then change all commands to pip3.11. We also want to know which Python 3.11 are you using?
    Also a common mistake that most make is using the AIX-based Python instead of the toolbox Python. 

    So there could be a variety of reasons for not working. We need more details.



    ------------------------------
    Aditya Kamath
    ------------------------------



  • 4.  RE: AIX 7.2 binary core dumps on 7.3

    Posted 22 hours ago

    Hi,

    OBJECT_MODE=64

    AR=ar -X64

    (venv) bash-5.2$ pip list
    Package                   Version
    ------------------------- -------
    altgraph                  0.17.4
    packaging                 25.0
    pip                       22.3.1
    pyinstaller               6.15.0
    pyinstaller-hooks-contrib 2025.9
    pyzmq                     27.1.0
    setuptools                65.5.0
    (venv) bash-5.2$ python --version
    Python 3.11.3

    Python 3.11.3 is built by us, we use pyenv to maintain multiple python versions.

    For pyinstaller I did the below:

    bash-5.2$ /opt/IBM/xlC/16.1.0/bin/xlclang --version
    IBM XL C/C++ for AIX, V16.1.0  (5725-C72, 5765-J12)
    Version: 16.01.0000.0021

    CC=/opt/IBM/xlC/16.1.0/bin/xlclang

    CXX=/opt/IBM/xlC/16.1.0/bin/xlclang++

    pip download -d venv/ pyinstaller==6.15.0

    then did export CFLAGS="-D_THREAD_SAFE, -D_THREAD_SAFE_ERRNO" as suggested by pyinstaller folks

    cd pyinst_debug/7_3/venv/pyinstaller-6.15.0

    bootloader/waf distclean all -vv

    For pyzmq the below is changed

     export CXX="/opt/freeware/bin/g++"
     export CFLAGS="-maix64 -pthread -fPIC"
     export CXXFLAGS="-maix64 -pthread -fPIC" 

    export AR="ar -X64"

    export ORIG_PATH=$PATH

    export PATH=$PATH:/opt/freeware/bin  
    export LDFLAGS="-Wl,-bnoipath /opt/pyenv/versions/3.11.3/lib/libpython3.11.so"  
    $PIP_CMD install pyzmq==27.1.0 --no-binary pyzmq --no-cache-dir
    export LDFLAGS=""
    export PATH=$ORIG_PATH

    gcc is version 10

    We use the same steps on AIX 7.3 except for setting CFLAGS before pyinstaller

    Even the python pyenv is copied and pasted on the AIX 7.3 setup from the 7.2 and the binary works without issues.

    libzmq is downloaded from opensource toolbox

    To reproduce the issue you can copy the code I shared in the pyinstaller ticket and run the pub code which connects to pyinstaller binary, it needs to be run multiple times for the core dump to occur

    sometimes close to 30 hours.



    ------------------------------
    Avinesh Benjamin
    ------------------------------



  • 5.  RE: AIX 7.2 binary core dumps on 7.3

    Posted an hour ago

    Hi,

    First of all, it is not a good idea to mix compilers for different modules since we have seen in the past that at times, due to compatibility issues, Python modules can core dump.

    So I would advice sticking to GCC or only openXL or XLC. 

    The next thing I want to ask is, since you mentioned you build Python on your own, what are the flags/compilers you used to do the same? Did you build a 64-bit or a 32 bit? What is output of file /opt/pyenv/versions/3.11.3/lib/libpython3.11.so?

    Lastly,

    >pip download -d venv/ pyinstaller==6.15.0

    >then did export CFLAGS="-D_THREAD_SAFE, -D_THREAD_SAFE_ERRNO" as suggested by pyinstaller folks

    >cd pyinst_debug/7_3/venv/pyinstaller-6.15.0

    Did you pass -q64 here?

    I would recommend you stick to one compiler for now, which is GCC, and build these one by one. 
    pip3.11 cache purge
    pip3.11 uninstall pyzmq pyinstaller
    export CXX="g++ -pthread"
    export CXXFLAGS=-maix64
    export OBJECT_MODE=64
    export LDFLAGS="-maix64"
    export CC="gcc -pthread"
    export CFLAGS=-maix64
    pip3.11 install pyinstaller
    pip3.11 install pyzmq 



    Your flags for pyzmq is correct. So that should have been correctly built. Let us go one by one, and let me know the answer to all the questions, and check if everything built with GCC works or not?. 



    ------------------------------
    Aditya Kamath
    ------------------------------



  • 6.  RE: AIX 7.2 binary core dumps on 7.3

    Posted 16 hours ago

    @Aditya Kamath Let me know if you need detailed build steps for the pyinstaller binary.



    ------------------------------
    Avinesh Benjamin
    ------------------------------