Open Source Development

Power Open Source Development

Explore the open source tools and capabilities for building and deploying modern applications on IBM Power platforms including AIX, IBM i, and Linux.


#Power


#Power

 View Only
  • 1.  AIX 7.1 - GCC issues when installing matplotlib for Python

    Posted Fri October 12, 2018 04:09 PM

    Originally posted by: Teej42


    This is weird.  Matplotlib is a C++ program that is used by Python to plot a graph based on a set of data, very common tool and highly useful.  However, it is exposing errors within the library header files.  I am using GCC from this site (for AIX 7.1 - GCC 6.3.0):

     

    The first error is:  

      /usr/bin/gcc -maix32 -D_LARGE_FILES -pthread -fno-strict-aliasing -D_GNU_SOURCE -fPIC -fno-strict-aliasing -fwrapv -D_LINUX_SOURCE_COMPAT -I/usr/include -I/opt/freeware/include -I/opt/freeware/include/ncurses -DNDEBUG -D_GNU_SOURCE -fPIC -fno-strict-aliasing -fwrapv -DFREETYPE_BUILD_TYPE=system -DPY_ARRAY_UNIQUE_SYMBOL=MPL_matplotlib_ft2font_ARRAY_API -DNPY_NO_DEPRECATED_API=NPY_1_7_API_VERSION -D__STDC_FORMAT_MACROS=1 -I/opt/freeware/lib/python2.7/site-packages/numpy/core/include -I/opt/freeware/lib/python2.7/site-packages/numpy/core/include -I/opt/freeware/include -I/opt/freeware/include/freetype2 -I/usr/local/include -I/usr/include -I. -I/opt/freeware/include/python2.7 -c src/ft2font.cpp -o build/temp.aix-7.1-2.7/src/ft2font.o

      In file included from /opt/freeware/lib/gcc/powerpc-ibm-aix7.1.0.0/6.3.0/include/c++/math.h:36:0,
                       from /opt/freeware/include/python2.7/pyport.h:325,
                       from /opt/freeware/include/python2.7/Python.h:61,
                       from src/mplutils.h:31,
                       from src/ft2font.cpp:10:
      /opt/freeware/lib/gcc/powerpc-ibm-aix7.1.0.0/6.3.0/include/c++/cmath:646:11: error: '::isnan' has not been declared
         using ::isnan;
                 ^~~~~
      error: command '/usr/bin/gcc' failed with exit status 1

    This was an easy fix, adding std before ::isnan; resolved this issue, although std is defined earlier in this code.  However, once I recompiled, this one stumped me:

      /usr/bin/gcc -maix32 -D_LARGE_FILES -pthread -fno-strict-aliasing -D_GNU_SOURCE -fPIC -fno-strict-aliasing -fwrapv -D_LINUX_SOURCE_COMPAT -I/usr/include -I/opt/freeware/include -I/opt/freeware/include/ncurses -DNDEBUG -D_GNU_SOURCE -fPIC -fno-strict-aliasing -fwrapv -DFREETYPE_BUILD_TYPE=system -DPY_ARRAY_UNIQUE_SYMBOL=MPL_matplotlib_ft2font_ARRAY_API -DNPY_NO_DEPRECATED_API=NPY_1_7_API_VERSION -D__STDC_FORMAT_MACROS=1 -I/opt/freeware/lib/python2.7/site-packages/numpy/core/include -I/opt/freeware/lib/python2.7/site-packages/numpy/core/include -I/opt/freeware/include -I/opt/freeware/include/freetype2 -I/usr/local/include -I/usr/include -I. -I/opt/freeware/include/python2.7 -c src/ft2font_wrapper.cpp -o build/temp.aix-7.1-2.7/src/ft2font_wrapper.o
      In file included from /opt/freeware/lib/gcc/powerpc-ibm-aix7.1.0.0/6.3.0/include/c++/powerpc-ibm-aix7.1.0.0/pthread/bits/gthr-default.h:30:0,
                       from /opt/freeware/lib/gcc/powerpc-ibm-aix7.1.0.0/6.3.0/include/c++/powerpc-ibm-aix7.1.0.0/pthread/bits/gthr.h:148,
                       from /opt/freeware/lib/gcc/powerpc-ibm-aix7.1.0.0/6.3.0/include/c++/ext/atomicity.h:35,
                       from /opt/freeware/lib/gcc/powerpc-ibm-aix7.1.0.0/6.3.0/include/c++/bits/basic_string.h:39,
                       from /opt/freeware/lib/gcc/powerpc-ibm-aix7.1.0.0/6.3.0/include/c++/string:52,
                       from /opt/freeware/lib/gcc/powerpc-ibm-aix7.1.0.0/6.3.0/include/c++/stdexcept:39,
                       from src/py_exceptions.h:7,
                       from src/ft2font_wrapper.cpp:4:
      /opt/freeware/lib/gcc/powerpc-ibm-aix7.1.0.0/6.3.0/include/c++/powerpc-ibm-aix7.1.0.0/pthread/bits/gthr-posix.h:118:1: error: 'pthread_mutex_timedlock' was not declared in this scope
       __gthrw(pthread_mutex_timedlock)
       ^
      /opt/freeware/lib/gcc/powerpc-ibm-aix7.1.0.0/6.3.0/include/c++/powerpc-ibm-aix7.1.0.0/pthread/bits/gthr-posix.h: In function 'int __gthread_mutex_timedlock(__gthread_mutex_t*, const __gthread_time_t*)':
      /opt/freeware/lib/gcc/powerpc-ibm-aix7.1.0.0/6.3.0/include/c++/powerpc-ibm-aix7.1.0.0/pthread/bits/gthr-posix.h:768:69: error: '__gthrw_pthread_mutex_timedlock' cannot be used as a function
           return __gthrw_(pthread_mutex_timedlock) (__mutex, __abs_timeout);
                                                                           ^
      error: command '/usr/bin/gcc' failed with exit status 1

    I have no idea how to resolve this one.

    Is there a known issue with both of those header files?  Why?

    To reproduce, get all dependencies set up:

    GCC/G++
    Python 2.7
    All dependencies detailed here:  https://matplotlib.org/users/installing.html#dependencies
    Include /opt/freeware/bin in your PATH

    Then run:  pip install matplotlib==2.2.3

    Other option is to download the source from here (https://github.com/matplotlib/matplotlib) and try to compile ft2font.cpp and ft2font_wrapper.cpp using the above options.

    Please advise on what I may be missing, or whether this whole problems are caused by a bug somewhere else (bad/old GCC header files?)


    #AIX-Open-Source-Software
    #AIXOpenSource


  • 2.  Re: AIX 7.1 - GCC issues when installing matplotlib for Python

    Posted Fri October 12, 2018 04:44 PM

    Originally posted by: Teej42


    Note, I discovered that there was a new version of GCC RPMs and I installed those, only to see the same issues.

     

    $ ls -1 done |grep 6.3.0-2                                                                                       
    gcc-6.3.0-2.aix7.1.ppc.rpm
    gcc-cplusplus-6.3.0-2.aix7.1.ppc.rpm
    gcc-cpp-6.3.0-2.aix7.1.ppc.rpm
    gcc-gfortran-6.3.0-2.aix7.1.ppc.rpm
    libgcc-6.3.0-2.aix7.1.ppc.rpm
    libstdcplusplus-6.3.0-2.aix7.1.ppc.rpm
    libstdcplusplus-devel-6.3.0-2.aix7.1.ppc.rpm


    #AIX-Open-Source-Software
    #AIXOpenSource


  • 3.  Re: AIX 7.1 - GCC issues when installing matplotlib for Python

    Posted Mon October 15, 2018 06:49 AM

    Originally posted by: AyappanP


    Just checked /usr/include/pthread.h. pthread_mutex_timedlock is guarded by _XOPEN_SOURCE >=600. 

    #if (_XOPEN_SOURCE >= 600)

    extern int
    pthread_mutex_timedlock __((pthread_mutex_t *__restrict__,
                                 const struct timespec *__restrict__));
    #endif

     

    Tried setting it via python _sysconfigdata.py doesn't seems to be working. Need to look deeper into this.

    Removing this check from the header file makes the installation successful.


    #AIX-Open-Source-Software
    #AIXOpenSource


  • 4.  Re: AIX 7.1 - GCC issues when installing matplotlib for Python

    Posted Mon October 15, 2018 10:57 AM

    Originally posted by: Teej42


    I can confirm this address the original issue, but the two modifications are of concern.  

     

    1.  Why std::isnan; and not ::isnan;?  Is this a bug in GCC's header file?

    2.  Can we have a better solution than modifying the AIX's pthread.h file?

     

    Please let us know.  Thanks!


    #AIXOpenSource
    #AIX-Open-Source-Software


  • 5.  Re: AIX 7.1 - GCC issues when installing matplotlib for Python

    Posted Tue October 16, 2018 10:40 AM

    Originally posted by: AyappanP


    Both issues seems to be coming because of these conditions in matplotlib source code.

    # grep -r "XOPEN_SOURCE" *
    extern/ttconv/pprdrv_tt.cpp:#ifdef _XOPEN_SOURCE
    extern/ttconv/pprdrv_tt.cpp:#    undef _XOPEN_SOURCE
    src/_png.cpp:   Undefining _POSIX_C_SOURCE and _XOPEN_SOURCE stops a couple
    src/_png.cpp:#   ifdef _XOPEN_SOURCE
    src/_png.cpp:#       undef _XOPEN_SOURCE
    src/mplutils.h:#ifdef _XOPEN_SOURCE
    src/mplutils.h:#    undef _XOPEN_SOURCE
    src/numpy_cpp.h:#ifdef _XOPEN_SOURCE
    src/numpy_cpp.h:#    undef _XOPEN_SOURCE

     

     

    There are undefining _XOPEN_SOURCE . Because of which the "isnan" from math.h and "pthread_mutex_timedlock" from pthread.h are missing out. 

    If you remove these conditions or put "#ifndef _AIX" around these , then things seems to be working. 


    #AIXOpenSource
    #AIX-Open-Source-Software


  • 6.  Re: AIX 7.1 - GCC issues when installing matplotlib for Python

    Posted Tue October 16, 2018 10:48 AM

    Originally posted by: Teej42


    I will ping the matplotlib team on this resolution.  Any details on the GCC issue?

     

    Thanks!

     

    -T.J.


    #AIX-Open-Source-Software
    #AIXOpenSource


  • 7.  Re: AIX 7.1 - GCC issues when installing matplotlib for Python

    Posted Tue October 16, 2018 11:05 AM

    Originally posted by: AyappanP


    As i mentioned in the earlier thread, both has same root cause.

    Undefining _XOPEN_SOURCE takes out isnan from math.h

    #if (_XOPEN_SOURCE<600) || defined(_ALL_SOURCE)
    extern     int      isnan();
     

    So the global declaration is gone and "using ::isnan" will not work. 

     

     

     


    #AIXOpenSource
    #AIX-Open-Source-Software


  • 8.  Re: AIX 7.1 - GCC issues when installing matplotlib for Python

    Posted Tue October 16, 2018 11:26 AM

    Originally posted by: Teej42


    Ticket opened with Matplotlib team - https://github.com/matplotlib/matplotlib/issues/12535


    #AIX-Open-Source-Software
    #AIXOpenSource


  • 9.  Re: AIX 7.1 - GCC issues when installing matplotlib for Python

    Posted Mon March 04, 2019 02:24 AM

    Originally posted by: sanket


    Here is the tutorial how to install ML tools (NumPy, SciPy, Matplotlib, Pandas, scikit-learn and other on AIX)

    https://developer.ibm.com/tutorials/machine-learning-with-python-on-aix/

     

    I hope this will be helpful.


    #AIXOpenSource
    #AIX-Open-Source-Software