AIX Open Source

AIX Open Source

Share your experiences and connect with fellow developers to discover how to build and manage open source software for the AIX operating system

 View Only

  • 1.  Rust cargo and openssl on AIX

    Posted 22 days ago

    Hi, 

    last week Rust community edition went online on AIX toolbox. For us, it was just-in-time :-)
    See post https://community.ibm.com/community/user/discussion/rust-on-aix#bm785615a7-5e28-46f0-a2e2-0196fd80350d

    But we have a problem with openssl when we want install package cargo-pgrx with cargo. And I assume other packages will have the same problem. We are new to Rust and don't know the development environment of rust well.

    Short foray: We are going to use a vector database functionality on AIX and we've found only pgvectorscale (based on pgvector) for PostgreSQL database as a performant and efficient solution. pgvectorscale is based on Rust. So here we are.

    Now to the problem: To build and install pgvectorscale cargo-pgrx is required. But cargo-pgrx has a dependency on openssl. The problem is that the cargo package searches openssl only in the aixtoolbox packages (/opt/freeware/pkgconfigs). But, as we know, openssl is a lpp source on AIX and aixtoolbox is built on that.

    # cargo install --locked cargo-pgrx --version $(cargo metadata --format-version 1 | jq -r '.packages[] | select(.name == "pgrx") | .version')

        Updating crates.io index
      Installing cargo-pgrx v0.12.9
        Updating crates.io index
        Updating crates.io index
       Compiling proc-macro2 v1.0.83
       Compiling libc v0.2.155
    ...<truncated for clarity>...
       Compiling backtrace v0.3.69
    error: failed to run custom build command for `openssl-sys v0.9.99`

    Caused by:
      process didn't exit successfully: `/tmp/cargo-installNOVf2W/release/build/openssl-sys-69fa8e7e61581bee/build-script-main` (exit status: 101)
      --- stdout
      cargo:rerun-if-env-changed=POWERPC64_IBM_AIX_OPENSSL_LIB_DIR
      POWERPC64_IBM_AIX_OPENSSL_LIB_DIR unset
      cargo:rerun-if-env-changed=OPENSSL_LIB_DIR
      OPENSSL_LIB_DIR unset
      cargo:rerun-if-env-changed=POWERPC64_IBM_AIX_OPENSSL_INCLUDE_DIR
      POWERPC64_IBM_AIX_OPENSSL_INCLUDE_DIR unset
      cargo:rerun-if-env-changed=OPENSSL_INCLUDE_DIR
      OPENSSL_INCLUDE_DIR unset
      cargo:rerun-if-env-changed=POWERPC64_IBM_AIX_OPENSSL_DIR
      POWERPC64_IBM_AIX_OPENSSL_DIR unset
      cargo:rerun-if-env-changed=OPENSSL_DIR
      OPENSSL_DIR unset
      cargo:rerun-if-env-changed=OPENSSL_NO_PKG_CONFIG
      cargo:rerun-if-env-changed=PKG_CONFIG_powerpc64-ibm-aix
      cargo:rerun-if-env-changed=PKG_CONFIG_powerpc64_ibm_aix
      cargo:rerun-if-env-changed=HOST_PKG_CONFIG
      cargo:rerun-if-env-changed=PKG_CONFIG
      cargo:rerun-if-env-changed=OPENSSL_STATIC
      cargo:rerun-if-env-changed=OPENSSL_DYNAMIC
      cargo:rerun-if-env-changed=PKG_CONFIG_ALL_STATIC
      cargo:rerun-if-env-changed=PKG_CONFIG_ALL_DYNAMIC
      cargo:rerun-if-env-changed=PKG_CONFIG_PATH_powerpc64-ibm-aix
      cargo:rerun-if-env-changed=PKG_CONFIG_PATH_powerpc64_ibm_aix
      cargo:rerun-if-env-changed=HOST_PKG_CONFIG_PATH
      cargo:rerun-if-env-changed=PKG_CONFIG_PATH
      cargo:rerun-if-env-changed=PKG_CONFIG_LIBDIR_powerpc64-ibm-aix
      cargo:rerun-if-env-changed=PKG_CONFIG_LIBDIR_powerpc64_ibm_aix
      cargo:rerun-if-env-changed=HOST_PKG_CONFIG_LIBDIR
      cargo:rerun-if-env-changed=PKG_CONFIG_LIBDIR
      cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR_powerpc64-ibm-aix
      cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR_powerpc64_ibm_aix
      cargo:rerun-if-env-changed=HOST_PKG_CONFIG_SYSROOT_DIR
      cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR
      run pkg_config fail:
      pkg-config exited with status code 255
      > PKG_CONFIG_ALLOW_SYSTEM_CFLAGS=1 pkg-config --libs --cflags openssl

      The system library `openssl` required by crate `openssl-sys` was not found.
      The file `openssl.pc` needs to be installed and the PKG_CONFIG_PATH environment variable must contain its parent directory.
      The PKG_CONFIG_PATH environment variable is not set.

      HINT: if you have installed the library, try setting PKG_CONFIG_PATH to the directory containing `openssl.pc`.


      --- stderr

      thread 'main' panicked at /h/alpo/.cargo/registry/src/index.crates.io-7f555b6b8ccf4919/openssl-sys-0.9.99/build/find_normal.rs:190:5:


      Could not find directory of OpenSSL installation, and this `-sys` crate cannot
      proceed without this knowledge. If OpenSSL is installed and this crate had
      trouble finding it,  you can set the `OPENSSL_DIR` environment variable for the
      compilation process.

      Make sure you also have the development packages of openssl installed.
      For example, `libssl-dev` on Ubuntu or `openssl-devel` on Fedora.

      If you're in a situation where you think the directory *should* be found
      automatically, please open a bug at https://github.com/sfackler/rust-openssl
      and include information about your system as well as this message.

      $HOST = powerpc64-ibm-aix
      $TARGET = powerpc64-ibm-aix
      openssl-sys = 0.9.99


      note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
    warning: build failed, waiting for other jobs to finish...
    error: failed to compile `cargo-pgrx v0.12.9`, intermediate artifacts can be found at `/tmp/cargo-installNOVf2W`.
    To reuse those artifacts with a future compilation, set the environment variable `CARGO_TARGET_DIR` to that path.

    I've tried to create an openssl.pc with configuration to the lpp sources and set PKG_CONFIG_PATH. But with no luck. Maybe I've made still a mistake.

    Can anybody confirm this problem?
    Is there any solution for that?
    Is it a problem how rust is built for aixtoolbox?
    Maybe the lpp Rust SDK works?



    ------------------------------
    Emanuel Reisinger
    ------------------------------


  • 2.  RE: Rust cargo and openssl on AIX

    Posted 22 days ago

    The recent openssl lpp fileset is shipping .pc files. You can set PKG_CONFIG_PATH=/usr/lib/pkgconfig and try again. 
    Going forward,  there is a plan to rebuild pkg-config package in Toolbox to automatically search /usr/lib/pkgconfig path also for .pc files. 



    ------------------------------
    Ayappan P
    ------------------------------

    Original Message


  • 3.  RE: Rust cargo and openssl on AIX

    Posted 22 days ago

    This sounds good, but...

    I've installed the latest openssl and set PKG_CONFIG_PATH but the same error still exists.

    # openssl version
    OpenSSL 3.0.15 3 Sep 2024 (Library: OpenSSL 3.0.15 3 Sep 2024)
    # ls -l /usr/lib/pkgconfig
    total 24
    -rw-r--r--    1 root     system          760 Dec 24 08:02 libcrypto.pc
    -rw-r--r--    1 root     system          704 Dec 24 08:02 libssl.pc
    -rw-r--r--    1 root     system          659 Dec 24 08:02 openssl.pc

    # cargo install --locked cargo-pgrx --version $(cargo metadata --format-version 1 | jq -r '.packages[] | select(.name == "pgrx") | .version')
        Updating crates.io index
      Installing cargo-pgrx v0.12.9
        Updating crates.io index
        Updating crates.io index
       Compiling libc v0.2.155
    ...<truncated for clarity>...
       Compiling backtrace v0.3.69
    error: failed to run custom build command for `openssl-sys v0.9.99`

    Caused by:
      process didn't exit successfully: `/tmp/cargo-installIlOzux/release/build/openssl-sys-69fa8e7e61581bee/build-script-main` (exit status: 101)
      --- stdout
      cargo:rerun-if-env-changed=POWERPC64_IBM_AIX_OPENSSL_LIB_DIR
      POWERPC64_IBM_AIX_OPENSSL_LIB_DIR unset
      cargo:rerun-if-env-changed=OPENSSL_LIB_DIR
      OPENSSL_LIB_DIR unset
      cargo:rerun-if-env-changed=POWERPC64_IBM_AIX_OPENSSL_INCLUDE_DIR
      POWERPC64_IBM_AIX_OPENSSL_INCLUDE_DIR unset
      cargo:rerun-if-env-changed=OPENSSL_INCLUDE_DIR
      OPENSSL_INCLUDE_DIR unset
      cargo:rerun-if-env-changed=POWERPC64_IBM_AIX_OPENSSL_DIR
      POWERPC64_IBM_AIX_OPENSSL_DIR unset
      cargo:rerun-if-env-changed=OPENSSL_DIR
      OPENSSL_DIR unset
      cargo:rerun-if-env-changed=OPENSSL_NO_PKG_CONFIG
      cargo:rerun-if-env-changed=PKG_CONFIG_powerpc64-ibm-aix
      cargo:rerun-if-env-changed=PKG_CONFIG_powerpc64_ibm_aix
      cargo:rerun-if-env-changed=HOST_PKG_CONFIG
      cargo:rerun-if-env-changed=PKG_CONFIG
      cargo:rerun-if-env-changed=OPENSSL_STATIC
      cargo:rerun-if-env-changed=OPENSSL_DYNAMIC
      cargo:rerun-if-env-changed=PKG_CONFIG_ALL_STATIC
      cargo:rerun-if-env-changed=PKG_CONFIG_ALL_DYNAMIC
      cargo:rerun-if-env-changed=PKG_CONFIG_PATH_powerpc64-ibm-aix
      cargo:rerun-if-env-changed=PKG_CONFIG_PATH_powerpc64_ibm_aix
      cargo:rerun-if-env-changed=HOST_PKG_CONFIG_PATH
      cargo:rerun-if-env-changed=PKG_CONFIG_PATH
      cargo:rerun-if-env-changed=PKG_CONFIG_LIBDIR_powerpc64-ibm-aix
      cargo:rerun-if-env-changed=PKG_CONFIG_LIBDIR_powerpc64_ibm_aix
      cargo:rerun-if-env-changed=HOST_PKG_CONFIG_LIBDIR
      cargo:rerun-if-env-changed=PKG_CONFIG_LIBDIR
      cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR_powerpc64-ibm-aix
      cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR_powerpc64_ibm_aix
      cargo:rerun-if-env-changed=HOST_PKG_CONFIG_SYSROOT_DIR
      cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR
      run pkg_config fail:
      pkg-config exited with status code 255
      > PKG_CONFIG_PATH=/usr/lib/pkgconfig PKG_CONFIG_ALLOW_SYSTEM_CFLAGS=1 pkg-config --libs --cflags openssl

      The system library `openssl` required by crate `openssl-sys` was not found.
      The file `openssl.pc` needs to be installed and the PKG_CONFIG_PATH environment variable must contain its parent directory.
      PKG_CONFIG_PATH contains the following:
          - /usr/lib/pkgconfig

      HINT: you may need to install a package such as openssl, openssl-dev or openssl-devel.


      --- stderr

      thread 'main' panicked at /h/alpo/.cargo/registry/src/index.crates.io-7f555b6b8ccf4919/openssl-sys-0.9.99/build/find_normal.rs:190:5:


      Could not find directory of OpenSSL installation, and this `-sys` crate cannot
      proceed without this knowledge. If OpenSSL is installed and this crate had
      trouble finding it,  you can set the `OPENSSL_DIR` environment variable for the
      compilation process.

      Make sure you also have the development packages of openssl installed.
      For example, `libssl-dev` on Ubuntu or `openssl-devel` on Fedora.

      If you're in a situation where you think the directory *should* be found
      automatically, please open a bug at https://github.com/sfackler/rust-openssl
      and include information about your system as well as this message.

      $HOST = powerpc64-ibm-aix
      $TARGET = powerpc64-ibm-aix
      openssl-sys = 0.9.99


      note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
    warning: build failed, waiting for other jobs to finish...
    error: failed to compile `cargo-pgrx v0.12.9`, intermediate artifacts can be found at `/tmp/cargo-installIlOzux`.
    To reuse those artifacts with a future compilation, set the environment variable `CARGO_TARGET_DIR` to that path.



    ------------------------------
    Emanuel Reisinger
    ------------------------------

    Original Message


  • 4.  RE: Rust cargo and openssl on AIX

    Posted 22 days ago

    I think the pkg-config command is failing in the machine because of some library loading issue. Try running "PKG_CONFIG_PATH=/usr/lib/pkgconfig PKG_CONFIG_ALLOW_SYSTEM_CFLAGS=1 pkg-config --libs --cflags openssl" and see what is failing. 



    ------------------------------
    Ayappan P
    ------------------------------

    Original Message


  • 5.  RE: Rust cargo and openssl on AIX

    Posted 22 days ago

    Nothing fails, only output for compile and linkage:

    # PKG_CONFIG_PATH=/usr/lib/pkgconfig PKG_CONFIG_ALLOW_SYSTEM_CFLAGS=1 pkg-config --libs --cflags openssl
    -I/usr/include -lssl -lcrypto



    ------------------------------
    Emanuel Reisinger
    ------------------------------

    Original Message


  • 6.  RE: Rust cargo and openssl on AIX

    Posted 22 days ago

    Okay. I think since this command is executed through cargo and cargo don't have "/opt/freeware/lib" in its library search path, pkg-config binary ended up picking the libraries from /usr/lib and it is not loading (error code: 255). This is something we can bring it up to the rust team and fix it. For the time being, you can do "export LIBPATH=/opt/freeware/lib" and check . 



    ------------------------------
    Ayappan P
    ------------------------------

    Original Message


  • 7.  RE: Rust cargo and openssl on AIX

    Posted 22 days ago

    Unfortunately LIBPATH doesn't help.

    Why should "export LIBPATH=/opt/freeware/lib" help finding libssl? openssl library is located in /usr/lib and not in /opt/freeware/lib

    "export LIBPATH=/usr/lib" doesn't help either.



    ------------------------------
    Emanuel Reisinger
    ------------------------------

    Original Message


  • 8.  RE: Rust cargo and openssl on AIX

    Posted 21 days ago

    I suggest exporting LIBPATH to /opt/freeware/lib because pkg-config command itself is failing to get executed through cargo in this case. It's nothing to do with openssl libraries. 
    Can you do "export OPENSSL_DIR=/usr" and then try again ? 



    ------------------------------
    Ayappan P
    ------------------------------

    Original Message


  • 9.  RE: Rust cargo and openssl on AIX

    Posted 21 days ago
    Edited by Emanuel Reisinger 7 days ago

    That's it. It works!

    Conclusion (all steps you can read above):

    • I've installed the latest openssl (lpp source), because latest openssl lpp fileset has the pkgconfig openssl.pc included.

    # openssl version
    OpenSSL 3.0.15 3 Sep 2024 (Library: OpenSSL 3.0.15 3 Sep 2024)

    • Set PKG_CONFIG_PATH that pkg-config package searches in /usr/lib/pkgconfig, too. As Ayappan said (see above): Going forward,  there is a plan to rebuild pkg-config package in Toolbox to automatically search /usr/lib/pkgconfig path also for .pc files. I future it will not be required any more.

    export PKG_CONFIG_PATH=/usr/lib/pkgconfig

    • In a second run it worked without exporting LIBPATH:

    export LIBPATH=/usr/lib

    • Required that openssl will be found--I assume--by pkg-config. Maybe it is a configuration matter in pkg-config like PKG_CONFIG_PATH.

    export OPENSSL_DIR=/usr

    • Additionally I've to set the correct compiler because we have several compilers installed and LLVM/clang is required for Rust.

    export CC=ibm-clang

    Thank you Ayappan for your support!



    ------------------------------
    Emanuel Reisinger
    ------------------------------

    Original Message


Related Content