AIX Open Source

AIX Open Source

Share your experiences and connect with fellow developers to discover how to build and manage open source software for the AIX operating system

 View Only
Back to discussions
Expand all | Collapse all

bitbucket clone via ssh fails on aix 7.1

  • 1.  bitbucket clone via ssh fails on aix 7.1

    Posted Mon January 14, 2019 06:52 PM

    Originally posted by: appd


    I'm facing issue while trying to clone a test repo(public) from bitbucket as shown below. Strangely the ssh auth is not going through for bitbucket and works fine for github when same public key is added to both accounts. I'm assuming that there is some sort of ssh negotiation variation between github and bitbucket which doesn't work with version of opnessh on my aix 7.1 server. Any clues what can be done here to find the root cause?
    Please note that the https clone on bitbucket works just fine.


    # ssh and git version

        max-08$ ssh -V
        OpenSSH_7.5p1, OpenSSL 1.0.2p  14 Aug 2018
       
        max-08$ git --version
        git version 2.18.0 

    # permissions on .ssh directory
        max-08$ ls -lrt .ssh
        total 40
        -rw-r--r--    1 max staff            57 Jan 14 16:36 config
        -rwx------    1 max staff          1989 Jan 14 16:38 authorized_keys
        -rw-------    1 max staff          3243 Jan 14 16:40 id_rsa
        -rw-r--r--    1 max staff           755 Jan 14 16:40 id_rsa.pub
        -rwx------    1 max staff          2040 Jan 14 17:00 known_hosts

     

      

    max-08$ GIT_SSH_COMMAND="ssh -vvv -i /home/max/.ssh/id_rsa" git clone git@bitbucket.org:max/testing.git
Cloning into 'testing'...
OpenSSH_7.5p1, OpenSSL 1.0.2p  14 Aug 2018
debug1: Reading configuration data /home/max/.ssh/config
debug1: /home/max/.ssh/config line 1: Applying options for *
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: Failed dlopen: /usr/krb5/lib/libkrb5.a(libkrb5.a.so): Could not load module /usr/krb5/lib/libkrb5.a(libkrb5.a.so).
System error: No such file or directory

debug1: Error loading Kerberos, disabling Kerberos auth.
debug2: resolving "bitbucket.org" port 22
debug2: ssh_connect_direct: needpriv 0
debug1: Connecting to bitbucket.org [18.205.93.2] port 22.
debug1: Connection established.
debug1: identity file /home/max/.ssh/id_rsa type 1
debug1: key_load_public: No such file or directory
debug1: identity file /home/max/.ssh/id_rsa-cert type -1
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_7.5
debug1: Remote protocol version 2.0, remote software version conker_1.1.17-9953920 app-133
debug1: no match: conker_1.1.17-9953920 app-133
debug2: fd 3 setting O_NONBLOCK
debug1: Authenticating to bitbucket.org:22 as 'git'
debug3: hostkeys_foreach: reading file "/home/max/.ssh/known_hosts"
debug3: record_hostkey: found key type RSA in file /home/max/.ssh/known_hosts:3
debug3: load_hostkeys: loaded 1 keys from bitbucket.org
debug3: order_hostkeyalgs: prefer hostkeyalgs: ssh-rsa-cert-v01@openssh.com,rsa-sha2-512,rsa-sha2-256,ssh-rsa
debug3: send packet: type 20
debug1: SSH2_MSG_KEXINIT sent
debug3: receive packet: type 20
debug1: SSH2_MSG_KEXINIT received
debug2: local client KEXINIT proposal
debug2: KEX algorithms: curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha256,diffie-hellman-group14-sha1,ext-info-c
debug2: host key algorithms: ssh-rsa-cert-v01@openssh.com,rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,ssh-ed25519-cert-v01@openssh.com,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ssh-ed25519
debug2: ciphers ctos: aes128-ctr,aes192-ctr,aes256-ctr,chacha20-poly1305@openssh.com,aes128-gcm@openssh.com,aes256-gcm@openssh.com,aes128-cbc,aes192-cbc,aes256-cbc
debug2: ciphers stoc: aes128-ctr,aes192-ctr,aes256-ctr,chacha20-poly1305@openssh.com,aes128-gcm@openssh.com,aes256-gcm@openssh.com,aes128-cbc,aes192-cbc,aes256-cbc
debug2: MACs ctos: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: MACs stoc: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: compression ctos: none,zlib@openssh.com,zlib
debug2: compression stoc: none,zlib@openssh.com,zlib
debug2: languages ctos:
debug2: languages stoc:
debug2: first_kex_follows 0
debug2: reserved 0
debug2: peer server KEXINIT proposal
debug2: KEX algorithms: curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1
debug2: host key algorithms: ssh-dss,ssh-rsa
debug2: ciphers ctos: aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,chacha20-poly1305@openssh.com,arcfour256,arcfour128
debug2: ciphers stoc: aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,chacha20-poly1305@openssh.com,arcfour256,arcfour128
debug2: MACs ctos: hmac-sha2-256-etm@openssh.com,hmac-sha2-256,hmac-sha1,hmac-sha1-96
debug2: MACs stoc: hmac-sha2-256-etm@openssh.com,hmac-sha2-256,hmac-sha1,hmac-sha1-96
debug2: compression ctos: none
debug2: compression stoc: none
debug2: languages ctos:
debug2: languages stoc:
debug2: first_kex_follows 0
debug2: reserved 0
debug1: kex: algorithm: curve25519-sha256@libssh.org
debug1: kex: host key algorithm: ssh-rsa
debug1: kex: server->client cipher: aes128-ctr MAC: hmac-sha2-256-etm@openssh.com compression: none
debug1: kex: client->server cipher: aes128-ctr MAC: hmac-sha2-256-etm@openssh.com compression: none
debug3: send packet: type 30
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug3: receive packet: type 31
debug1: Server host key: ssh-rsa SHA256:zzXQOXSRBEiUtuE8AikJYKwbHaxvSc0ojez9YXaGp1A
debug3: hostkeys_foreach: reading file "/home/max/.ssh/known_hosts"
debug3: record_hostkey: found key type RSA in file /home/max/.ssh/known_hosts:3
debug3: load_hostkeys: loaded 1 keys from bitbucket.org
debug3: hostkeys_foreach: reading file "/home/max/.ssh/known_hosts"
debug3: record_hostkey: found key type RSA in file /home/max/.ssh/known_hosts:3
debug3: load_hostkeys: loaded 1 keys from 18.205.93.2
debug1: Host 'bitbucket.org' is known and matches the RSA host key.
debug1: Found key in /home/max/.ssh/known_hosts:3
debug3: send packet: type 21
debug2: set_newkeys: mode 1
debug1: rekey after 4294967296 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug3: receive packet: type 21
debug1: SSH2_MSG_NEWKEYS received
debug2: set_newkeys: mode 0
debug1: rekey after 4294967296 blocks
debug2: key: /home/max/.ssh/id_rsa (2008c568), explicit
debug3: send packet: type 5
debug3: receive packet: type 6
debug2: service_accept: ssh-userauth
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug3: send packet: type 50
debug3: receive packet: type 51
debug1: Authentications that can continue: publickey
debug3: start over, passed a different list publickey
debug3: preferred publickey,keyboard-interactive,password
debug3: authmethod_lookup publickey
debug3: remaining preferred: keyboard-interactive,password
debug3: authmethod_is_enabled publickey
debug1: Next authentication method: publickey
debug1: Offering RSA public key: /home/max/.ssh/id_rsa
debug3: send_pubkey_test
debug3: send packet: type 50
debug2: we sent a publickey packet, wait for reply
debug3: receive packet: type 60
debug1: Server accepts key: pkalg ssh-rsa blen 535
debug2: input_userauth_pk_ok: fp SHA256:QHbX3ieex06ZuF8XtkQcBFoCfpCooxh2noxyGdTE1uI
debug3: sign_and_send_pubkey: RSA SHA256:QHbX3ieex06ZuF8XtkQcBFoCfpCooxh2noxyGdTE1uI
debug3: send packet: type 106
debug1: Sent ALLOW_PKCS12_KEYSTORE_CLIENT_FLAG packet
debug3: send packet: type 50
Authentication failed.
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists

     



  • 2.  Re: bitbucket clone via ssh fails on aix 7.1

    Posted Tue January 15, 2019 02:38 AM

    Originally posted by: AyappanP


    I just tried this in a AIX 6.1 machine and didn't face any issues.

    SSH version is

    # ssh -V
    OpenSSH_6.0p1, OpenSSL 1.0.2p  14 Aug 2018

     

    May be you can try with a new sample repo and check again.



  • 3.  Re: bitbucket clone via ssh fails on aix 7.1

    Posted Mon January 28, 2019 01:20 PM

    Originally posted by: appd


    @ayappanp I tried downgrading my OpenSSH to your version on AIX7.1 and even then it doesn't work. Please note that my git cloning works just fine for github so I'm guessing there is some sort of problem specific to Atlassian products.

    Could you please try cloning public repo: git clone git@bitbucket.org:nhanda/testing.git if you have attlasian account. Since it's a public repo anybody with atlassian account should be able to clone it. I can even add your pub key if required for RO access.

    One more important observation git clone on https endpoint works fine. The problem is just with ssh endpoint. 

     

    Works: nhanda@bitbucket.org/nhanda/testing.git">https://nhanda@bitbucket.org/nhanda/testing.git

    Doesn't work: git@bitbucket.org:nhanda/testing.git

     



  • 4.  Re: bitbucket clone via ssh fails on aix 7.1

    Posted Tue January 29, 2019 02:05 AM

    Originally posted by: AyappanP


    I just did a clone.

    # git clone git@bitbucket.org:nhanda/testing.git
    Cloning into 'testing'...
    remote: Counting objects: 3, done.
    remote: Compressing objects: 100% (2/2), done.
    remote: Total 3 (delta 0), reused 0 (delta 0)
    Receiving objects: 100% (3/3), done.

    (0) root @ aixoss-automation-3: 6.1.0.0: /atlassian_test
    # ls
    testing

    (0) root @ aixoss-automation-3: 6.1.0.0: /atlassian_test
    # cd testing

    (0) root @ aixoss-automation-3: 6.1.0.0: /atlassian_test/testing
    # ls
    README.md

    (0) root @ aixoss-automation-3: 6.1.0.0: /atlassian_test/testing
    # git remote -v
    origin  git@bitbucket.org:nhanda/testing.git (fetch)
    origin  git@bitbucket.org:nhanda/testing.git (push)
     

     



  • 5.  Re: bitbucket clone via ssh fails on aix 7.1

    Posted Tue January 29, 2019 02:30 AM

    Originally posted by: AyappanP


    Awesome !! . In another machine , it failed with an error similar to yours.

    # git clone git@bitbucket.org:nhanda/testing.git
    Cloning into 'testing'...
    Warning: Permanently added 'bitbucket.org,18.205.93.1' (RSA) to the list of known hosts.
    Permission denied (publickey).
    fatal: Could not read from remote repository.

    Please make sure you have the correct access rights
    and the repository exists.
     

     



  • 6.  Re: bitbucket clone via ssh fails on aix 7.1

    Posted Tue January 29, 2019 02:31 AM

    Originally posted by: AyappanP


    Will analyze this and let you know what is the issue.



  • 7.  Re: bitbucket clone via ssh fails on aix 7.1

    Posted Tue January 29, 2019 04:23 AM

    Originally posted by: AyappanP


    Okay. I didn't had the public key from the machine added to bitbucket. And that's the reason it failed. 

    Once i added it , git clone succeeded. Tested this in AIX 7.1 as well.

    (0) root @ fvt-p7a2-lp16: /
    # oslevel -s
    7100-04-06-1806
    (0) root @ fvt-p7a2-lp16: /
    # git clone git@bitbucket.org:nhanda/testing.git
    Cloning into 'testing'...
    remote: Counting objects: 3, done.
    remote: Compressing objects: 100% (2/2), done.
    remote: Total 3 (delta 0), reused 0 (delta 0)
    Receiving objects: 100% (3/3), done.
     

    Not sure what's the problem in your env. 



  • 8.  Re: bitbucket clone via ssh fails on aix 7.1

    Posted Fri February 01, 2019 02:30 AM

    Originally posted by: appd


    @AyappanP 

    Here is what did the trick 

    AllowPKCS12keystoreAutoOpen=no

    I'm not sure why this is enabled by default or why only Atlassian products were not able to handle it while github could.

    Any insights would help.



  • 9.  Re: bitbucket clone via ssh fails on aix 7.1

    Posted Fri February 01, 2019 03:48 AM

    Originally posted by: AyappanP


    Is it enabled by default in your AIX machine (/etc/ssh/sshd_config) ?



  • 10.  Re: bitbucket clone via ssh fails on aix 7.1

    Posted Fri February 01, 2019 03:41 PM

    Originally posted by: appd


    Nope .. I don't see that property in any of my ssh config files. Still it behaves that way and causes problem only with Atlassian products. Somebody else confirmed the same issue when they upgraded from AIX 7.1 to 7.2. So it's not just me; something has really changed.



  • 11.  Re: bitbucket clone via ssh fails on aix 7.1

    Posted Mon February 04, 2019 12:52 AM

    Originally posted by: appd


    Related bug: https://issues.apache.org/jira/browse/SSHD-886



  • 12.  Re: bitbucket clone via ssh fails on aix 7.1

    Posted Mon February 04, 2019 01:26 AM

    Originally posted by: AyappanP


    You are right. After checking with the AIX security team, it seems like to support an RFE this option is made enabled by default.