Cognos Analytics

Cognos Analytics

Connect, learn, and share with thousands of IBM Cognos Analytics users! 


#Product
#Analytics
#CognosAnalytics
#Analyticstools
#TechXchange Partner
#TechXchange Session
#TechXchange Presenter
#AI
 View Only
Expand all | Collapse all

Jupyter Notebook Kernel not found

  • 1.  Jupyter Notebook Kernel not found

    Posted 12/22/19 09:35 AM
    Hi all,

    Installed CA 11.1.5 with a clean installation off Jupyter Notebook (centos 7.6).

    When creating a New Notebook, I see an error in the GUI; see att.
    The Sample Report is working, but i don't like errors in my environement.

    Any hint how I can investergate what's wrong ?

    Kind Regards,




    Kind Regards,


    ------------------------------
    Chris Bos
    ------------------------------

    #CognosAnalyticswithWatson


  • 2.  RE: Jupyter Notebook Kernel not found

    Posted 12/29/19 05:22 AM
    Hi All,

    I think I found my issue.

    Looking at the kernel.json setting it looking for some software located at /opt/anaconda3/bin/python but it's not there.
    So I think my Installation went wrong....

    I will continue with this and will post the outcome to you.

    Kind Regards,


    ------------------------------
    Chris Bos
    ------------------------------



  • 3.  RE: Jupyter Notebook Kernel not found

    Posted 12/30/19 11:29 AM
    Hi Chris,

    sorry for the late reply!

    It would be concerning if the installation didn't succeeed somehow. With kernel errors, the root cause is typically a misconfiguration of the cognos config, the jupyter server, or the gateway.

    Do you have a gateway in front of Cognos by any chance?
    In your cognos config under environment, are the URLs fully qualified domain names?
    In your jupyter installation directory in config.conf, is the HOST_NAME variable set to $(hostname)?
    If you run the "hostname" command on your centos machine, does it return the fully qualified domain name of the machine?

    Regards,
    Rob

    ------------------------------
    ROBERT MCAULEY
    ------------------------------



  • 4.  RE: Jupyter Notebook Kernel not found

    Posted 01/01/20 10:52 AM
    Hi Robert,

    Don't worry, it was Christmas / New Year time, so I wasn't expected any fast response :-).

    I have done some more investergation and I think Cognos is expecting the Python-3, but I can't see it's running or available.

    1 - Right now I don't have a Gateway installed.
    2 - Right now I use a IP address, not a FQDN in Cognos Environment (http://<IP>:8000)
    3 - In my config.conf on my Jupyter CentOS environment, the setting is set to $(hostname)
    4 - Command Hostname is giving me the right hostname of my Jupyter hostname

    From my Cognos Analytics environment I am able to ping to my Jupyter environment.
    From my Jupyter environment, I am able to pint to my Cognos Analytics environment.

    -- so, not firewall issue there -- 

    I am new with Docker / Python, but I think my issue is that Cognos expect an running Python-3 and I can't see that version running on my Jupyter right now.

    I stared with a clean fresh CentOS 7.6.
    Installed Docker-CE
    Tested it's running 
    Did a [ python --version ] and Python 2.7.5 was running.
    In all the 'kernel.json' files I see reference to a Python-3 , but I can't see where or how it's running / installed.
    Perhaps, this is my issue ?

    I can install Python-3, but I don't know if Cognos Jupyter works with a 'default' installation of Python.

    docker inspect <ID> | grep IP

    docker inspect <ID> | grep HOST

    docker inspect <ID> | grep Port

    All gives me the expected information of my environment, so I don't think this is the issue.

    When I look into the kernel.json:
    kernel.json:
    {
    "argv": [
    "/opt/anaconda3/bin/python",
    "-m",
    "ipykernel_launcher",
    "-f",
    "{connection_file}"
    ],
    "display_name": "Python 3",
    "language": "python"

    But my /opt:

    drwx--x--x. 4 root root 28 Dec 29 04:24 containerd
    drwxrwxr-x. 3 root root 20 Dec 29 04:33 ibm
    drwxr-xr-x. 2 root root 6 Oct 30 2018 rh

    Any hints how and what I can test with the Python-3 is weolcome.

    Kind Regards,





















    ------------------------------
    Chris Bos
    ------------------------------



  • 5.  RE: Jupyter Notebook Kernel not found

    Posted 01/02/20 04:25 AM
    Hi Robert,

    Sorry, but sometimes Google is giving the wrong answer.....

    There is a Python 3.6.5 version running on my system.

    (python3 -- version)


    Kind Regards,


    ------------------------------
    Chris Bos
    ------------------------------



  • 6.  RE: Jupyter Notebook Kernel not found

    Posted 01/02/20 02:08 PM
    Hi Chris, 

    Thanks for all the info above. Everything looks like it's in good condition.

    Two things I noticed:

    1) You're using IP addresses to communicate between jupyter and Cognos, but the websocket request (that starts the python 3 kernel connection) is using your Centos' hostname. Do you think that may pose a problem? I would try pinging the jupyter server from Cognos using the Centos hostname, just to make sure the hostname is reachable :-)

    If it isn't, feel free to modify the config.conf file in your jupyter installation directory to use the IP address of the jupyter server instead of the $(hostname) variable. Don't forget to rebuild the server once you make that change.

    2) In your above post you tried python3 --version on your Centos machine I imagine, but you'd need to verify that python 3 is running in the docker container itself. Once you open a new notebook, it should spawn the docker container. Verify that python 3 is running by doing:

    docker ps -a (this will let you see the name of the newly spawned container)
    docker exec -it <name of container> bash
    python --version (or python3 --version, I'm not sure which will work)


    If you're still running into issues, let's take a look at the docker logs. There could be something interesting in there :-)

    Regards,
    Rob

    ------------------------------
    ROBERT MCAULEY
    ------------------------------



  • 7.  RE: Jupyter Notebook Kernel not found

    Posted 01/02/20 05:05 PM
    Hi Robert,

    Thanks for the hints.

    1: Using the FQDN or the IP gives me the same result. I am able to ping on FQDN and IP from and to the Cognos Server and/or Jupyter Server.
    2: Modified the config.conf $(hostname) from HOST_NAME to IP and did a uninstall / install, startup with the same result.
        (right now I am back on the default setting HOST_NAME and not the IP)
    3: I think Python is running in Docker (see att)

    Question:
    1: how can I check the kernel, running and version ?
    2: from Cognos you can restart the Kernel, it looks nothing happens, but can I look in a log file what's going on when restarting?
        Can't see anything in the Cognos log files, but perhaps I should look into some Jupyter logfiles on my CentOS ?
    3: where am I looking for when investergate the docker logs as you mentioned ?



    Kind Regards,

    ------------------------------
    Chris Bos
    ------------------------------



  • 8.  RE: Jupyter Notebook Kernel not found

    Posted 01/06/20 09:14 AM

    Hi Chris,

    Let's take a look at the Docker logs to see what went wrong with your kernel :-) 


    This is what you'd need:

    1) Start a new notebook in Cognos

    2) On CentOS: docker ps -a (so you can find the name of your running notebook container - it should look like ca_jupyter-<some alpha-numeric string>)

    3) On CentOS: docker logs <name of your running notebook container>

    This will give you the logs of your currently running notebook. If there's a kernel error, my bet is there will be an error message there.

    I tried reproducing your issue in my own environment, but I wasn't able to unfortunately. Would you be able to share the exact version of the jupyter notebook kit that you installed? I would like to take a closer look.

    As for getting the version of your kernel, you can check the ipykernel package version using:

    1) on CentOS: docker ps -a (to get the name of your notebook container once again - it should look like ca_jupyter-<some alpha-numeric string>)

    2) on CentOS: docker exec -it <name of your container> bash

    3) Within the docker container now: conda list | grep ipykernel

    As for getting the list of your running kernels, unfortunately there isn't a command that allows you to do that as the kernels are managed internally by the notebook. 

    Regards,
    Rob



    ------------------------------
    ROBERT MCAULEY
    ------------------------------



  • 9.  RE: Jupyter Notebook Kernel not found

    Posted 01/06/20 03:00 PM
    Hi Robert,

    Thanks again for your reply.

    And many thanks for the commando's I can use for (future) issue tracing :-)
     
    Followed your tips and I found something in the log file what could be the issue ?
    -> Blocking Cross Origin API request for .... <-

    SingleUserNotebookApp handlers:480] Blocking Cross Origin API request for /bi/v1/jupyter/user/xTERBUDp1OnVpZD1jZmJvcyxvdT1wZW9wbGU_/api/sessions. Origin: http://[Cognos Server]:[port], Host: IBM-JUPSRV2-DEMO:8000

    Searched on the internet and found some posts (https://github.com/twosigma/beakerx/issues/1963) within the Python world.  
    In this post a person was trying with another browser (FireFox) and there he faced not this error.
    When I use FireFox, I also don't get this error, so looks like it's brwoser related ?

    So when I use Chroom, I see this error in de log file. 
    When I use FireFox, I don't see this error in the log file.
    Before testing I did use the prune.sh / stop.sh / startup.sh scripts making sure it's a fresh log.

    1) Do you agree this could be my issue ?
    2) And do you have a direction how I can solve this so it works also with Chroom ?

    Kind Regards,








    ------------------------------
    Chris Bos
    ------------------------------



  • 10.  RE: Jupyter Notebook Kernel not found

    Posted 01/06/20 03:20 PM
    Hi Chris,

    I'm glad you got it working with Fire Fox! 

    Maybe we can try one thing: in your Cognos Configuration under Environment -> Gateway URI, can you try using the Cognos FQDN instead
    of its IP address? Make sure to restart Cognos afterward.

    Let me know if that works :-) 

    Regards,
    Rob

    ------------------------------
    ROBERT MCAULEY
    ------------------------------



  • 11.  RE: Jupyter Notebook Kernel not found

    Posted 01/06/20 06:19 PM
    Hi Robert,

    Within my Cognos Configuration I prefer to use always FQDN and not the IP numbers.
    Right now the Gateway URI is set to the NAME and not the IP.

    Kind Regards,

    ------------------------------
    Chris Bos
    ------------------------------



  • 12.  RE: Jupyter Notebook Kernel not found

    Posted 01/08/20 03:45 PM

    Hi Chris,

    Looking at the CORS problem in the logs, it's possible that Firefox is a little more lenient than Chrome. Here's some information with respect to CORS handling that you may find useful:

    The browser's origin (the host that you connect to) is compared to the Gateway URI value in your cognos config to verify that the request came from the same domain. For example, if you connect to cognos with http://mycognos:9300 and have 10.202.32.120:9300 for example in the Gateway URI, the jupyter server will notice that "mycognos:9300" is not equal to "10.202.32.120:9300" and will raise a CORS error. In this case, you would need to have "mycognos:9300" in your Gateway URI, OR you could access cognos via http://10.202.32.120:9300 in the browser. Both of these would solve the CORS problem.

    The takeaway would be to make sure the host and port you enter in the browser is the same as what is set in the Gateway URI. 

    Regards,

    Rob



    ------------------------------
    ROBERT MCAULEY
    ------------------------------



  • 13.  RE: Jupyter Notebook Kernel not found

    Posted 01/07/20 12:59 PM


  • 14.  RE: Jupyter Notebook Kernel not found

    Posted 01/07/20 03:02 PM
    Hi Jerzy,

    Thank you for your reply.

    I did tried using the IP and not the HOST_NAME=$(hostname) setting, but with the same result.
    Perhaps I am wrong, but when Browser 1 is not working and Browser 2 is working, to me it sound not like a network issue.
    If there is something wrong on the network IP / FQDN I can understand, but than it should always give me an error, agree ?

    Right now I am more searching for Windows / Browser issue and not the Cognos (Jupyter) Server Configuration.

    Or am I missing some steps ?

    Right now it is working with FireFox for me, so I can investergate Jupyter Notebook with Cognos.
    And when 11.1.5 comes available (again) perhaps with a new fresh installation it will also working with Chroom ?

    Kind Regards,








    Kind Regards






    ------------------------------
    Chris Bos
    ------------------------------



  • 15.  RE: Jupyter Notebook Kernel not found

    Posted 01/07/20 03:08 PM
    PS:

    I wrote a small installation steps document, how to install your own Jupyter Notebook Server: http://bit.ly/2ZYhtgZ

    Kind Regards,

    ------------------------------
    Chris Bos
    ------------------------------



  • 16.  RE: Jupyter Notebook Kernel not found

    Posted 01/08/20 04:33 AM
    Hi Chris, 
    Have you ever run Jupiter with previous versions of CA (11.1R4 or 11.1R3)?
    For the moment I have not installed 11.1.5, but previously I made the Jupiter 11.1R3 work with CA 11.1.4. Maybe you could try installing the previous version of notebook .
    And the last thing, does your Centos machine belong to the domain? When - during my first installation - I used a CentOS server which belongs to the same AD domain as my Cognos server everything went very well without the need for additional configurations.

    Otherwise, your doc is fine, just a question. Do you know how to start the Jupiter server automatically after the machine reboots?

    Regards,
    Jerzy

    ------------------------------
    Jerzy Konarski
    ------------------------------



  • 17.  RE: Jupyter Notebook Kernel not found

    Posted 01/10/20 08:14 AM
    Hi Jerzy,

    Yes, when I connect my Jupyter Notebook 11.1.4 it's working with the same combination without any error.
    I will trace more at my 11.1.5 environment and post my results over here.... thanks for all the help right now.

    For starting all after a reboot I have done it like below (don't know if it it's how it should, but for me it's working...)

    cd /etc/rc.d
    chmod +x rc.local

    vi rc.local and add 4 lines at the bottem:

    systemctl start docker
    sleep 10  -- don't know if this helps, but I want to make sure docker is running ;-)
    cd /opt/ibm/cognos/jupyter/dist/scripts/unix
    bash startup.sh

    when you reboot your environment everything is up and running.

    Kind Regards,


    ------------------------------
    Chris Bos
    ------------------------------



  • 18.  RE: Jupyter Notebook Kernel not found

    Posted 10/15/20 12:13 PM
    Hello,

    We ran into the same problem on our environment. I found this thread by googling for "Cognos Jupiter Blocking Cross Origin API request". We also use the 11.1.5 version.

    Right now, Firefox gives the same error. So, switching from Chrome to Firefox is not a solution anymore.

    We tried a lot of combinations using IP address and fully qualified domain names in the configuration, but we always get this error. We also modified the hosts files on the servers and included the other machine.

    Our Jupyter Server is running on CentOS 7.8, and the Cognos Server is running on Windows 2012.

    Does anyone have any suggestion? It seems to be a network and browser related error, but I have no idea how to fix this error.

    Regards,

    Balazs



    ------------------------------
    Balázs Zelenák
    ------------------------------



  • 19.  RE: Jupyter Notebook Kernel not found

    Posted 10/15/20 12:13 PM

    Hello,

    We ran into the same problem on our environment. I found this thread by googling for "Cognos Jupiter Blocking Cross Origin API request". We also use the 11.1.5 version.

    Right now, Firefox gives the same error. So, switching from Chrome to Firefox is not a solution anymore.

    We tried a lot of combinations using IP address and fully qualified domain names in the configuration, but we always get this error. We also modified the hosts files on the servers and included the other machine.

    Our Jupyter Server is running on CentOS 7.8, and the Cognos Server is running on Windows 2012.

    Does anyone have any suggestion? It seems to be a network and browser related error, but I have no idea how to fix this error.

     

    Regards,

    Balazs



    ------------------------------
    Balázs Zelenák
    ------------------------------



  • 20.  RE: Jupyter Notebook Kernel not found

    Posted 10/15/20 12:34 PM
    Hello, thanks for reaching out.

    This error usually occurs when there's a mismatch between the "Gateway URI" value and the URI you use to access Cognos in the browser.
    For example, in Cognos configuration, if your "Gateway URI" is "https://my-cognos:9300/bi/v1/disp", you'll need to enter "https://my-cognos:9300/bi" in the browser before creating a notebook. 

    Let me know if this is already the case, and we can dig further into the issue. 

    Regards,
    Rob

    ------------------------------
    ROBERT MCAULEY
    ------------------------------



  • 21.  RE: Jupyter Notebook Kernel not found

    Posted 10/16/20 03:45 AM

    Hello Robert,

    Thank you for your help. We found a similar suggestion.
    Our Gateway URI parameter: http://my-cognos:80/ibmcognos/bi/v1/disp. So, we tried to access Cognos using http://my-cognos:80/ibmcognos/bi/.

    It seems to get further. In the Notebook the kernel doesn't gives immediately an error, but stucks on connecting to kernel. After a couple of minutes, it gives a timeout error. In the docker logs there is no Cross Origin message.

    When the timeout occurs in the console log of the browser, we see WebSocket connection to: 'ws:my-cognos/ibmcognos/bi/v1/Jupiter/user/….'failed. Error during WebSocket handshake. Unexpected response code: 502.

    The response code varies. It can be 400 and 502.


    Regards,

    Balazs



    ------------------------------
    Balázs Zelenák
    ------------------------------



  • 22.  RE: Jupyter Notebook Kernel not found

    Posted 11/17/20 01:49 PM
    Hi Balázs,

    Could you solve your problem? I have a similar installation to yours and I have the same problem.
    Can anybody help me.
    Thank you.
    Sebastián.

    ------------------------------
    Sebastian Zaballa
    ------------------------------



  • 23.  RE: Jupyter Notebook Kernel not found

    Posted 11/18/20 04:02 AM
    Hello Sebastián,

    We still don't have a solution.
    I opened a support ticket for this problem. Tried out a lot of things with IBM support without success.
    Currently waiting for a response from the support engineer.

    Regards,
    Balazs

    ------------------------------
    Balázs Zelenák
    ------------------------------



  • 24.  RE: Jupyter Notebook Kernel not found

    Posted 11/18/20 11:48 AM
    Hi Sebastián and Balázs,
    Do you have an SSL enabled environment?
    If so, please try to use fully qualified domain name for Jupyter Service Location.
    Manage -> Configuration -> System -> Environment.
    We used to have a similar issue when we enabled SSL and it is resolved after I updated all the URLS with FQDN.
    Hope that works for you as well.

    Regards,
    Nag Ravulapalli.

    ------------------------------
    Nag Ravulapalli
    ------------------------------



  • 25.  RE: Jupyter Notebook Kernel not found

    Posted 11/18/20 09:42 AM
    With containers, you can't do things like edit the hosts file or use IP addresses. That doesn't sync into the container when it is started. You must use resolvable addresses. Try tailing the Jupyter hub container logs with docker logs -f <Juptyer hub container> and then try to open a notebook to see what the error might be. I bet it is an address resolution issue.

    ------------------------------
    Robert Hofstetter
    ------------------------------



  • 26.  RE: Jupyter Notebook Kernel not found

    Posted 11/19/20 11:54 AM
    We are also experiencing a similar issue after trying to upgrade CA Notebooks from 11.1.4 to 11.1.7.  We currently do not have gateway installed and configured  - is it now required when configuring Notebooks?

    In ref to the suggestion to trail Jupyter hub container logs,  we've done this and are receiving a similar error log entry that Chris shared a year ago:

    [I 2020-11-19 03:47:36.425 SingleUserNotebookApp log:158] 200 GET /bi/v1/jupyter/user/xSmFiaWxfQUQ6dToxMzI2OTQ1YTlkMzExNzQ3YWQ3ZjMxNWFmYmIyZDgwOQ__.31ed9e61476190dcb2676c14a8bad891/api/contents/new6/checkpoints?_=1605757635407 (xSmFiaWxfQUQ6dToxMzI2OTQ1YTlkMzExNzQ3YWQ3ZjMxNWFmYmIyZDgwOQ__.31ed9e61476190dcb2676c14a8bad891@::ffff:10.0.2.2) 1.65ms
    [W 2020-11-19 03:47:36.436 SingleUserNotebookApp handlers:484] Blocking Cross Origin API request for /bi/v1/jupyter/user/xSmFiaWxfQUQ6dToxMzI2OTQ1YTlkMzExNzQ3YWQ3ZjMxNWFmYmIyZDgwOQ__.31ed9e61476190dcb2676c14a8bad891/api/sessions. Origin: https://my-cognos.com, Gateway: http://my-cognos.com, Host: my-cognos.com:8000



    ------------------------------
    Jeffrey Qureshy
    ------------------------------



  • 27.  RE: Jupyter Notebook Kernel not found

    Posted 11/19/20 12:35 PM
    A gateway isn't required to run notebooks. I do see a mismatch between the "Origin" and "Gateway" values though; the origin has HTTPS and the gateway has HTTP. If you open your Cognos configuration and switch the "Gateway URI" from HTTP to HTTPS the issue should be resolved. Hope that helps!

    ------------------------------
    ROBERT MCAULEY
    ------------------------------



  • 28.  RE: Jupyter Notebook Kernel not found

    Posted 11/19/20 12:57 PM
    I noticed the gateway referenced HTTP as well but I double checked our Cognos Configuration and the Gateway URI is correctly set to HTTPS not HTTP.  Not sure where the HTTP reference is coming from.

    ------------------------------
    Jeffrey Qureshy
    ------------------------------



  • 29.  RE: Jupyter Notebook Kernel not found

    Posted 11/19/20 01:20 PM
    Hello everyone!
    I pass all the information that I have been able to collect to see if someone can detect my failure and perhaps that of other users who are in the same condition.
    The files with a CA_ prefix are those of the cognos analytics server, the JU_ files are those of the linux server where Cognos Jupyter is installed and the two images are of my machine doing tests from IE and Firefox.

    From already thank you very much!!!


    Regards.
    Sebastian.

    ------------------------------
    Sebastian Zaballa
    ------------------------------

    Attachment(s)

    log
    JU_ca_jupyter_hub.log   1 KB 1 version
    log
    JU_install.log   14 KB 1 version
    xml
    CA_cogstartup.xml   142 KB 1 version
    txt
    JU_cmplst.txt   618 B 1 version
    conf
    JU_config.conf   2 KB 1 version
    txt
    CA_cmplst.txt   13 KB 1 version


  • 30.  RE: Jupyter Notebook Kernel not found

    Posted 11/20/20 11:23 AM
    Hey Jeffrey,

    I noticed in your cogstartup.xml that the "Gateway" is set to "http://132.1.0.55:9300/ibmcognos/bi/v1/disp", but in the browser you're navigating to "http://132.1.0.55/ibmcognos/bi/v1/disp" (notice that the port 9300 is missing). In your cogstartup, try replacing the port 9300 with port 80 for the "Gateway" and restart cognos. 

    If that doesn't work, the problem might be in your IIS gateway config. It needs to be able to reroute the kernel websocket from the browser to the jupyter server. Step 6d is what you're looking for here:

    https://www.ibm.com/support/knowledgecenter/SSEP7J_11.1.0/com.ibm.swg.ba.cognos.inst_cr_winux.doc/t_gateway_iis.html


    ------------------------------
    ROBERT MCAULEY
    ------------------------------



  • 31.  RE: Jupyter Notebook Kernel not found

    Posted 11/25/20 06:45 AM
    Hello, Robert

    You were right, the problem was on that side.
    The problem was generated when I installed the CA server that I put the option of a single machine and I installed everything except the gateway and how it worked I did not realize that that part was missing.
    I uninstalled it and reinstalled it but selecting what I wanted to install.
    Finally configure the environment well, restart everything (CA and Jupyter) and it worked without problems !!!
    Thank you very much to all for the tips.
    Regards.

    ------------------------------
    Sebastian Zaballa
    ------------------------------



  • 32.  RE: Jupyter Notebook Kernel not found

    Posted 11/19/20 03:49 PM
    Hi Jeffrey,
    Do you have IIS configured on your Gateway Server?
    If so, make sure if you have any URL Rewrite properties for Jupyter and if so, they are referencing to HTTPS.

    ------------------------------
    Nag Ravulapalli
    ------------------------------