Stan's Corner

 View Only
Expand all | Collapse all

Need help creating a VERY simple setup Python call REST API -> view results in Instana

  • 1.  Need help creating a VERY simple setup Python call REST API -> view results in Instana

    Posted Wed May 11, 2022 06:50 AM
    I am new to Instana.

    Need to create an extremely SIMPLE setup to show to the customer:

    1) Call Instana REST API from Python script, the REST API call will send custom health check values (availability, response time).
    2) Show the results in Instana GUI.

    Questions:
    1) Is there an Instana playground online available for this or do I need to install Instana (for example in my laptop)
    2) Need Python code example. Script will be extremely simple, 20-30 lines tops.

    https://instana.github.io/openapi/


    ------------------------------
    Claudio Nunez-Giordano
    ------------------------------


  • 2.  RE: Need help creating a VERY simple setup Python call REST API -> view results in Instana

    Posted Thu May 12, 2022 11:35 AM
    Hi Claudio,

    About your points:
    1) Call Instana REST API from Python script, the REST API call will send custom health check values (availability, response time).
    Re: You cannot use the REST API to send metrics. Instana only receives metrics from sensors, SDK & OpenTelemetry using the agent. You may send metrics to the agent using statsd (https://www.ibm.com/docs/en/obi/current?topic=technologies-monitoring-statsd), or create traces with information about latency using the Trace SDK Webservice (https://www.ibm.com/docs/en/obi/current?topic=apis-host-agent-rest-api#trace-sdk-web-service)

    2) Show the results in Instana GUI.
    Re: Statsd metrics display in the host dashboard. Trace SDK generates traces, the Service view will show the golden signals (calls, latency, errors).

    And your questions:
    1) Is there an Instana playground online available for this or do I need to install Instana (for example in my laptop)
    Re: No. Instana on-premise needs a lot of power (64 GB RAM, 16 cores, 1 TB). Use the SaaS trial.

    2) Need Python code example. Script will be extremely simple, 20-30 lines tops.
    Re: Take a look at the docs.

    Good luck,


    ------------------------------
    Luis Sanchez
    luis@impulseit.com
    Chile
    ------------------------------



  • 3.  RE: Need help creating a VERY simple setup Python call REST API -> view results in Instana

    Posted Thu May 12, 2022 02:09 PM

    Hi Luis,
    Thanks a lot for your response
    About your points:
    1) Re: You cannot use the REST API to send metrics. Instana only receives metrics from sensors, SDK & OpenTelemetry using the agent. You may send metrics to the agent using statsd (www.ibm.com/docs/en/obi/...), or create traces with information about latency using the Trace SDK Webservice (www.ibm.com/docs/en/obi/...
    Claudio comment: Great, Luis, thanks. Will have a look at the documentation. Unfortunately from tomorrow until the end of May I will be spending a holiday with the family. Will continue working on the issue from 1st of June.   

    2) Show the results in Instana GUI.
    Re: Statsd metrics display in the host dashboard. Trace SDK generates traces, the Service view will show the golden signals (calls, latency, errors).
    Claudio comment: Sounds good. Not sure if I grasp everything will study the StatsD documentation in detail.

    And your questions:
    1) Is there an Instana playground online available for this or do I need to install Instana (for example in my laptop)
    Re: No. Instana on-premise needs a lot of power (64 GB RAM, 16 cores, 1 TB). Use the SaaS trial.
    Claudio comment: Yes, I received an invitation and have accessed the SaaS trial have been studying the gui as well.

    2) Need Python code example. Script will be extremely simple, 20-30 lines tops.
    Re: Take a look at the docs.
    Comment: With all your information I am closer to a solution. Thanks a lot!

    Good luck,


    ------------------------------
    Luis Sanchez
    luis@impulseit.com
    Chile
    ------------------------------



    ------------------------------
    Claudio Nunez-Giordano
    ------------------------------



  • 4.  RE: Need help creating a VERY simple setup Python call REST API -> view results in Instana

    Posted Mon May 30, 2022 05:00 AM
    Edited by Claudio Nunez-Giordano M.Sc. Mon May 30, 2022 05:30 AM
    Hi Instana users,

    Sorry my delay, I've been 15 days out of office.

    I seem to be closer have installed the Instana Agent in my laptop. I would like someone to give me an example of Python code to send a string of data to the agent and be able to see it in the SaaS trial GUI which I have access to.

    Installed agent in my Windows laptop

    I am running on Windows and see in the documentation that 'Python AutoTrace is: supported on Linux 64bit binaries only'
    https://www.ibm.com/docs/en/obi/current?topic=package-python-configuration-configuring-instana#host-agent-communication 

    Could someone explain step-by-step how to setup and code a short python script that sends data in Windows 10 OS with StatsD or Trace SDK Webservice to the agent installed in the same Windows laptop so that I can see the result displayed in the SaaS trial interface?

    At the moment I just need to send a hardcoded string (maybe with random or date parts) just to get the end-to-end solution up and running.

    Later it will send real availability and response time of some services.

    Thanks a lot in advance. 
    '

    ------------------------------
    Claudio Nunez-Giordano M.Sc.
    Sweden
    ------------------------------



  • 5.  RE: Need help creating a VERY simple setup Python call REST API -> view results in Instana

    Posted Mon May 30, 2022 10:52 AM
    Hi Claudio,

    With the agent installed you should see Infrastructure Metrics coming from your Windows laptop. Keep in mind the Instana agent isn't a passive agent waiting for data. It's an active agent with continuous discovery of components, automatic & dynamic application of sensors and metrics trasmissión to the backend.

    Having said so, the agent is running and sending data. You mentioned some custom data to send via REST API, as a said before this operation is not allowed. You could send data through the agent using statsd. Configure the statsd in the agent, https://www.ibm.com/docs/en/obi/current?topic=technologies-monitoring-statsd, restart the agent and then run a script like the following:
    # Mini statsd sender
    # LaS (c) 2022
    import socket
    from random import randrange
    from time import sleep

    # Send data with label
    def send_temp(label,temp):
      byte_message = bytes(f"{label}:{temp}|g", "utf-8")
      opened_socket = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
      opened_socket.settimeout(2.0)
      opened_socket.sendto(byte_message, ("127.0.0.1", 8125))


    label="dc1-temp"

    print("Mini statsd sender. Press Ctrl-C to stop")

    # Infinite loop, send data each second
    while True:
      temp=randrange(15) + 15
      print (f"{label}:{temp}|g")
      send_temp(label, temp)
      sleep(10)

    The script sends a random value between 15 and 29 to the agent every 10 seconds. The resulting metrics timeline is available in the host dashboards of your laptop, after TCP activity:
    Screen Shot 2022-05-30 at 10.39.02.png

    Regards,

    Luis








  • 6.  RE: Need help creating a VERY simple setup Python call REST API -> view results in Instana

    Posted Mon May 30, 2022 01:16 PM
    Edited by Claudio Nunez-Giordano M.Sc. Mon May 30, 2022 01:59 PM
    Thanks a lot, Luis. It's clear for me I can't use REST API. StatsD will do just fine.
    Almost everything in place:

    1) configuration.yaml corrected according to the instruction with StatsD section activated
    2) Agent in laptop restarted
    3) 'Mini statsd sender' Python script implemented and running in cmd

    However I am unsure where to see the displayed metrics and under what 'name' (dc1-temp maybe?)

    I've been searching in the SaaS trial gui in 'Live' mode, however have not been able to find the displayed metrics.

    Feel closer to a working solution though. Thanks again for your input.

    https://postimg.cc/8FVXD0Lh

    https://postimg.cc/v1H9W7sJ

    https://i.postimg.cc/BZP2rmL9/Instana-configurationyaml.jpg
    ------------------------------
    Claudio Nunez-Giordano M.Sc.
    Sweden
    ------------------------------



  • 7.  RE: Need help creating a VERY simple setup Python call REST API -> view results in Instana

    Posted Tue May 31, 2022 04:22 PM
      |   view attached
    A picture is worth a thousand words:
    custom-metrics.gif

    Regards,

    Luis








  • 8.  RE: Need help creating a VERY simple setup Python call REST API -> view results in Instana

    Posted Tue May 31, 2022 05:36 PM
    Edited by Claudio Nunez-Giordano M.Sc. Tue May 31, 2022 05:46 PM
    Thanks. I cannot see my server (or any other Windows server by the way) in section Instana Infrastructure.

    However I can see in section Instana Event that something is coming in from my agent.
    Don't know if 'Server Down' was when I restarted the agent. After 23:22:09 seems to be coming in normally.

    Maybe this tells you what can be wrong. Thank you.


    Brgds


    ------------------------------
    Claudio Nunez-Giordano M.Sc.
    ------------------------------



  • 9.  RE: Need help creating a VERY simple setup Python call REST API -> view results in Instana

    Posted Tue May 31, 2022 05:50 PM
    Hi,

    Review the agent log, it is located at <Instana-Agent-Home>/data/log/agent.log.

    Regards,

    Luis








  • 10.  RE: Need help creating a VERY simple setup Python call REST API -> view results in Instana

    Posted Wed June 01, 2022 05:43 AM
    Edited by Claudio Nunez-Giordano M.Sc. Wed June 01, 2022 05:45 AM
    Instana SaaS trial GUI displays the agent just fine now. However only with standard entities Instana monitors: CPU Load, System Memory, Garbage Collection, etc.

    The agent detects the running Python script but does not display the metrics sent to the agent by the python script.



    The metrics sent from the Python script are nowhere to be found. Perhaps its caused by the 'Monitoring Issues' displayed in the Dashboard:




    ------------------------------
    Claudio Nunez-Giordano M.Sc.
    ------------------------------



  • 11.  RE: Need help creating a VERY simple setup Python call REST API -> view results in Instana

    Posted Wed June 01, 2022 08:59 AM
    I have succeeded with the complete end-to-end solution including sending custom metrics to Instana SaaS trial GUI and having it displayed in the Dashboard. Let me thank you for your input that was of great help.



    ------------------------------
    Claudio Nunez-Giordano M.Sc.
    ------------------------------



  • 12.  RE: Need help creating a VERY simple setup Python call REST API -> view results in Instana

    Posted Wed June 01, 2022 09:15 AM
    The discovery of the Python script is expected, this is the nature of Instana. However this isn't the point, the script is not interesting at all, it's mission is sending metrics with statsd.

    If you enable the statsd in the config and restarted the agent, the port is open. The script is sending packets through the port, and if everything is right with your configuration the metric is there, in your host dashboard. Double check everything because the issue is on your side.

    Regards,

    Luis








  • 13.  RE: Need help creating a VERY simple setup Python call REST API -> view results in Instana

    Posted Fri June 03, 2022 06:43 AM
    Yes, yes, it is exactly as you say. When agent was installed it immediately started sending standard data from my laptop to Instana, like CPU, memory, etc
    and it included info that a python script was running but StatsD was still at that point not enabled.

    As soon as I updated configuration.yaml uncommenting the StatsD section it started sending the Custom Metrics.
    I didn't need to restart the agent, changes in config are hot reloaded by the agent, the StatsD data was displayed immediately in Instana Dashboard. Thanks a lot!

    ------------------------------
    Claudio Nunez-Giordano M.Sc.
    ------------------------------