Hi,
We have experienced weird behaviour occasionally with our ISVA containers. Liveness or readiness- probes have failed with following message
Error> an invalid response code was received from the service. cat: /tmp/health_check.hdr: No such file or directory
Our liveness and readiness- probes were both configured to run with 10 s interval. When looking at the health_check.sh, script is storing curl- commands results to these files. After curl is executed, script checks the outcome from the files.
# Some temporary files used by this script.
result_file=/tmp/health_check.out
error_file=/tmp/health_check.err
hdr_file=/tmp/health_check.hdr
cert_file=/tmp/health_check.pem
The script also contains this trap- command, clearing the files
trap "rm -f $result_file $error_file $hdr_file" EXIT
We cannot be sure if the problem is in fact on above, but when we modified the probe intervals so that liveness is executed every 10s and readiness every 7s, problem vanished.
Just by looking at the script, it seems that there is in fact possiblity that if both liveness and readiness- probes are executing at same time with just small time difference, the first execution clears the output files (on exit) which the second execution has just written - causing the second execution failing.
Could somebody confirm this is the case?
Edit: The script file seems to have other references to various fixed temporary files under /tmp, which may equally well either ruin the runs or even provide wrong results if both probes execute same time. Migth be good to try insulate runs for liveness and readiness-probe to use fully separate tmp files.
Br Jan
------------------------------
Jan Lindstam
------------------------------