Hello,

I am not able to find out the issue where the exactly issue is, My vm backup failed last night. When I saw the log it indicate schedule is not started but I have no clue what is the issue why it is not started although when I started manually it is working fine.

I am attaching the error snapshot for better understanding. Backup is failed on 3rd of April 2016.

Regards

Prem

Prem,
It would be helpful to have more information like the platform and TSM verison of the client and Spectrum Protect (TSM) Server. But from reviewing the log you provided it appears to be a connectivity issue. There are numerous messages that indicate a problemm for instance:

ANS1815E Either the window has elapsed or the schedule has been deleted

Check the details of your schedule, you may need to increase the duration and/or duration units.

ANS1351E Session rejected: All server sessions are currently in use

Check you maximun sessions and maximum scheduled sessions parameter on the backup server by issueing the "q stat" command and increasing them as needed. This message is saying when the client is trying to perform the backup all available sessions are being used, you will need to increase the number of allowed scheduled sessions, or re-schedule the backup at a time when the backup server is not so busy.

The error I see most is obviously:

ANS5216E Could not establish a TCP/IP connection with address '10.50.59.73:1500'. The TCP/IP error is 'Unknown error' (errno = 10061). 

This is saying for whatever reason the machine cannot access the IP address and/or port specified. To troubleshoot this issue refer to the following link:

http://www-01.ibm.com/support/docview.wss?uid=swg21623552 

-Rick Adamson

edited: corrected response by changing period and period units to duration and duration units.

Rick,

Firsty I'll thank you for your response. As you mentioned probably that time session is busy it could be one reason of failure. I would like to tell you that time no backup is schedule. schedule is not delted because as i mentioned manully i am able to do it.I have added all the snapshot for your reference for better understanding, pls suggest if any changes required.

Regards

Prem Singh

Singh,

The first thing is to determine why the client is not connecting with the TSM Server.

The newest error messages from your original log show the backup clientt is trying to connect to "10.50.59.73" but in the "Q STAT" data you provided it shows the TSM Server's IP address as "172.30.1.24"

Here is the erro message:

02/05/2016 23:59:32 ANS5216E Could not establish a TCP/IP connection with address '10.50.59.73:1500'. The TCP/IP error is 'Unknown error' (errno = 10061).

Assuming that the TSM Server's IP address is in fact "172.30.1.24" you will need to update the client's configuration with the correct address. To do this edit the "dsm.opt" file located in the "baclient" directory where the TSM client is installed. Specifically look for the option tcpserveraddress, it should look like this:

TCPServeraddress  172.30.1.24

The TSM Server hostname can also be used as long as the client can resolve it.

After the change be sure to restart the backup service to pick up the changes.

Attempt a backup and post any new errors here.

-Rick Adamson

Rick,

Frankly speaking if you will ask me why it is showing I have no clue but I am sharing the snapshot where it's mentioned although this ip is not in use at all anywhere as far as my knowledge.

Regards

Prem Singh

You need to update the TSM configuration on the machine that is failing the back up with the correct address.

Edi the file "dsm.opt" in the TSM client install directory. Look for the option tcpserveraddress, and update it to look like this:

TCPServeraddress  172.30.1.24

Save the change and attempt a backup.

It may be helpfule to review the client documentation on the knowledge center for more info:

https://www.ibm.com/support/knowledgecenter/SSGSG7_6.2.0/com.ibm.itsm.nav.doc/r_pdf_clients.html?lang=en

-Rick Adamson

Rick,

My only point is if any configuration issue is there then how other schedule and backup is happening and communcation with server.

Regards

Prem

Hi.

First of all try telnet on the port 1500 of your tsm server ip address from the windows server. May be your tsm server ip address changed so you have to update the dsm.opt of your client.

Regards.

Hi Ruben,

Thanks for your response but no configuration has changed recently in atleast one year. I am able to telnet that server from any machine and windows server also.

Regards

Prem

I'd presume that the scheduled event is to run a script.  The RC=1 suggests that the script doesn't exist in the subdirectory where the CAD or Scheduler is running or the path given in the scheduled event is incorrect.

I used to like specifying fully qualified paths but that led to team confusion as AIX/Linux clients typically do not use spaces in subdirectory names where Windows does and thus causes fully qualified paths to have to be doubly quoted ('"filespec"') so now we use relative paths like ..\TDPSql\<filename> which are relative to the baclient subdirectory where most CADs/Schedulers are started.

Need futher help?  Please provide output of 'Query SCHedule <domain> <schedule_name> F=D'.

Hi Ano,

I am attching the details of command which you requested, one more thing full path has been given. pls let me know if any other details required.

Regards

Prem

Prem,
My bad, my response was regarding all the communication errors that were in the original log you posted. After revisiting it I noticed that they are not current error messages.
I would have to agree with Ano, from your latest post of the schedule details the schedule is trying to execute command “e:\vcb\Scripts\av-db-mgmt.cmd”.

Have you confirmed that the file av-db-mgmt.cmd exists in that location.


-Rick Adamson
904.783.5264

From: Prem Singh [mailto:storage-ti@lists.imwuc.org]
Sent: Friday, April 08, 2016 4:07 AM
To: storage-ti@lists.imwuc.org
Subject: [storage-ti] - RE: Schedule Backup is Failed


Rick,

My only point is if any configuration issue is there then how other shcudule and backup is happening and communcation with server.

Regards

Prem

-----End Original Message-----

Rick,

Firsty I'll thank you all of you for investing such a precious time with me thank you for that. Yes I verified that is there only that's why with the help of this script I am able to take the backup manually once again additionally for testing purpose I have changed the schedule to shift in another day becuase that is a monthly backup it runs only once in a month. That work's perfectly it means automatically also when script runs as per schedule that is working. That happened in the past also in few other backups  that time also I am not able to pin point the issue. I have attached the path snapshot also to just confirm it.

Regards

Prem

Prem,

as Rick posted the first thing to do is verify that the file exists:

dir <filespec from schedule>

If that works then you need to verify what account the CAD/Scheduler is running under (typically System) and that that account has permission to execute the file.

If the schedule ever worked then something has changed.

Ano,

Thanks for the response, yes I verified the file exist, and schedule is running under system only for this I am attaching a snashot also for clear understanding. yes schedule worked earlier also, after getting this issue for testing when I changed the schdule and when it ran automatically that time also it worked perfectly. moreover nothing changed typically in the system or configuration becuase no one has the autority to access this server, as i replied to Rick even I wonder many times why it's faling frequently.

Regards

Prem

Here are a couple of links that might apply but without access i dont see much more that can be done.

This link is a good way to test but if you cant access the client then it wont help:
http://www-01.ibm.com/support/docview.wss?uid=swg21284034
This one may help if you are reusing an internal log file within the script:
https://adsm.org/forum/index.php?threads/tdp-for-sql-schedule-fails-with-ans1909e-rc-1.12740/

 Do you have a copy of the script?  Presuming you can post it.

Ano,

I am attaching the script and service snapshot meanwhile I am checking the link which you shared.The only thing I wanted to update before posting script I have edited the user name and password.

Regards

Prem

Prem,
To help solve the issue can you change the schedule to run everyday? Then you can search the error,schedule and actlog for all occurances for that client.

On the morning after the schedule is suppseed to run query the TSM server actlog for all entries for the client.

For example: 

-open the TSM Server administrative command line

-enter this command replacing "client-name" with the actual client TSM node name: query actlog begint=-24 search=client-name

-post the results along with any new error messages in the client error log so I can review.

-Rick Adamson

Hi Rick,

Thanks for bearing with me so long, Practically that is not possible to change the schedule to daily in my environment becuase from vcb roxy backup is coming on local server then it's moving into tape and in local server I don't have much space to manage multiple backups daily.moreover I don't have space available on storage also.I will monitor the montly backup for sure anything else which i can do it with your guidance please let  me know apart from managing the space.

Regards

Prem

Hi Rick,

I am attaching a snapshot of daily backup which failed yesterday only I capture the actlog also. Please let me know anything else which i need to share.

Regards

Prem

Prem,

The ANE4005E "file not found" messages are usually caused by transient data. 

Before TSM backs up data it inspects the file space and generates a list of objects that need to be backed up, then it goes back and performs that actual backup. The ANE4005E message is saying that after the inspection was performed the data was removed before TSM could back it up.

This is usually temporary, or transient, data which can be excluded from the backup process. You may prefer to confirm before actually excluding it.

Also, notice in your error log file you are still receiving the TCPIP errors:

04/12/2016 15:09:05 ANS5216E Could not establish a TCP/IP connection with address '10.50.59.73:1500'. The TCP/IP error is 'Unknown error' (errno = 10061).
04/12/2016 15:09:05 ANS9020E Could not establish a session with a TSM server or client agent. The TSM return code is -50.
04/12/2016 15:09:05 ANS1017E Session rejected: TCP/IP connection failure

You will need to determine why these are happening, assuming you are using the default port of 1500 on both the backup server and client make sure that a client firewall is not blocking it. May be best to try and establish a telnet session using that port from the client.

-Rick Adamson

Rick,

I tried telnet earlier also it is happening and moreover I have not made any changes today backup is successful as you said we can exclude temporary, or transient, data will that impact my production. I check all the relevant port is allowed on firewall. Yes as of now I am using the default port 1500. The most important thing is in my environment it is happening with different backup schedule on frequently. That is why i am not able to reach to any conclusion what is the exact cause for it.

Regards

Prem