Hi Expert,

I am unable to understand why that error is coming checked on log attaching it if anyone can give me any clue.

dsmerror-

08/11/2016 05:47:12 ANS0361I DIAG: sessSendVerb: Error sending Verb, rc: -50
08/11/2016 05:47:12 ANS1809W A session with the TSM server has been disconnected. An attempt will be made to reestablish the connection.
08/11/2016 05:47:13 ANS1809W A session with the TSM server has been disconnected. An attempt will be made to reestablish the connection.
08/11/2016 09:47:58 ANS1577I The Windows console event handler received a 'Close' console event.

dsmsched-

08/11/2016 11:35:25 ANS1898I ***** Processed   374,500 files *****
08/11/2016 11:35:28 ANS1898I ***** Processed   375,000 files *****
08/11/2016 11:35:30 ANS1898I ***** Processed   375,500 files *****
08/11/2016 11:35:35 ANS1898I ***** Processed   376,000 files *****
08/11/2016 11:35:37 ANS1898I ***** Processed   376,500 files *****
08/11/2016 11:35:41 ANS1898I ***** Processed   377,000 files *****
08/11/2016 11:35:44 ANS1898I ***** Processed   377,500 files *****
08/11/2016 11:35:47 ANS1898I ***** Processed   378,000 files *****
08/11/2016 11:35:51 ANS1898I ***** Processed   378,500 files *****
08/11/2016 11:35:55 ANS1898I ***** Processed   379,000 files *****
08/11/2016 11:36:02 ANS1898I ***** Processed   379,500 files *****
08/11/2016 11:36:05 ANS1898I ***** Processed   380,000 files *****

Session of node-

tsm: TSMSERVER>q sess

  Sess Comm.  Sess     Wait   Bytes   Bytes Sess    Platform Client Name
Number Method State    Time    Sent   Recvd Type
------ ------ ------ ------ ------- ------- ------- -------- ----------------
 2,432 Tcp/Ip IdleW  16.7 M 546.8 M   8.5 M Node    WinNT    COGNOS
 2,443 Tcp/Ip RecvW   11 S   27.1 K  29.1 G Node    WinNT    COGNOS

All i can see still session is running sched is at 1:30 am and session is running till 6 PM still running and files also processing not able to understand what's going wrong.

Regards

Prem

Hi,

Waiting for the response.

Regards

Prem

A network issue ?

Hi Philippe,

I was guessing the same becuase after repeated attempts it is still failing but then I found my guess was wrong because other backup is happening and completing successfully. TSM server is located on cloud and other server also on cloud  but privately and publicly on a different subnet. we have assigned 2 TB's 2 lun to tsm server which we are using as a storage in sort all the node backup is moving into TSM server only. let me show you the actlog.

Actlog-

-------------------- ----------------------------------------------------------
08/16/2016 12:29:36  ANR2017I Administrator ADMIN issued command: QUERY ACTLOG
                      begind=-1 se=cognos  (SESSION: 2793)
08/16/2016 12:44:54  ANR2017I Administrator ADMIN issued command: QUERY ACTLOG
                      begind=-1 se=cognos  (SESSION: 2794)
08/16/2016 13:26:46  ANR2017I Administrator ADMIN issued command: QUERY
                      SCHEDULE windom daily_incr f=d node=cognos  (SESSION:
                      2795)
08/16/2016 13:27:03  ANR2017I Administrator ADMIN issued command: QUERY NODE
                      cognos f=d  (SESSION: 2795)
08/16/2016 14:45:07  ANR2017I Administrator ADMIN issued command: QUERY
                      SCHEDULE windom daily_incr f=d node=cognos  (SESSION:
                      2798)
08/16/2016 15:10:34  ANR0406I Session 2800 started for node COGNOS (WinNT)
                      (Tcp/Ip 10.66.233.80(51678)). (SESSION: 2800)
08/16/2016 16:09:58  ANR0481W Session 2779 for node COGNOS (WinNT) was
                      terminated. The client did not respond within 3600
                      seconds. (SESSION: 2779)
08/17/2016 02:50:01  ANR2578W Schedule DAILY_INCR in domain WINDOM for node
                      COGNOS has missed its scheduled start up window.
08/17/2016 08:26:52  ANR0406I Session 2814 started for node COGNOS (WinNT)
                      (Tcp/Ip 10.66.233.80(64632)). (SESSION: 2814)
08/17/2016 09:10:55  ANR2017I Administrator ADMIN issued command: QUERY ACTLOG
                      begind=-1 se=cognos  (SESSION: 2815)
08/17/2016 09:11:50  ANR2017I Administrator ADMIN issued command: QUERY ACTLOG
                      begintime=-24 se=cognos  (SESSION: 2815)
08/17/2016 09:26:28  ANR0481W Session 2800 for node COGNOS (WinNT) was
                      terminated. The client did not respond within 3600
                      seconds. (SESSION: 2800)
08/17/2016 12:43:11  ANR0406I Session 2830 started for node COGNOS (WinNT)
                      (Tcp/Ip 10.66.233.80(57989)). (SESSION: 2830)
08/17/2016 13:42:35  ANR0481W Session 2814 for node COGNOS (WinNT) was
                      terminated. The client did not respond within 3600
                      seconds. (SESSION: 2814)
08/17/2016 16:49:07  ANR0406I Session 2840 started for node COGNOS (WinNT)
                      (Tcp/Ip 10.66.233.80(50655)). (SESSION: 2840)
08/17/2016 16:50:56  ANR2017I Administrator ADMIN issued command: QUERY ACTLOG
                      se=cognos begind=-2  (SESSION: 2839)
08/17/2016 17:48:42  ANR0481W Session 2830 for node COGNOS (WinNT) was
                      terminated. The client did not respond within 3600
more...   (<ENTER> to continue, 'C' to cancel)

Session-

tsm: TSMSERVER>q sess

  Sess Comm.  Sess     Wait   Bytes   Bytes Sess    Platform Client Name
Number Method State    Time    Sent   Recvd Type
------ ------ ------ ------ ------- ------- ------- -------- ------------------
 2,860 Tcp/Ip RecvW    0 S    1.6 K  14.5 G Node    WinNT    COGNOS

Regards

Prem

Hi,

I did't cancel the session it is still running but in event log it shows servered. here is the snapshot below./

tsm: TSMSERVER>q eve * * begind=-4

Scheduled Start      Actual Start         Schedule Name Node Name     Status
-------------------- -------------------- ------------- ------------- ---------
08/16/2016 01:50:00  08/16/2016 01:51:04  DAILY_INCR    BACKUP01      Completed
08/16/2016 01:50:00  08/16/2016 02:00:03  DAILY_INCR    COGNOS        Severed
08/16/2016 01:50:00  08/16/2016 01:50:41  DAILY_INCR    CONTENT_NEW   Completed
08/16/2016 02:30:00  08/16/2016 02:30:49  DAILY_INCR    BD01          Completed

 Session-

tsm: TSMSERVER>q sess f=d

              Sess Number: 2,883
             Comm. Method: Tcp/Ip
               Sess State: RecvW
                Wait Time: 0 S
               Bytes Sent: 10.1 K
              Bytes Recvd: 125.0 G
                Sess Type: Node
                 Platform: WinNT
              Client Name: COGNOS
      Media Access Status: Current output volumes:  WINFILESTG,H:\TSM_FILECLAS-
                            S\0000028D.BFS,(7428 Seconds)
                User Name:
Date/Time First Data Sent: 08/19/2016 08:17:33
   Proxy By Storage Agent:
                  Actions: BkIns
            Failover Mode: No

              Sess Number: 2,918
             Comm. Method: Tcp/Ip
               Sess State: Run
                Wait Time: 0 S
               Bytes Sent: 110.6 K
              Bytes Recvd: 2.0 K
                Sess Type: Admin
                 Platform: WinNT
              Client Name: ADMIN
      Media Access Status:
                User Name:
Date/Time First Data Sent:
   Proxy By Storage Agent:
                  Actions:
            Failover Mode: No

which session showing severed after that this is new session number which i pasted above still backup is running not able to understand why it's not completing after running 4 days continuosly. it can't be a network issue becuase other server backup is happening which is on same subnet.

Regards

Prem

Hi,

I have cancel the session yesterday today it is started again still not completed. here is the details of running session.

Sess-

tsm: TSMSERVER>q sess f=d

              Sess Number: 2,967
             Comm. Method: Tcp/Ip
               Sess State: RecvW
                Wait Time: 0 S
               Bytes Sent: 46.9 K
              Bytes Recvd: 96.1 G
                Sess Type: Node
                 Platform: WinNT
              Client Name: COGNOS
      Media Access Status: Current output volumes:  WINFILESTG,H:\TSM_FILECLAS-
                            S\000002A3.BFS,(838 Seconds)
                User Name:
Date/Time First Data Sent: 08/21/2016 02:07:36
   Proxy By Storage Agent:
                  Actions: BkIns BkDel
            Failover Mode: No

on Action it is showing Bkins and Bkdel seems few are deleted from server and few are taking backup as a part of incremental not sure what's going wrong.

Regards

Prem

This sounds like something that needs level 2 TSM support - they can set up a trace and find out what is going wrong.

Hi,

Pls find the latest log still it's failing waited many days not able to understand when it's keep on transferring the data still it shows it is backed up less than lakh object .

Log:-

08/24/2016 07:44:59 Total number of objects inspected:    7,604,116
08/24/2016 07:44:59 Total number of objects backed up:       93,442
08/24/2016 07:44:59 Total number of objects updated:              0
08/24/2016 07:44:59 Total number of objects rebound:              0
08/24/2016 07:44:59 Total number of objects deleted:              0
08/24/2016 07:44:59 Total number of objects expired:            544
08/24/2016 07:44:59 Total number of objects failed:               5
08/24/2016 07:44:59 Total number of objects encrypted:            0
08/24/2016 07:44:59 Total number of subfile objects:              0
08/24/2016 07:44:59 Total number of objects grew:                 0
08/24/2016 07:44:59 Total number of retries:                    113
08/24/2016 07:44:59 Total number of bytes inspected:         483.44 GB
08/24/2016 07:44:59 Total number of bytes transferred:       379.99 GB
08/24/2016 07:44:59 Data transfer time:                  268,001.21 sec
08/24/2016 07:44:59 Network data transfer rate:            1,486.74 KB/sec
08/24/2016 07:44:59 Aggregate data transfer rate:          1,423.66 KB/sec
08/24/2016 07:44:59 Objects compressed by:                        0%
08/24/2016 07:44:59 Total data reduction ratio:               21.40%
08/24/2016 07:44:59 Subfile objects reduced by:                   0%
08/24/2016 07:44:59 Elapsed processing time:               77:44:35
08/24/2016 07:44:59 --- SCHEDULEREC STATUS END
08/24/2016 07:44:59 ANS1369E Session Rejected: The session was canceled by the server administrator.

08/24/2016 07:44:59 --- SCHEDULEREC OBJECT END DAILY_INCR 08/21/2016 01:50:00
08/24/2016 07:44:59 ANS1512E Scheduled event 'DAILY_INCR' failed.  Return code = 12.
08/24/2016 07:44:59 Sending results for scheduled event 'DAILY_INCR'.
08/24/2016 07:45:04 Session established with server TSMSERVER: Windows
08/24/2016 07:45:04   Server Version 7, Release 1, Level 3.0
08/24/2016 07:45:04   Server date/time: 08/24/2016 07:44:21  Last access: 08/24/2016 07:44:16

08/24/2016 07:45:04 Results sent to server for scheduled event 'DAILY_INCR'.

Regards

Prem

Are you trying to start another session while the first is still processing?

Hi Cheryl,

No I am not the situtation is when that schedule should run in my case that is cognos it'e keep on running I have check what all data needs to backed up that is around 600 + GB it is incremental backup but first time it will take the complete backup which is not completing earlier i was cancelling the session now from 16th Aug I have not cancel the session it is still running. the data is moving over cloud.

Regards

Prem