Tape Storage

 View Only
Expand all | Collapse all

Errors with LTO-8 drive

  • 1.  Errors with LTO-8 drive

    Posted Wed December 29, 2021 04:33 PM
    Hello to the support community.

    So I am looking to see if anyone might be able to help with some issues I am having with a new LTO-8 drive (TS2280).  I had a different thread going where I was having some software issues with the drive on a Linux machine. I moved the drive over to a Windows 10 computer to see if that would work better.

    With this new setup, I am getting errors sometimes when writing files to tape.  I am able to sometimes write files with no issues.  But often the drive will write files and then an error will come up.  On the front of the drive I see a '6' yellow error light.  I eject the tape.  I then need to run the cleaning tape because it will then say 'C'.  After that.. the error lights are gone and I can run another test.  I will still have the same issue.

    I'm using an internal ATTO SAS card (the ATTO H1244) and have installed their latest driver and firmware for the card.

    I've tried a few different blank LTO-8 Fuji tapes, and also putting the drive into a few different computers.  I also try different files to write to the tape.. so it's not always the same files.  I'm just using the IBM Spectrum SDE 2.4.4 software. I've also tried 2 different new LTO-8 internal SAS drives.  Both have given the '6' error.

    I just can't figure out why it will error out and give me the '6' message.  Can anyone shed any light or offer any suggestions?  Thank you so much.

    -Peter Berg



    ------------------------------
    Peter Berg
    ------------------------------


  • 2.  RE: Errors with LTO-8 drive

    IBM Champion
    Posted Thu December 30, 2021 03:05 AM
    Hi Peter,

    LTO tape libraries and drives is works with server operating systems like Win2016, Win2019 etc. It is not supported on Win10
    you can check compatible operating systems, connectivity and device drivers on IBM System Storage Interoperation Center (SSIC)
    https://www-03.ibm.com/systems/support/storage/ssic/interoperability.wss

    Storage Family >> IBM System Storage LTO Ultrium Tape
    Storage Model >> TS2280 with Ultrium 8 HH Drives
    Storage Version >> TS2280 with LTO8 HH (MA71)
    Host Platform >> x86 & x86_64
    Connection Protocol >> SAS
    Server Model >> ALL x86_64 Compatible Servers

    ------------------------------
    NEZIH BOYACIOGLU
    ------------------------------



  • 3.  RE: Errors with LTO-8 drive

    Posted Thu December 30, 2021 04:37 AM
    Thank you Nezih.

    I had found this info in the Installation page for Spectrum SDE 2.4.4.  Since it said Windows 10 on that page.. I guess I assumed that meant it could be used with that OS.  I guess SDE can be used with Windows 10.. but the actual LTO drive (TS2280) Needs Windows Server.  And Seeing what it says here for SDE... I guess Windows Server 2016 would be the best choice?

    I also see my ATTO SAS HBA isn't listed on the Interoperation Center.  Maybe I should also try the LSI or Broadcom?

    Distribution packages

    Distribution packages are available for IBM Spectrum® Archive Single Drive Edition for Linux®, Mac, or Windows.

    IBM Spectrum Archive Single Drive Edition is available in the following forms:

    Windows

    • An installation package (.exe) for the following distributions:
      • Windows 10 (Anniversary update or later)
      • Windows Server 2016 (64-bit)



    -Peter Berg

    ------------------------------
    Peter Berg
    ------------------------------



  • 4.  RE: Errors with LTO-8 drive

    Posted Thu December 30, 2021 11:47 AM
    Peter,
       Go to the following site and put in your configuration. Suspect that HBA you are using is the issue cause have seen problems with it.

    https://www-03.ibm.com/systems/support/storage/ssic/interoperability.wss

    Bobby

    ------------------------------
    Bobby Bohuslav
    ------------------------------



  • 5.  RE: Errors with LTO-8 drive

    Posted Thu December 30, 2021 02:32 PM
    @Peter Berg There are debug recommendations in the IBM TS2280 Setup, Operator, and Service Guide, Machine Type 3580 Model H8S. Look at Appendix A under SCD 6.
    https://www.ibm.com/support/pages/ibm-ts2280-setup-operator-and-service-guide-machine-type-3580-model-h8s

    Also, the IBM Tape Diagnostic Tool may be of use. There are some tests it can run against the drive to confirm its health. It can also be used to capture a drive dump (i.e., debug log) that can then be looked at in ITDT and subsequently sent to IBM Support if your drive is under support.
    https://www.ibm.com/docs/en/spectrum-archive-ee/1.2.5.0?topic=tools-downloading-tape-diagnostic-tool

    ------------------------------
    Kevin Butt
    ------------------------------



  • 6.  RE: Errors with LTO-8 drive

    Posted Thu December 30, 2021 03:24 PM
    @Peter Berg
    Debug instructions for various SCD values (e.g., 6) are found in Annex A of the IBM TS2280 Setup, Operator, and Service Guide, Machine Type 3580 Model H8S found at https://www.ibm.com/support/pages/ibm-ts2280-setup-operator-and-service-guide-machine-type-3580-model-h8s

    The IBM Tape Diagnostic Tool https://www.ibm.com/docs/en/spectrum-archive-ee/1.2.5.0?topic=tools-downloading-tape-diagnostic-tool contains some test for determining the health of a Tape Drive. It also provides a means to extract a drive dump (i.e., error log) and then view some parts of it. If that is not enough, then the drive dump can be sent through IBM Support for analysis, assuming your drive is covered by support.​

    ------------------------------
    Kevin Butt
    ------------------------------



  • 7.  RE: Errors with LTO-8 drive

    Posted Thu December 30, 2021 04:16 PM
    Thank you Kevin for the tips.  I have done some of that troubleshooting and I did run some tests with the diagnostic tool and it didn't find any obvious errors with the drive.

    Maybe that points to me using Windows 10, and also possibly the ATTO HBA card as the source of the problems.  I will work on getting one of the approved/tested Broadcom cards and also switching the OS to Windows Server 2016 and see if that resolves the issues.  I'll try to report back with what I figure out.

    All your help is much appreciated.

    -Peter Berg

    ------------------------------
    Peter Berg
    ------------------------------



  • 8.  RE: Errors with LTO-8 drive

    Posted Thu December 30, 2021 04:57 PM
    Peter, I would be surprised if an SCD '6' is caused by an HBA. SCD '6' indicates either a media problem or a hardware problem. An interface problem would report as an SCD '8'.

    ------------------------------
    Kevin Butt
    ------------------------------



  • 9.  RE: Errors with LTO-8 drive

    Posted Thu December 30, 2021 05:02 PM
    Sorry, I posted sooner than I wanted to. Log Page 16: Tape Diagnostic Data has a listing of Media/Hardware errors that have been returned. If you can read that live, or If you get a drive dump via ITDT and then view it in ITDT, you might find useful information about the specific error being seen. I don't know if the Spectrum Archive log contains anything specific about what error was returned.

    ------------------------------
    Kevin Butt
    ------------------------------



  • 10.  RE: Errors with LTO-8 drive

    Posted Fri December 31, 2021 08:32 AM
    Thanks Kevin.

    That is also what I saw with the '6' error.  here is the confusing part.

    I tried my write tests with 3 different pieces of tape stock.  2 Fuji branded tapes and one HP.  seems sort of unlikely that all 3 would be bad.

    So I then bought a second new LTO-8 drive.  same model number. I got this one from a different seller.  Both give me the same error.  Really has me baffled.  I did run the ITDT tool on one of the drives.  With the basic test, it didnt report any errors.  I could try to do more tests in there if that might help figure out the error.

    The OS and HBA seemed to maybe make sense since those stayed consistant with my testing.  But if its not either of those.. Im totatly stumped again.  I do know that both drives have older firmware/microcode on them.  seems that I cant update that, as I dont have a current IBM support contract.  Unfortunate that those firmware files cant be downloaded without a service contract.  But maybe its nothing to do with firmware.

    I appreaciate all the assistance.  We have an IBM LTO-5 drive that was pretty easy to setup and has been running great for years.  these two new LTO-8 drives just have me baffled.  Thanks much.

    Peter

    ------------------------------
    Peter Berg
    ------------------------------



  • 11.  RE: Errors with LTO-8 drive

    Posted Fri December 31, 2021 12:21 PM
    Peter,

    It does seem confusing. The suggestion from Bobby Bohuslav should be followed. He has a lot of experience responding to IBM Tape drive issues.
      The other thing that can be done is to follow the instructions in the service guide for SCD '6' (https://www.ibm.com/support/pages/ibm-ts2280-setup-operator-and-service-guide-machine-type-3580-model-h8s) and manually run the diagnostic tests recommended.
      Aside from this, and the previous suggestion to look at Log Page 16, I don't know what else can be done without involving IBM Support.

    ------------------------------
    Kevin Butt
    ------------------------------



  • 12.  RE: Errors with LTO-8 drive

    Posted Mon January 03, 2022 03:05 PM
    Edited by Peter Berg Mon January 03, 2022 03:07 PM
    Thank you Kevin.  I'm going to try the approved HBA and approved OS and see how that goes.  I will try to get more details from ITDT if I still have issues.  I did find these errors which were reported in the IBM LTFS properties of the drive after it came up with the '6' error previously.  Not sure if it sheds any light.  They seem somewhat generic and maybe the new HBA or OS will help resolve them.  I know it mentions read-only issues.. but that seems to happen after the error is logged (it goes into write protect mode and then I need to eject and run the cleaning tape before I can write again).  The tape will write fine (and I can format it) for hours and then the error will pop up.  These are not WORM tapes and the write protect tab is not set (I don't think I could format the tape and start writing if it wasn't). 







    Thanks.

    -Peter Berg

    ------------------------------
    Peter Berg
    ------------------------------



  • 13.  RE: Errors with LTO-8 drive

    Posted Mon January 03, 2022 08:42 PM
    @Kevin Butt,
    Just for your awareness, there was similar topic on another thread for the same HBA but on Linux.
    https://community.ibm.com/community/user/storage/communities/community-home/digestviewer/viewthread?MessageKey=a0acf0fd-9223-42b2-9d63-09c3d8247f0e

    According to Abe-san's analysis in that thread, ATTO's 12Gb SAS HBA won't be able to return the valid residual length at the read the block less than 512KB.  This does not occur on their 6Gb SAS HBA (I believe 3Gb HBA also have the same problem), so his previous suggestion was recompiling the open source LTFS with a flag turned on.  Unfortunately, Windows SDE is not open source, the only option is switching to HBA from different vendor.

    ------------------------------
    TAKESHI ISHIMOTO
    ------------------------------



  • 14.  RE: Errors with LTO-8 drive

    Posted Tue January 04, 2022 12:02 PM
    @TAKESHI ISHIMOTO Thanks!​

    ------------------------------
    Kevin Butt
    ------------------------------



  • 15.  RE: Errors with LTO-8 drive

    Posted Tue January 04, 2022 12:27 PM
    @Peter Berg, Some errors will "Fence" the drive from writing (i.e., make them read-only until ejected).


    ------------------------------
    Kevin Butt
    ------------------------------



  • 16.  RE: Errors with LTO-8 drive

    Posted Mon January 10, 2022 05:50 AM
    Hello All.

    I wanted to report back on this issue..and unfortunately I am still having the issue after trying to correct a few issues.

    I first switched our Windows machine over to Server 2016 Standard Edition (that was listed as supported for the drive and Spectrum SDE software).  I then have installed a SAS HBA that was listed as supported on the IBM System Storage Interoperation Center (SSIC).  I installed the Broadcom/LSI 9300-4i4E.

    Hard for me to believe.. but the drive will still report the '6' error after writing for awhile.  It wrote one file successfully.. but after maybe about 30 mins.. I heard it eject the tape, take the take back in and then it didn't make the regular writing notes.  Eventually it errored out and I got the '6' on the front of the drive again.  So hard to believe considering everything I have tried and this is the 2nd LTO drive that I have tested.

    I then did run the ITDT and did a drive dump.  I do see some errors that it is reporting.. but I can't quite tell what to try next.  Could anyone offer any help here?  I uploaded the logs that were generated during that drive dump to my Google drive.  Could anyone see if these shed any light on the issue:


    https://drive.google.com/drive/folders/1GbuV-iHQ1b4bJkA8eMDA2Hkv8JzB3bZC?usp=sharing


    I'm just at a loss.  Could it be something with the LTO drive firmware/microcode, SAS HBA firmware version or BIOS version?  I wouldn't think so.. but I don't know what else to try.   I could try to contact Broadcom.. but I wouldn't think it was a defective HBA.

    Thanks for any help you could provide.

    -Peter


    ------------------------------
    Peter Berg
    ------------------------------



  • 17.  RE: Errors with LTO-8 drive

    Posted Mon January 10, 2022 12:11 PM
    @Peter Berg, It appears that your drive is getting a Sense Key 3 ASC/ASCQ of 3/​0900. This is listed on www.t10.org https://www.t10.org/lists/asc-num.htm#ASC_09 as a Track Following Error. The TapeAlerts returned are 3 and 4, which indicate Hard Error and Media. Since multiple media have failed, this points to a drive issue.

    ------------------------------
    Kevin Butt
    ------------------------------



  • 18.  RE: Errors with LTO-8 drive

    Posted Tue January 11, 2022 05:43 PM
    Thanks Kevin for looking into that.  I'm checking with my reseller about possibly getting an exchange or doing a return.  I guess I just got very unlucky. To have two drives that both showed the '6' error just seems so unlikely and strange.  But I guess it can happen.. maybe some manufacturing issues with a small batch of drives?  Hopefully if I get a replacement... it won't have this same error come up when using it.

    Appreciate all the help with these many rounds of troubleshooting.  Super helpful.

    -Peter

    ------------------------------
    Peter Berg
    ------------------------------



  • 19.  RE: Errors with LTO-8 drive

    Posted Wed February 02, 2022 09:49 AM
    Hi All.

    I just wanted circle back with an update on this issue.  So I received a replacement drive (actually my 2nd replacement drive) and the good thing is this new drive has not reported any errors in all my initial testing (with everything else being the same).

    So it seems that I just got unlucky and got 2 drives that were reporting that '6' error and now with this new drive, those errors have not returned.

    I just wanted to thank everyone for the helpful tips in trying to narrow down the issues I was having.  Thanks again and have a great day.

    -Peter

    ------------------------------
    Peter Berg
    ------------------------------



  • 20.  RE: Errors with LTO-8 drive

    Posted Wed February 02, 2022 11:50 AM
    @Peter Berg That's good to hear!​

    ------------------------------
    Kevin Butt
    ------------------------------