Primary Storage

 View Only
Expand all | Collapse all

Flashsystem host unmap not working ¿?

  • 1.  Flashsystem host unmap not working ¿?

    Posted Fri October 22, 2021 03:30 PM
    Hi
    can somebody help me to understand the UNMAP function on the flashsystem.
    Or what i am doing wrong.
    I want to use UNMAP for my hosts ( generic hosts Windows & AIX), i got spectrum virtualize 8.4.0.5. Flashsystem 7200 with all flash FCM2 modules.
    I understand that UNMAP must be enable by default.
     lssystem command shows that host_unmap is not enable
    > lssystem
    ...
    host_unmap off
    ...
    So i enabled it:
    > chsystem -hostunmap on 
    now lssystem shows it enabled.

    For testing i type in a windows 2019 host:
    PS:> defrag D: /L
    but i got an error
    "the operation requested is not supported by the hardware backing the volume"
    same error with command.  PS:> Optimize-volume

    what am i doing wrong?

    thank you in advance for any help

    ------------------------------
    Arturo Garfias
    ------------------------------


  • 2.  RE: Flashsystem host unmap not working ¿?

    Posted Mon October 25, 2021 10:43 AM
    Hi Arturo.

    I hope this article could help to clarify the usage for UNMAP SCSI for Spectrum Virtualize.

    https://www.ibm.com/support/pages/scsi-unmap-support-spectrum-virtualize-systems

    Kind regards.

    Luis Fer Becerra.

    ------------------------------
    Luis Fernando Becerra Lebaza
    ------------------------------



  • 3.  RE: Flashsystem host unmap not working ¿?

    Posted Mon October 25, 2021 06:01 PM
    Hello Arturo

    AFAIK the command would be: svctask chsystem -unmap on  (The hostunmap is not a valid syntax)


    Then, remember that SCI unmap applies to format, create operations. I don´t sure about defrag. But i hope the above lines can help you.

    Regards

    ------------------------------
    Humberto Sosa
    ------------------------------



  • 4.  RE: Flashsystem host unmap not working ¿?

    Posted Tue October 26, 2021 03:54 AM

    Hi,
    on newer firmwares there are two commands

    SCSI Unmap can be disabled using the CLI command:
    svctask chsystem -backendunmap off
    svctask chsystem -hostunmap off

    To re-enable SCSI Unmap, use the
    svctask chsystem -backendunmap on
    svctask chsystem -hostunmap on

    On Standard pools you will only see that the used capacity of the FCM goes down, not the size of the thin provisioned volumes.

    https://docs.microsoft.com/en-us/windows-hardware/drivers/storage/thin-provisioning

    https://www.vembu.com/blog/reclaiming-space-using-trim-unmap/ 



    ------------------------------
    Bernd Albrecht
    ------------------------------



  • 5.  RE: Flashsystem host unmap not working ¿?

    Posted Thu August 10, 2023 02:43 PM

    Hi Bernd,

    will there be a change in the behavior in newer versions as this is pretty ugly? 

    #On Standard pools you will only see that the used capacity of the FCM goes down, not the size of the thin provisioned volumes.#

    thanks



    ------------------------------
    Sebastian Hofmann
    ------------------------------



  • 6.  RE: Flashsystem host unmap not working ¿?

    Posted Fri August 11, 2023 02:59 AM

    Hi Sebastian,

    this won't change for standard pools, i.e. non-DR-pools, as those do not support SCSI UNMAP at all.



    ------------------------------
    Best regards, 

    Christian Schroeder
    IBM Storage Virtualize Support with Passion
    ------------------------------



  • 7.  RE: Flashsystem host unmap not working ¿?

    Posted Fri August 11, 2023 03:04 AM

    Hi Christian,

    this is not correct. STD Pools support SCSI UNMAP ... but the Volumes in the Pool do not show the reclaimed Space. The Pool usage will go down if eg a big VM is deleted 



    ------------------------------
    Sebastian Hofmann
    ------------------------------



  • 8.  RE: Flashsystem host unmap not working ¿?

    Posted Fri August 11, 2023 03:50 AM

    Hi Sebastian, 

    what is happen on happen on STD pools is, the SCSI UNMAP free up the blocks on Flash drives internal. On Flash less physical Storage is used. On FCMs  is what you see on pool usage. 

    Best regards.



    ------------------------------
    Bernd Albrecht
    ------------------------------



  • 9.  RE: Flashsystem host unmap not working ¿?

    Posted Sat August 19, 2023 02:36 PM

    a simple and accurate answer



    ------------------------------
    Sandeep Sharma
    ------------------------------



  • 10.  RE: Flashsystem host unmap not working ¿?

    User Group Leader
    Posted Mon August 21, 2023 05:43 AM

    Not entirely accurate.

    All pools support the host unmap commands.  DRPs track and account for capacity changes as a result, standard pools do not and cannot (That is the primary difference and the purpose of Data Reduction Pools).



    ------------------------------
    Evelyn Perez
    IBM Senior Technical Staff Member
    IBM Storage Virtualize Software Architect for SVC and FlashSystem
    ------------------------------



  • 11.  RE: Flashsystem host unmap not working ¿?

    Posted Tue August 22, 2023 02:01 AM

    Hi All,

    thanks for clarification. TBH the implementation on STD Pools is pretty ugly. As per my knowledege ( using FCM3 module ) its better to utilize STD Pools ( no dedup & no software compression ) also what i faced is that creating a LUN in a DRP pool u need to utilize either Compression and / or Dedup. Is there a way to utilize DRP Pool without software compression and deduduplication ( only the native FCM compression )

    Thanks



    ------------------------------
    Sebastian Hofmann
    ------------------------------



  • 12.  RE: Flashsystem host unmap not working ¿?

    User Group Leader
    Posted Tue August 22, 2023 04:52 AM

    Keep in mind that Standard or Legacy pools as we sometimes call them are about over 20 years old.  The technology pre-dates the unmap command and even flash in enterprise storage, and thin provisioning was added to it long before it was industry standard.

    The architecture required to do the tracking for capacity at this level is a log structured array, this data structure is what is used by the FCMs themselves and by the Data Reduction Pools.  DRP exists BECAUSE of this problem, the original name for DRP internally was "Unmap".

    In Software, it is this log structured array that adds latency to a DRP overprovisioned volume because of the transformations and the io amplification required to keep the meta data updated (not the compression, not the deduplication, those are all offloaded to the Intel Quick Assist Compression Offload Hardware on all boxes except the 50XX).

    So in answer to your original question:  If you require capacity returns reported at the pool level in a FlashSystem Box, you need to use Data Reduction Pools with overprovisioning ( Because a fully allocated volume is ALWAYS a fully allocated volume regardless of it's pool, it will never return partial capacity to the pool).

    If your workload is unsuitable for DRP, then the recommendation is to be aware of the behaviour (as you are now) and know the new capacity will be taken from unmapped capacity first, and as I said before, the re-thinning technique works for folks who are concerned about a lot of unmap without overwrite.

    For capacity planning in general, there are some very good practices, foremost is getting SI as it comes included with your FlashSystem and can provide a lot of capacity tracking information.

    I'm trying to convince Barry and Andrew to put their very good advice on how to manage this better in their blog, so keep an eye out for it.

    I hope that helps.



    ------------------------------
    Evelyn Perez
    IBM Senior Technical Staff Member
    IBM Storage Virtualize Software Architect for SVC and FlashSystem
    ------------------------------



  • 13.  RE: Flashsystem host unmap not working ¿?

    User Group Leader
    Posted Fri August 11, 2023 04:59 AM

    Just to clarify what is happening here:


    There are several capacities at play here:
    Written Capacity - how much actual data has been written to the backend storage media
    Physical or Usable Capacity - how much data Can be written to the media in the absence of any data reduction or overprovisioning (the amount of physical media blocks/sectors/etc written)
    Logical/Addressable or Effective capacity - how much Capacity can be presented to the Host as addressable space including overprovisioning or data reduction techniques.


    In a Data reduction Pool:   Unmap commands sent from the Host will be marked in the Pool and the internal data structures updated to mark this for garbage collection.  Garbage collection runs and unmaps or write same(0's) to the backend to free the Physical capacity and the Logical capacity is returned to to the pool.  This occurs IF and only if  the volume is overprovisioned or data reduced: OR. it is an entire volume delete operation for any type of volume.  (Fully allocated volumes will take up the space in the pool regardless of how much has actually been written).

    In a Standard Pool:  The host unmap is just passed directly to the backend.  In the case of Fully Allocated volume deletion, the capacity will be returned to the pool.  In the case of unmap to a thin provisioned/overallocated volume that capacity is not tracked at the pool (because it is not a DRP), so it is not freed or released at the pool level.  However, additionally created capacity will re-use this allocated capacity.

    Tracking unmap behaviour in capacity reporting is a very large part of what DRP provides and is not possible in Standard Pools.

    However - In standard pools it is possible to reclaim this capacity if it is a problem (which mostly it isn;t) by doing a re-thin technique where you mirror the volume to a new thin provisioned copy and then remove the old copy.  This will free that used capacity from the old thin volume and leave you with a new thin volume that accurately reflects the written capacity.

    I hope that helps to clarify the behaviour.



    ------------------------------
    Evelyn Perez
    IBM Senior Technical Staff Member
    IBM Storage Virtualize Software Architect for SVC and FlashSystem
    ------------------------------



  • 14.  RE: Flashsystem host unmap not working ¿?

    Posted Mon December 11, 2023 10:12 AM

    this is not correct. STD Pools support SCSI UNMAP ... but the Volumes in the Pool do not show the reclaimed Space

    Shoreline Excellence Revealed. The Pool usage will go down if eg a big VM is deleted.



    ------------------------------
    fadi mughal
    ------------------------------



  • 15.  RE: Flashsystem host unmap not working ¿?

    Posted Tue December 12, 2023 01:49 AM

    you need to reboot the host in order to rediscover the new unmap capabilities on the storage side.



    ------------------------------
    Adrian Nicolae
    ------------------------------