File and Object Storage

File and Object Storage

Software-defined storage for building a global AI, HPC and analytics data platform 

 View Only

How many Files on my Filesystem?

By Smita Raut posted Mon May 07, 2018 10:47 AM

  
Many a times the administrators need to know the total number of files on IBM Spectrum Scale Filesystem. With IBM Spectrum Scale one can do it in in couple of ways ... Here is how.

1. mmdf gives you total number of inodes used in the system - but this contains files + directories
If you are using multiple file systems, one will have to run multiple mmdf commands.

2. If you want list of just files and/or number of files - you will need to use ILM policy rules supported by IBM Spectrum Scale. Policy rules can be used with spectrum scale mmapplypolicy. Keep in mind that it scans and lists all files in the filesystem - so can take time and put some workload on the metadata - so advisable not to run at peak hours. Detailed steps given below-

a. Create a policy rule file consisting of this rule
RULE 'all' LIST 'allfiles' where name like '%'

b. Run the command-
mmapplypolicy

It will output something like-

[root@srnode1 ~]# mmapplypolicy fs1 -P policyfile
[I] GPFS Current Data Pool Utilization in KB and %
Pool_Name KB_Occupied KB_Total Percent_Occupied
system 3878912 4194304 92.480468750%
[I] 5706 of 113664 inodes used: 5.020059%.
[W] Attention: In RULE 'all' LIST name 'allfiles' appears but there is no corresponding "EXTERNAL LIST 'allfiles' EXEC ... OPTS ..." rule to specify a program to process the matching files.
[I] Loaded policy rules from policyfile.
Evaluating policy rules with CURRENT_TIMESTAMP = 2018-04-18@14:17:35 UTC
Parsed 1 policy rules.
RULE 'all' LIST 'allfiles' where name like '%'
[I] 2018-04-18@14:17:35.590 Directory entries scanned: 1617.
[I] Directories scan: 1182 files, 435 directories, 0 other objects, 0 'skipped' files and/or errors.
[I] 2018-04-18@14:17:36.806 Parallel-piped sort and policy evaluation. 1617 files scanned.
[I] 2018-04-18@14:17:36.818 Piped sorting and candidate file choosing. 1166 records scanned.
[I] Summary of Rule Applicability and File Choices:
Rule# Hit_Cnt KB_Hit Chosen KB_Chosen KB_Ill Rule
0 1166 44360 1166 44360 0 RULE 'all' LIST 'allfiles' WHERE(.)

[I] Filesystem objects with no applicable rules: 435.

[I] GPFS Policy Decisions and File Choice Totals:
Chose to list 44360KB: 1166 of 1166 candidates;
Predicted Data Pool Utilization in KB and %:
Pool_Name KB_Occupied KB_Total Percent_Occupied
system 3883008 4194304 92.578125000%
[I] 2018-04-18@14:17:36.834 Policy execution. 1166 files dispatched.
[I] A total of 1166 files have been migrated, deleted or processed by an EXTERNAL EXEC/script;
0 'skipped' files and/or errors.



The highlighted number in bold is the #of files in given filesystem.
#Softwaredefinedstorage
0 comments
7 views

Permalink