AIX

AIX

Connect with fellow AIX users and experts to gain knowledge, share insights, and solve problems.

 View Only

  • 1.  what would cause lrud to be heaviest cpu process

    Posted Mon November 21, 2011 12:23 PM

    Originally posted by: admin2805


    lrud on an lpar running oracle is now the highest cpu process. were running aix 6.1TL5. What would cause this?


  • 2.  Re: what would cause lrud to be heaviest cpu process

    Posted Tue November 22, 2011 07:42 AM

    Originally posted by: MarkTaylor


    Poor memory tuning or over commited memory .. paste some data

    lparstat -i
    vmstat 1 20
    lsps -a
    vmo -Fa

    Rgds
    Mark Taylor


  • 3.  Re: what would cause lrud to be heaviest cpu process

    Posted Tue November 22, 2011 01:20 PM

    Originally posted by: admin2805


    Partition Number : 3
    Type : Dedicated-SMT
    Mode : Capped
    Entitled Capacity : 4.00
    Partition Group-ID : 32771
    Shared Pool ID : -
    Online Virtual CPUs : 4
    Maximum Virtual CPUs : 24
    Minimum Virtual CPUs : 1
    Online Memory : 61439 MB
    Maximum Memory : 102656 MB
    Minimum Memory : 1280 MB
    Variable Capacity Weight : -
    Minimum Capacity : 1.00
    Maximum Capacity : 24.00
    Capacity Increment : 1.00
    Maximum Physical CPUs in system : 64
    Active Physical CPUs in system : 19
    Active CPUs in Pool : -
    Shared Physical CPUs in system : 0
    Maximum Capacity of Pool : 0
    Entitled Capacity of Pool : 0
    Unallocated Capacity : -
    Physical CPU Percentage : 100.00%
    Unallocated Weight : -
    Memory Mode : Dedicated
    Total I/O Memory Entitlement : -
    Variable Memory Capacity Weight : -
    Memory Pool ID : -
    Physical Memory in the Pool : -
    Hypervisor Page Size : -
    Unallocated Variable Memory Capacity Weight: -
    Unallocated I/O Memory entitlement : -
    Memory Group ID of LPAR : -
    Desired Virtual CPUs : 4
    Desired Memory : 61440 MB
    Desired Variable Capacity Weight : -
    Desired Capacity : 4.00
    Target Memory Expansion Factor : -
    Target Memory Expansion Size : -
    <root>lpar1/>

    vmstat 1 20

    System configuration: lcpu=8 mem=61439MB

    kthr memory page faults cpu
    -----------
    ------------
    r b avm fre re pi po fr sr cy in sy cs us sy id wa
    2 4 12816266 6726 0 0 0 3894 6188345 9 2539 36804 8293 15 31 27 27
    2 6 12817385 6151 0 0 0 3518 6076608 8 2869 39812 9822 12 33 24 31
    0 5 12817482 6465 0 0 0 2699 4650122 7 1945 31482 7043 13 24 31 32
    4 4 12817594 6089 0 0 0 1768 5810950 8 2292 33296 5889 12 30 29 29
    0 4 12816753 6833 0 0 0 2055 6008069 9 2223 38615 6593 13 31 25 31
    0 0 12817768 7139 0 0 0 3338 5445429 7 2318 31473 7733 11 29 29 31
    0 3 12818619 6871 0 0 0 2769 3872177 6 2797 48323 9103 17 21 44 18
    2 4 12817320 7008 0 0 0 146 3001686 4 2164 37110 6204 13 18 52 16
    1 3 12816481 8710 0 0 0 2281 4739861 7 2116 30302 6565 10 25 45 20
    1 5 12814428 8181 0 0 0 0 0 0 3262 48549 7584 16 6 45 33
    3 5 12815480 6566 0 0 0 1845 6009098 8 2989 51518 9057 15 31 22 32
    3 4 12813571 8045 0 0 0 1371 6072152 9 2384 48570 6737 19 30 28 23
    4 5 12810665 8673 0 0 0 59 2131549 3 2980 41803 7843 15 14 42 30
    2 5 12810843 7442 0 0 0 3291 4396850 6 3691 46942 12210 20 26 22 31
    3 5 12810934 6489 0 0 0 2865 6203781 9 3095 28023 9440 12 33 20 35
    2 4 12808749 7940 0 0 0 2389 3421491 4 2580 47023 8751 15 21 28 36
    2 3 12812287 6968 0 0 0 4779 5800331 8 2370 45753 10331 14 32 26 28
    3 4 12813128 6276 0 0 0 2390 5602190 8 2777 74427 8165 19 30 25 25
    0 2 12806332 11212 0 0 0 193 965394 2 3303 47135 8222 13 11 49 27
    0 0 12797458 16859 0 0 0 0 0 0 4642 72056 11614 25 10 33

    lsps -a
    Page Space Physical Volume Volume Group Size %Used Active Auto Type Chksum
    paging00 hdisk13 pagingvg 34752MB 2 yes yes lv 0
    hd6 hdisk0 rootvg 6912MB 7 yes yes lv 0
    vmo -Fa
    ame_cpus_per_pool = n/a
    ame_maxfree_mem = n/a
    ame_min_ucpool_size = n/a
    ame_minfree_mem = n/a
    ams_loan_policy = n/a
    enhanced_affinity_affin_time = 1
    enhanced_affinity_vmpool_limit = 10
    force_relalias_lite = 0
    kernel_heap_psize = 65536
    lgpg_regions = 0
    lgpg_size = 0
    low_ps_handling = 1
    maxfree = 1088
    maxperm = 13718058
    maxpin = 12677034
    maxpin% = 80
    memory_frames = 15728624
    memplace_data = 2
    memplace_mapped_file = 2
    memplace_shm_anonymous = 2
    memplace_shm_named = 2
    memplace_stack = 2
    memplace_text = 2
    memplace_unmapped_file = 2
    minfree = 960
    minperm = 457267
    minperm% = 3
    nokilluid = 0
    npskill = 83328
    npswarn = 333312
    numpsblks = 10665984
    pinnable_frames = 5053770
    relalias_percentage = 0
    scrub = 0
    v_pinshm = 0
    vmm_default_pspa = 0
    wlm_memlimit_nonpg = 1
    ##Restricted tunables
    ame_sys_memview = n/a
    cpu_scale_memp = 8
    data_stagger_interval = 161
    defps = 1
    enhanced_affinity_attach_limit = 100
    enhanced_affinity_balance = 100
    enhanced_affinity_private = 20
    enhanced_memory_affinity = 0
    framesets = 2
    htabscale = n/a
    kernel_psize = 65536
    large_page_heap_size = 0
    lru_file_repage = 0
    lru_poll_interval = 10
    lrubucket = 131072
    maxclient% = 90
    maxperm% = 90
    mbuf_heap_psize = 65536
    memory_affinity = 1
    npsrpgmax = 666624
    npsrpgmin = 499968
    npsscrubmax = 666624
    npsscrubmin = 499968
    num_sem_undo_lists = 0
    num_spec_dataseg = 0
    page_steal_method = 1
    psm_timeout_interval = 20000
    rpgclean = 0
    rpgcontrol = 2
    scrubclean = 0
    soft_min_lgpgs_vmpool = 0
    spec_dataseg_int = 512
    strict_maxclient = 1
    strict_maxperm = 0
    vm_modlist_threshold = -1
    vmm_fork_policy = 1
    vmm_mpsize_support = -1
    vmm_vmap_policy = 0


  • 4.  Re: what would cause lrud to be heaviest cpu process

    Posted Mon July 21, 2014 06:14 AM

    Originally posted by: Mohammed Farooqui


    Hi,

     I am also experiencing the same in our production Database and normally it is seen when SAP-business objects application is accessing the Database (remote server)  However both servers are on same VLAN, in fact they are the LPARs of the same node.

    ibmsupt@pdws[/home/ibmsupt] lparstat -i
    Node Name                                  : pdws
    Partition Name                             : PDWS
    Partition Number                           : 1
    Type                                       : Shared-SMT-4
    Mode                                       : Uncapped
    Entitled Capacity                          : 2.10
    Partition Group-ID                         : 32769
    Shared Pool ID                             : 0
    Online Virtual CPUs                        : 3
    Maximum Virtual CPUs                       : 3
    Minimum Virtual CPUs                       : 1
    Online Memory                              : 20480 MB
    Maximum Memory                             : 25600 MB
    Minimum Memory                             : 18432 MB
    Variable Capacity Weight                   : 128
    Minimum Capacity                           : 1.00
    Maximum Capacity                           : 3.00
    Capacity Increment                         : 0.01
    Maximum Physical CPUs in system            : 32
    Active Physical CPUs in system             : 16
    Active CPUs in Pool                        : 16
    Shared Physical CPUs in system             : 16
    Maximum Capacity of Pool                   : 1600
    Entitled Capacity of Pool                  : 1350
    Unallocated Capacity                       : 0.00
    Physical CPU Percentage                    : 70.00%
    Unallocated Weight                         : 0
    Memory Mode                                : Dedicated
    Total I/O Memory Entitlement               : -
    Variable Memory Capacity Weight            : -
    Memory Pool ID                             : -
    Physical Memory in the Pool                : -
    Hypervisor Page Size                       : -
    Unallocated Variable Memory Capacity Weight: -
    Unallocated I/O Memory entitlement         : -
    Memory Group ID of LPAR                    : -
    Desired Virtual CPUs                       : 3
    Desired Memory                             : 20480 MB
    Desired Variable Capacity Weight           : 128
    Desired Capacity                           : 2.10
    Target Memory Expansion Factor             : -
    Target Memory Expansion Size               : -
    Power Saving Mode                          : Disabled

     

    ibmsupt@pdws[/home/ibmsupt] lsps -a
    Page Space      Physical Volume   Volume Group Size %Used Active  Auto  Type Chksum
    hd6             hdisk0            rootvg        8192MB     1   yes   yes    lv     0

     

    ibmsupt@pdws[/home/ibmsupt] su -
    root's Password:
    3004-501 Cannot su to "root" : Authentication is denied.
    ibmsupt@pdws[/home/ibmsupt] su -
    root's Password:
    root@pdws[/home/root] vmo -Fa
                 ame_cpus_per_pool = n/a
                   ame_maxfree_mem = n/a
               ame_min_ucpool_size = n/a
                   ame_minfree_mem = n/a
                   ams_loan_policy = n/a
      enhanced_affinity_affin_time = 1
    enhanced_affinity_vmpool_limit = 10
                    esid_allocator = 0
               force_relalias_lite = 0
                 kernel_heap_psize = 65536
                      lgpg_regions = 0
                         lgpg_size = 0
                   low_ps_handling = 1
                           maxfree = 1088
                           maxperm = 4566872
                            maxpin = 4226295
                           maxpin% = 80
                     memory_frames = 5242880
                     memplace_data = 0
              memplace_mapped_file = 0
            memplace_shm_anonymous = 0
                memplace_shm_named = 0
                    memplace_stack = 0
                     memplace_text = 0
            memplace_unmapped_file = 0
                           minfree = 960
                           minperm = 152227
                          minperm% = 3
                         nokilluid = 0
                           npskill = 16384
                           npswarn = 65536
               num_locks_per_semid = 1
                         numpsblks = 2097152
                   pinnable_frames = 4237212
               relalias_percentage = 0
                             scrub = 0
                          v_pinshm = 0
                  vmm_default_pspa = 0
                    vmm_klock_mode = 0
                wlm_memlimit_nonpg = 1
    ##Restricted tunables
                      ame_hw_accel = n/a
                   ame_sys_memview = n/a
                         batch_tlb = 1
                    cpu_scale_memp = 8
             data_stagger_interval = 161
                             defps = 1
    enhanced_affinity_attach_limit = 100
         enhanced_affinity_balance = 100
         enhanced_affinity_private = 40
          enhanced_memory_affinity = 1
                         framesets = 2
                         htabscale = n/a
                      kernel_psize = 65536
              large_page_heap_size = 0
                   lru_file_repage = 0
                 lru_poll_interval = 10
                         lrubucket = 131072
                        maxclient% = 90
                          maxperm% = 90
                   mbuf_heap_psize = 65536
                   memory_affinity = 1
              multiple_semid_lists = 0
                     munmap_npages = 16384
                         npsrpgmax = 131072
                         npsrpgmin = 98304
                       npsscrubmax = 131072
                       npsscrubmin = 98304
                num_sem_undo_lists = 0
                 num_sems_per_lock = 1
                  num_spec_dataseg = 0
                    numperm_global = 1
                 page_steal_method = 1
              psm_timeout_interval = 20000
                 relalias_lockmode = 1
                   relalias_nlocks = 96
                          rpgclean = 0
                        rpgcontrol = 2
                        scrubclean = 0
                    shm_1tb_shared = 12
               shm_1tb_unsh_enable = 0
                  shm_1tb_unshared = 256
             soft_min_lgpgs_vmpool = 0
                  spec_dataseg_int = 512
                  strict_maxclient = 1
                    strict_maxperm = 0
                       sync_npages = 4096
                     thrpgio_inval = 1024
                    thrpgio_npages = 1024
                    vm_16m_lruopts = 0
                   vm_mmap_areload = 0
              vm_modlist_threshold = -1
                  vm_pvlist_dohard = 0
                  vm_pvlist_szpcnt = 0
                   vmm_fork_policy = 1
                vmm_mpsize_support = 3
                   vmm_vmap_policy = 0
                      vtiol_avg_ms = 200
                      vtiol_minreq = 25
                vtiol_minth_active = 1
                        vtiol_mode = 0
                   vtiol_pgin_mode = 2
                  vtiol_pgout_mode = 2
                   vtiol_q_cpu_pct = 2500
              vtiol_thread_cpu_pct = 5000
                   wlm_rmem_filter = 0

    ---------------------------------------------------------------

    We have not configured concurrent I/O. Could this be a solution?



  • 5.  Re: what would cause lrud to be heaviest cpu process

    Posted Tue July 22, 2014 09:00 AM

    Originally posted by: YannickBergeron


    Please also note that AIX 6.1 TL5 is unsupported and it would be good idea to be on a supported technology level (TL7, TL8 or TL9) at their latest SP