Checking num_cmd_elems for VFC adapters with kdb

Publicado por el


This entry is similar in theme to one of my previous posts about verifying your hdisk queue_depth settings with kdb. This time we want to check if an attribute for a Virtual FC (VFC) adapter has been modified and whether or not AIX has been restarted since the change. The attribute I’m interested in is num_cmd_elems. This value is often changed from its default settings, in AIX environments, to improve I/O performance on SAN attached storage.

From kdb you can identify the VFC adapters configured on an AIX system using the vfcs subcommand. Not only does this tell you what adapters you have, but it also identifies the VIOS each adapter is connected to and the corresponding vfchost adapter. Nice!

(0)> vfcs
NAME      ADDRESS             STATE   HOST    HOST_ADAP  OPENED NUM_ACTIVE
fcs0      0xF1000A00103D4000  0x0008  vio1    vfchost10  0x01   0x0000
fcs1      0xF1000A00103D6000  0x0008  vio2    vfchost10  0x01   0x0000

You can view the current (running) configuration of a VFC adapter using the kdb vfcs subcommand and the name of the VFC adapter, for example fcs1:

(0)> vfcs fcs1
adapter: fcs0   partition_num: 0x14     partition_name: aixlpar1
num_cmd_elems: 0xC8     location_code: U9119.FHA.87654A1-V20-C10-T1
state: 0x8  opened: 0x1  flags: 0x1
host_name: vio1       host_device: vfchost10
maxCmds: FA     maxDMALength: 100000    SCSIid: 6F1E9A
portName: C050760399100168      nodeName: C050760399100168      nport_id:
6F1E9A
cmd_q.base_addr: 0xF1000A00102D0000     asyncq.base_addr: 0xF1000A0019761000
crq[0]: 0x8001000000000000      crq[1]: 0x8013960
ctl_head: 0x0   ctl_tail: 0x0
cmd_head: 0x0   cmd_tail: 0x0
pool_head: 0xF1000A001975BD08   pool_tail: 0xF1000A0019759D88
head_active: 0x0        tail_active: 0x0
cancel_head: 0xF1000A0019758090 cancel_tail: 0xF1000A0019758E58
num_cmds_allowed: 0xFA  num_cmds_active: 0x0    num_cmds_pending: 0x0
head_flush_q: 0x0       tail_flush_q: 0x0       cmd_flush_q: 0x0
no_elem: 0x0  no_dma: 0x0  no_sglist: 0x0  bad_mad: 0x0  qfull: 0x0
h_dropped: 0x0  link_down_cnt: 0x0  num_frames: 0x0  tx_lock: 0x0
rcv_lock: 0x0   ctl_lock: 0x0   lock: 0xFFFFFFFFFFFFFFFF
ctl_event: 0xFFFFFFFFFFFFFFFF   open_event: 0xFFFFFFFFFFFFFFFF
reset_cmd: 0x0  log_cmd: 0x0
link_cmd: 0x0   ctl_cmd->cmd_state: 0x0
inp_reqs: 0x12594       out_reqs: 0x1A7FD4      ctrl_reqs: 0x44
inp_bytes: 0x14973F90  out_bytes: 0xEC14EC18  npiv_logout_sent: 0x0
npiv_logout_rcvd: 0x0  act_hi_water: 0x10  pend_hi_water: 0x1

Using the output from this command we can determine the current (running) value for a number of VFC attributes, including num_cmd_elems.

So I start with an adapter with a num_cmd_elems value of 200. Both the lsattr command and kdb report 200 (C8 in hex) for num_cmd_elems.

# lsattr -El fcs1 -a num_cmd_elems
num_cmd_elems 200 Maximum Number of COMMAND Elements True

# echo vfcs fcs1 | kdb | grep num_cmd_elems
num_cmd_elems: 0xC8     location_code: U9119.FHA.87654A1-V20-C10-T1

I change num_cmd_elems to 400 with chdev –P (remember, the –P flag only updates the AIX ODM, and not the running configuration of the device in the AIX kernel. You must either reboot for this change to take effect or offline &online the device).

# chdev -l fcs1 -a num_cmd_elems=400 -P
fcs1 changed

Now the lsattr command reports num_cmd_elems is set to 400 in the ODM.

# lsattr -El fcs1 -a num_cmd_elems
num_cmd_elems 400 Maximum Number of COMMAND Elements True

But kdb still has num_cmd_elems set to C8 (200).

# echo vfcs fcs1 | kdb | grep num_cmd_elems
num_cmd_elems: 0xC8     location_code: U9119.FHA.87654A1-V20-C10-T1

Using this technique I now have a way of checking if an AIX system has been restarted since an attribute was changed on a VFC adapter.

At this point I could suggest a reboot of the system. Or I could take the adapter offline and then online for the changes to take effect. For example:

# lspath
Enabled hdisk0 fscsi0
Enabled hdisk1 fscsi0
Enabled hdisk0 fscsi1
Enabled hdisk1 fscsi1

# rmpath -l hdisk0 -p fscsi1
# rmpath -l hdisk1 -p fscsi1

# lspath
Enabled hdisk0 fscsi0
Enabled hdisk1 fscsi0
Defined hdisk0 fscsi1
Defined hdisk1 fscsi1

# rmdev -Rl fcs1
fscsi1 Defined
fcs1 Defined

# cfgmgr

# lspath
Enabled hdisk0 fscsi0
Enabled hdisk1 fscsi0
Enabled hdisk0 fscsi1
Enabled hdisk1 fscsi1

# lsattr -El fcs1 –a num_cmd_elems
num_cmd_elems 400      Maximum Number of COMMAND Elements True

# echo vfcs fcs1 | kdb | grep num_cmd_elems
num_cmd_elems: 0x190    location_code: U9119.FHA.87654A1-V20-C10-T1

;; 0x190 hex = 400 decimal

Please note: Be careful when using the kdb command. If used incorrectly, you can crash an AIX system!