DA pairs and array sites: plan your arrays before you build
Every array you build on a DS8000 sits behind one DS8000 DA pair, and you pick that pair the moment you choose an array site. Most people never think about it on day one, because the commands work no matter which site you take, and the trouble shows up months later when one pair of adapters is busy and the other is sitting idle.
So before Lesson 13, where you build arrays and ranks properly, this lesson slows down at the step people rush. You’ll see what a device adapter is on the G10 hardware, how the system turns installed drives into array sites, and how to read the DA Pair column so your plan spreads work over both adapters and both servers.
Configuration on a DS8000 is a chain: array site, array, rank, pool, LSS or LCU, volume. That chain is the same for open systems (FB) and for IBM Z (CKD), and you’ll walk both in the lab.
| Time | You need | By the end you can |
|---|---|---|
| 60 to 75 minutes | The practice shell from Lesson 2, Python 3 on your laptop, paper and a pencil | Read lsarraysite -l, lsdacard and lsstgencl together, plan arrays across DA pairs and rank groups, build FB and CKD ranks on that plan, and check the spread with a small script |
What it is, in one line
A DA pair is two flash RAID adapters, one in each I/O enclosure of a pair and each tied to a different CPC, that together own one HPFE Gen3 enclosure pair and every array site in it.
In Simple Words
Two school buses and their stops
Picture a school with two buses. Bus A covers the stops on the east side and Bus B covers the west side, and each bus has two drivers who sit in different depots, so if one driver is sick the other still runs the route.
When new children join, the office has to decide which bus they ride. If the clerk puts every new child on Bus A because its list is on top of the pile, Bus A is packed and late every morning while Bus B drives around half empty.
A DA pair is one of those buses with its two drivers, and the array sites are its stops. Your job is to share the children out before the term starts.
Engineer's View
Device adapters on the DS8A00
On the 10th generation (DS8A00, Release 10.1) the device adapter is a flash RAID adapter that talks NVMe to the drives. The Redbook SG24-8559 gives these rules:
- Where they sit: DAs go in slots C1 and C2 of the I/O enclosures. The two DAs of a pair are in different I/O enclosures of an enclosure pair, and each one has allegiance to a different CPC.
- How many: one DA pair for every HPFE Gen3 pair, and up to two DA pairs per I/O enclosure pair. DS8A80 and DS8A50 take 1 to 8 DA pairs; DS8A10 takes 1 to 4.
- Paths: each controller has four ports, one to each of the four ESMs (enclosure service modules) in the HPFE pair, so a DA pair has eight paths. Every drive has two connections, one to each NVMe switch in its enclosure.
- Dual active: both controllers can reach every array in the DA pair, and on the DS8A00 each CPC can connect directly to both adapters, so a failed adapter doesn’t force arrays to change owner.
- Array sites: eight identical drives each. An HPFE pair holds 16, 32 or 48 drives, which makes 2, 4 or 6 array sites.
- Spares: two global spares per HPFE pair (that is, per DA pair), taken from the first 16 drives. The first two RAID 6 arrays on the pair come out as 5+P+Q+S and later ones as 6+P+Q. A failed drive can only be spared by a drive of equal or larger capacity on the same DA pair.
- No intermix: in Release 10.1 you can’t mix FlashCore Modules with Flash Tier 1 or Flash Tier 2 drives in one HPFE pair, or in one pool.
The DS CLI guide adds one useful hint: DA pair IDs are decimal numbers, and an even ID means the first DA pair in an I/O enclosure pair while an odd ID means the second. Check lsdacard on your own system rather than guessing from the number.
On DS8900F: the same idea, HPFE Gen2 pairs behind flash RAID adapter pairs, with different limits. The older loop-based design comes at the end of “How it works”.
How it works, step by step
Here is the path a read takes, and why you never have only one way to a drive.
1A DA pair is two adapters in two I/O enclosures
I/O enclosures come in pairs. One DA of the pair sits in each enclosure, in slot C1 or C2, and each has allegiance to a different CPC, so DA pair 0 has one adapter working for server0 and its partner working for server1.
2Both adapters cable to both HPFE enclosures
The HPFE Gen3 enclosures also come in pairs, and each enclosure has two ESMs with an NVMe switch inside. Each DA runs one cable to each of the four ESMs, which is eight paths for the pair, and each drive plugs into both switches of its enclosure.
3Losing one piece is fine
If an ESM is pulled, the other switch in that enclosure still reaches every drive and both adapters. If one DA loses both of its paths, its partner still sees every drive in the enclosure pair, which is the whole reason they come in twos.
4The system makes array sites, you never do
When IBM installs a drive set (16 drives) in an HPFE pair, the system groups them into two array sites of eight identical drives. You’ll see them as S1, S2 and so on in lsarraysite -l, and there is no command to create, split or move an array site.
Because the drives of an array site live in one HPFE pair, every array site belongs to one DA pair and only one. That’s what the DA Pair column of lsarraysite -l shows, next to the drive capacity and the disk class (FlashCoreModule, FlashTier2 and so on).
5Three more commands tell the same story from the hardware side
lsdacard lists the adapters with their I/O enclosure, state and DA pair. lsstgencl (newer name lsstgenclosure) lists the storage enclosures and which DA pair serves each one, and lsdrive lists every drive with its storage enclosure, class and the array it belongs to.
Read them together and you can follow a line from adapter to enclosure to drive to array site. When an engineer asks “which arrays are on the adapter that logged an error last night?”, this is how you answer in two minutes. The older names lsda and lsddm still run, but the G10 names are the ones to learn.
6The configuration chain
Logical configuration on a DS8000 always runs in this order, and the DA pair decision is made right at the start:
- Array site: pick by DA pair and disk class.
- Array:
mkarray -raidtype 6 -arsite S1. RAID 6 is the default, and the G10 refuses RAID 5. - Rank:
mkrank -array A0 -keygrp 1 -stgtype fb, or-stgtype ckdfor IBM Z. With encryption on, the key group is required. - Extent pool:
mkextpool -rankgrp 0or-rankgrp 1, thenchrank -extpool. Rank group 0 is server0 and rank group 1 is server1. - LSS or LCU: for FB the LSS appears by itself when you make the first volume; for CKD you create the LCU with
mklcu. Even IDs pair with rank group 0 pools, odd with rank group 1. - Volumes:
mkfbvolfor open systems,mkckdvolfor IBM Z.
CKD is not a separate world. It’s the same chain with -stgtype ckd on the rank and the pool and one extra step for the LCU, which is why a mainframe shop and an open systems shop can share one DS8000 as long as each pool keeps one storage type.
7So what do you “configure” on a DA pair?
Nothing with a command. IBM installs DA pairs together with the HPFE pairs, and there is no mkda or similar. What you control is placement, and the DS CLI guide is clear that the system won’t help: array site position and DA pair are not factors when a rank goes into a pool, so any rank can go anywhere.
- Spread arrays of the same drive class over the DA pairs that have them, so no pair does all the work.
- Spread ranks over rank group 0 and 1 pools, so both servers get work. Two pools fed from two DA pairs is the usual starting point.
- Keep one drive class per pool. FCM and Flash Tier 2 never share a pool on Release 10.1 anyway.
- Leave the spares to the system. It takes them from the first arrays on each DA pair, which is why the first two arrays are 5+P+Q+S.
- Check after you build with
lsarray -landlsrank -l. IBM’slsperfrescrptalso accepts a DA pair ID such as DP0 for performance reports on a real system.
8On older systems: loops and enclosure pairs
On the DS8870 and the hybrid DS8880 models, a DA was an 8 Gbps, four-port Fibre Channel RAID adapter, and the DA pair drove switched FC-AL loops out to standard drive enclosures (24 small or 12 large SAS drives each). Enclosures came in pairs and were daisy-chained one after another down the loop, up to four enclosure pairs per DA pair.
The upper enclosure of a pair sat on one dual loop and the lower one on the other, and arrays were built across both, which IBM called “array across loops”. Each drive had four paths. In those racks the DA pair number told you which group of enclosures in which frame it served, so planning arrays per DA pair also meant planning per stack of enclosures.
Spares were also stricter there: four per DA pair on standard enclosures instead of two. If you inherit one of these boxes, read DS8880 architecture and DS8000 internal connections before you add capacity, and don’t reuse G10 output examples for them.
Hands-on lab: plan and build arrays across two DA pairs
Before you run any command, double-check it
Read every command twice before you press Enter. Check the target: the system or account, the IDs, the names and the sizes. These commands are for learning and were tested in our lab, not on your system. At work, run the read-only list commands first, follow your change process, make sure a backup exists, and ask a second person to check anything that changes or deletes data. You run them at your own risk.
You’ll read the practice DS8A50 the way you would read a new box at work, plan where three arrays go, build them across both DA pairs and both servers, and finish the chain with one FB volume and one CKD volume. At the end a small Python check on your laptop tells you whether the layout is balanced.
Practice shell note: the practice shell is our teaching tool, not IBM software, and real DS CLI output looks very much like this but isn’t identical. Messages marked as practice shell messages are ours. The shell’s DA pair 2 holds both FlashCore Modules and Flash Tier 2 drives so you can try FB and CKD on a small box; a real DS8A00 on Release 10.1 doesn’t mix those in one HPFE pair. All outputs below are what the shell printed.
1Reset and read the hardware
Start the shell in your lab folder, log in, and look at the box from four sides.
Run this:
simctl reset
lsarraysite -l
lsdacard
lsstgencl
lsdriveYou should see this:
SIM0100I simctl: factory state restored (users and passwords kept; use simctl reset -all to reset them too).
Date/Time: 25 October 2026 04:45:20 UTC IBM DSCLI Version: 7.10.10.348 DS: IBM.2107-78SV001
arsite DA Pair dkcap (10^9B) diskrpm State Array diskclass encrypt
===============================================================================
S1 0 4800.0 65000 Unassigned - FlashCoreModule supported
S2 0 4800.0 65000 Unassigned - FlashCoreModule supported
S3 0 4800.0 65000 Unassigned - FlashCoreModule supported
S4 0 4800.0 65000 Unassigned - FlashCoreModule supported
S5 2 4800.0 65000 Unassigned - FlashCoreModule supported
S6 2 4800.0 65000 Unassigned - FlashCoreModule supported
S7 2 3840.0 65000 Unassigned - FlashTier2 supported
S8 2 3840.0 65000 Unassigned - FlashTier2 supported
ID I/O enclosure Frame State DA pair Location code S/N
=====================================================================================================
IBM.1400-1B1-78SV0/R-1-P1-C6 IBM.1400-1B1-78SV0 1 Online 0 U1400.1B1.78SV0-P1-C6 YA30SV0010
IBM.1400-1B2-78SV0/R-1-P1-C6 IBM.1400-1B2-78SV0 1 Online 0 U1400.1B2.78SV0-P1-C6 YA30SV0020
IBM.1400-1B1-78SV0/R-1-P1-C8 IBM.1400-1B1-78SV0 1 Online 2 U1400.1B1.78SV0-P1-C8 YA30SV0012
IBM.1400-1B2-78SV0/R-1-P1-C8 IBM.1400-1B2-78SV0 1 Online 2 U1400.1B2.78SV0-P1-C8 YA30SV0022
ID Frame Enclnum Loc FC Interfaces State DA pair
======================================================================
IBM.2107-D04-8SV01/R1 1 1 R1-D1 HPFE Gen3 Normal 0
IBM.2107-D04-8SV01/R2 1 2 R1-D2 HPFE Gen3 Normal 0
IBM.2107-D04-8SV02/R1 1 3 R1-D3 HPFE Gen3 Normal 2
IBM.2107-D04-8SV02/R2 1 4 R1-D4 HPFE Gen3 Normal 2
ID Storage enclosure Frame State Class Capacity Array Location Interface Firmware S/N
===============================================================================================================================
IBM.2107-D04-8SV01/R1-P1-D1 IBM.2107-D04-8SV01 1 Normal FlashCoreModule 4800.0 - R1-P1-D1 NVMe 4.1.2 SVS101
IBM.2107-D04-8SV01/R1-P1-D2 IBM.2107-D04-8SV01 1 Normal FlashCoreModule 4800.0 - R1-P1-D2 NVMe 4.1.2 SVS102
...
IBM.2107-D04-8SV02/R1-P1-D17 IBM.2107-D04-8SV02 1 Normal FlashTier2 3840.0 - R1-P1-D17 NVMe 4.1.2 SVS701
(output shortened: 64 drives in all, 32 in each storage enclosure ID)Eight array sites, all Unassigned. S1 to S4 are on DA pair 0 and S5 to S8 on DA pair 2, and lsdacard shows each DA pair as two cards in two different I/O enclosures (1B1 and 1B2). lsstgencl shows which storage enclosures each DA pair serves, and in lsdrive the enclosure ID 8SV01 is the pair behind DA pair 0 while 8SV02 is behind DA pair 2.
✅ Checkpoint: You can say which DA pair owns S1 (0) and S7 (2), and which disk class S7 has (FlashTier2).
2Plan on paper before you type
Write a small table. You want FlashCore Module work on both DA pairs and FB ranks in both rank groups, plus one CKD rank for the mainframe team on the Flash Tier 2 drives.
| Array site | DA pair | Disk class | Storage type | Pool (rank group) |
|---|---|---|---|---|
| S1 | 0 | FlashCoreModule | fb | fcm_fb_0 (0) |
| S5 | 2 | FlashCoreModule | fb | fcm_fb_1 (1) |
| S7 | 2 | FlashTier2 | ckd | ft2_ckd_0 (0) |
Notice the cross-over: the FCM array on DA pair 0 feeds server0, and the one on DA pair 2 feeds server1. If both came from DA pair 0, server balance would look fine while one adapter pair did all the FCM work.
✅ Checkpoint: Your plan has one FCM array on each DA pair and one FB pool in each rank group.
3Build the arrays on two DA pairs
Run this:
lskeygrp
mkarray -raidtype 6 -arsite S1
mkarray -raidtype 6 -arsite S5
mkarray -raidtype 6 -arsite S7
lsarray -lYou should see this:
Date/Time: 25 October 2026 04:45:21 UTC IBM DSCLI Version: 7.10.10.348 DS: IBM.2107-78SV001
ID state reckeystate reckeydate datakeydate keyprotocol type name
=======================================================================
1 accessible disabled - 10/01/2026 LOCAL DAR DAR_1
CMUC00004I mkarray: Array A0 successfully created.
CMUC00004I mkarray: Array A1 successfully created.
CMUC00004I mkarray: Array A2 successfully created.
Array State Data RAIDtype arsite Rank DA Pair DDMcap (10^9B) diskclass encrypt
================================================================================================
A0 Unassigned Normal 6 (5+P+Q+S) S1 - 0 4800.0 FlashCoreModule supported
A1 Unassigned Normal 6 (5+P+Q+S) S5 - 2 4800.0 FlashCoreModule supported
A2 Unassigned Normal 6 (5+P+Q+S) S7 - 2 3840.0 FlashTier2 supportedThe array IDs come from the system (A0, A1, A2) and don’t match the site numbers, so always read the arsite and DA Pair columns of lsarray -l instead of assuming. Key group 1 is accessible, which matters in the next step.
✅ Checkpoint: lsarray -l shows A0 on DA pair 0 and A1 and A2 on DA pair 2, all RAID 6.
4Create the ranks: two FB, one CKD
Encryption is on in this box (lskeygrp showed key group 1), so every rank must name its key group with -keygrp 1. Leave it out and the shell stops you with a practice shell message, CMUN02805E.
Run this:
mkrank -array A0 -keygrp 1 -stgtype fb
mkrank -array A1 -keygrp 1 -stgtype fb
mkrank -array A2 -keygrp 1 -stgtype ckdYou should see this:
CMUC00007I mkrank: Rank R0 successfully created.
CMUC00007I mkrank: Rank R1 successfully created.
CMUC00007I mkrank: Rank R2 successfully created.✅ Checkpoint: R0 and R1 are fb ranks and R2 is a ckd rank.
5Pools for rank group 0 and 1
Run this:
mkextpool -rankgrp 0 -keygrp 1 -stgtype fb fcm_fb_0
mkextpool -rankgrp 1 -keygrp 1 -stgtype fb fcm_fb_1
mkextpool -rankgrp 0 -keygrp 1 -stgtype ckd ft2_ckd_0
chrank -extpool P0 R0
chrank -extpool P1 R1
chrank -extpool P2 R2
lsrank -lYou should see this:
CMUC00000I mkextpool: Extent pool P0 successfully created.
CMUC00000I mkextpool: Extent pool P1 successfully created.
CMUC00000I mkextpool: Extent pool P2 successfully created.
CMUC00008I chrank: Rank R0 successfully modified.
CMUC00008I chrank: Rank R1 successfully modified.
CMUC00008I chrank: Rank R2 successfully modified.
Date/Time: 25 October 2026 04:45:22 UTC IBM DSCLI Version: 7.10.10.348 DS: IBM.2107-78SV001
ID Group State datastate Array RAIDtype extpoolID extpoolnam stgtype exts usedexts keygrp marray extsize totalLogicalCap(MiB/cyl) usedLogicalCap(MiB/cyl) totalPhysicalCap(MiB/cyl) usedPhysicalCap(MiB/cyl) compressionRatio compressionType
=================================================================================================================================================================================================================================================
R0 0 Normal Normal A0 6 P0 fcm_fb_0 fb 6372928 0 1 MA1 16MiB 101966848 1024 22659301 1024 1.00 compression
R1 1 Normal Normal A1 6 P1 fcm_fb_1 fb 6372928 0 1 MA2 16MiB 101966848 1024 22659301 1024 1.00 compression
R2 0 Normal Normal A2 6 P2 ft2_ckd_0 ckd 1064924 0 1 MA3 21cyl 22363404 1344 22363405 1344 1.00 noneIn lsrank -l the Group column is the rank group: R0 works for server0 and R1 for server1, and the two came from different DA pairs. The CKD rank shows 21cyl extents and its capacity in cylinders, while the FB ranks show 16MiB extents.
✅ Checkpoint: lsrank -l shows R0 in group 0 (P0), R1 in group 1 (P1) and R2 in group 0 (P2, ckd).
6Finish the chain: one FB and one CKD volume
FB volumes go straight in, because the LSS (20 here) is created for you. CKD needs its LCU first; LCU 80 is even, so it pairs with the rank group 0 CKD pool.
Run this:
mkfbvol -extpool P0 -cap 10 -sam ese -name fb_0_#h 2000
mklcu -qty 1 -id 80 -ss 8080
mkckdvol -extpool P2 -cap 3339 -name ckd_#h 8000You should see this:
CMUC00025I mkfbvol: FB volume 2000 successfully created.
CMUC00017I mklcu: LCU 80 successfully created.
CMUC00021I mkckdvol: CKD Volume 8000 successfully created.✅ Checkpoint: Volume 2000 (FB) and volume 8000 (CKD, 3,339 cylinders) exist.
7Check the spread with dacheck.py
Save this on your laptop as dacheck.py, in the lab folder next to ds8sim.py. It runs there, not on the DS8000, and talks to the box only through the DS CLI.
A script can’t answer a password prompt, so if security.dat and auto.profile aren’t in your lab folder yet from an earlier lesson, create them first (use your own password both times).
Run this (PowerShell):
python ds8sim.py -hmc1 10.0.0.1 -user admin -passwd YourPassw0rd managepwfile -action add -name admin -pw YourPassw0rd -pwfile security.datThen save these eight lines as auto.profile:
hmc1: 10.0.0.1
username: admin
pwfile: security.dat
devid: IBM.2107-78SV001
banner: off
header: on
format: delim
delim: ,File dacheck.py:
# dacheck.py: runs on your laptop or jump server, not on the DS8000.
# It talks to the DS8000 only through the DS CLI (auto.profile: banner off, header on, format delim).
# It counts arrays per DA pair (per drive class) and ranks per rank group (per storage type).
import csv, subprocess, sys
from collections import Counter
DSCLI = ["python", "ds8sim.py", "-cfg", "auto.profile"] # at work: ["dscli", "-cfg", "auto.profile"]
def dscli(*args):
"""Run one DS CLI command and return its rows as dictionaries."""
out = subprocess.run(DSCLI + list(args), capture_output=True, text=True).stdout
lines = [l for l in out.splitlines() if l and not l.startswith("SaatVedha") and not l.startswith("CMUC")]
return list(csv.DictReader(lines))
sites = dscli("lsarraysite", "-l")
arrays = dscli("lsarray", "-l")
ranks = dscli("lsrank", "-l")
warn = 0
print("Arrays per DA pair")
for cls in sorted({s["diskclass"] for s in sites}):
pairs = sorted({s["DA Pair"] for s in sites if s["diskclass"] == cls}, key=int)
used = Counter(a["DA Pair"] for a in arrays if a["diskclass"] == cls)
counts = [used[p] for p in pairs]
text = " ".join("DA%s=%d" % (p, used[p]) for p in pairs)
if len(pairs) > 1 and max(counts) - min(counts) > 1:
print(" WARN %-16s %s (put the next array on the quieter DA pair)" % (cls, text)); warn += 1
elif len(pairs) == 1:
print(" INFO %-16s %s (only one DA pair has these drives)" % (cls, text))
else:
print(" OK %-16s %s" % (cls, text))
print("Ranks per rank group (0 = server0, 1 = server1)")
for stg in sorted({r["stgtype"] for r in ranks}):
grp = Counter(r["Group"] for r in ranks if r["stgtype"] == stg)
text = "group0=%d group1=%d not in a pool=%d" % (grp["0"], grp["1"], grp["-"])
if abs(grp["0"] - grp["1"]) > 1:
print(" WARN %-4s %s" % (stg, text)); warn += 1
else:
print(" OK %-4s %s" % (stg, text))
print("Result:", "WARN (%d)" % warn if warn else "balanced")
sys.exit(1 if warn else 0)
The script reads lsarraysite -l to learn which DA pairs hold each disk class, counts the arrays per DA pair from lsarray -l, and counts ranks per rank group from lsrank -l. A gap of more than one array or rank gets a WARN and exit code 1, which a scheduler at work can turn into a ticket.
Run this:
python dacheck.py
echo $LASTEXITCODEYou should see this:
Arrays per DA pair
OK FlashCoreModule DA0=1 DA2=1
INFO FlashTier2 DA2=1 (only one DA pair has these drives)
Ranks per rank group (0 = server0, 1 = server1)
OK ckd group0=1 group1=0 not in a pool=0
OK fb group0=1 group1=1 not in a pool=0
Result: balanced
0
The CKD line says OK with one rank in group 0 and none in group 1, because a gap of one is allowed. When the mainframe team asks for more space, the next CKD rank goes into a rank group 1 pool, and you’ll do that in Break It.
✅ Checkpoint: dacheck.py prints “Result: balanced” and exit code 0.
On macOS or Linux run python3 dacheck.py and change "python" to "python3" in the DSCLI line. At work you swap the DSCLI line for the real dscli and run it from cron or Task Scheduler on the jump server after every capacity change.
Break It, Then Fix It
Carry on from the lab state. First you’ll make the mistake that hurts slowly (everything on one DA pair), then one the DS CLI stops at once (a CKD rank pushed into an FB pool).
1Break it: a rushed build on DA pair 0
The application team wants more FB space today. You take “the next unassigned sites”, which are S2 and S3, and put both ranks into the pool you used last time.
Run this:
mkarray -raidtype 6 -arsite S2
mkarray -raidtype 6 -arsite S3
mkrank -array A3 -keygrp 1 -stgtype fb
mkrank -array A4 -keygrp 1 -stgtype fb
chrank -extpool P0 R3
chrank -extpool P0 R4You should see this:
CMUC00004I mkarray: Array A3 successfully created.
CMUC00004I mkarray: Array A4 successfully created.
CMUC00007I mkrank: Rank R3 successfully created.
CMUC00007I mkrank: Rank R4 successfully created.
CMUC00008I chrank: Rank R3 successfully modified.
CMUC00008I chrank: Rank R4 successfully modified.Every command worked, and that’s the trap. Now run the check from your laptop.
Run this:
python dacheck.py
echo $LASTEXITCODEYou’ll get this:
Arrays per DA pair
WARN FlashCoreModule DA0=3 DA2=1 (put the next array on the quieter DA pair)
INFO FlashTier2 DA2=1 (only one DA pair has these drives)
Ranks per rank group (0 = server0, 1 = server1)
OK ckd group0=1 group1=0 not in a pool=0
WARN fb group0=3 group1=1 not in a pool=0
Result: WARN (2)
1
Why: S2 and S3 are both on DA pair 0, so three of the four FCM arrays now hang off one adapter pair, and three of the four FB ranks feed server0. Nothing stopped you, because the DS8000 lets any rank join any pool.
Fix: R4 has no volumes yet, so take it back out, release S3 and build the next array on the quieter DA pair (S6), placing it in the rank group 1 pool.
Run this:
chrank -unassign R4
rmrank -quiet R4
rmarray -quiet A4
mkarray -raidtype 6 -arsite S6
mkrank -array A4 -keygrp 1 -stgtype fb
chrank -extpool P1 R4
python dacheck.pyYou should see this:
CMUC00008I chrank: Rank R4 successfully modified.
CMUC00010I rmrank: Rank R4 successfully deleted.
CMUC00006I rmarray: Array A4 successfully deleted.
CMUC00004I mkarray: Array A4 successfully created.
CMUC00007I mkrank: Rank R4 successfully created.
CMUC00008I chrank: Rank R4 successfully modified.
Arrays per DA pair
OK FlashCoreModule DA0=2 DA2=2
INFO FlashTier2 DA2=1 (only one DA pair has these drives)
Ranks per rank group (0 = server0, 1 = server1)
OK ckd group0=1 group1=0 not in a pool=0
OK fb group0=2 group1=2 not in a pool=0
Result: balancedThe system reused the IDs A4 and R4, but this A4 sits on S6 behind DA pair 2. On a real box, rmrank formats the drives first, so plan for that time before you undo a rank, and never unassign one that holds volumes.
✅ Checkpoint: dacheck.py shows FlashCoreModule DA0=2 DA2=2 and fb group0=2 group1=2.
2Break it: a CKD rank in an FB pool
The mainframe team needs a second CKD rank, so you build one on S8 and, in a hurry, assign it to P1 because “P1 is the rank group 1 pool”.
Run this:
mkarray -raidtype 6 -arsite S8
mkrank -array A5 -keygrp 1 -stgtype ckd
chrank -extpool P1 R5You’ll get this (the last line is a practice shell message):
CMUC00004I mkarray: Array A5 successfully created.
CMUC00007I mkrank: Rank R5 successfully created.
CMUN02020E chrank: R5: The storage type of rank R5 (ckd) does not match the storage type of extent pool P1 (fb).Why: a pool has one storage type, set when you create it. P1 is fb, R5 is ckd, and the two can never mix, which is the same reason an FB volume can’t land in a CKD pool.
Fix: make a CKD pool for rank group 1 and put R5 there. That also gives the mainframe its second server.
Run this:
mkextpool -rankgrp 1 -keygrp 1 -stgtype ckd ft2_ckd_1
chrank -extpool P3 R5
python dacheck.pyYou should see this:
CMUC00000I mkextpool: Extent pool P3 successfully created.
CMUC00008I chrank: Rank R5 successfully modified.
Arrays per DA pair
OK FlashCoreModule DA0=2 DA2=2
INFO FlashTier2 DA2=2 (only one DA pair has these drives)
Ranks per rank group (0 = server0, 1 = server1)
OK ckd group0=1 group1=1 not in a pool=0
OK fb group0=2 group1=2 not in a pool=0
Result: balanced✅ Checkpoint: R5 is in P3 (ckd, group 1) and dacheck.py shows ckd group0=1 group1=1, Result: balanced.
One more classic, for reference only: mkarray -raidtype 5 on a G10 shows a warning (CMUC00536W) and then CMUN81522E, “RAID 5 is not supported by the device adapter pair”. You met that one in the arrays lesson; it’s a DA pair rule too.
This happened at work
The month-end that ran on one adapter pair
A mid-sized private bank ran its core banking database on a DS8A50 with two HPFE pairs, so two DA pairs. The box had been set up well, with arrays spread across both pairs and pools in both rank groups.
In the second year they ordered two more drive sets. Both went into the first HPFE pair because it had empty slots, and nobody on the storage team looked at the order layout.
The admin ran the old build script, which took the four new unassigned array sites in ID order and added all four ranks to the database pool pair. Every new array was on DA pair 0.
The first month-end after that, the interest and statement batch ran late into the morning. The database team saw higher read latency on the new data files, while the older ones were fine. The first guess was the database itself, and they spent a night tuning queries that hadn’t changed.
The storage engineer who finally looked compared the DS8000 performance reports per DA pair and saw DA pair 0 far busier than DA pair 1. lsarray -l showed why in one screen: every new array had DA pair 0 in its row.
They couldn’t move drives quickly, so they moved the batch staging volumes to a pool pair built only on DA pair 1 over two weekends, and asked IBM to put the next drive sets in the second HPFE pair. Since then every capacity change has a one-page placement plan, per DA pair and per rank group, signed off before anyone runs mkarray, and a balance check runs after the build.
Common mistakes and troubleshooting
| Symptom | Likely cause | What to check / fix |
|---|---|---|
| One DA pair busy, the other quiet | New arrays all built on one DA pair | lsarray -l DA Pair column; plan the next arrays on the other pair |
| One server much busier than the other | Most ranks in rank group 0 pools | lsrank -l Group column; add new ranks to rank group 1 pools |
| chrank fails with a storage type mismatch | CKD rank into an FB pool or the reverse | Make a pool of the right -stgtype, or use the other rank |
| mkrank asks for a key group | Encryption is on | lskeygrp, then mkrank -keygrp 1 |
| mkarray -raidtype 5 refused | G10 DA pairs do not do RAID 5 | Use RAID 6 (default) or RAID 10 |
| Array is 5+P+Q+S, not 6+P+Q | First arrays on a DA pair hold the spares | Normal; later arrays on that pair are 6+P+Q |
| FCM and Flash Tier 2 ranks will not share a pool | No intermix on Release 10.1 | One pool per drive class |
Habit: paste lsarraysite -l into the change record before any build, and lsarray -l plus lsrank -l after it. The DA Pair and Group columns side by side tell a reviewer in ten seconds whether the plan was followed.
Interview questions
Click a question to open it. Each one is written the way an interviewer would ask it. Under it you will see why they ask it and one way you could answer, in your own words.
Fresher level
1. What is a DA pair on a DS8A00, and why are there always two adapters?
Why they ask this: They want to see that you know the hardware under the arrays, not only the commands.
How you might answer: A DA pair is two flash RAID adapters in slots C1 or C2 of two different I/O enclosures, each with allegiance to a different CPC. Together they own one HPFE Gen3 pair, with four ports each to the four ESMs, so eight paths. If one adapter or path fails, the partner still reaches every drive.
2. You run lsarraysite -l on a new box. Which columns do you look at before building anything?
Why they ask this: A quick check that you plan before you type.
How you might answer: I look at DA Pair, diskclass and dkcap first, then State. That tells me which sites share an adapter pair and which drives can go into the same pool, so I can spread arrays over the DA pairs and keep FCM and Flash Tier 2 apart.
2–5 years experience
3. Walk me through adding four array sites of new capacity to a running DS8000.
Why they ask this: They are checking for a repeatable, safe process with a plan and a check.
How you might answer: First I run lsarraysite -l and lsarray -l to see where the new sites are and how the existing arrays are spread. I write a plan that puts the new arrays across the DA pairs and the ranks across rank group 0 and 1 pools, keeping one drive class per pool.
Then mkarray -raidtype 6, mkrank with the key group and the right -stgtype, chrank -extpool, and finally I compare lsarray -l and lsrank -l with the plan. In my last project we ran a small script for that check after every change.
4. What is the difference between balancing across DA pairs and balancing across rank groups? When does each one bite?
Why they ask this: They want to hear that these are two separate dimensions.
How you might answer: Rank groups balance the two servers: rank group 0 pools are worked by server0 and rank group 1 by server1. DA pairs balance the back end, the adapters and drives. You can be perfect on one and terrible on the other, for example all ranks split 50/50 between servers but every array on DA pair 0, and that shows up as back-end latency under heavy batch.
Senior level
5. A database team says latency doubled after last month’s expansion. Nothing else changed. How do you investigate?
Why they ask this: They are testing whether you connect a symptom to the physical layout.
How you might answer: I would compare lsarray -l before and after, looking at the DA Pair column for the new arrays, and pull performance reports per DA pair and per rank, for example with lsperfrescrpt on DP0 and DP1. If the new arrays all sit on one DA pair that is also busy, that is my first suspect.
The fix depends on whether volumes already live there: move the busiest volumes to pools on the quieter pair, and make sure the next capacity order puts drives behind the other DA pair.
6. Design rules for a mixed FB and CKD DS8A50 with two HPFE pairs. What do you agree up front?
Why they ask this: Senior people are expected to set rules others follow.
How you might answer: I agree which drive classes go where, one class per pool, and that each storage type gets a pool in both rank groups. FB and CKD both get arrays on both DA pairs where the drives allow it, and CKD LCUs and FB LSSs sit in separate address groups. I write it down as a placement table per DA pair and make the post-build check part of the change.
Quick quiz
Q1. Who creates array sites on a DS8A00?
- You, with mkarray
- The system, from installed drives
- The host team
- Easy Tier
Show answer
B — The system groups each 16-drive set into two 8-drive array sites. There is no command to make one.
Q2. How many DA pairs does one HPFE Gen3 pair need?
- One
- Two
- Four
- None
Show answer
A — One DA pair is required for each HPFE Gen3 pair.
Q3. Which column tells you which DA pair an array site belongs to?
- State in lsextpool
- DA Pair in lsarraysite -l
- Group in lsrank -l
- encrypt in lsarray -l
Show answer
B — lsarraysite -l and lsarray -l both show a DA Pair column.
Q4. chrank -extpool P1 R5 fails: rank ckd, pool fb. What do you do?
- Use RAID 5
- Create a CKD pool and assign R5 there
- Recreate P1 as ckd with the FB ranks inside
- Change the DA pair
Show answer
B — A pool has one storage type. CKD ranks need a CKD pool.
Q5. Why are the first two RAID 6 arrays on a DA pair 5+P+Q+S?
- A bug
- They donate the two spares for that DA pair
- They are RAID 5
- FCM drives are smaller
Show answer
B — Each HPFE pair (DA pair) gets two global spares from its first arrays.
Quick recap
- A DA pair is two flash RAID adapters in different I/O enclosures, each tied to a different CPC, owning one HPFE Gen3 pair over eight paths.
- The system makes array sites (8 drives each) from installed drives; each site belongs to one DA pair.
- Read lsarraysite -l (DA Pair, diskclass), lsdacard, lsstgencl and lsdrive together.
- Chain: array site → array → rank (fb or ckd) → pool (rank group 0/1) → LSS (auto) or LCU (mklcu) → mkfbvol or mkckdvol.
- No command configures a DA pair. You configure placement: arrays across DA pairs, ranks across rank groups, one drive class per pool.
- Check after every build with lsarray -l, lsrank -l and a small script such as dacheck.py.
Next lesson
Lesson 13: Create arrays and ranks → You build RAID 6 arrays and encrypted ranks on a plan like the one you made here.
Further reading: DS8000 DA pair
Want to go deeper into DS8000 DA pair? IBM Redbook SG24-8559 (DS8A00) and the DS CLI guide G10-C53-00 cover the adapters and the commands, and DS8000 internal connections shows the hardware. You can also see the full plan for this course on the IBM DS8000 Implementation course page.