HowTo DELETE & RECREATE a Tape Library in Spectrum Protect

Sometimes, we need to delete and recreate all references to a tape library under Spectrum Protect, maybe because we have replaced the HW (even if it is for the same model of library), or because we are running a Disaster Recovery test,
or like on my particular case: because I had a tape library logically partitioned, and I removed the partition and assigned all the tape library resources to my Spectrum Protect server.

The process is not extremely complex, nor trivial, therefore I will post the steps needed to achieve the full change.

My setup is a Spectrum Protect server v8.1.5 running under an LPAR with AIX v7.2, and my Tape Library is an IBM (Now Lenovo) TS3200 with 4 LTO Tape Drives.
Since I had the TS3200 partitioned in 2 logicalis libraries, the name of the tape library was TS3200_LL2 in my Spectrum Protect server (called spectre), and had 2 LTO tape drives assigned (DRIVE3 & DRIVE4).
After the change, the library under Spectrum Protect will be called TS3200, and will have all 4 drives assigned (DRIVE1 to DRIVE4).

I have put an easy to follow index, step-by-step, first I indicate where is the action performed (SP for Spectrum Protect, AIX for the OS, TS3200 for the physical library GUI interface, and MANUAL: hmm, for -pen & paper!-).

INDEX
1.- SP – DELETE TAPE DEVS
2.- SP – DEFINE LIB SP
3.- AIX – DELETE TAPE DEVICES
4.- AIX – RECREATE TAPE DEVICES
5.- AIX – Get the WWNs from the AIX DEVs
6.- TS3200 – Get the WWNs from the TS3200
7.- MANUAL – CORRELATE WWNs y DEVs
8.- AIX – RENAME TAPEDEVs to follow HW’s ORDER
9.- AIX – CHECK DEVs
10.- SP – DEFINE LIBRARY’s CONTROL PATH
11.- SP – DEFINE LIBRARY’s DRIVES
12.- SP – DEFINE LIBRARY’s PATHS
13.- SP – VERIFY (LOGICAL)
14.- SP – REVISE DEVCLASSES
15.- SP – REVISE SCRIPTS
16.- SP – FINAL VERIFY (PHYSICAL)
16.1.- Try the freshly modified scripts
16.2.- Try to use all the tape drives
16.3.- Check Tape Library HW
16.4.- Check and Backup SP Tape Library Definitions

NOTES
A.- What is a Tape Library Control Path
B.- Final Thoughts

bdr

1.- SP – DELETE TAPE DEVS

First, we delete all references to the old devices, so we find out what we have by issuing <query path>, <query drive> and <query library>, and then, we delete the old devices:

Protect: SERVER1>delete path SERVER1 DRIVE4 srctype=server desttype=drive library=TS3200_LL2
ANR1721I A path from SERVER1 to TS3200_LL2 DRIVE4 has been deleted.
Protect: SERVER1>delete path SERVER1 DRIVE3 srctype=server desttype=drive library=TS3200_LL2
ANR1721I A path from SERVER1 to TS3200_LL2 DRIVE3 has been deleted.
Protect: SERVER1>delete path SERVER1 TS3200_LL2 srctype=server desttype=library
ANR1721I A path from SERVER1 to TS3200_LL2 has been deleted.
Protect: SERVER1>delete drive TS3200_LL2 DRIVE4
ANR8412I Drive DRIVE4 deleted from library TS3200_LL2.
Protect: SERVER1>delete drive TS3200_LL2 DRIVE3
ANR8412I Drive DRIVE3 deleted from library TS3200_LL2.
Protect: SERVER1>delete library TS3200_LL2
ANR8410I Library TS3200_LL2 deleted.

2.- SP – DEFINE LIB SP

Then, we define the new library name, it’s only a high level object, as it doesn’t actually link to the HW until we define the Control Path (if you don’t know what a control path is, you can look at the NOTES section A.- What is a Tape Library Control Path at the bottom of this article).

Protect: SERVER1>define library TS3200 libtype=scsi serial=autodetect RESETDrives=yes shared=yes
ANR8400I Library TS3200 defined.

3.- AIX – DELETE TAPE DEVICES

[root@spectre:/]cfgmgr

[root@spectre:/]lsdev -c tape
rmt0 Available 13-T1-01 IBM 3580 Ultrium Tape Drive (FCP)
rmt1 Available 14-T1-01 IBM 3580 Ultrium Tape Drive (FCP)
rmt3 Available 14-T1-01 IBM 3580 Ultrium Tape Drive (FCP)
rmt4 Available 13-T1-01 IBM 3580 Ultrium Tape Drive (FCP)
smc0 Available 14-T1-01 IBM 3573 Tape Medium Changer (FCP)
smc1 Available 14-T1-01 IBM 3573 Tape Medium Changer (FCP)

[root@spectre:/]rmdev -Rdl rmt0
rmt0 deleted
[root@spectre:/]rmdev -Rdl rmt1
rmt1 deleted
[root@spectre:/]rmdev -Rdl rmt3
rmt3 deleted
[root@spectre:/]rmdev -Rdl rmt4
rmt4 deleted
[root@spectre:/]rmdev -Rdl smc0
smc0 deleted
[root@spectre:/]rmdev -Rdl smc1
smc1 deleted

[root@spectre:/]lsdev -c tape
[root@spectre:/]

4.- AIX – RECREATE TAPE DEVICES

[root@spectre:/]cfgmgr

[root@spectre:/]lsdev -c tape
rmt0 Available 13-T1-01 IBM 3580 Ultrium Tape Drive (FCP)
rmt1 Available 13-T1-01 IBM 3580 Ultrium Tape Drive (FCP)
rmt2 Available 14-T1-01 IBM 3580 Ultrium Tape Drive (FCP)
rmt3 Available 14-T1-01 IBM 3580 Ultrium Tape Drive (FCP)
smc0 Available 14-T1-01 IBM 3573 Tape Medium Changer (FCP)

5.- AIX – Get the WWNs from the AIX DEVs

[root@spectre:/]lsdev -c tape -F "name class location physloc description"
rmt0 tape 13-T1-01 U9009.42A.7803790-V5-C13-T1-W2005000E1115B46F-L0 IBM 3580 Ultrium Tape Drive (FCP)
rmt1 tape 13-T1-01 U9009.42A.7803790-V5-C13-T1-W200B000E1115B46F-L0 IBM 3580 Ultrium Tape Drive (FCP)
rmt2 tape 14-T1-01 U9009.42A.7803790-V5-C14-T1-W2002000E1115B46F-L0 IBM 3580 Ultrium Tape Drive (FCP)
rmt3 tape 14-T1-01 U9009.42A.7803790-V5-C14-T1-W2008000E1115B46F-L0 IBM 3580 Ultrium Tape Drive (FCP)
smc0 tape 14-T1-01 U9009.42A.7803790-V5-C14-T1-W2002000E1115B46F-L1000000000000 IBM 3573 Tape Medium Changer (FCP)

6.- TS3200 – Get the WWNs from the TS3200

DEVICE WWNN             WWPN
DRIVE1 2001000E1115B46F-2002000E1115B46F
DRIVE2 2004000E1115B46F-2005000E1115B46F
DRIVE3 2007000E1115B46F-2008000E1115B46F
DRIVE4 200A000E1115B46F-200B000E1115B46F 

7.- MANUAL – CORRELATE WWNs y DEVs

rmt0 - drive2
rmt1 - drive4
rmt2 - drive1
rmt3 - drive3, OK
smc0 - drive1, OK, it's the one with the Control Path & appears as WWN-L1000... and identified as Tape Medium Changer.

8.- AIX – RENAME TAPEDEVs to follow HW’s ORDER

Call me finicky, but I cannot stand to have a device called rmt2 in AIX and DRIVE4 in the tape library.

This doesn’t usually happen when you just deploy a brand new tape library, since the serials & WWNs should be correlative, however, a couple of years down the line, and a couple of hardware replacements done, and the serials/WWNs are not correlative anymore, and therefore cfgmgr just creates the devices following an order which is not what we need or want. Most people will leave them as is, but I cannot, it produces me severe itch ;o)

Looking at the relationship we did on the previous step:

smc0 = OK (smc0 it’s fine, we only have 1 drive with a control path, so we leave it as-is)
rmt2 = rmt1 / drive1  (we need to rename rmt2 as rmt1)
rmt0 = rmt2 / drive2  (rmt0 as rmt2)
rmt3 = OK (Bonus! one of the devices matches the right drive out of pure probability!)
rmt1 = rmt4 / drive4  (and finally, rmt1 as rmt4)

[root@spectre:/]chdev -l rmt1 -a new_name=rmt4
rmt1 changed
[root@spectre:/]chdev -l rmt2 -a new_name=rmt1
rmt2 changed
[root@spectre:/]chdev -l rmt0 -a new_name=rmt2
rmt0 changed

9.- AIX – CHECK DEVs

We will check that the renaming of the devices matches the hardware descriptions:

[root@spectre:/]lsdev -c tape
rmt1 Available 14-T1-01 IBM 3580 Ultrium Tape Drive (FCP)
rmt2 Available 13-T1-01 IBM 3580 Ultrium Tape Drive (FCP)
rmt3 Available 14-T1-01 IBM 3580 Ultrium Tape Drive (FCP)
rmt4 Available 13-T1-01 IBM 3580 Ultrium Tape Drive (FCP)
smc0 Available 14-T1-01 IBM 3573 Tape Medium Changer (FCP)
 [root@spectre:/]lsdev -c tape -F "name class location physloc description"
rmt1 tape 14-T1-01 U9009.42A.7803790-V5-C14-T1-W2002000E1115B46F-L0 IBM 3580 Ultrium Tape Drive (FCP)
rmt2 tape 13-T1-01 U9009.42A.7803790-V5-C13-T1-W2005000E1115B46F-L0 IBM 3580 Ultrium Tape Drive (FCP)
rmt3 tape 14-T1-01 U9009.42A.7803790-V5-C14-T1-W2008000E1115B46F-L0 IBM 3580 Ultrium Tape Drive (FCP)
rmt4 tape 13-T1-01 U9009.42A.7803790-V5-C13-T1-W200B000E1115B46F-L0 IBM 3580 Ultrium Tape Drive (FCP)
smc0 tape 14-T1-01 U9009.42A.7803790-V5-C14-T1-W2002000E1115B46F-L1000000000000 IBM 3573 Tape Medium Changer (FCP)

10.- SP – DEFINE LIBRARY’s CONTROL PATH

Protect: SERVER1>define path SERVER1 TS3200 srctype=server desttype=library device=/dev/smc0 online=yes autodetect=yes
ANR1720I A path from SERVER1 to TS3200 has been defined.

11.- SP – DEFINE LIBRARY’s DRIVES

Spectrum protect just uses the drives as a logical object for a device, it’s not until you create the PATHs that the physical tape device get associated with a drive.

Protect: SERVER1>define drive TS3200 DRIVE1
ANR8404I Drive DRIVE1 defined in library TS3200.
Protect: SERVER1>define drive TS3200 DRIVE2
ANR8404I Drive DRIVE2 defined in library TS3200.
Protect: SERVER1>define drive TS3200 DRIVE3
ANR8404I Drive DRIVE3 defined in library TS3200.
Protect: SERVER1>define drive TS3200 DRIVE4
ANR8404I Drive DRIVE4 defined in library TS3200.

12.- SP – DEFINE LIBRARY’s PATHS

Here is where we associate the OS tape devices with the SP drive objects

Protect: SERVER1>define path SERVER1 DRIVE1 srctype=server desttype=drive library=TS3200 online=yes device=/dev/rmt1 autodetect=yes
ANR1720I A path from SERVER1 to TS3200 DRIVE1 has been defined.
Protect: SERVER1>define path SERVER1 DRIVE2 srctype=server desttype=drive library=TS3200 online=yes device=/dev/rmt2 autodetect=yes
ANR1720I A path from SERVER1 to TS3200 DRIVE2 has been defined.
Protect: SERVER1>define path SERVER1 DRIVE3 srctype=server desttype=drive library=TS3200 online=yes device=/dev/rmt3 autodetect=yes
ANR1720I A path from SERVER1 to TS3200 DRIVE3 has been defined.
Protect: SERVER1>define path SERVER1 DRIVE4 srctype=server desttype=drive library=TS3200 online=yes device=/dev/rmt4 autodetect=yes
ANR1720I A path from SERVER1 to TS3200 DRIVE4 has been defined.

13.- SP – VERIFY (LOGICAL)

Verify that everything looks OK from the logical perspective

Protect: SERVER1>q path
Source Name Source Type Destination Destination On-Line
----------- ----------- ----------- ----------- ----------
SERVER1 SERVER TS3200 LIBRARY Yes
SERVER1 SERVER DRIVE1 DRIVE Yes
SERVER1 SERVER DRIVE2 DRIVE Yes
SERVER1 SERVER DRIVE3 DRIVE Yes
SERVER1 SERVER DRIVE4 DRIVE Yes Protect: SERVER1>q drive
Library Name Drive Name Device Type On-Line
------------ ------------ ----------- -------------------
TS3200 DRIVE1 LTO Yes
TS3200 DRIVE2 LTO Yes
TS3200 DRIVE3 LTO Yes
TS3200 DRIVE4 LTO Yes Protect: SERVER1>q library
Library Name Library Type Shared TS3200  SCSI       Yes

14.- SP – REVISE DEVCLASSES

Protect: SERVER1>q devclass
Device Cl Device Acc Storage Device Ty Format Est/Max Mount
ass Name ess Strate Pool C pe Capacity Limit
gy ount (MB)
--------- ---------- ------- --------- ------ -------- ------
DBBACK_FI Sequential 0       FILE      DRIVE  51,200.0     32
LEDEV
DISK Random 1
LTO_6 Sequential 3 LTO DRIVE DRIVES
 Protect: SERVER1>q devclass LTO_6 f=d
Device Class Name: LTO_6
Device Access Strategy: Sequential
Storage Pool Count: 3
Device Type: LTO
Format: DRIVE
...
Library: TS3200_LL2
Directory:
Server Name:
... Protect: SERVER1>update devclass LTO_6 library=TS3200
ANR2205I Device class LTO_6 updated.

15.- SP – REVISE SCRIPTS

Protect: SERVER1>q scr
Name            Description                     Managing profile
--------------- ------------------------------------- ----------
AUDIT_LIB       SP - Syncro Tape inventory with Tape library
BACKUP_DB       SP - BACKUP DB & Config
CHECKIN_ALL_LIB SP - CHECKIN ALL Tapes in the Library
CHECKIN_PRIVATE SP - CHECKIN Private Tapes
CHECKIN_SCRATCH SP - CHECKIN Scratch Tapes
CONTAINER_COPY  Run container copy pool operation
CONTAINER_RECL  Run container-copy reclamation
LABEL_TAPES     SP - LABEL New Tapes
PATHS_DOWN      SP - Bring DOWN PATHS & DRIVES of the Tape Library
PATHS_UP        SP - Bring UP PATHS & DRIVES of the Tape Library

As usual, we have a good number of scripts to perform actions with tapes, and as SP forces us to specify the LIB name in each command, we will have to change a few of this scripts to point to the new tape devices.

Also, as we now have 4 drives instead of 2, we will need to modify a couple of scripts to account for the extra tape drives. So, let’s go ahead and change three as an example:

Protect: SERVER1>q scr AUDIT_LIB f=d
...
Name: AUDIT_LIB
Line Number: 5
Command: audit library TS3200_LL2 checklabel=barcode refresh=yes
Last Update by (administrator): CIJALBA
Last Update Date/Time: 06/09/17 10:18:54
Protect: SERVER1>upd scr AUDIT_LIB "audit library TS3200 checklabel=barcode refresh=yes" line=5
ANR1456I UPDATE SCRIPT: Command script AUDIT_LIB updated. Protect: SERVER1>q scr CHECKIN_ALL_LIB f=d
...
Name: CHECKIN_ALL_LIB
Line Number: 10
Command: checkin libvolume TS3200_LL2 status=scratch search=yes checklabel=barcode
Name: CHECKIN_ALL_LIB
Line Number: 20
Command: checkin libvolume TS3200_LL2 status=private search=yes checklabel=barcode
Protect: SERVER1>upd scr CHECKIN_ALL_LIB "checkin libvolume TS3200 status=scratch search=yes checklabel=barcode" line=10
ANR1456I UPDATE SCRIPT: Command script CHECKIN_ALL_LIB updated.
Protect: SERVER1>upd scr CHECKIN_ALL_LIB "checkin libvolume TS3200 status=private search=yes checklabel=barcode" line=20
ANR1456I UPDATE SCRIPT: Command script CHECKIN_ALL_LIB updated. Protect: SERVER1>q scr PATHS_DOWN f=d
...
Name: PATHS_DOWN
Line Number: 1
Command: upd path SERVER1 DRIVE3 srcty=server destt=drive library=TS3200_LL2 online=no
Name: PATHS_DOWN
Line Number: 5
Command: upd path SERVER1 DRIVE4 srcty=server destt=drive library=TS3200_LL2 online=no
Name: PATHS_DOWN
Line Number: 10
Command: upd drive TS3200_LL2 DRIVE3 online=no
Name: PATHS_DOWN
Line Number: 15
Command: upd drive TS3200_LL2 DRIVE4 online=no
Protect: SERVER1>upd scr PATHS_DOWN "upd drive TS3200 DRIVE4 online=no" line=40
ANR1456I UPDATE SCRIPT: Command script PATHS_DOWN updated.
Protect: SERVER1>upd scr PATHS_DOWN "upd drive TS3200 DRIVE3 online=no" line=35
ANR1456I UPDATE SCRIPT: Command script PATHS_DOWN updated.
Protect: SERVER1>upd scr PATHS_DOWN "upd drive TS3200 DRIVE2 online=no" line=30
ANR1456I UPDATE SCRIPT: Command script PATHS_DOWN updated.
Protect: SERVER1>upd scr PATHS_DOWN "upd drive TS3200 DRIVE1 online=no" line=25
ANR1456I UPDATE SCRIPT: Command script PATHS_DOWN updated.
Protect: SERVER1>upd scr PATHS_DOWN "upd path SERVER1 DRIVE4 srcty=server destt=drive library=TS3200 online=no" line=20
ANR1456I UPDATE SCRIPT: Command script PATHS_DOWN updated.
Protect: SERVER1>upd scr PATHS_DOWN "upd path SERVER1 DRIVE3 srcty=server destt=drive library=TS3200 online=no" line=15
ANR1456I UPDATE SCRIPT: Command script PATHS_DOWN updated.
Protect: SERVER1>upd scr PATHS_DOWN "upd path SERVER1 DRIVE2 srcty=server destt=drive library=TS3200 online=no" line=10
ANR1456I UPDATE SCRIPT: Command script PATHS_DOWN updated.
Protect: SERVER1>upd scr PATHS_DOWN "upd path SERVER1 DRIVE1 srcty=server destt=drive library=TS3200 online=no" line=5
ANR1456I UPDATE SCRIPT: Command script PATHS_DOWN updated.
Protect: SERVER1>upd scr PATHS_DOWN "upd path SERVER1 TS3200 srcty=server destt=library online=no" line=1
ANR1456I UPDATE SCRIPT: Command script PATHS_DOWN updated.

At the end, I had to change a few scripts, but if you want to save yourself some time, or have a lot more scripts than I do, then it will be more efficient to redirect all scripts to a text file and manipulate it from the OS (In fact, this is a Best Practice which I recommend to do from time to time: Export your SP Scripts out of SP).

This is easily done with:

Protect: SERVER1>q scr * f=d > /tmp/scripts.txt
Output of command redirected to file '/tmp/scripts.txt'

And then just do a grep from the OS, you can check for the old name and the new name, util, you have modified all the scripts:

[root@spectre:/tmp]grep -c TS3200 scripts.txt
23
[root@spectre:/tmp]grep -c TS3200_LL2 scripts.txt
0

16.- SP – FINAL VERIFY (PHYSICAL)

16.1.- Try the freshly modified scripts:

Protect: SERVER1>run PATHS_DOWN
ANR1722I A path from SERVER1 to TS3200 has been updated.
ANR1722I A path from SERVER1 to TS3200 DRIVE1 has been updated.
ANR1722I A path from SERVER1 to TS3200 DRIVE2 has been updated.
ANR1722I A path from SERVER1 to TS3200 DRIVE3 has been updated.
ANR1722I A path from SERVER1 to TS3200 DRIVE4 has been updated.
ANR8467I Drive DRIVE1 in library TS3200 updated.
ANR8467I Drive DRIVE2 in library TS3200 updated.
ANR8467I Drive DRIVE3 in library TS3200 updated.
ANR8467I Drive DRIVE4 in library TS3200 updated.
ANR1462I RUN: Command script PATHS_DOWN completed successfully.
Protect: SERVER1>q path
Source Name Source Type Destination Destination On-Line
Name Type
----------- ----------- ----------- ----------- ----------
SERVER1 SERVER TS3200 LIBRARY No
SERVER1 SERVER DRIVE1 DRIVE No
SERVER1 SERVER DRIVE2 DRIVE No
SERVER1 SERVER DRIVE3 DRIVE No
SERVER1 SERVER DRIVE4 DRIVE No
Protect: SERVER1>q drive
Library Name Drive Name Device Type On-Line
------------ ------------ ----------- -------------------
TS3200 DRIVE1 LTO No
TS3200 DRIVE2 LTO No
TS3200 DRIVE3 LTO No
TS3200 DRIVE4 LTO No
Protect: SERVER1>run PATHS_UP
ANR1722I A path from SERVER1 to TS3200 has been updated.
ANR1722I A path from SERVER1 to TS3200 DRIVE1 has been updated.
ANR1722I A path from SERVER1 to TS3200 DRIVE2 has been updated.
ANR1722I A path from SERVER1 to TS3200 DRIVE3 has been updated.
ANR1722I A path from SERVER1 to TS3200 DRIVE4 has been updated.
ANR8467I Drive DRIVE1 in library TS3200 updated.
ANR8467I Drive DRIVE2 in library TS3200 updated.
ANR8467I Drive DRIVE3 in library TS3200 updated.
ANR8467I Drive DRIVE4 in library TS3200 updated.
ANR1462I RUN: Command script PATHS_UP completed successfully.
Protect: SERVER1>q path
Source Name Source Type Destination Destination On-Line
Name Type
----------- ----------- ----------- ----------- ----------
SERVER1 SERVER TS3200 LIBRARY Yes
SERVER1 SERVER DRIVE1 DRIVE Yes
SERVER1 SERVER DRIVE2 DRIVE Yes
SERVER1 SERVER DRIVE3 DRIVE Yes
SERVER1 SERVER DRIVE4 DRIVE Yes
Protect: SERVER1>q drive
Library Name Drive Name Device Type On-Line
------------ ------------ ----------- -------------------
TS3200 DRIVE1 LTO Yes
TS3200 DRIVE2 LTO Yes
TS3200 DRIVE3 LTO Yes
TS3200 DRIVE4 LTO Yes
Protect: SERVER1>run AUDIT_LIB
ANR1462I RUN: Command script AUDIT_LIB completed successfully.
Protect: SERVER1>q libv
ANR2034E QUERY LIBVOLUME: No match found using this criteria.
ANS8001I Return code 11.
Protect: SERVER1>run CHECKIN_ALL_LIB
ANR1462I RUN: Command script CHECKIN_ALL_LIB completed successfully.
Protect: SERVER1>q libv
Library Name Volume Name Status Owner Last Use Home El Device
ement Type
------------ ----------- ---------------- ---------- --------- ------- ------
TS3200 000001L6 Private SERVER1 4,118 LTO
TS3200 000004L6 Private SERVER1 4,123 LTO
TS3200 000006L6 Private SERVER1 4,119 LTO
TS3200 000007L6 Private SERVER1 4,125 LTO
TS3200 000009L6 Private SERVER1 4,136 LTO
TS3200 000010L6 Scratch 4,102 LTO
TS3200 000011L6 Private SERVER1 4,139 LTO
...

The scripts work OK.
OK!!!!!!!!!!

16.2.- Try to use all the tape drives:

If we are lucky, SP might launch a Space Reclamation process which will use 2 drives, otherwise by using a MOVE DATA command, we will use 2 tape drives at the same time one for READ and another for WRITE, so by issuing a couple of MOVE DATAs, we will try the 4 tape drives at once.

Protect: SERVER1>q vol stg=tapepool
Volume Name Storage Poo Device Cla Estimated Pct U Volume S
l Name ss Name Capacity til tatus
------------------------ ----------- ---------- --------- ----- --------
000002L6 TAPEPOOL LTO_6 8.9 T 10.7 Filling
000009L6 TAPEPOOL LTO_6 5.7 T 0.4  Filling
000014L6 TAPEPOOL LTO_6 9.5 T 0.0  Full
000019L6 TAPEPOOL LTO_6 5.7 T 0.0  Filling
000024L6 TAPEPOOL LTO_6 5.7 T 8.2  Filling
000031L6 TAPEPOOL LTO_6 5.7 T 37.9 Filling
...
Protect: SERVER1>move data 000014L6
ANR2232W This command will move all of the data stored on volume 000014L6 to other volumes within the same storage pool; the data
will be inaccessible to users until the operation completes.
Do you wish to proceed? (Yes (Y)/No (N)) Y
ANS8003I Process number 5 started.
Protect: SERVER1>q pr
Process Process Description Process Status
Number
-------- -------------------- -----------------------------------------------
5 Move Data Volume 000014L6 (storage pool TAPEPOOL), Target Pool TAPEPOOL, Moved Files: 0, 
Moved Bytes: 0 bytes, Deduplicated Bytes: 0 bytes, Unreadable Files: 0, Unreadable Bytes: 0
 bytes. Current Physical File (bytes): 2,033 bytes Waiting for mount of scratch volume (1 seconds).
Protect: SERVER1>q req
ANR8352I Requests outstanding:
ANR8308I 001: LTO volume 000014L6 is required for use in library TS3200; CHECKIN LIBVOLUME required within 20 minutes.
Protect: SERVER1>reply 1
ANR8499I Command accepted.

Update the status of the tapes in the library to be READWRITE (depends how how many tapes you have, careful since in the following example, I have made READW all my tapes, but might not be wise for your system if you have a big tape library, or different tape libraries –a better example should have been a and update each vol individually, but I am pressed for time ;o) –)

Protect: SERVER1>upd vol * access=readw
ANR2207I Volume 000001L6 updated.
ANR2207I Volume 000002L6 updated.
ANR2207I Volume 000003L6 updated.
...
ANR2207I Volume 000061L6 updated.
ANR2207I Volume 000062L6 updated.
ANR2207I Volume 000063L6 updated.

After a while, all the 4 drives had a tape mounted and where doing operations, so: The drives work fine.
OK!!!!!!!!!!

16.3.- Check Tape Library HW

Bad tapes or problems with barcodes can be checked using the SHOW SLOTS undocumented cmd:

Protect: SERVER1>show slots ts3200
PVR slot information for library TS3200.
Library : TS3200
Product Id : 3573-TL
Support module : 2
Mount count : 1
Drives : 4
Slots : 44
Changers : 1
Import/Exports : 3
.
Device : /dev/smc0
.
Drive 0, element 256
Drive 1, element 257
Drive 2, element 258
Drive 3, element 259
.
Changer 0, element 1
.
ImpExp 0, element number 16
ImpExp 1, element number 17
ImpExp 2, element number 18
Slot 0, status Allocated, element number 4096, barcode present, barcode value , devT=LTO, mediaT=436, elemT=ANY
Slot 1, status Allocated, element number 4097, barcode present, barcode value , devT=LTO, mediaT=436, elemT=ANY
Slot 2, status Allocated, element number 4098, barcode present, barcode value , devT=LTO, mediaT=436, elemT=ANY
...
Slot 42, status Allocated, element number 4138, barcode present, barcode value , devT=LTO, mediaT=436, elemT=ANY
Slot 43, status Allocated, element number 4139, barcode present, barcode value , devT=LTO, mediaT=436, elemT=ANY
.
slot element range 4096 - 4139

No problems in tapes or barcodes found.
OK!!!!!!!!!!

16.4.- Check and Backup SP Tape Library Definitions:

Now that we have redefined the tape library configuration and loaded the tapes, Issue a BACKUP DEVCONFIG and a BACKUP VOLHIST.

Protect: SERVER1> BACKUP VOLHISTORY
ANR2463I BACKUP VOLHISTORY: Server sequential volume history information was written to all configured history files. Protect: SERVER1> BACKUP DEVCONFIG
ANR2394I BACKUP DEVCONFIG: Server device configuration information was written to all device configuration files.

We should go to Spectrum Protects installation directory (by default /home/tsminst1), and look at the devconfig file (devconf.dat).

[root@spectre:/home/tsminst1]cat devconf.dat
/* Device Configuration */
DEFINE DEVCLASS DBBACK_FILEDEV DEVT=FILE FORMAT=DRIVE SHARE=NO MAXCAP=52428800K MOUNTL=32 DIR=/tsminst1/TSMbkup00,/tsminst1/TSMbkup01
DEFINE DEVCLASS LTO_6 DEVT=LTO FORMAT=DRIVE MOUNTL=DRIVES MOUNTWAIT=20 MOUNTRETENTION=5 PREFIX=ADSM LIBRARY=TS3200 WORM=NO DRIVEENCRYPTION=ALLOW LBPROTECT=NO
DEFINE SERVER SPECTRE COMMMETHOD=TCPIP HLADDRESS=10.1.1.207 LLADDRESS=1500
SET SERVERNAME SERVER1
DEFINE LIBRARY TS3200 LIBTYPE=SCSI WWN="2001000E1115B46F" SERIAL="A0L4U78W5927_LL0" SHARED=YES AUTOLABEL=NO RESETDRIVE=YES
DEFINE DRIVE TS3200 DRIVE1 ELEMENT=256 ONLINE=Yes WWN="2001000E1115B46F" SERIAL="A0WT025496"
DEFINE DRIVE TS3200 DRIVE2 ELEMENT=257 ONLINE=Yes WWN="2004000E1115B46F" SERIAL="A0WT038765"
DEFINE DRIVE TS3200 DRIVE3 ELEMENT=258 ONLINE=Yes WWN="2007000E1115B46F" SERIAL="A0WT046112"
DEFINE DRIVE TS3200 DRIVE4 ELEMENT=259 ONLINE=Yes WWN="200A000E1115B46F" SERIAL="A0WT045812"
/* LIBRARYINVENTORY SCSI TS3200 000001L6 4118 101*/
/* LIBRARYINVENTORY SCSI TS3200 000004L6 4123 101*/
...
/* LIBRARYINVENTORY SCSI TS3200 000062L6 4121 101*/
/* LIBRARYINVENTORY SCSI TS3200 000063L6 4105 101*/
DEFINE PATH SERVER1 TS3200 SRCTYPE=SERVER DESTTYPE=LIBRARY DEVICE=/dev/smc0 ONLINE=YES
DEFINE PATH SERVER1 DRIVE1 SRCTYPE=SERVER DESTTYPE=DRIVE LIBRARY=TS3200 DEVICE=/dev/rmt1 ONLINE=YES
DEFINE PATH SERVER1 DRIVE2 SRCTYPE=SERVER DESTTYPE=DRIVE LIBRARY=TS3200 DEVICE=/dev/rmt2 ONLINE=YES
DEFINE PATH SERVER1 DRIVE3 SRCTYPE=SERVER DESTTYPE=DRIVE LIBRARY=TS3200 DEVICE=/dev/rmt3 ONLINE=YES
DEFINE PATH SERVER1 DRIVE4 SRCTYPE=SERVER DESTTYPE=DRIVE LIBRARY=TS3200 DEVICE=/dev/rmt4 ONLINE=YES
SERVERBACKUPNODEID 1

OK!!!!!!!!!!

NOTES

A.- What is a Tape Library Control Path

A Tape Library Control Path it’s a logical path for a SCSI Medium Changer to send commands over to tape drives.

Each tape library has at least one control path, and for example on an AIX OS, the tape drive with the control path, will create 2 devices, one for Tape Drive and one for the Tape Medium Changer (in this example rmt1 and smc0 are really the same physical device):

[root@spectre:/tmp]lsdev -c tape
rmt1 Available 14-T1-01 IBM 3580 Ultrium Tape Drive (FCP)
rmt2 Available 13-T1-01 IBM 3580 Ultrium Tape Drive (FCP)
rmt3 Available 14-T1-01 IBM 3580 Ultrium Tape Drive (FCP)
rmt4 Available 13-T1-01 IBM 3580 Ultrium Tape Drive (FCP)
smc0 Available 14-T1-01 IBM 3573 Tape Medium Changer (FCP)

There is a catch here, and that is: if the tape drive with the control path is taken down, all the tape librarie’s drives will stop working, as the communications bus with the library is down.

In that case, and while we replace/repair the hardware, we will have to change the control path over to another drive, and perhaps reconfigure the device in AIX and Spectrum Protect.

We can have more than one control path in a library to eliminate single points of failure (as this is a clear SPOF in a several drives tape library), however it comes at a price, as at least in IBM libraries, an extra licence must be purchased, to enable Control Path Failover (CPF). In some cases, having CPF also enables Data Path Failover (DFP), which includes load balancing of the HBAs.

B.- Final Thoughts

Well, Phewww, now that was a bit of a long ride, wasn’t it? It’s not actually complex, it’s just a matter or order, and if done in the right sequence (and after having devoured a few red books, and technical guides) it’s pretty straight-forward.

Just try not perform this procedure very often, as it does take a few hours work, and while the process is being done Spectrum Protect cannot use the library (you can do it quicker if you script the lot, of course, and for Disaster Recovery it is recommended, because one or two hours saved in time might make a huge difference).

I hope you have enjoyed the procedure, and any comments or steps which can be done better are always welcome, so if you have suggestions, post them here to <<Give the sysadmin a shout!>>

Advertisements

Update Cisco Servers CIMC’s Firmware

Cisco’s CIMC updates are done using the HUU – Cisco Host Upgrade Utility, which comes in ISO form so can be burned on physical media or mounted remotely via the KVM on the CICM interface. Quoting Cisco’s manual:

<<Upgrading BIOS and Cisco IMC Firmware, Cisco provides the Cisco Host Upgrade Utility to assist you in upgrading the BIOS, Cisco IMC, CMC LOM, LSI storage controller, and Cisco UCS Virtual Interface Cards firmware to compatible levels.

Always upgrade the BIOS, the Cisco IMC and CMC from the HUU ISO. Do not upgrade individual components (only BIOS or only Cisco IMC or CMC), since this could lead to unexpected behavior. If you choose to upgrade BIOS, the Cisco IMC and the CMC individually and not from the HUU ISO, make sure to upgrade both Cisco IMC, BIOS and CMC to the same container release. If the BIOS, CMC and the Cisco IMC versions are from different container releases, it could result in unexpected behavior. Cisco recommends that you use the Update All option from the Host Upgrade Utility to update the firmware versions of Cisco IMC, BIOS, CMC and all other server components (VIC, RAID Controllers, PCI devices, and LOM) together.>>

CIMC_01

1.- Connect to Cisco Download Software website & download the ISO update.

https://software.cisco.com/download/navigator.html

Example of direct dowload HUU 3.03e

https://software.cisco.com/download/release.html?mdfid=286281356&flowid=71443&softwareid=283850974&release=3.0(3e)&relind=AVAILABLE&rellifecycle=&reltype=latest

Easiest way to find the latest software for your Server if you have updated it before:

From Download Software > My Download History > UCS C240 Rack Server Software > Latest Release > Download File

2.- Perform a Backup

Backup of the FW.

NOTE: in CIMC this step is performed automatically. Backup FW versions can be perused in CIMC > Admin > Firmware Management

Backup of the BIOS.

Navigation Panel (Top Left Icon) > Compute > BIOS > Configure BIOS Profile > Take Backup

3.- Revise NTP & TimeZone

Set NTP:

Navigation Panel (Top Left Icon) > Admin > Networking > NTP Setting > NTP Properties
NTP Enabled x
Server 1: 0.pool.ntp.org
Server 2: 1.pool.ntp.org
Server 3: 2.pool.ntp.org
Server 4: 3.pool.ntp.org
> Save Changes

Set TimeZone:

Navigation Panel (Top Left Icon) > Chassis > Summary >Cisco Integrated Management Controller (Cisco IMC) Information > Timezone: Europe/Madrid

Check Time:

Navigation Panel (Top Left Icon) > Chassis > Summary >Cisco Integrated Management Controller (Cisco IMC) Information > Local Time

Check NTP (Note: leave at least 15 minutes between configuration and verification –to avoid getting a stratum 16):

Navigation Panel (Top Left Icon) > Admin > Networking > NTP Setting > NTP Properties > Status: Values 1-15

4.- Start Console

Navigation Panel (Top Left Icon) > Admin > Firmware Management > Firmware Management Review the versions stored.

Open a KVM (Java or HTML), map the ISO downloaded to the server as Virtual Media, CD.
NOTE: if we choose the Java KVM, we can record the whole installation in a video (for documentation purposes), the HTML KVM option can only take snaps, not videos.

If the UCS is being managed from a vCenter or a OVMmanager, move the VMs off the server first and put into maintenance or stopped mode.

5.- Reboot from HUU

On Boot, press <F6>, to choose the Boot Menu, and once there, select boot device Cisco vKVM-Mapped vDVD1.22. Now it will start from the HUU software.

ISOLINUX 4.04… image will start booting, followed by a GUI named Cisco UCS Host Upgrade Utility, with an animated loading circle, which does a few full turns.

6.- Update FW Packages

Once HUU has finished copying the FW to the CIMC, it will display the Agreement page, which we will confirm, and after that we can review the FW levels to be updated.

We will select the Update All option, and Are you sure? Yes

It will mark all the packages in the inventory as SCHEDULED, and it will go through each one as IN PROGRESS, PASS.
NOTE: The BIOS is updated last, as after doing that step, the system reboots.

7.- Verify Upgrade & System Status

Once all the packages have a green PASS update status, we can press the Exit button, wait 5 minutes and login into the new CIMC.

Review that the CIMC is correct via HOME > Chassis > Summary, note that the CIMC starts pretty quick, but the server takes longer to start, and it will show as powered off from the CIMC until it has boot up, so it’s best to wait for a while to see if the server comes up OK, before issuing a host power on from the CIMC.

8.- Now, go and grab a coffee… ‘coz we’ve finished!

 

UNIX/Linux everywhere???

In our days most of the appliances that support a terminal session and provide a CLI or command line interface behave like UNIX/Linux.

This is good, as at least there is a wide standard for basic system commands like:

 ls
 dir
 cat
 df
 uname
 hostname
 ps
 etc...

However the most common implementation of a CLI in appliances that have an embedded OS, is not really a Linux environment or shell, but an implementation of BusyBox.

BusyBox is a multi-call binary that combines many common Unix/Linux commands and utilities into a single executable, therefore creating links or alias to all the individual commands pointing to the same executable, saving space in disk and memory.

The latest version of BusyBox (v1.26.2 as of 10/01/2017) has all the following commands built in:

 [, [[, ar, awk, base64, basename, bunzip2, bzcat, bzip2, cal, cat,
 catv, chat, chgrp, chmod, chown, chpasswd, chpst, chroot, chrt,
 cksum, clear, cmp, comm, cp, cpio, cryptpw, cut, date, dc, dd,
 devmem, df, diff, dirname, dnsd, dnsdomainname, dos2unix, du, echo,
 ed, egrep, env, envdir, envuidgid, expand, expr, fakeidentd, false,
 fgrep, find, fold, fsync, ftpd, ftpget, ftpput, fuser, getopt, grep,
 groups, gunzip, gzip, hd, head, hexdump, hostid, hostname, httpd,
 id, inetd, install, iostat, ipcalc, kill, killall, killall5, less,
 ln, logger, logname, logread, lpd, lpq, lpr, ls, lzcat, lzma, lzop,
 lzopcat, makemime, man, md5sum, mesg, microcom, mkdir, mkfifo,
 mknod, mkpasswd, mktemp, more, mpstat, mt, mv, nc, nice, nmeter,
 nohup, nslookup, od, patch, pgrep, pidof, pipe_progress, pkill,
 popmaildir, printenv, printf, ps, pscan, pstree, pwd, pwdx,
 readlink, realpath, reformime, renice, reset, resize, rm, rmdir,
 rpm, rpm2cpio, run-parts, runsv, runsvdir, sed, sendmail, seq,
 setsid, setuidgid, sha1sum, sha256sum, sha512sum, sleep, smemcap,
 softlimit, sort, split, start-stop-daemon, stat, strings, stty, sum,
 sv, svlogd, sync, sysctl, syslogd, tac, tail, tar, tcpsvd, tee,
 telnet, telnetd, test, tftp, tftpd, time, timeout, touch, tr, true,
 tty, ttysize, udpsvd, uname, uncompress, unexpand, uniq, unix2dos,
 unlzma, unlzop, unxz, unzip, usleep, uudecode, uuencode, vlock,
 volname, watch, wc, which, whoami, whois, xargs, xz, xzcat, yes,
 zcat

If you want to try busybox on real time, you can go to this page, where there is a JavaScript PC emulator with BusyBox v1.20.0:

http://www.busybox.net/live_bbox/live_bbox.html

If you want to know more about this excellent and widely-used Open-Source project, visit:

http://www.busybox.net/about.html

Traditionally, to try a UNIX like environment on a windows PC, you had a couple of options: install microsoft’s own windows unix support, or install cygwin.

Well it’s good to know that now we have another option, and that it’s easier to install.
But in any case, my all-time favourite until today, is the use of MobaXterm.

This is a product to handle all sort of sessions & connections, and the best of it is that it lets you setup connections with terminals with SSH and Telnet, but also handles RDP, HTTP, etc. It even imports settings from PuTTY.

The best of all, is that it lets you open a local terminal session in windows, and that local session uses our new friend busybox, so you can also type windows commands (not built-ins, only externals, but hey!) or unix-like commands, at your leisure.

It’s specially good to develop unix or linux scripts in your PC, instead of doing so in a real server (really handy when you don’t have a development server in your environment, or if you are developing or fixing scripts in a laptop while you travel, on a hotel, etc.).

So there you go, if you want to have the best of linux in windows you can use busybox, cygwin, or MobaXterm. In Windows 10 mind, we also have the ubuntu bash shell, which is also very nice, and supported by Microsoft, excellent.

HowTo: Actualizar FW de librerías de cintas IBM TS3100-TS3200.

Pre-requisitos:

a.- Conocer el Hardware de nuestra librería de cintas: Modelo, Serial Number, FW version numbers de la librería y de los drives. Saber si tenemos drives Full Hight (FH) o Half Hight (HH) y si su conexión es SAS (directa) o FC (fibra SAN).

[Nota: si tenemos una TS3100, podemos tener 1 drive FH o 2 HH, si es una TS3200, podemos tener 2 drives FH o 4 HH].  Para conocer mejor los detalles del HW de cada librería, revisar sus datos técnicos en la web de Storage de IBM:

http://www-03.ibm.com/systems/storage/tape/ts3100

http://www-03.ibm.com/systems/storage/tape/ts3200

003_modelfw

b.- Bajar el pack de FW de la librería de “IBM Download Center”, el cual incluye tanto el FW de la librería, como el FW de los drives: se debe siempre instalar el pack, ya que se comprueba en los laboratorios de IBM para que la version del FW de los drives funcione correctamente junto con el FW de la librería, y actualizar el FW de la librería o de los drives por separado no está recomendado (a no ser que se reemplace un drive, en cuyo caso su nivel de FW se debe poner al mismo que el resto de drives en la librería -mismo baselevel-).

  • Desde Fix Central, seleccionar las siguientes opciones:

001_fix_central

  • escoger el modelo y versión de pack FW correcto para tu librería:

002_fix_central

c.- Leer el fichero README del pack de actualización de FW por si vienen instrucciones especiales que puedan ser diferentes de las especificadas en este documento.

d.- Disponer de los usuarios de la librería:

· user  (este usuario es para operadores, y no permite actualizar FW).
· admin (en realidad solo se necesita este usuario, pero éste es un buen momento para verificar si tenemos todos los usuarios registrados. por defecto admin/secure para FW posterior a la v1.90 y admin/adm001 para FW anterior).
· service (probar con: service/sev001 para TS3100 y service/Sev001 para TS3200 -Unconfirmed-)

Si no tenemos la contraseña de “admin”, seguir la siguiente documentación oficial de IBM para devolverlo a valores de fábrica:

http://www-01.ibm.com/support/docview.wss?uid=ssg1S1003110

e.- Inhabilitar el uso de la librería desde nuestro servidor/software de backups, ya que si los drives o el brazo robótico están en uso, y la actualización de FW reinicia la librería, los dispositivos en uso pueden estropearse (si un drive usando una cinta se reinicia, la cinta se quedará encajada dentro del drive y se perderá el drive y la cinta -en cuyo caso, abrá que abrir caso a soporte HW IBM-).

 

Procedimiento:

1.- Es buena idea el configurar las alertas por email de la librería de cintas, ya que si la actualización FW falla, o la ventana del navegador se cuelga, recibiremos una notificación por email de los fallos o sucesos a nuestro correo en el PC, portátil o móvil (muy útil y recomendado).

2.- Primero se actualiza el FW de la librería  luego el de los drives, y solo se pueden actualizar de uno en uno. El proceso comprende la subida del fichero de FW primero al dispositivo, y luego su actualización. Ya que estos envíos se hacen a través de interfaces de red a 100 Mbps, son muy lentos y en ocasiones la conexión se cae; cuando esto pasa, falla la actualización del FW, en cuyo caso la probaremos de nuevo, hasta que funcione correctamente (o mejoramos la estabilidad de la conexión).

10_tape_drive_upgrade

 

Verificar que el status de la librería y los drives están en Ready o Idle, y no en reading, writing, loading, unloading, inventory.

Para actualizar el FW, ir a > Service Library > Upgrade Firmware, seleccionar Choose File desde la sección “Upgrade Library Firmware”. Cargamos el fichero .fbi con el FW y pulsamos el botón “Update”.

Comprobaremos el estado de la actualización revisando primero la subida del FW a la librería (mirando la barra de estado en la parte inferior de la ventana) y luego la carga al dispositivo revisando la sección “System Status” a la derecha.

Una vez actualizado el FW de la librería, procederemos a subir el FW de cada drive, siguiendo el mismo procedimiento, pero seleccionando los ficheros .fmrz apropiados para nuestros drives (FH/HH/SAS/FC).

12_tape_drive_upgrade

14_tape_library-drives_upgraded

3.- Una vez finalizado todas las actualizaciones de FWs, hay que probar los drives, el brazo robótico y revisar los logs por errores (una vez revisado todo OK, podemos borrar los logs, para tener el sistema estable y controlado, o dejarlos tal cual por motivos de auditoría).

Para comprobar la librería, podemos lanzar movimientos de cintas desde el panel de control de la librería, o desde el GUI, pero lo mejor es lanzar algún proceso de inventario, consolidación de cintas o traspaso de datos entre cintas desde el software de backup.

4.- Actualizar nuestros sistemas de monitorización (Nagios, Zabbix, Netcool, etc) con el MIB adjunto al FW de la librería, ya que cada actualización de FW de un dispositivo HW cambia los OIDs a monitorizar por SNMP, por lo que hay que actualizar el nuevo MIB, y reconfigurar los OIDs o Trap Events que pudiéramos tener monitorizados por los nuevos.

Notas:

No es necesario realizar actualizaciones de FW correlativas para llegar a la última versión disponible, se puede hacer directamente la subida a la última versión, no obstante el procedimiento recomendado por IBM es subir de versión en versión, paso a paso.

Al hacer la actualización al último nivel disponible, podremos ahorrarnos un montón de tiempo, ya que las actualizaciones (especialmente las de los drives) son lentas y se tarda mucho tiempo, tiempo en el cual la librería no está operativa para el software de backups. (Puede minimizarse el tiempo actualizando la librería y 1 drive, devolver el acceso al software de backup para dicho drive, y a medida que vamos actualizando cada drive, devolver el acceso al software, drive a drive).

En el caso de que la subida directa al último FW falle, probaremos el método recomendado por IBM de subir versión a versión (lógicamente, cuando el atajo no te lleva donde pensabas, lo mejor es por el camino largo y tortuoso pero seguro…).

Las actualizaciones firmware del hardware de almacenamiento enterprise son muy seguras y rara vez fallan hasta tal punto de dejar el dispositivo offline (siempre y cuando se hayan seguido las instrucciones dadas en esta guía, recordar que si a la librería se le da orden de efectuar cualquier acción de backup/restore mientras se está actualizando la librería, es muy posible que después de actualizarse se reinicie y nos cause una avería HW, perdiendo uno o varios drives y cintas, así que tener especial CUIDADO!).

-Para que no exista afectación alguna a los sistemas productivos siempre se aconseja realizar las intervenciones fuera de horas de afectación al negocio (te ahorrarás sorpresas, dolores de cabeza, y trabajarás de manera mas profesional).-

 

HowTo: Install a Fibre Switch (Brocade) in 12 to 15 steps.

Instructions to install a Brocade switch (in our days most of SAN fibre switches are Brocade: IBM, EMC, DELL, FUJITSU, HP, HITACHI, NETAPP, SUN, and ORACLE use OEMized Brocade Switches, and Brocade also sells switches directly, other manufacturers of fibre switches are QLogic and CISCO).

We assume that you have unboxed the switch, put in the rails, mounted it on the rack, populated the FC ports with the SFPs that came in the box, and put the cable or cables and switched it on.

To do the Initial configuration of the switch, We will need the following:

  • A laptop with windows, where we will install the EzSwitchSetup software that comes in a CD with the switch.
  • An Ethernet (normal – no crossover needed) cable to plug directly to the switch on the Management port (We can also use the Serial cable that comes included with some switches, but it’s a lot easier with a CAT5 cable).
  • And now to sit down for a minute and think: prepare/collect the following information to use on the initial switch config:

MAC WWN: _____________________
Hostname: _____________________
IP: _____________________
Mask: _____________________
Gateway: _____________________
NTP Server: _____________________
root passwd: _____________________
factory passwd: _____________________
admin passwd: _____________________
user passwd: _____________________

Once we have this details and prerequisites, we can carry on with the initial config:

SAN_SW1

1.- We get our laptop and connect it directly to the ethernet management port of our first switch.

2.- Start EzSwitchSetup software

3.- Select the first part of the switch’s MAC address from the drop down list

4.- Complete the MAC address withthe last 3 bytes of our model (it will be printed on a sticker on the switch, or on a plastic tongue on the front left side).

5.- Once our switch is identified, we can proceed to change it’s IP address (the switch will show as default values):

IP: 10.77.77.77
MASK: 255.255.255.0
GTW: 0.0.0.0
IP6:

6.- Once the switch has been successfully changed to it’s final IP, we will unplug our laptop, and connect the switch to it`s final network connection.

7.- Now connect remotely to the switch as admin, and when presented with the message:

Please change passwords for switch default accounts now.
Use Control-C to exit or press 'Enter' key to proceed.

Warning: Access to the Root and Factory accounts may be required for
proper support of the switch. Please ensure the Root and Factory
passwords are documented in a secure location. Recovery of a lost Root
or Factory password will result in fabric downtime.

for user - root
Changing password for root
Enter new password:
Re-type new password:
passwd: all authentication tokens updated successfully
Please change passwords for switch default accounts now.
for user - factory
Changing password for factory
Enter new password:
Re-type new password:
passwd: all authentication tokens updated successfully
Please change passwords for switch default accounts now.
for user - admin
Changing password for admin
Enter new password:
Re-type new password:
passwd: all authentication tokens updated successfully
Please change passwords for switch default accounts now.
for user - user
Changing password for user
Enter new password:
Re-type new password:
passwd: all authentication tokens updated successfully
Saving passwords to stable storage.
Passwords saved to stable storage successfully

8.- Change the default 10 minute timeout value to 60 minutes:

timeout 60

9.- Configure our switch hostname with the command:

switchname $HOSTNAME

10.- Configure the timezone:

tstimezone --interactive

11.- Configure the time and the NTP server:

date "0213101215"
Fri Feb 13 10:12:59 CET 2015

Note: Date and time are specified as a string in the format: “mmddhhmmyy” (month day hour minutes year).

tsClockServer "$NTP_IP"
Updating Clock Server configuration...done.
Updated with the NTP servers

12.- Wait 5 minutes and verify the time has adjusted accordingly:

date
Fri Feb 13 10:12:59 CET 2015

13- Reboot (Note: this step is not really needed, I do it to verify the boot process and that settings are applied correctly):

reboot

14.- Extra step: If you want to be thorough, you can Check the SFPs are correct with the provided FC Loopback.

For that you need to plug a loopback in a SFP port and issue the command:

switchshow

You can change the loopback to another port and issue switchshow again, until we have tried all the SFPs. As usually you will setup switches in pairs, you will have 2 loopbacks, wich makes the check quicker.

15.- As a Final Check, issue the command:

switchstatusshow

and verify that all the hardware sensors and that the switchstate are all in HEALTHY status, and none of them are in MARGINAL.

 

And that’s it, the switch is now configured, and ready to plug the fibres to the SAN and the Servers, then the zoning will be needed, and maybe extend the fabric to another switch… but that’s material for another post… Thanx for reading!

 

HowTo: detect and connect to Oracle / Sun Microsytems ILOMs (Integrated Lights Out Manager)

Prereqs:

· A laptop PC (with windows)
· NetScan from Softperfect (it’s free for personal & professional use)
· An ethernet cable (ie: cat5-6, normal cable -no crossover needed-)

Steps:

1.- If you haven’t done it already, go to Softperfects web and download the latest version of NetScanner.
Why I like this program? well, it’s free, it’s portable (I love portables BTW), it’s one of the best scanners that I have seen, and it’s free (did I say that already?).

2.- Conect your laptop to the “Net Mgmt” ethernet port of your server, and change your laptop’s local IP address to 192.160.1.2, with a gateway of 192.168.1.1.

3.- Open NetScan, let it identify your NICs IP network, and start the scan.

ILOM_detect_01

4.- As soon as you have 2 entries displayed on NetScan, you can save time and pause the scan process.

ILOM_detect_02

5.- Select the last IP shown, and right click, open as Secure HTTP (HTTPS).

ILOM_detect_03

6.- Voilá, You may get a certificate error, ignore that, and you will be able to login;

ILOM_detect_04

a browser windows should have opened with the ILOM login’s page.

ILOM_detect_05
remember that the default user/password to access is root/changeme.

7.- Now you can do what you wanted to do in the ILOM, log off, and go for another server…

ILOM_detect_06

Oracle Hardware – How to replace a Blade from a Sun Blade 8000 chassis.

Today I will do a post with some nice photos of blades hardware ;o)

Well, on this post we’re going to describe the process of replacing a faulty blade, for a good one.

Prerrequisites:

-To have the replacement blade, equal to the one that we are replacing, would be a good start.

Blade X8400_5

To replace a Blade, we basically need to follow this steps:

1.- Power off blade,

2.- Remove the blade from the chassis, -the chassis without blade will look like this:

Sun Blade 8000 Chassis

3.- Take off the blade’s cover,

4.- Remove and set aside the HDs of the blade (better leave them in top of an anti-static mat)

5.- Remove and set aside the RAM SIMMs of the blade (and leave them in the anti-static mat)

6.- Remove and set aside the CPUs Heat Dissipators of the blade (and leave them in the anti-static mat)

7.- Clean the CPUs and the Heat Dissipators of the heat conducting paste with a special cleaning cloth
(you can buy them on and IT shop, next to the heat conducting paste)

Sun Blade 8000 Chassis_1New

8.- Remove and set aside the CPUs of the blade (and leave them in the anti-static mat)

9.- Prepare the new blade, and take off it’s cover,

10.- Insert the CPUs from the old blade,

11.- Apply new heat conducting paste to each CPU (with the syringe: I usually put a big dot in the center, but if you really want to get into it, there are lots of pages on the internet about how to corrrectly apply heat dissipator paste to a CPU),

12.- Put the CPU Heat Dissipators (careful with the pressure applied to the screws, you don’t want one of this broken – in fact, I always screw them crossed like the car tires: if we have screws 1,2,3,4, I will half screw 1, 3, 2, 4, and then finish off 1,3,2,4 again, that ensures even pressure has been applied to the heat dissipator),

13.- Put the RAM DIMMs,

14.- Put the HDs,

Sun Blade 8000 Chassis_3New

15.- Put the cover, and reseat the blade on the chassis,

Sun Blade 8000 Chassis_6End

16.- Power On, and check the lights for green ( if an orange light appears, look at the ILOM, you may have a problem ;o(  )

The old blade will look all empty and sad on one side:

Sun Blade 8000 Chassis_4Old

This instructions are from a Sun Blade 8000, but they are pretty simmilar for an IBM BladeCenter blade.

In any case, this procedure is usually performed by an Oracle Field Service Engineer (FSE), as a blade or motherboard is a Field Replaceable Unit (FRU), not a Customer Replaceable Unit (CRU); but it may be of use to those who have a blade at home, or to the inquisitive type sysadmins.

And that’s about the size of it,

thanks for reading.

 

Create a free website or blog at WordPress.com.

Up ↑