How to check and change hdisk attributes on AIX LPAR using script.

Today I am sharing script to change disk attributes using script.

precheck

Create directory with name aix_automation

#mkdir aix_automation

#cd aix_automation

Inside this directory create file with name list_hdisk

#touch list_hdisk

#vi list_hdisk

hdisk10

hdisk11

hdisk12

hdisk13

hdisk14

Add hdisk name in this file and save it .

#for i in 'cat list_hdisk'

do

echo $i

chdev -l $i -a algorithm=round_robin -a reserve_policy=no_reserve

sleep 2

lsattr -El $i |egrep -i 'algorithm | reserve_policy'

done

Using this simple script AIX admin can change and display multiple attributes and display.

Thanks !!!

How to check and change VIO shared ethernet adapter status,ha_mode using script

Use following scripts for shared ethernet adapter failover, status and priority check

Let's start

1.

Check ha_mode on multiple shared ethernet adapter in VIO .

for i in `lsdev -Cc adapter | grep -i shared |awk '{print $1}'`

do

echo "-------shared SEA $i  ha mode------ "

lsattr -El $i |grep -i ha_mode

sleep 2

done

≠==≠=========

Output will be:

All shared ethernet adapter in VIO and their ha_mode

2.Script to display shared ethernet adapter "state, priority and active status.

for i in `lsdev -Cc adapter | grep -i shared |awk '{print $1}'`

do

echo "-------shared SEA $i  state priority and active status ------ "

entstat -d $i | grep -E  "^ *state|priority.*Active"

echo "---------------------"

sleep 2

done

3.Script to change shared ethernet adapter "ha_mode" . This script useful for failover and failback.

If Unix admin execute following script on vio server on which shared ethernet adapter is primary it will become backup.

for i in `lsdev -Cc adapter | grep -i shared |awk '{print $1}'`

do

echo "-------shared SEA $i  failover to partner VIO------ "

chdev -l $i -a ha_mode=standby

echo "---------------------"

sleep 2

done

4.If Unix admin execute following script on primary VIO where shared ethernet adapter already fail overed to its partner VIO because of some incident and admin wanted to failback from parter VIO or secondary VIO to primary VIO .

for i in `lsdev -Cc adapter | grep -i shared |awk '{print $1}'`

do

echo "-------shared SEA $i  failback to primary VIO------ "

chdev -l $i -a ha_mode=auto

echo "---------------------"

sleep 2

done

Thanks!!!!

How to gzip thousands of files using for loop

UNIX admin may get task like housekeep Unix filesystem.

 In such situations he may housekeep existing files in the file system or increase file system. Few day back I got such ticket but there was no scope for increasing file system size, so decided to find which files occupying space on that file system.

Finally found more than  2000 files, now question is how to gzip those??

Solution:

Step 1 created directory named like following,

#mkdir /tmp/housekeep

Then created file "listfiles" using touch command and redirected all files inside that file. Cross check done whether it contain correct files only , which we are going to gzip.

Step 2

#touch listfiles

#vi listfiles // add all files which need to be gzip

Step 3

Use following for loop

for i in `cat listfiles`

do

gzip $i

done

UNIX admin can check whether gzip happening or not by executing a following command  in target directory

#ls -lrt *.gz |wc -l

Or by simply take another session of server and execute while loop with above command

While true

do

ls -lrt *.gz |wc -l

sleep 1

done

**Before doing gzip make sure that it will not impact any production application or database.

Usefull command to find files modified in last hours

Print files which are modified in last 24 hrs

#find /var -mtime -1 -print

Thanks !!

Move logical partition using “migratelp” command to preserve cross site mirroring

 

Lets first understand what is cross site mirroring and why it necessary??

Answer: In cross site mirroring disks are coming from two different site. If on AIX LPAR there is logical volume “backuplv” which having 2 disk hdisk0 and hdisk1. Because of some disaster at “Disk-Mirror site 1” we lost hdisk0 availability, then we can access same data which is present on hdisk0 from other site disk “hdisk1”.

 

Disk-Mirror site 1- hdisk0, hdisk2

Disk-Mirro-site 2- hdisk1

Each site have unique identifier. AIX admin can identify Lun/disk from correct site by engaging storage admin and providing them your requirement that you need disk from cross site for file system.

 

Generally production environment have cross site mirroring, for development and test environment filesystem have only single copy of logical partition.

 

Here I am going to discuss 1 scenario where, we need had issue for “cross-site” mirroring. This incident was occurred because of admin mistake.

 

Issue description:

“backuplv”  data copy 1 and data copy 2 pointing to same site disk. To solve this issue admin need to use “migratelp” .

 

LV name : backuplv

File system name : /backup

PV- hdisk0 hdisk1 hdisk 2 

VG – backupvg

#lslv  -m  backuplv

LP    PP1      PV1               PP2  PV2               PP3  PV3

0001   0001    hdisk0          0010   hdisk1

0002   0002    hdisk0          0011   hdisk1         

0003   0003    hdisk0          0012   hdisk2                

0004   0004    hdisk0          0013   hdisk2

 

From above output we can observe that, LP 03 and 04 data copy 1 and 2 belongs to Disk-Mirror site 1- hdisk0, hdisk2. Here we need to move these LP 2nd copy to hdisk1.

This will preserve cross site mirroring. AIX admin can move LP 03 and 04  2nd copy to free PP's on hdisk1 using following command.

 

#Migratelp backuplv/03/2 hdisk1/12  

-        Move 2^nd mirror copy of LP 03 of logical volume backuplv to hdisk1 PP 12.

#Migratelp backuplv/04/2 hdisk1/13  

-        Move 2^nd mirror copy of LP 04 of logical volume backuplv to hdisk1 PP 13.

 

After migratelp output of #lslv -m backuplv

 

Output:

 

#lslv  -m  backuplv

LP    PP1      PV1               PP2  PV2               PP3  PV3

0001   0001    hdisk0          0010   hdisk1

0002   0002    hdisk0     0011   hdisk1                 

0003   0003    hdisk0     0012   hdisk1                          

0004   0004    hdisk0     0013   hdisk1

 

Here we have moved 2^nd copy of backuplv to correct site, to preserve cross site mirroring.

 

So if in future any issue while accessing Disk-Mirror site 1, then same data will be accessible from other site disk hdisk1. If cross site mirroring not preserved in production environment it will cause application and database outage, if that file system having critical data on it.

 

 

 

 

Why PP from same disk mirror site got allocated to “backuplv”

 

Answer:

 

Some time admin just use “chfs” command to increase file system size, without extending LP from Disk-Mirror site 1 and Disk-Mirror site 2.

 

 

 

 

Good practice :

 

Identify disk from Disk-Mirror site 1 and Disk-Mirror site 2

Select those disk (hdisk0 and hdisk1)

Extend desired LP for backuplv

Increase file system size.

 

 

 

Thanks  !!!!

 

 

 

 

 

 

 

 

 

 

 

 

                   

 

How to use screen command in Linux

--> To open a new screen:

#screen -S "Screenname"
Eg.: 

#screen -S "testscreen"

--> To deattach the screen:

Cntrl+A then D   //it will run after detach also
--->    Ctrl-a followed by k: close the current windows (kill)

--> To list the screen:

screen -list

--> To reattach the screen:

screen -r ScreenID------- From screen -list



Example:

# screen -S "testscreen"   // create screen name testscreen
Agent pid 21981

Now we are running small test script in "testscreen".

#/test.sh >>data.txt &  //ran script test.sh to append hello and date/time in data.txt
[1] 4917

Cntrl+A then D   //it will detach

[detached]
# screen -list   //list currently running screen
There is a screen on:
        3015.testscreen (Detached)
1 Socket in /var/run/uscreens/S-alhatul.

Reattch screen using following command
screen -r ScreenID
#screen -r 3015   //reattach screen
Agent pid 21981

output of screen command

Output of test.sh
===data.txt ======

#cat data.txt
hello
Fri Aug 30 15:21:00 CEST 2019
hello
Fri Aug 30 15:21:02 CEST 2019
hello
Fri Aug 30 15:21:04 CEST 2019
hello

Ctrl-a followed by k:
It will terminate current screen session.

#screen -list
No Sockets found in /var/run/uscreens/S-alhatul.

Reduce XFS filesystem on Linux

Hello friend ,

today i am going to discuss on XFS file system Limitation and that limitation is Linux admin cant reduce XFS file system online. after knowing this limitation i am still thinking about what is that preventing XFS shrink ? also is there any alternative way we can reduce size of XFS file system.

Yes, we can reduce size of XFS , but for this downtime needed.

Step 1: Backup existing file system backup using "xfsdump" 

Step 2: Then umount XFS file system.

Step 3: Then remove LV .

Step 4: Create required size LV.

Step 5: Mount File system.

But here Linux admin require downtime.

.........

Copy pedbg zip file using SCP from HMC to NIM server.

How to copy pedbg file after pedbg file created on HMC

Pedbg collected using "hscpe" user.

#pedbg -r 

Above command will remove previous data

Now collect fresh pedbg logs using following command

#pedbg -q 4 -c

This will generate file in /dump with name like /dump/hmclogs.hmcabc.zip

Copy thes to remote NIM or AIX Lpar using SCP command.

Login to NIM , create folder in /TMP like "pedbglogs"

#cd /tmp

#mkdir pedbglogs

#cd pedbglogs

# scp hscpe@hmcIP:/dump/hmclogs.hmcabc.zip  .

Here . Means current directory on NIM server. (/tmp/pedbglogs)

** Make sure /tmp have enough space to hold this pedbg file.

Thanks !!!!!

AIX lpar not connecting ,ssh process issue troubleshoot

Got incident for AIX LPAR not connecting from outside using ssh.

To solve this issue followed following approach.

First did pre-check like tried to connect from AIX NIM server and CyberArk, so found that its not connecting from both NIM and CyberArk.

so finally decided to try from putty, that also failed.

so, decided to take console from HMC and took session. upon checking found that "ssh" process is in inoperative state. so tried to start that process but its not starting using following command. :(

#startstc -s sshd 

#refresh -s sshd

so decided to check which process is holding that port 22. checked this using lsof command

#lsof -i :22

this command showing process id where socket status is "closed_wait",

so decided that will go for kill that process.

killed that process by command #kill -9 PID

and started "sshd" process and finally it work.

AIX lpar we able access from cyberARK ,putty and NIM.

Finally we conclude that, ssh process was in "hung" state and sshd process refresh and stop, start also not worked, then finally after clearing PID associated with that process solve issue.

Thanks !!!!

AIX HACMP ,GPFS and veritas Cluster state check command

 AIX HACMP ,GPFS and veritas Cluster state check command

AIX HACMP resource group state check command

#clRGinfo

How to check cluster is in  stable state or not

#lssrc -ls clstrmgrES |grep -i state

if output is showing "ST_STABLE" ,then cluster is in stable state .

#lssrc -g cluster   // Show process status in cluster group

GPFS cluster information check command

#mmgetstate -aLs

following command will show information about gpfs cluster

#mmlscluster

Veritas Cluster state check command

#hastatus -summ

Thanks !!!!!

How to replace faulty disk on AIX VIO rootvg using DIAG HOT plug task

 How to replace faulty disk on AIX VIO rootvg using DIAG HOT plug task??

AIX admin can replace faulty hard drive using

"DIAG" procedure. here faulty disk must support 

hot swap operation.

Before replacement we need to do some pre-check,

1. IS this part of rootvg and its in mirror ?

Ans : if yes then ,identify that disk and unmirror 

that disk from rootvg using following command.

#umirrovg rootvg hdisk1

#reducevg rootvg hdisk1

2. Identify disk location using lscfg command

#lscfg -vpl hdisk1

3. make sure that disk is removed from rootvg.

Once all precheck done we can identify hdisk1 by following procdure,

PART I

DIAG--->Certify media Task---->Hot plug task--->scsi and scsi RAID hot plug RAID manager----->Identify disk

Select hdisk1 and make sure location and disk is current and press enter key.

Example . 

XXX.XXX.XX.P2-C9-D5    //disk location

once "Enter key pressed" LED will blink, which set disk to identify mode.

once disk identified, you can exit from this menu by pressing "enter" and come to previous menu by "ESC +3"

PART II

Next is Remove and replace hdisk1

===============================

DIAG--->Certify media Task---->Hot plug task--->scsi and scsi RAID hot plug RAID manager----->Identify disk-->Replace / Remove a Device Attached to An SCSI Hot Swap Enclosure Device

select hdisk1 and make sure location and disk is current and press enter key.

XXX.XXX.XX.P2-C9-D5 

now disk is ready for remove and replace.

now you can ask remote engineer to perform disk replacement.

PART III

Once disk replaced you need to detect that disk on AIX LPAR.

follow below procedure it will configure newly replaced disk on LPAR.

DIAG--->Certify media Task---->Hot plug task--->scsi and scsi RAID hot plug RAID manager-->Configure Added/Replaced Devices

identify new disk by lspv and lscfg command 

#lscfg -vpl hdisk    // it will have new serial number.

Thanks !!!!