Versioned WPAR on AIX 7.1

Hello everyone today i am going to share knowledge on how we can create versioned WPAR on AIX 7.1.

For creating versioned WPAR you need to pay some cost to IBM for versioned WPAR fileset .Version WPAR are supported on AIX 7.1 only.

So question is why we need version WPAR ??

Generally versioned WPAR is AIX 5.3 and AIX 5.2 instance running on AIX 7.1 .When client application or database having operating system version dependency then version WPAR come in picture.

 

Minimum OS requirement for creating versioned WPAR 

Global LPAR OS version :AIX 7.1 TL01 SP05 recommended .

Minimum AIX 7.1 TL01 SP04.

WPAR extention OS version :AIX 5.3 TL12 SP 05(minimum) and AIX 5.2 TL10 SP08(minimum).

Lets see step by step procedure how to create versioned WPAR on AIX 7.1 TL03 SP04.we going to create AIX 5.3 versioned WPAR on AIX 7.1

Hardware : IBM POWER8

Step 1:

First create fresh MKSYSB backup of  client LPAR OS (AIX 5.3 TL12 SP05).

Step 2: Create target AIX 7.1 TL03 SP04 LPAR and then install following licensed fileset on that GLOBAL LPAR.

licensed fileset for versioned WPAR.

vwpar.images.53            1.1.2.0  COMMITTED  Versioned WPAR 5.3 Support

vwpar.images.base          1.1.2.0  COMMITTED  Versioned WPAR Base Support

vwpar.sysmgt               1.1.2.0  COMMITTED  Versioned WPAR Management

bos.wpars                 7.1.3.30  COMMITTED  AIX Workload Partition

copy required versioned wpar fileset(IBM Licensed fileset)  on /tmp/wpar_5.3 path,which you purchased from IBM for creating versioned WPAR.

Step 3:Then

#cd  /tmp/wpar_5.3

#inutoc .

use smit install to install all fileset in path for supporting versioned WPAR of AIX 5.3.

Step 4:

After versioned wpar fileset installation completed check installed fileset using command.

#lslpp -l |grep -i wpar

Step 5:

Copy AIX 5.3  MKSYSB using NFS share to GLOBAL LPAR in file system /mksysb_AIX_LPAR/image/ path.

/mksysb_AIX_LPAR/image/lparb.mksysb

Step 6:

Assign 70 GB SAN disk on Global AIX 7.1 LPAR for WPAR install.

cd to path /mksysb_AIX_LPAR/image

Execute mkwpar command with following flag

# mkwpar -D rootvg=yes devname=hdisk1 -n wparb -h wparb -N interface=en0 netmask=255.255.255.0 address=XX.XX.XX.XX -r -C -B lparb.mksysb –l

Flag 

D  configure device from global LPAR to WPAR when system start

devname  device name to allocate to the versioned WPAR

rootvg=yes   use specified device as rootvg.

h   hostname

n   wpar name

N  interface name for IP configuration

r  Duplicate network configuration file from global LPAR

C Create versioned WPAR

B workload partition backup image (mksysb image)

l   Creates private and writable versions of the /usr and /opt file systems

Above command will take around 20-25 minutes

Once command completes it will show versioned wpar has been created successfully.

  

Step 7:Check versioned WPAR is active or not by following command

Following are the state of WPAR.

A  Active

D defined

Step 8:

#startwpar  wparb      

above command will start WPARB.for accessing wparb it must be in ACTIVE state.

Once wpar started login to WPAR using  command

#clogin  wparb

It will show AIX 5.3 versioned WPAR console

Thanks !!!

Fork function failed on AIX versioned WPAR.

Hello everyone today i am going to share knowledge about the error "fork function failed".
I faced this issue on AIX 5.3  versioned WPAR on AIX 7.1 TL03 SP4.

fork function - "it create new process"

On IBM POWER8 we had created versioned WPAR of AIX 5.3.so here you must be thinking that why this guy didn't install directly AIX 5.3 on POWER8 BOX ??
so answer to this query is ,we cant directly install AIX 5.3 on POWER8,so first we need to install AIX 7.1 TL03 and SP4 and on that need to create AIX 5.3 versioned WPAR by using AIX 5.3 MKSYSB image,also because of database dependency ,we not able to install latest version of AIX on POWER8.

Let me answer why we created Versioned WPAR 5.3 on AIX 7.1
1st reason is that we cant directly install 5.3 on POWER 8
2nd ,we need IBM support for versioned WPAR in case of any critical issue and IBM support AIX 5.3 Versioned WPAR on AIX 7.1.

Overall setup was like below.

BASE OS
AIX 7.1 TL03 SP4.
on top of this we created AIX 5.3 versioned WPAR which hosting oracle 9i database for critical application.After restoration of database on AIX versioned WPAR oracle team started database and
after 30-40 minutes ,oracle team was complaining ,when they trying to login oracle account they are getting error :

"The fork function failed. Too many processes already exist"

after doing analysis of this error we increased memory of AIX LPAR and again asked DBA team to start database again and check whether error is sorted out or not.but no success getting same error again.
so what next ??

from OS end our thinking was like if we restore OS MKSYSB then no need to change any OS tuning parameter,but when we started checking parameter for "MAXUPROC" ,we found culprit which was causing this error."MAXUPROC" wasn't correctly set on AIX global LPAR and versioned WPAR  and which causing fork function failed error.on source AIX  server it was 2048 and on target it was 128 on both BASE OS and versioned WPAR.

MAXUPROC = maximum number of oracle/application process a user can able run on AIX OS.
so because of wrong setting at target after migration database throwing error of reaching maximum limit 128 process on versioned WPAR.

our solution to this was like ,first change maxuproc on AIX 7.1 GLOBAL LPAR and then do changes on AIX 5.3 versioned WPAR. Here note one thing that if AIX admin didn't set maxuproc for WPAR then non-root user inherits maximum process limit from global AIX 7.1 LPAR.

Command to change "MAXUPROC"

# chdev -l sys0 -a maxuproc=2048 
and then reboot AIX 7.1 global LPAR
and after reboot change "totalProcessess" on wpar by command,
#chwpar -R totalProcessess=2048 wparname.
then stopwpar and startwpar once .

after setting correct MAXUPROC limit we sorted out fork function failed error .

at last we can say that maxuproc limit changes according  to database and application requirement,so 
its always wise decision to refer database and application tuning document for performance related issue.

Thanks !!!!

Passwordless ssh configuration for ORACLE RAC

                                         

While doing oracle RAC configuration on AIX 6.1 TL 08 ,oracle RAC admin  team had requested some prerequisite and one of them is to configure password less configuration between 2 user of oracle RAC. These 2 user exist on 2 different AIX LPAR.

Here is the scenario,

There are 2 AIX LPAR, where oracle RAC DBA going to configure ORACLE RAC cluster.

Hostname are 

1.aixorarac1

2.aixorarac2

and  user  is "racora" on both AIX node.AIX OS team going to configure password less ssh .

you must be wondered why need password less configuration for oracle RAC ???

Answer to this query is ,it is one of the mandatory prerequisite before doing oracle RAC configuration on AIX or Linux server.

While doing oracle RAC configuration oracle RAC admin stuck with error,password less connection not working for user "racora" .After this he contacted our team and request to sort out this issue.

we immediately copied public key of user "racora" from  aixorarac1 to aixorarac2 in .ssh path of user "racora" on 2^nd node.when again installation started for oracle RAC again getting error that password less connection from aixorarac1 to aixorarac2 is ok but from aixorarac2 to aixorarac1 is not correct.

So here we got that we need to configure both way password less communication for oracle RAC user "racora".

Lets see how to configure both way passoword less communication for "racora" user on 2 AIX node which going to host oracle RAC cluster.

Passwordless ssh configuration for oeacle node aixorarac1 and aixorarac2

=============================================================

Steps :

Login to aixorarac1 using user "racora" and execute following command,

aixorarac1#ssh-keygen -t rsa (execute same command on aixorarac2)  //perform this step on both node then proceed for next part.

aixorarac1#cd .ssh (.ssh permission must be 700)

aixorarac1#cat id_rsa.pub >> authorized_keys

aixorarac1#scp authorized_keys aixorarac2:/home/racora/.ssh

============================================================

Now switch to aixorarac2

steps:

aixorarac2#cd .ssh (.ssh permission must be 700)

aixorarac2#cat id_rsa.pub >> authorized_keys

aixorarac2#scp authorized_keys aixorarac1:/home/racora/.ssh

Here what we done is ,First generated rsa key on both node,

Then added id_rsa.pub to authorized_keys file on aixorarac1.

Then SCP authorized_keys file to aixorarac2 home directory "aixorarac2:/home/racora/.ssh."

Then on node aixorarac2 added public key to authorized_keys by command cat id_rsa.pub >> authorized_keys and scp that file to aixorarac1.

So in this way on aixorarac2 home directory(aixorarac2:/home/racora/.ssh) we having authorized_keys file which contain both AIX node public key and same file we copied to “aixorarac1.

after this we simply tested whether passwordless connection is working or not from both end by following command,

aixorarac1# ssh aixorarac2 date

Fri Jun 29 11:11:31 UTC 2018

================================

aixorarac2#ssh aixorarac1 date

Fri Jun 29 11:11:50 UTC 2018

After executing above both command on oracle RAC cluster output  must show date without asking for password.

That’s all  ,Needed to configure password less connection between 2 oracle RAC node on AIX .

Thanks !!!

Recover niminfo on AIX NIM MASTER

How to recover niminfo file on NIM master in AIX environment ??

What if anyone from your admin team by mistake remove /etc/niminfo on NIM MASTER .how will you recover that file ?what will be your approach on this ??

Lets see first , what is /etc/niminfo  on AIX NIM MASTER ??

 why it is so important ??

Answer to this query is ,it is NIM configuration file which contain nim configuration information and it’s is edited by only NIM MASTER only.it contain information like NIM MASTER name,NIM MASTER port like 1058(nim) and 1059(nimreg).1059 register client on NIM MASTER and 1058 is used by process

“nimesis”,which is responsible for accepting connection from NIM client and respond to them. nimesis always run on NIM MASTER.

Also this file contain BOS image path on NIM MASTER, apart from this NIMclient route information and host IP address.

Here is the short answer for recovering niminfo using SMIT.

1.Using smitty AIX admin can recover file by following below steps.

1.     SMIT

2.     Perform NIM Administration Tasks

3.     Rebuild the niminfo File on the Master

2.Command line option is like below.

Rebuild nimnfo on MASTER server by using command

#nimconfig -r

Thanks !!!

Change Timezone on SUSE Linux

While doing administration Linux admin came across situation where need is to change time zone.

This situation arise because of following reason

1. Application/database requirement for time-zone change.

2. Wrongly configured Time zone by admin.

So here procedure for changing time zone.

You must be thinking why i am changing time zone, in my case while doing new installation for one of the application server i forgot to change time zone.

Let’s see ...........

While changing  time zone for server please do proper backup of setting which you are changing.

Example. Time zone setting before changing.in case of reverting back this will help you.

So generally all time zone files are in path /usr/share/zoninfo/.

Here we need to change  timezone from UTC to Pacific Time.

Step 1. Change directory to /etc

Execute command #date before changing time zone file.

# date

Mon Sep 17 22:59:24 UTC 2017

here we can see that time zone is UTC.

Step 2: Change directory to cd /etc and remove file “localtime”

#rm localtime

Step 3:

To change time zone from UTC to Pacific timezone execute following command

# cd /etc

# ln -s /usr/share/zoneinfo/US/Pacific localtime

To confirm changes again execute date command.

# date

Mon Sep 17 23:20:14 PDT 2017

Now here we changed timezone from UTC to PDT.

Thanks !!!!

How to take MKSYSB of AIX LPAR from NIM MASTER

How to take MKSYSB of AIX LPAR from NIM MASTER

During my struggling days for job when I was appeared for interview, interviewer asked me how to take backup from NIM master, after hearing I was completely blank. After this incident I collected all knowledge from my friend and team-mate for how to take backup from NIM MASTER. So this blog is for those who want to know how to take backup of AIX LPAR from NIM MASTER.

Generally command for taking AIX LPAR backup from locally is like below.

#mksysb –iX /tmp/aixbackup/hostname_backup

But what will procedure if AIX admin want to take backup from NIM MASTER.

Step by step procedure to take MKSYSB backup from NIM MASTER on AIX LPAR.

1. On NIM MASTER enter AIX LPAR hostname/IP entry in /etc/hosts file.

2. Enable rshd on AIX LPAR by following command.

# Chsubserver -a -v shell -p tcp6 -r inetd

          # refresh –s inetd

          # cd  /

          # rm .rhosts

          # vi .rhosts

                +

Here if AIX admin want to take backup of multiple AIX LPAR then simply add + in .rhosts file.

If admin want to take selective AIX LAPR backup then only add hostname of these LPAR in .rhosts file.

          # chmod 600 .rhosts

3. Copy /etc/niminfo file from NIM MASTER to client /etc location

4. Execute following command to take backup of AIX LPAR from NIM MASTER.

# nim -o define -t mksysb -a server=master \

-a location=/backup/AIXLpar -a mk_image=yes \

-a source=AIX_Lpar_name  mksysb_AIX71TL04

Flag

-t                      Type of resource here it is MKSYSB.

Server             Specify “master”.

Location         Path on NIM master where AIX_LPAR image will be stored.

source                   Name of AIX client (hostname of AIX LPAR)

Mksysb_AIX71TL04        MKSYSB resource name

5. Create spot from existing MKSYSB resource Mksysb_AIX71TL04.

#nim -o define -t spot -a source= Mksysb_AIX71TL04 -a server=master -a location=/export/spot  spot_ AIX71TL04

By using MKSYSB resource and SPOT AIX admin can restore AIX client to same AIX level.

Thanks!!!