Ads 468x60px

Pages

Subscribe:

Featured Posts

Friday, November 22, 2013

 Creating a simple SMF service in solaris 11.1 using svcbundle



Solaris 10 introduced SMF but not all services are SMF managable , SMF services give the simple way of configuring or managing a service .

It becomes our requirement sometimes to create a SMF manageable service in case we need a startup script executed on bootup like setting ndd parameters . SMF services use a  manifest a xml files that has all the configurations settings for that particular service , this xml file needs to be created before defining a SMF service



In solaris 10 to create a SMF service you need to manually edit and define an xml file that could be tiresome and may need some xml knowledge . With imporved Solaris 11.1 we have  svcbundle utility which gives us easy way to create SMF manifest and profiles .



This post gives a simple procedure to create a transient service (service that doesnt need a stop script) to accomplish setting ndd parameters for /dev/ip.



Start with writing a simple shell script that sets ndd parameters when executed . Here I am a two line script to set igmp version.(Currently this cannot be set with ipadm set-prop)



Step 1:

root@sol11u1:~# cat /lib/svc/method/ndd-igmp.sh
#!/usr/sbin/sh
#
ndd -set /dev/ip igmp_max_version 1
ndd -set /dev/ip mld_max_version 1



Step:2

#svcbundle -o net-tune.xml -s service-name=network/net-tune -s start-method=/lib/svc/method/ndd-igmp.sh



Option -o  the output xml file name and properties for the servcie can be defined with multiple -s options like above service-name name of the service as displayed with svcs -a and start-method that give the script name with complete path . (I prefer to create script under /lib/svc/method/ as all the scripts defined with default SMF services are here). By default if you dont mention -s model which defines the serivce type(transient,deamon etc..) , svcbundle considers it a transient service .



Here is the xml file created after executing the svcbundle .

root@sol11u1:/lib/svc/method# more net-tune.xml

<?xml version="1.0" ?>

<!DOCTYPE service_bundle

SYSTEM '/usr/share/lib/xml/dtd/service_bundle.dtd.1'>

<!--

Manifest created by svcbundle (2013-Nov-20 07:17:48+0530)

-->

<service_bundle type="manifest" name="network/net-tune">

<service version="1" type="service" name="network/net-tune">

<!--

The following dependency keeps us from starting until the

multi-user milestone is reached.

-->

<dependency restart_on="none" type="service"

name="multi_user_dependency" grouping="require_all">

<service_fmri value="svc:/milestone/multi-user"/>

</dependency>

<exec_method timeout_seconds="60" type="method" name="start"

exec="/lib/svc/method/ndd-igmp.sh"/>

<!--

The exec attribute below can be changed to a command that SMF

should execute to stop the service. See smf_method(5) for more

details.

-->

<exec_method timeout_seconds="60" type="method" name="stop"

exec=":true"/>

<!--

The exec attribute below can be changed to a command that SMF

should execute when the service is refreshed. Services are

typically refreshed when their properties are changed in the

SMF repository. See smf_method(5) for more details. It is

common to retain the value of :true which means that SMF will

take no action when the service is refreshed. Alternatively,

you may wish to provide a method to reread the SMF repository

and act on any configuration changes.

-->

<exec_method timeout_seconds="60" type="method" name="refresh"

exec=":true"/>

<property_group type="framework" name="startd">

<propval type="astring" name="duration" value="transient"/>

</property_group>

<instance enabled="true" name="default"/>

<template>

<common_name>

<loctext xml:lang="C">

<!--

Replace this comment with a short name for the

service.

-->

</loctext>

</common_name>

<description>

<loctext xml:lang="C">

<!--

Replace this comment with a brief description of

the service

-->

</loctext>

</description>

</template>

</service>

</service_bundle>





Step 3: Copy the xml file under  /lib/svc/manifest/site/

root@sol11u1:/#   cp net-tune.xml /lib/svc/manifest/site/

root@sol11u1:/lib/svc/method# svcs net-tune
svcs: Pattern 'net-tune' doesn't match any instances
STATE          STIME    FMRI



Import the created SMF service using svcadm restrat manifest-import . This will create the service and can be seein with svcs now .



root@sol11u1:/lib/svc/method# svcadm restart manifest-import



Great to see our service with svc but that is in maintenance . Lets troubleshoot in our traditional manner starting with SMF log file



root@sol11u1:/lib/svc/method# svcs net-tune
STATE          STIME    FMRI
maintenance     7:19:28 svc:/network/net-tune:default

root@sol11u1:/lib/svc/method# svcs net-tune
STATE          STIME    FMRI
maintenance     7:19:59 svc:/network/net-tune:default
root@sol11u1:/lib/svc/method# svcs -l network/net-tune
fmri         svc:/network/net-tune:default
enabled      true
state        maintenance
next_state   none
state_time   November 20, 2013 07:19:59 AM IST
logfile      /var/svc/log/network-net-tune:default.log
restarter    svc:/system/svc/restarter:default
manifest     /lib/svc/manifest/site/net-tune.xml
dependency   require_all/none svc:/milestone/multi-user (online)



Wow we do have a log file created for our services , which clearly shows permission denied

root@sol11u1:/lib/svc/method# more /var/svc/log/network-net-tune:default.log
[ Nov 20 07:19:28 Enabled. ]
[ Nov 20 07:19:28 Executing start method ("/lib/svc/method/ndd-igmp.sh"). ]
/usr/sbin/sh[1]: exec: /lib/svc/method/ndd-igmp.sh: cannot execute [Permission denied]
[ Nov 20 07:19:28 Method "start" exited with status 126. ]
[ Nov 20 07:19:28 Executing start method ("/lib/svc/method/ndd-igmp.sh"). ]
/usr/sbin/sh[1]: exec: /lib/svc/method/ndd-igmp.sh: cannot execute [Permission denied]
[ Nov 20 07:19:28 Method "start" exited with status 126. ]
[ Nov 20 07:19:28 Executing start method ("/lib/svc/method/ndd-igmp.sh"). ]
/usr/sbin/sh[1]: exec: /lib/svc/method/ndd-igmp.sh: cannot execute [Permission denied]

[ Nov 20 07:19:28 Method "start" exited with status 126. ]





Check the permissions on the script we have create ... Always a shell script need executable permissions .



root@sol11u1:/lib/svc/method# ls -l /lib/svc/method/ndd-igmp.sh
-rw-r--r--   1 root     root          93 Nov 20 07:17 /lib/svc/method/ndd-igmp.sh



Change the permissions to have it executed .



root@sol11u1:/lib/svc/method# chmod 755 /lib/svc/method/ndd-igmp.sh



Lets see what our settings currently are before we test if our SMF is working or not . And here we have the ndd parameter  igmp_max_version set to 2 (our goal with the SMF service is to set it to 1)

root@sol11u1:/lib/svc/method# ndd /dev/ip igmp_max_version
2



As our service is in maintenance just clearing the services should make it online as we maded the script executable now .

root@sol11u1:/lib/svc/method# svcadm clear net-tune
root@sol11u1:/lib/svc/method# svcs net-tune
STATE          STIME    FMRI
online          7:21:11 svc:/network/net-tune:default



Check if our goal with SMF service is fullfilled



root@sol11u1:/lib/svc/method#  ndd /dev/ip igmp_max_version
1
root@sol11u1:/lib/svc/method#  ndd /dev/ip mld_max_version
1



That makes us success in creating a SMF service manageable with svcadm . Follow the similar process to create your own SMF service , play with the service using svcadm,svccfg etc …

Thursday, September 1, 2011

CREATING A SPARSE ROOT ZONE



Pre-requisite before creating zone :

Plan for, how your zone should and where its path on global zone .
Below are the few required values which require while creating zones.
zonepath : Is the location or path on global zone where the lofs filesystems are located . (I am using here /myzone) .And the zone path should have the permissions 700 as shown below (otherwise zone installation will fail) .

[root@solaris1 /]#ls -ld /myzone
drwx------   5 root     root        1024 Aug  3 21:18 /myzone


Inherited-pkg-dir: Are the directories which are loop back mounted from global zone. (/lib,/platform,/sbin and /usr are default dir that are loopback from global zone to local zone ).Further you can add other directories to the list also ... (As it will be shown in this post)
CREATING A SPARSE ROOT ZONE :
In this post I will be creating as sparse root zone under /myzone directory with all default inherited dir and an extra dir (/opt ) . Creating a zone is straight forward with zonecfg command .
For the first time of creting myzone a message appears similar to below . This occurs becuase myzone is not already created and not in any configuration files .
[root@solaris1 /]#zonecfg -z myzone
myzone: No such zone configured
Use 'create' to begin configuring a new zone.
zonecfg:myzone>
Using the zonecfg it will take you to a zone configuration special prompt zonecfg: from where you can configure the zone .
For the first time you need to use the 'create' to start with new zone configuration . Below output shows the initial config of a zone without any customization .
zonecfg:myzone> create
zonecfg:myzone> info
zonename: myzone
zonepath:
brand: native
autoboot: false
bootargs:
pool:
limitpriv:
scheduling-class:
ip-type: shared
inherit-pkg-dir:
        dir: /lib
inherit-pkg-dir:
        dir: /platform
inherit-pkg-dir:
        dir: /sbin

Now the very first config value will be zonepath for which I am using /myzone which is already created with 700 permission in global zone . And apart from default inherit-pkg-dir here I am adding one more dir that needs to be lofs mounted in local zone .


zonecfg:myzone> set zonepath=/myzone
zonecfg:myzone>
zonecfg:myzone> add inherit-pkg-dir
zonecfg:myzone:inherit-pkg-dir> set dir=/opt
zonecfg:myzone:inherit-pkg-dir> info
inherit-pkg-dir:
        dir: /opt

Below is the zone configuration after you add extra inherit-pkg-dir and set zonepath.

zonecfg:myzone> info
zonename: myzone
zonepath: /myzone
brand: native
autoboot: false
bootargs:
pool:
limitpriv:
scheduling-class:
ip-type: shared
inherit-pkg-dir:
        dir: /lib
inherit-pkg-dir:
        dir: /platform
inherit-pkg-dir:
        dir: /sbin
inherit-pkg-dir:
        dir: /usr
inherit-pkg-dir:
        dir: /opt



Further you can add network info to the zone  with ‘add net’ attributes required are address , which is the IP address assigned to zone and the physical, which is the interface on which a VIP is configured .


zonecfg:myzone> add net
zonecfg:myzone:net> info
net:
        address not specified
        physical not specified
        defrouter not specified
zonecfg:myzone:net> set address=192.168.15.100
zonecfg:myzone:net> set physical=e1000g0
zonecfg:myzone:net> end


Note: before you exit from the zone prompt , save this configuration with ‘commit’ because all the configuration set before commit are not permanently saved .




This completes our configuration of zones and below is the configured zone info . (You can further see the same outside the zone prompt with command  zoneinfo –z info )


zonecfg:myzone> info
zonename: myzone
zonepath: /myzone
brand: native
autoboot: false
bootargs:
pool:
limitpriv:
scheduling-class:
ip-type: shared
inherit-pkg-dir:
        dir: /lib
inherit-pkg-dir:
        dir: /platform
inherit-pkg-dir:
        dir: /sbin
inherit-pkg-dir:
        dir: /usr
inherit-pkg-dir:
        dir: /opt
net:
        address: 192.168.15.100
        physical: e1000g0
        defrouter not specified
zonecfg:myzone> commit
zonecfg:myzone> exit




Our zone is now in configured state and is ready to get installed .

[root@solaris1 /]#zoneadm list -cv
  ID NAME             STATUS     PATH                           BRAND    IP   
   0 global           running    /                              native   shared
   - myzone           configured /myzone                        native   shared

You can install the zone with zone admin command ‘zoneadm –z install’

[root@solaris1 /]#zoneadm -z myzone install
Preparing to install zone .
Creating list of files to copy from the global zone.
Copying <1969> files to the zone.
Initializing zone product registry.
Determining zone package initialization order.
Preparing to initialize <953> packages on the zone.
Initialized <953> packages on zone.                               
Zone is initialized.
Installation of <2> packages was skipped.
The file contains a log of the zone installation.



Yes now we are done with the zone installation without any problems and now the status of zone changes to installed.


[root@solaris1 /]#zoneadm list -cv
  ID NAME             STATUS     PATH                           BRAND    IP   
   0 global           running    /                              native   shared
   - myzone           installed  /myzone                        native   shared


No You are ready to boot the zone and you that with zoneadm command as shown below .

[root@solaris1 /]#zoneadm -z myzone boot

As you boot the zone status will be changed to running .

[root@solaris1 /]#zoneadm list -cv
  ID NAME             STATUS     PATH                           BRAND    IP   
   0 global           running    /                              native   shared
   1 myzone           running    /myzone                        native   shared

For the first time boot of zone just after installation , you need to login to zone console using zlogin and “-C” options , because there are few details that you need to provide for zone to operate .(Like Language info , timezone and so on . These questions are same as it show up while installing solaris OS .)

[root@solaris1 /]#zlogin -C myzone
[Connected to zone 'myzone' console]
Reading ZFS config: done.

Select a Language

  0. English
  1. Japanese
  2. Korean
  3. Simplified Chinese
  4. Traditional Chinese

Please make a choice (0 - 4), or press h or ? for help:

[NOTICE: Zone rebooting]


Above are few messages that appear in the process when you connect to console of zone .  After answering all , zone boots again and now it is complete and stable .

SunOS Release 5.10 Version Generic_137138-09 64-bit
Copyright 1983-2008 Sun Microsystems, Inc.  All rights reserved.
Use is subject to license terms.
Hostname: myzone
Reading ZFS config: done.

myzone console login:
myzone console login: ~.


Note : To exit from zone console session use the key sequence “~.“

And now zone is perfect to use .

Below are few of the outputs after zone installation .

[root@solaris1 /]#zlogin myzone
[Connected to zone 'myzone' pts/2]
Sun Microsystems Inc.   SunOS 5.10      Generic January 2005

# uname -a
SunOS myzone 5.10 Generic_137138-09 i86pc i386 i86pc

# df -h
Filesystem             size   used  avail capacity  Mounted on
/                      1.0G    97M   870M    10%    /
/dev                   1.0G    97M   870M    10%    /dev
/lib                   940M   623M   261M    71%    /lib
/opt                   1.9G   490M   1.4G    27%    /opt
/platform              940M   623M   261M    71%    /platform
/sbin                  940M   623M   261M    71%    /sbin
/usr                   2.9G   2.3G   560M    81%    /usr
proc                     0K     0K     0K     0%    /proc
ctfs                     0K     0K     0K     0%    /system/contract
mnttab                   0K     0K     0K     0%    /etc/mnttab
objfs                    0K     0K     0K     0%    /system/object
swap                   968M   288K   968M     1%    /etc/svc/volatile
/usr/lib/libc/libc_hwcap1.so.1
                       2.9G   2.3G   560M    81%    /lib/libc.so.1
fd                       0K     0K     0K     0%    /dev/fd
swap                   968M     4K   968M     1%    /tmp
swap                   968M    16K   968M     1%    /var/run

Wednesday, July 20, 2011

Online VCS configuration of Apache http application

Onilne VCS configuration

As a part of my VCS learning, I was able to configure my first Service group to make my apache http a Highly Available aplication under Veritas Cluster .

This post is meant to show the procedure I followed to configure an application . Below is the hierarchy for my

service group apacheSG .

Application that I consider here is Apache http (You can find the installation procedure for apache http in my previous post).

Before we go with the steps to configure, check the pre-requisites that are required to configure a application under VCS . Below are the required setup

for any cluster build .

1. Two servers running Solaris 10.(You can consider other versions too, but

I have tried this on solaris 10)

2. Shared storage , a disk attached to both the servers .(It is optional for the setup

in which application is installed on both the servers).

3. An IP address (This is the IP address we use to access the application configured under VCS)

In my setup I have Apache installed under /apachevol Filesystem which is created from veritas volume 'apachevol' , apachevol is created under a shared disk group apachedg .

Below are the few screen shots that shows the diskgroup and veritas volume configured .

[root@solaris1 /]#vxdisk list

DEVICE TYPE DISK GROUP STATUS

c0t0d0s2 auto:none - - online invalid

c1t0d0s2 auto:cdsdisk disk apachedg online

[root@solaris1 /]#vxprint -g apachedg

TY NAME ASSOC KSTATE LENGTH PLOFFS STATE TUTIL0 PUTIL0

dg apachedg apachedg - - - - - -

dm disk c1t0d0s2 - 338576 - - - -

v apachevol fsgen ENABLED 307200 - ACTIVE - -

pl apachevol-01 apachevol ENABLED 307200 - ACTIVE - -

sd disk-01 apachevol-01 ENABLED 307200 0 - - -

[root@solaris1 /]#df -k /apachevol

Filesystem kbytes used avail capacity Mounted on

/dev/vx/dsk/apachedg/apachevol

153600 18488 126759 13% /apachevol

Note: Apache http is installed under /apachevol .

Before we start configuring , we need to make the configuration files (main.cf) read write . This can be done with haconf command as below .

#haconf -makerw

Procedure to create a service group has the flow which starts with creating service group .

i. Creating Service Group apacheSG:

Create a Service group and modify the SystemList attribute to include the server on which the service group needs to be HA .

#hagrp -add apacheSG

#hagrp -modify apacheSG SystemList solari1 0 solaris2 1

Set the AutoStartList Attribute to define the server on which SG should be made online on VCS startup .

#hagrp -modify apacheSG AutoStartList solaris1

ii. Creating resources to be under service group apacheSG :

Now create the Resources that are to be controlled by VCS . As per the hierarchy we will be creating the resources with names apacheApp , apacheNIC , apacheIP , apacheDG, apacheVol and apacheMnt .

You can create the resources in any order there is no specific order to follow .

We use hares command to add a new resource or modify an existing resource. Below is the syntax for the same .

hares –add

hares –modify

You can find the Resource types available with command ‘hatype –list’ and the resource attributes with ‘hatype –display

Ex: hatype –display Volume (will display the attributes for volume resource type)

1. Create NIC resource apacheNIC – Resource Type NIC

NIC is the physical device and so it is a persistent resource which mean it will be online on both the servers and it cannot be made offline as this requires to configure IP .

#hares -add apacheNIC NIC apacheSG

#hares -modify apacheNIC Critical 0

#hares -modify apacheNIC Device e1000g0

2. Create IP resource apacheIP – Resource Type ‘IP’

Public IP cannot be same for two servers in a cluster and so if we need to access the application we need to know first on which server the application is active as server IP changes when application failovers to other server in cluster . So we need an IP which is moved to the server where application is online . So service group needs to control that IP address when failover is initiated . so that application can be accessed with one constant IP irrespective of , on which server it is active .

Below are the commands to configure a IP resource and bind it to the interface .

#hares -add apacheIP IP apacheSG

#hares -modify apacheIP Critical 0

#hares -modify apacheIP Device e1000g0

#hares -modify apacheIP Address "xxx.xxx.xxx.xxx"

#hares -modify apacheIP NetMask "255.255.255.0"

3. Create Diskgroup resource apacheDG – Resource Type ‘DiskGroup’

Disk group resource is required as VCS will do import or deport of DG while Service Group failover and this is pre-requisite resource for Volume and Mount point . (As the Veritas disk group has this feature to import the DG and inturn the disks under the DG). We create the DG resource by specifying ‘DiskGroup’ type . Here the diskgroup name used is apachedg

#hares -add apacheDG DiskGroup apacheSG

#hares -modify apacheDG Critical 0

#hares -modify apacheDG DiskGroup apachedg

4. Create Volume resource apacheVol – Resource Type ‘Volume’

Volume resource is the pre-requisite for Mount point as we have the filesystem created on veritas volume created from disk group . Type we specify for this resource is Volume and the volume name used is apachevol

#hares -add apacheVol Volume apacheSG

#hares -modify apacheVol Critical 0

#hares -modify apacheVol Volume apachevol

#hares -modify apacheVol DiskGroup apacheDG

Note:Their might be more than one disk group on the server, so VCS need to know from which Disk group volume comes from , for that we have the DiskGroup attribute .

5. Create MountPoint Resource apacheMnt – Resource Type ‘MountPoint’

Mount point here is the one on which the application is installed and this mount point needs to be mounted before application needs to be started . Resource Type for this is ‘Mount’ and MountPoint used here is “/apachevol” on which apache http is used .

hares -add apacheMnt Mount apacheSG

hares -modify apacheMnt Critical 0

hares -modify apacheMnt MountPoint /apachevol

hares -modify apacheMnt BlockDevice /dev/vx/dsk/apachedg/apachevol

hares -modify apacheMnt FSType vxfs

hares -modify apacheMnt FsckOpt %-y

6. Create Apache Resource apacheApp –Resource Type ‘ Apache’

Now configure the Apache resource which holds the application start and stop process . Required attributes should be configured before resource can be enabled .

hares -add apacheApp Apache apacheSG

hares -modify apacheApp Critical 0

hares -modify apacheApp httpdDir /apachevol/bin

hares -modify apacheApp PidFile /apachevol/logs/http.pid

hares -modify apacheApp ConfigFile /apachevol/conf/httpd.conf

hares -modify apacheApp EnvFile /apachevol/bin/envvars

Now Enable all the resources so that resources can be OFFLINE or ONLINE .

#hares –modify apacheApp Enable

#hares –modify apacheMnt Enable

#hares –modify apacheDG Enable

#hares –modify apacheIP Enable

#hares –modify apacheNIC Enable

#hares –modify apacheVol Enable

Testing the VCS SG :

You would have observed in resources configuration , we mentioned all the resources as Non-Critical by specifying Critical attribute to ‘0’ . Generally Critical attribute should be set to ‘1’ when that attribute is required for the Service Group to be completely online , if that Critical attribute is ‘0’ it specifies that the Service Group can be online even that resource is in offline state . Use the below command to online the resources manually .

hares –online -sys

Prior to changing the Critical attribute try to make the resources online on one server and check if all the resources comes online , if their exists any issues (like resource become FAULTED) , re-check the configuration and try again until all the resources comes online without any issues (Please follow the dependency hierarchy while making the individual resource online) . Once the SG shows online status , we are now ready to define the dependencies so that VCS handles the process of online/offline resources .

Link the VCS Resources :

We are now ready to define the resource dependencies and link the resource , this is required as VCS needs to identify which resource should be online before other resources are made online . In the figure you can find the hierarchy of resource dependencies .

#hares –link apacheApp apacheIP

#hares –link apacheApp apacheMnt

#hares –link apacheIP apacheNIC

#hares –link apacheMnt apacheVol

#hares –link apacheVol apacheDG

Now you can use the hagrp commands to control the service group . General VCS Service group operation involves switch , online or offline resources . Below are the commands you can use to perform these operations .

hares –online -sys

hares –offline -sys

hares –switch -to

Final Review of apacheSG :

Hmm atlast we are done with configuring and testing of our apacheSG . Below commands shows some final review of the service group .

[root@solaris2 /]#hastatus -sum

-- SYSTEM STATE

-- System State Frozen

A solaris1 RUNNING 0

A solaris2 RUNNING 0

-- GROUP STATE

-- Group System Probed AutoDisabled State

B apacheSG solaris1 Y N OFFLINE

B apacheSG solaris2 Y N ONLINE

Output defining online resources .

[root@solaris2 /]#hares -display -attribute State -group apacheSG -sys solaris2

#Resource Attribute System Value

apacheApp State solaris2 ONLINE

apacheDG State solaris2 ONLINE

apacheIP State solaris2 ONLINE

apacheMnt State solaris2 ONLINE

apacheNIC State solaris2 ONLINE

apacheVol State solaris2 ONLINE

Output defining dependencies .

[root@solaris2 /]#hares -dep

#Group Parent Child

apacheSG apacheApp apacheMnt

apacheSG apacheApp apacheIP

apacheSG apacheIP apacheNIC

apacheSG apacheMnt apacheVol

apacheSG apacheVol apacheDG