Solaris 10 : Configuring Interprocess communication Parameters
Other Learning Articles that you may like to read
Free Courses We Offer
Paid Training Courses we Offer
Many system administrators, particularly those managing large database servers, have had difficulty tuning shared memory and semaphore parameters in /etc/system. Often, these difficulties are experienced before starting an application instance. The Solaris 10 Operating System addresses this issue by integrating the System V IPC tunables into a powerful and flexible framework for easy administration.
Solaris 10 raises the system defaults for these parameters so that most applications can run out of the box. Solaris 10 also obsoletes many of the older IPC tunables.
The Solaris 10 Operating System extends the resource control (rctl) facility, introduced in the Solaris 9 Operating System, to incorporate the IPC tunables. These new IPC rctls can now be modified within the Solaris Resource Management framework using the standard resource management commands and features. They can be administered on a per-process or per-project basis to provide fine grained control.
The Solaris 10 Operating System introduces the following new resource controls to replace the old /etc/system tunables:
Old Old New New Resource control tunable default Max value default ---------------------- ------------- ------- --------- ---------- process.max-msg-qbytes msginfo_msgmnb 4096 ULONG_MAX 65536 process.max-msg-messages msginfo_msgtql 40 UINT_MAX 8192 process.max-sem-ops seminfo_semopm 10 INT_MAX 512 process.max-sem-nsems seminfo_semmsl 25 SHRT_MAX 512 project.max-shm-memory shminfo_shmmax 0x800000 UINT64_MAX 1/4 physmem project.max-shm-ids shminfo_shmmni 100 2**24 128 project.max-msg-ids msginfo_msgmni 50 2**24 128 project.max-sem-ids seminfo_semmni 10 2**24 128
As the names suggest, these rctls are attributes of either processes or projects.
The following tunables are now obsolete in the Solaris 10 Operating System:
Shared Memory Semaphores Message-Queue -------------- -------------- --------------- shminfo_shmseg seminfo_semmns msginfo_msgmax shminfo_shmmin seminfo_semvmx msginfo_msgssz shminfo_shmmax* seminfo_semmnu msginfo_msgmni* shminfo_shmmni* seminfo_semaem msginfo_msgtql* seminfo_semume msginfo_msgmnb* seminfo_semusz msginfo_msgmap seminfo_semmap msginfo_msgseg seminfo_semmsl* seminfo_semopm* seminfo_semmni*
values ending in * are technically obsolete, but if they are present in the /etc/system file then at boot the kernel will translate the values into global resource controls.
Configuring InterProcess Communication parameters
The /etc/system file is no longer the preferred way to set IPC tunables. The Resource Manager framework provides several methods to administer resource controls:
1. Persistent settings using the project database and the project file. The last field of a project entry in the /etc/project file is a semi-colon separated list of “attributes” of a project, which includes rctls. Attributes are specified in a “name=value” syntax, and with rctls the “value” field is a comma-separated 3-tuple called an “action clause”. An action clause consists of:
- a privilege level that dictates who can manipulate the rctls
- the threshold value of the rctl Threshold values in /etc/project must always be in the “base” unit for the rctl. For example, the base unit for project.max-shm-memory is a byte.
- the action to be taken when the threshold is exceeded The default deny action denies any resource requests greater than the threshold.
Specifying a limit in the /etc/project file extends that limit to all processes belonging to the project. Here is an example of an /etc/project file containing IPC settings for both the oracle_oltp and oracle_dss projects:
oracle_oltp:100:Oracle OLTP:oracle:: project.max-shm-memory=(privileged,51539607552,deny); process-max-sem-nsems=(privileged,300,deny) oracle_dss:101:Oracle DSS:oracle:: project.max-shm-memory=(privileged,17179869184,deny)
This sets a limit of 48GB per shared memory segment and 300 semaphores for all processes in the oracle_oltp project, and a 16GB shared memory segment limit for all processes in the oracle_dss project.
Note: lines in the /etc/project file must be one line only. The previous examples have been broken into multiple lines for readability purposes only.
The recommended method for modifying the /etc/project file is to use the “proj*” commands, such as projadd for creating a project and projmod for modifying a project.
2. Resource controls can be changed dynamically using prctl , assuming the user has the correct privileges.
The syntax of prctl can, at first, seem complex. Some common usages are:
# prctl -i process <pid> to list all resource controls for process <pid>
# prctl -i project <project> to list all resource controls for project <project>
# prctl -n <rctl> -i process <pid> lists only the resource control named <rctl> for process <pid>
# prctl -n <rctl> -r -v <value> -i process <pid> replaces (-r) the named rctl setting with the value <value> for process <pid>
Unlike the /etc/project file, prctl allows the use of “scale factors” to simplify resource control management. Values specified with the -v switch can be “human readable” values such as 48GB instead of the 51539607552 bytes required in the project database. For example, assuming the preceding /etc/project file we can check the values for the Shared Memory setting for the oracle_dss project:
% prctl -n project.max-shm-memory -i project oracle_dss project: 101: oracle_dss NAME PRIVILEGE VALUE FLAG ACTION RECIPIENT project.max-shm-memory privileged 16.0GB - deny - system 16.0EB max deny -
Should we need to temporarily increase the setting to 24GB:
% prctl -n project.max-shm-memory -r -v 24GB -i project oracle_dss % prctl -n project.max-shm-memory -i project oracle_dss project: 101: oracle_dss NAME PRIVILEGE VALUE FLAG ACTION RECIPIENT project.max-shm-memory privileged 24.0GB - deny - system 16.0EB max deny
Resource controls can be expressed as in units of size (bytes), time (seconds), or as a count (integer). See the resource_controls(5) manpage for more information about available resource controls, their units and possible string modifiers which can be used in the prctl, projadd, and projmod commands.
3. Programmatically using the system calls getrctl(2), setrctl(2) and libc functions such as rctlblk_set_value. Applications can use these system calls and library functions to check and set their own resource controls. These methods are beyond the scope of this document.
# projadd -c “Oracle” ‘user.oracle’
Once the project is created, assign resource controls corresponding to the Solaris 9 values:
This value is less than the new default for project.max-sem-ids: 128. As such, we could artificially lower the value to 100, or simply ignore it. We chose the latter.
Again, this value is less than the new default for process.max-sem-nsems: 512. In this case, however, we wish to limit Oracle to only 256 semaphores per process:
# projmod -s -K "process.max-sem-nsems=(privileged,256,deny)" 'user.oracle'
This system has 8GB of memory so this value (4GB, in bytes) is larger than new default for project.max-shm-memory (1/4 physmem, or 2GB on this system) and so an rctl must be created:
# projmod -s -K "project.max-shm-memory=(privileged,4GB,deny)" 'user.oracle'
Again, this value is larger than the new default for project.max-shm-ids so a rctl needs to be created:
# projmod -s -K “project.max-shm-ids=(privileged,256,deny)” ‘user.oracle’
There are no more lines from /etc/system that pertain to IPC, so we remove the old lines:
# /bin/cp /etc/system /etc/system.solaris9 # /bin/egrep -v "semsys:|shmsys:|msgsys:" /etc/system > /etc/system.solaris10 # /bin/mv /etc/system.solaris10 /etc/system
and our final /etc/project file:
# cat /etc/project system:0:::: user.root:1:::: noproject:2:::: default:3:::: group.staff:10:::: user.oracle:100:Oracle:::process.max-sem-nsems=(privileged,256,deny);project.max-shm-ids=(privileged,256,deny);project.max-shm-memory=(privileged,4294967296,deny)
Note, however, that these project rctls will not affect any processes run by a currently logged in userid ‘oracle’. We can check that any new oracle logins will receive the correct rctls by running:
# su oracle -c ‘sh -c “prctl -n process.max-sem-nsems -i process $$”‘
process: 3652: csh -c prctl -n process.max-sem-nsems -i process $$ NAME PRIVILEGE VALUE FLAG ACTION RECIPIENT process.max-sem-nsems privileged 256 - deny - system 32.8K max deny -
Which verifies that the new limit of 256 semaphores is present for the oracle user. We could then verify the other limits are also correct individually as above, or run:
# su oracle -c 'sh -c "prctl -i process $$"'
to check all the rctls simultaneously.
Most applications should receive sufficient System V IPC resources out of the box with the new Solaris 10 Operating System defaults. Administrators also have powerful new mechanisms for constraining IPC resources among applications on large systems or tuning them for optimum performance.