Saturday, 10 December 2011


Oracle clusterware is a portable cluster infrastructure that provides High Availability(HA) to RAC databases and other applications. Here a cluster is a collection of two or more nodes where they share a common pool of storage used by Oracle clusterware system files (OCR and Voting disk), a common network interconnect, and a common operating system.


The primary purpose of voting disk is to help in situations where private network communication fail. So the voting disk is used to communicate the node state information used to determine which nodes will go offline.


The Oracle Cluster Registry (OCR) file is also a key component of oracle clusterware. It maintains information about high availability components in your cluster, such as the voting file is essentially used by Cluster synchronization services daemon for node-cluster node list , cluster database instance to node mapping, and CRS application resource profiles (such as services, Virtual interconnect protocol addresses and so on). This file is maintained by administrative tools such as SRVCTL. Its size is around 100 MB.

Oracle Clusterware Run-Time View

On UNIX, oracle clusterware stack is run from entries in /etc/inittab with respawn. Here is a basic description of each process :

Cluster Synchronization Service Daemon (OCSSD) : This process runs in both vendor clusterware and nonvendor clusterware environments. It integrates with existing vendor clusterware, when present. OCSSD's primary job is internode hea;th monitoring, primarily using the network interconnect as well as voting disks, and database/ASM instance endpoint discovery via group services. OCSSD runs as oracle user, and failure exit causes machine reboot to prevent data corruption in the event of a split brain.

Process Monitor Daemon (OPROCD) = This process is spawned in any nonvendor clusterware environment. If OPROCD detects problems, it kills a node. And yes it runs as root. This daemon is used to detect hardware and driver freezes on the machine. If a machine was frozen long enough for the other nodes to evict it from the cluster, it needs to kill itself to prevent any I/O from being reissued to the disk after the rest of the cluster has remastered locks.

Cluster ready services daemon (CRSD) : This process is the engine for High Availability operations.It manages Oracle clusterware registered applications and starts, stops, check and fails them over via special action scripts. CRSD spawns dedicated process called RACGIMON that monitor the health of the database and ASM instances and host various feature threads such as Fast Application Notification (FAN). One RACGIMON process is spawned for each instance. CRSD maintains configuration profiles as well as resource statuses in OCR (Oracle Cluster Registry). It runs as root and restarted automatically on failure. In addition, CRSD can spawn temporary children to execute particular actions such as : racgeut (execute under time), racgmd (Manage database), racgchsn (Change service Name), racgons (to add/remove ONS configuration to OCR), racgvip: to start/stop/check instance virtual IP.

Event management daemon (EVMD): This process forwards cluster events when things happen. It spawns a permanent child evmlogger that, on demand , spawns children such as racgevtf to invoke callouts. It runs as oracle, and is restarted automatically on failure.

NOTE :- The RACG infrastructure is used to deploy the oracle database in highly available clustered enviroment. This infrastructure in mainly implemented using the racgwrap script that invokes the racgmain program. It is used by CRS to execute actions for all node-centric resources as well as to proxy actions for all instance-centric resources to RACGIMON. Basically, this infrastructure is responsible for managing all ora.* resources.


Global Cache Service processes (GCS) are processes that, when spawned by Oracle, copy blocks directly from the holding instance's buffer cache and send a read consistent copy of the block to the requesting foreground process on the requesting instance to be placed into the buffer cache. GCS rolls back any uncommitted transactions for any blocks that are being requested for consistent read by the remote instance.

RAC software provides for up to 10 GCS processes (0 thru 9), depending on the amount of messaging traffic. However, there is, by default, one GCS process per pair of CPU. In general, the number of GCS processes varies depending on the amount of messaging traffic amongst nodes in the cluster. Note: Global Cache Services (GCS) were called Lock Manager Services (LMS) in Oracle Parallel Server (OPS).

Oracle 10g added a new initialization parameter called gcs_server_processes. This parameter allows you to set the number of GCS processes that are started to handle interconnect traffic. The default is 2, but can be increased on a per node basis if required.


Cluster Group Services (CGS) is a background process that monitors the entire cluster to manage global resources. By constantly probing the other instances, it checks and manages instance deaths and the associated recovery for GCS. When a node joins or leaves the cluster, it handles reconfiguration of locks and resources. In particular, CGS handles the part of recovery associated with global resources. Note: Cluster Group Services was called Lock Manager Monitor in OPS.


Global Enqueue Service Daemon is a background agent process that manages requests for resources to control access to blocks and global enqueues. It manages lock manager service requests for GCS resources and sends them to a service queue to be handled by the GCS process. The GES process also handles global deadlock detection and remote resource requests (remote resource requests are requests originating from another instance).


The Lock Process (LCK) manages non-cache fusion resource requests such as library and row cache requests and lock requests that are

local to the server. LCK manages instance resource requests and cross-instance call operations for shared resources. It builds a list of invalid lock elements and validates lock elements during recovery. Because the LMS process handles the primary function of lock management, only a single LCK process exists in each instance. There is only one LCK process per instance in RAC.


The Diagnosability Daemon (DIAG) background process monitors the health of the instance and captures diagnostic data about process failures within instances. The operation of this daemon is automated and updates an alert log file to record the activity that it performs.

Oracle 11g Processes

In addition, Oracle 11g added the following new processes:

ACMS: The Atomic Controlfile to Memory Service (ACMS) process insures SGA memory updates are updated on all nodes or none.

GTX: The Global Transaction Process supports XA transactions within RAC.

RMS: The RAC Management Process creates resources that allow new database instances to be added to the cluster.

RSM: The remote slave monitor creates slave processes for processes running on other RAC instances within the cluster.