Voting Disk Corrupted
Scenario 1 – All votedisks are corrupted
– move voting disk to data diskgroup – voting disk is automaticaly recovered using latest available backup of OCR.
RECOVER VOTING DISK
– To move voting disk to votedg diskgroup, ASM instance should be up and for ASM instance to be up, CRS should be up. Hence we will
– stop crs on all the nodes
– start crs in exclusive mode on one of the nodes (host01)
– start asm instance on host01 using pfile (since spfile of ASM instance is on ASM)
– create a new diskgroup votedg
– move voting disk to votedg diskgroup
– stop crs on host01(was running in exclusive mode)
– restart crs on host01
– start crs on rest of the nodes
– start cluster on all the nodes
Scenario 2 – Few votedisks are corrupted
– move voting disk to data diskgroup. voting disk will be automatically recovered using surviving copy of voting disk.
OCR Corrupted or Not Available?
This step is necessary only i It is not possible to directly restore a manual or automatic OCR backup if the OCR is located in an ASM disk group. This is caused by the fact that the command ‘ocrconfig -restore’ requires ASM to be up & running in order to restore an OCR backup to an ASM disk group. However, for ASM to be available, the CRS stack must have been successfully started. For the restore to succeed, the OCR also must not be in use (r/w), i.e. no CRS daemon must be running while the OCR is being restored.If OCR is also corrupted or otherwise unavailable, such as if OCR is on Oracle ASM and the disk group is no longer available.
When using an ASM disk group for CRS there are typically 3 different types of files located in the disk group that potentially need to be restored/recreated:
- the Oracle Cluster Registry file (OCR)
- the Voting file(s)
- the shared SPFILE for the ASM instances
If you store OCR on an Oracle ASM disk group and the disk group is not available, then you must recover and mount the Oracle ASM disk group.
1. Locate the latest automatic OCR backup
2. Make sure the Grid Infrastructure is shutdown on all nodes
# $CRS_HOME/bin/crsctl stop crs -f
3. Start the CRS stack in exclusive mode
# $CRS_HOME/bin/crsctl start crs –excl -184.108.40.206
This document assumes that the CRS diskgroup was completely lost, in which case the CRS daemon (resource ora.crsd) will terminate again due to the inaccessibility of the OCR – even if above message indicates that the start succeeded.
If this is not the case – i.e. if the CRS diskgroup is still present (but corrupt or incorrect) the CRS daemon needs to be shutdown manually using:
# $CRS_HOME/bin/crsctl stop res ora.crsd -init
otherwise the subsequent OCR restore will fail.
220.127.116.11 and above:
# $CRS_HOME/bin/crsctl start crs -excl –nocrs
4. Label the CRS disk for ASMLIB use
# /usr/sbin/oracleasm createdisk ASMD40 /dev/sdh1
5. Create the CRS diskgroup via sqlplus
SQL> create diskgroup CRS external redundancy disk ‘ORCL:ASMD40’ attribute ‘COMPATIBLE.ASM’ = ‘11.2’;
6. Restore the latest OCR backup
Now that the CRS disk group is created & mounted the OCR can be restored – must be done as the root user:
# cd $CRS_HOME/cdata/rac_cluster1/
# $CRS_HOME/bin/ocrconfig -restore backup00.ocr
7. Start the CRS daemon on the current node (18.104.22.168 only !)
Now that the OCR has been restored the CRS daemon can be started, this is needed to recreate the Voting file. Skip this step for 22.214.171.124.0.
# $CRS_HOME/bin/crsctl start res ora.crsd -init
CRS-2672: Attempting to start ‘ora.crsd’ on ‘racnode1’
CRS-2676: Start of ‘ora.crsd’ on ‘racnode1’ succeeded
8. Recreate the Voting file
The Voting file needs to be initialized in the CRS disk group:
# $CRS_HOME/bin/crsctl replace votedisk +CRS
9. Recreate the SPFILE for ASM (optional)
10. Shutdown CRS
11. Rescan ASM disks
If using ASMLIB rescan all ASM disks on each node as the root user:
12. Start CRS
As the root user submit the CRS startup on all cluster nodes:
# $CRS_HOME/bin/crsctl start crs
13. Verify CRS
To verify that CRS is fully functional again:
# $CRS_HOME/bin/crsctl check cluster -all
The note applies to 10gR2 and 11gR1, for 11gR2, refer to note 1062983.1 and note 1377349.1
OCR corrupted? and you also lost Backup of OCR?
Metalink Note: 399482.1
Below procedure can be used for recovery.
1) Execute rootdelete.sh script from All Nodes.
2) Execute rootdeinstall.sh from Primary Node.
3) Run root.sh from Primary node.
4) Run root.sh from all remaining nodes.
5) Execute remaining configurations (ONS,netca,register required resources)