This example will run a job only once:

In the above example the job “send_mail” runs only once “sysdate”.

to run the job more than once at a specific time we replace “sysdate” by our time :

e.g to run everyday once

1  BEGIN

2     DBMS_JOB.SUBMIT(:jobno,’Send_Mail;’,SYSDATE, ‘sysdate +1 ‘);

3     COMMIT;

4* END;

please note that your job will not run unless you commit it.

pmon : monitors the other background process. Whenever a server process or dispatcher process terminates abnormally, it performs the process recovery. It frees the resources that the client process was using. At the time of instance startup pmon checks if the listner is running or not. In case its not running pmon tries to contact it periodically.

NOTE: PMON is not doing any instance recovery , its just trying to know if the listener is   running.

SMON:  This is the process which does the instance recovery at the time of startup if needed.  It recovers the terminated transactions at instance crash when the tablespace or file is brought back online after recovery.

“One thing that should always be remembered is that the CKPT process updates the control file and the data file headers with the latest advancement of the checkpoint number based on the buffers written by the DBWR. ” This is the most important thing to be kept in mind.

Let’s see go and see now how there can be different cases of the control file loss and what we can do to recover it,

1)   Control file lost with a mirrored copy available>>

As like always, its always better to have a seperate copy of your control file. Its going to be very handy when you have lost your any one of them. Oracle recommends to have at least 3 copies of the file all the time. Please note that the copies can be actually more than 3 also but here, “the more the merrier” will not be the right thing to do. If you are going to make multiplexed copies of your control file and your database is doing huge amount of contol file updates, it would lead to the Control File Parallel Write waite event and also the Control File Enque will come into contention. So its always better to use wisely the number of copies needed.

If you have just lost one copy out of your mirrored files, you have very less to do. Here are the steps to recover the database ,

a) If database is already up, bring it down immediately without invoking the checkpoint.

SQL> shut abort;

b) Copy out of one available control files, one file to the location where the lost file is needed. This surely is mandatory that the location is intact. Use an operating system level command to do so,

$cp /u01/app/oracle/oradata/orcl/control02.ctl /u01/app/oracle/oradata/orcl/control01.ctl

c)    As you have already copied the file, just issue the instance startup to open the database.

SQL> startup

It may also be possible that even though you have the multiplexed control files and you have just lost out of N, only 1 control file but the underlying location of the missing control file is also damaged. If that’s going to be the case then you can’t restore the file to the same location. You can use the existing control files’s copy for sure but the location has to be different and this means, you now need to edit the initillization file of the database as well to update the Control_Files parameter. With all the steps mentioned above being the same, you need to one more step and that would be to edit the parameter with the new location with the corresponding missing file’s name. After you are done doing that, issue a startup and you would be having a running database.

2> Restoring control file with redo logs being intact >>

If you have lost all the copies of the control file but your datafiles and online redo logs are fine, you can use the backup of the control file to recover from the loss. Let’s create a small disaster where we would loss of our control files. Please note that we already took the backup and our database is already set into the archivelog mode.

Here is the listing of the files that we have in our database.

[oracle@dhcp-otp-phase2-floor-g-1-10-177-54-233 orcl]$ ls -l *ctl

-rw-r—–  1 oracle oinstall 9748480 Nov 17 12:51 control01.ctl

-rw-r—–  1 oracle oinstall 9748480 Nov 17 12:51 control02.ctl

-rw-r—–  1 oracle oinstall 9748480 Nov 17 12:51 control03.ctl

So , very rudely, we move them all to a folder called Control which is equivalent to losing all of them,

[oracle@dhcp-otp-phase2-floor-g-1-10-177-54-233 orcl]$ mv *.ctl control/

[oracle@dhcp-otp-phase2-floor-g-1-10-177-54-233 orcl]$ ls -ltr

total 1585940

-rw-r—–  1 oracle oinstall   5251072 Nov 17 08:27 users01.dbf

-rw-r—–  1 oracle oinstall  52429312 Nov 17 08:27 redo03.log

-rw-r—–  1 oracle oinstall  52429312 Nov 17 08:27 redo02.log

-rw-r—–  1 oracle oinstall 104865792 Nov 17 08:27 example01.dbf

-rw-r—–  1 oracle oinstall  20979712 Nov 17 09:26 temp01.dbf

-rw-r—–  1 oracle oinstall  41951232 Nov 17 12:52 undotbs01.dbf

-rw-r—–  1 oracle oinstall 723525632 Nov 17 12:52 system01.dbf

-rw-r—–  1 oracle oinstall 588849152 Nov 17 12:52 sysaux01.dbf

-rw-r—–  1 oracle oinstall  52429312 Nov 17 12:52 redo01.log

drwxr-xr-x  2 oracle oinstall      4096 Nov 17 12:52 control

And just to make sure that we do get an error, let’s to access some thing which would only come from Control file,

SQL> select * from V$database;

select * from V$database

*

ERROR at line 1:

ORA-00210: cannot open the specified control file

ORA-00202: control file: ‘/u01/app/oracle/oradata/orcl/control01.ctl’

ORA-27041: unable to open file

Linux Error: 2: No such file or directory

Additional information: 3

So there are no more control files there with us. Now, the first step would be to immediately issue shut down the database. Let’s do it,

SQL> shut abort;

ORACLE instance shut down.

Fortunately, we did take the backup using RMAN so we can use it. But before that we need to have at least the instance started as RMAN needs at least the instance to be up,

SQL> startup nomount

ORACLE instance started.

Total System Global Area  171573248 bytes

Fixed Size                  1298668 bytes

Variable Size             134221588 bytes

Database Buffers           29360128 bytes

Redo Buffers                6692864 bytes

And now we fire up RMAN and restore the control files from the backup,

RMAN> restore controlfile from autobackup;

Starting restore at 17-NOV-09

using target database control file instead of recovery catalog

allocated channel: ORA_DISK_1

channel ORA_DISK_1: SID=152 device

recovery area destination: /u01/app/oracle/flash_recovery_area

database name (or database unique name) used for search: ORCL

channel ORA_DISK_1: AUTOBACKUP /u01/app/oracle/flash_recovery_area/ORCL/autobackup/2009_11_17/o1_mf_s_703153678_5j449plt_.bkp found in the recovery area

AUTOBACKUP search with format “%F” not attempted because DBID was not set

channel ORA_DISK_1: restoring control file from AUTOBACKUP /u01/app/oracle/flash_recovery_area/ORCL/autobackup/2009_11_17/o1_mf_s_703153678_5j449plt_.bkp

channel ORA_DISK_1: control file restore from AUTOBACKUP complete

output file name=/u01/app/oracle/oradata/orcl/control01.ctl

output file name=/u01/app/oracle/oradata/orcl/control02.ctl

output file name=/u01/app/oracle/oradata/orcl/control03.ctl

Finished restore at 17-NOV-09

We can see that all the 3 control files have come up. And we can verify it as well,

[oracle@dhcp-otp-phase2-floor-g-1-10-177-54-233 orcl]$ ls *.ctl

control01.ctl  control02.ctl  control03.ctl

The next step would be to recover the  the database. We can do it with bringing the datbase in the mount stage and issue the Resetlogs command to open it up afterwards.

RMAN> recover database;

Starting recover at 17-NOV-09

using channel ORA_DISK_1

using channel ORA_DISK_2

using channel ORA_DISK_3

using channel ORA_DISK_4

starting media recovery

archived log for thread 1 with sequence 1 is already on disk as file /u01/app/oracle/oradata/orcl/redo01.log

archived log file name=/u01/app/oracle/oradata/orcl/redo01.log thread=1 sequence=1

media recovery complete, elapsed time: 00:00:00

Finished recover at 17-NOV-09

And now we can open the database happily,

SQL> alter database open resetlogs;

Database altered.

SQL> select status from V$instance;

STATUS

————

OPEN

2>Loss of control file with data files but redo logs being intact>>

This would a semi-disaster situation. Reason for adding here  “semi” is that you have almost lost everything with the loss of your datafiles and control files. Yet you have a ray of hope left as you are still left with your  current redo logs. In this case, as the current redo logs and archived logs , both are available , we can go ahead with the complete recovery.

Let’s try to have this issue for us. We shall delete the control files and data files and will see how we can recover this,

[oracle@dhcp-otp-phase2-floor-g-1-10-177-54-233 orcl]$ ls

control01.ctl  example01.dbf  redo03.log    temp01.dbf

control02.ctl  redo01.log     sysaux01.dbf  undotbs01.dbf

control03.ctl  redo02.log     system01.dbf  users01.dbf

[oracle@dhcp-otp-phase2-floor-g-1-10-177-54-233 orcl]$ mkdir control

[oracle@dhcp-otp-phase2-floor-g-1-10-177-54-233 orcl]$ mv *.ctl *.dbf control/

[oracle@dhcp-otp-phase2-floor-g-1-10-177-54-233 orcl]$ ls

control  redo01.log  redo02.log  redo03.log

So we have lost our control files and data files. Let’s stop our database and than start it in the nomount stage,

SQL>shut abort;

SQL>startup nomount;

Now, as we can be in the nomount stage, let’s restore first our control file,

RMAN> restore controlfile from autobackup;

Starting restore at 18-NOV-09

using target database control file instead of recovery catalog

allocated channel: ORA_DISK_1

channel ORA_DISK_1: SID=153 device

recovery area destination: /u01/app/oracle/flash_recovery_area

database name (or database unique name) used for search: ORCL

channel ORA_DISK_1: AUTOBACKUP /u01/app/oracle/flash_recovery_area/ORCL/autobackup/2009_11_18/o1_mf_s_703222747_5j67r3rd_.bkp found in the recovery area

AUTOBACKUP search with format “%F” not attempted because DBID was not set

channel ORA_DISK_1: restoring control file from AUTOBACKUP /u01/app/oracle/flash_recovery_area/ORCL/autobackup/2009_11_18/o1_mf_s_703222747_5j67r3rd_.bkp

channel ORA_DISK_1: control file restore from AUTOBACKUP complete

output file name=/u01/app/oracle/oradata/orcl/control01.ctl

output file name=/u01/app/oracle/oradata/orcl/control02.ctl

output file name=/u01/app/oracle/oradata/orcl/control03.ctl

Finished restore at 18-NOV-09

As we have control file with us, let’s mount the db,

SQL> alter database mount;

Database altered.
Now we need to restore all the datafiles as well. So time to restore the database it self.

RMAN> restore database;

Starting restore at 18-NOV-09

released channel: ORA_DISK_1

Starting implicit crosscheck backup at 18-NOV-09

allocated channel: ORA_DISK_1

channel ORA_DISK_1: SID=153 device

allocated channel: ORA_DISK_2

channel ORA_DISK_2: SID=151 device

allocated channel: ORA_DISK_3

channel ORA_DISK_3: SID=150 device

allocated channel: ORA_DISK_4

channel ORA_DISK_4: SID=149 device

Crosschecked 7 objects

Finished implicit crosscheck backup at 18-NOV-09

Starting implicit crosscheck copy at 18-NOV-09

using channel ORA_DISK_1

using channel ORA_DISK_2

using channel ORA_DISK_3

using channel ORA_DISK_4

Finished implicit crosscheck copy at 18-NOV-09

searching for all files in the recovery area

cataloging files…

cataloging done

List of Cataloged Files

=======================

File Name: /u01/app/oracle/flash_recovery_area/ORCL/archivelog/2009_11_18/o1_mf_1_5_5j6ml70l_.arc

File Name: /u01/app/oracle/flash_recovery_area/ORCL/archivelog/2009_11_18/o1_mf_1_6_5j6mlhdr_.arc

File Name: /u01/app/oracle/flash_recovery_area/ORCL/archivelog/2009_11_18/o1_mf_1_7_5j6mlr9s_.arc

File Name: /u01/app/oracle/flash_recovery_area/ORCL/autobackup/2009_11_18/o1_mf_s_703222747_5j67r3rd_.bkp

using channel ORA_DISK_1

using channel ORA_DISK_2

using channel ORA_DISK_3

using channel ORA_DISK_4

channel ORA_DISK_1: starting datafile backup set restore

channel ORA_DISK_1: specifying datafile(s) to restore from backup set

channel ORA_DISK_1: restoring datafile 00001 to /u01/app/oracle/oradata/orcl/system01.dbf

channel ORA_DISK_1: restoring datafile 00002 to /u01/app/oracle/oradata/orcl/sysaux01.dbf

channel ORA_DISK_1: restoring datafile 00003 to /u01/app/oracle/oradata/orcl/undotbs01.dbf

channel ORA_DISK_1: restoring datafile 00004 to /u01/app/oracle/oradata/orcl/users01.dbf

channel ORA_DISK_1: restoring datafile 00005 to /u01/app/oracle/oradata/orcl/example01.dbf

channel ORA_DISK_1: reading from backup piece /u01/app/oracle/flash_recovery_area/ORCL/backupset/2009_11_18/o1_mf_nnndf_TAG20091118T033237_5j67cxb0_.bkp

channel ORA_DISK_1: piece handle=/u01/app/oracle/flash_recovery_area/ORCL/backupset/2009_11_18/o1_mf_nnndf_TAG20091118T033237_5j67cxb0_.bkp tag=TAG20091118T033237

channel ORA_DISK_1: restored backup piece 1

channel ORA_DISK_1: restore complete, elapsed time: 00:01:05

Finished restore at 18-NOV-09

So we have all what we need. Let’s try to recover the database.

RMAN> recover database ;

Starting recover at 18-NOV-09

using channel ORA_DISK_1

using channel ORA_DISK_2

using channel ORA_DISK_3

using channel ORA_DISK_4

starting media recovery

archived log for thread 1 with sequence 3 is already on disk as file /u01/app/oracle/flash_recovery_area/ORCL/archivelog/2009_11_18/o1_mf_1_3_5j67hbwq_.arc

archived log for thread 1 with sequence 4 is already on disk as file /u01/app/oracle/flash_recovery_area/ORCL/archivelog/2009_11_18/o1_mf_1_4_5j67qx4v_.arc

archived log for thread 1 with sequence 5 is already on disk as file /u01/app/oracle/flash_recovery_area/ORCL/archivelog/2009_11_18/o1_mf_1_5_5j6ml70l_.arc

archived log for thread 1 with sequence 6 is already on disk as file /u01/app/oracle/flash_recovery_area/ORCL/archivelog/2009_11_18/o1_mf_1_6_5j6mlhdr_.arc

archived log for thread 1 with sequence 7 is already on disk as file /u01/app/oracle/flash_recovery_area/ORCL/archivelog/2009_11_18/o1_mf_1_7_5j6mlr9s_.arc

archived log for thread 1 with sequence 8 is already on disk as file /u01/app/oracle/oradata/orcl/redo02.log

archived log file name=/u01/app/oracle/flash_recovery_area/ORCL/archivelog/2009_11_18/o1_mf_1_3_5j67hbwq_.arc thread=1 sequence=3

archived log file name=/u01/app/oracle/flash_recovery_area/ORCL/archivelog/2009_11_18/o1_mf_1_4_5j67qx4v_.arc thread=1 sequence=4

archived log file name=/u01/app/oracle/flash_recovery_area/ORCL/archivelog/2009_11_18/o1_mf_1_5_5j6ml70l_.arc thread=1 sequence=5

archived log file name=/u01/app/oracle/flash_recovery_area/ORCL/archivelog/2009_11_18/o1_mf_1_6_5j6mlhdr_.arc thread=1 sequence=6

archived log file name=/u01/app/oracle/flash_recovery_area/ORCL/archivelog/2009_11_18/o1_mf_1_7_5j6mlr9s_.arc thread=1 sequence=7

archived log file name=/u01/app/oracle/oradata/orcl/redo02.log thread=1 sequence=8

media recovery complete, elapsed time: 00:00:33

Finished recover at 18-NOV-09

Please note that this is a complete recovery only. But we still need to open the database using Resetlogs.

SQL> alter database open resetlogs;

Database altered.

And we are done!

3>Loss of control files when data files are from the backup  with the loss of redo logs as well>>

This essentially would be a similar sort of recovery as we have seen above but(and this matters the most) will be an incomplete recovery as we would not be able to recover what we had in our current redo logs leading us to have only a handful only.

To illustrate the point, we created a table and put some data into it. After that we added  a lot more data into it.  Some of the the entries would be able to go to the redo logs and from there to the archive logs would come back to us. The remaining would be lost confirming the incomplete recovery to us.

First let’s create a table and add some data to it,

SQL> create table test_recovery(a number);

SQL> insert into test_recovery values(1);

SQL>commit;

And now we shall add some  more data to it.

SQL> insert into test_recovery values(1);

(output trimmed)

.

..

…

….

SQL>commit;

At this stage, we had this much of data in this table.

SQL>select count(*) from test_recovery;

COUNT(*)

———-

32

After this, we took a backup of the database and made sure that the current archive logs are in that backup. Once finished with that, we added a lot more to the table. And finally, we had these many rows in the table,

SQL>select count(*) from test_recovery;

COUNT(*)

———-

1048576

And this was the last output for the redo logs,

SQL> select * from V$log;

GROUP#    THREAD#  SEQUENCE#      BYTES    MEMBERS ARC STATUS

———- ———- ———- ———- ———- — —————-

FIRST_CHANGE# FIRST_TIM

————- ———

1          1          4   52428800          1 NO  CURRENT

593472 18-NOV-09

2          1          2   52428800          1 YES INACTIVE

593409 18-NOV-09

3          1          3   52428800          1 YES INACTIVE

593455 18-NOV-09

We can see that the Sequence# 4 is the number of the current log group of ours. At this moment, let’s introduce the crash.

[oracle@dhcp-otp-phase2-floor-g-1-10-177-54-233 orcl]$ ls

control        control03.ctl  redo02.log    system01.dbf   users01.dbf

control01.ctl  example01.dbf  redo03.log    temp01.dbf

control02.ctl  redo01.log     sysaux01.dbf  undotbs01.dbf

[oracle@dhcp-otp-phase2-floor-g-1-10-177-54-233 orcl]$ mv ** control/

mv: cannot move `control’ to a subdirectory of itself, `control/control’

[oracle@dhcp-otp-phase2-floor-g-1-10-177-54-233 orcl]$ ls

control

So we have nothing with us anymore. Naturally, the first step would be to stop the database and start in the Nomount stage. After this, the next step would be to restore the control file from the backup.

RMAN> restore controlfile from autobackup;

Starting restore at 18-NOV-09

using target database control file instead of recovery catalog

allocated channel: ORA_DISK_1

channel ORA_DISK_1: SID=153 device

recovery area destination: /u01/app/oracle/flash_recovery_area

database name (or database unique name) used for search: ORCL

channel ORA_DISK_1: AUTOBACKUP /u01/app/oracle/flash_recovery_area/ORCL/autobackup/2009_11_18/o1_mf_s_703262508_5j7globt_.bkp found in the recovery area

AUTOBACKUP search with format “%F” not attempted because DBID was not set

channel ORA_DISK_1: restoring control file from AUTOBACKUP /u01/app/oracle/flash_recovery_area/ORCL/autobackup/2009_11_18/o1_mf_s_703262508_5j7globt_.bkp

channel ORA_DISK_1: control file restore from AUTOBACKUP complete

output file name=/u01/app/oracle/oradata/orcl/control01.ctl

output file name=/u01/app/oracle/oradata/orcl/control02.ctl

output file name=/u01/app/oracle/oradata/orcl/control03.ctl

Finished restore at 18-NOV-09

Now, we need to bring the database in the Mount stage so that we can restore the rest of the database. And we shall also check here that whether we have got the control files or not(no need to doubt over RMAN but still),

[oracle@dhcp-otp-phase2-floor-g-1-10-177-54-233 orcl]$ ls

control  control01.ctl  control02.ctl  control03.ctl

[oracle@dhcp-otp-phase2-floor-g-1-10-177-54-233 orcl]$ sqlplus / as sysdba

SQL*Plus: Release 11.1.0.6.0 – Production on Wed Nov 18 14:44:39 2009

Copyright (c) 1982, 2007, Oracle.  All rights reserved.

Connected to:

Oracle Database 11g Enterprise Edition Release 11.1.0.6.0 – Production

With the Partitioning, OLAP, Data Mining and Real Application Testing options

SQL> alter database mount;

Database altered.

So indeed we have got them. Let’s proceed to the next step which is to restore the datafiles,

RMAN> restore database;

Starting restore at 18-NOV-09

released channel: ORA_DISK_1

Starting implicit crosscheck backup at 18-NOV-09

allocated channel: ORA_DISK_1

channel ORA_DISK_1: SID=153 device

allocated channel: ORA_DISK_2

channel ORA_DISK_2: SID=151 device

allocated channel: ORA_DISK_3

channel ORA_DISK_3: SID=150 device

allocated channel: ORA_DISK_4

channel ORA_DISK_4: SID=149 device

Crosschecked 14 objects

Finished implicit crosscheck backup at 18-NOV-09

Starting implicit crosscheck copy at 18-NOV-09

using channel ORA_DISK_1

using channel ORA_DISK_2

using channel ORA_DISK_3

using channel ORA_DISK_4

Finished implicit crosscheck copy at 18-NOV-09

searching for all files in the recovery area

cataloging files…

cataloging done

List of Cataloged Files

=======================

File Name: /u01/app/oracle/flash_recovery_area/ORCL/autobackup/2009_11_18/o1_mf_s_703262508_5j7globt_.bkp

using channel ORA_DISK_1

using channel ORA_DISK_2

using channel ORA_DISK_3

using channel ORA_DISK_4

channel ORA_DISK_1: starting datafile backup set restore

channel ORA_DISK_1: specifying datafile(s) to restore from backup set

channel ORA_DISK_1: restoring datafile 00003 to /u01/app/oracle/oradata/orcl/undotbs01.dbf

channel ORA_DISK_1: restoring datafile 00005 to /u01/app/oracle/oradata/orcl/example01.dbf

channel ORA_DISK_1: reading from backup piece /u01/app/oracle/flash_recovery_area/ORCL/backupset/2009_11_18/o1_mf_nnndf_TAG20091118T140610_5j7dhw3p_.bkp

channel ORA_DISK_2: starting datafile backup set restore

channel ORA_DISK_2: specifying datafile(s) to restore from backup set

channel ORA_DISK_2: restoring datafile 00002 to /u01/app/oracle/oradata/orcl/sysaux01.dbf

channel ORA_DISK_2: restoring datafile 00004 to /u01/app/oracle/oradata/orcl/users01.dbf

channel ORA_DISK_2: reading from backup piece /u01/app/oracle/flash_recovery_area/ORCL/backupset/2009_11_18/o1_mf_nnndf_TAG20091118T140610_5j7dhtyg_.bkp

channel ORA_DISK_3: starting datafile backup set restore

channel ORA_DISK_3: specifying datafile(s) to restore from backup set

channel ORA_DISK_3: restoring datafile 00001 to /u01/app/oracle/oradata/orcl/system01.dbf

channel ORA_DISK_3: reading from backup piece /u01/app/oracle/flash_recovery_area/ORCL/backupset/2009_11_18/o1_mf_nnndf_TAG20091118T140610_5j7dhtt6_.bkp

channel ORA_DISK_1: piece handle=/u01/app/oracle/flash_recovery_area/ORCL/backupset/2009_11_18/o1_mf_nnndf_TAG20091118T140610_5j7dhw3p_.bkp tag=TAG20091118T140610

channel ORA_DISK_1: restored backup piece 1

channel ORA_DISK_1: restore complete, elapsed time: 00:00:55

channel ORA_DISK_2: piece handle=/u01/app/oracle/flash_recovery_area/ORCL/backupset/2009_11_18/o1_mf_nnndf_TAG20091118T140610_5j7dhtyg_.bkp tag=TAG20091118T140610

channel ORA_DISK_2: restored backup piece 1

channel ORA_DISK_2: restore complete, elapsed time: 00:02:06

channel ORA_DISK_3: piece handle=/u01/app/oracle/flash_recovery_area/ORCL/backupset/2009_11_18/o1_mf_nnndf_TAG20091118T140610_5j7dhtt6_.bkp tag=TAG20091118T140610

channel ORA_DISK_3: restored backup piece 1

channel ORA_DISK_3: restore complete, elapsed time: 00:02:16

Finished restore at 18-NOV-09

And now we can try recovering the database.

RMAN> recover database;

Starting recover at 18-NOV-09

using channel ORA_DISK_1

using channel ORA_DISK_2

using channel ORA_DISK_3

using channel ORA_DISK_4

starting media recovery

archived log for thread 1 with sequence 2 is already on disk as file /u01/app/oracle/flash_recovery_area/ORCL/archivelog/2009_11_18/o1_mf_1_2_5j7dnrff_.arc

archived log for thread 1 with sequence 3 is already on disk as file /u01/app/oracle/flash_recovery_area/ORCL/archivelog/2009_11_18/o1_mf_1_3_5j7dw3cj_.arc

archived log for thread 1 with sequence 1 is already on disk as file /u01/app/oracle/flash_recovery_area/ORCL/archivelog/2009_11_18/o1_mf_1_1_5j7ghkx4_.arc

archived log for thread 1 with sequence 2 is already on disk as file /u01/app/oracle/flash_recovery_area/ORCL/archivelog/2009_11_18/o1_mf_1_2_5j7gkddx_.arc

archived log for thread 1 with sequence 3 is already on disk as file /u01/app/oracle/flash_recovery_area/ORCL/archivelog/2009_11_18/o1_mf_1_3_5j7glm85_.arc

archived log file name=/u01/app/oracle/flash_recovery_area/ORCL/archivelog/2009_11_18/o1_mf_1_2_5j7dnrff_.arc thread=1 sequence=2

archived log file name=/u01/app/oracle/flash_recovery_area/ORCL/archivelog/2009_11_18/o1_mf_1_3_5j7dw3cj_.arc thread=1 sequence=3

archived log file name=/u01/app/oracle/flash_recovery_area/ORCL/archivelog/2009_11_18/o1_mf_1_1_5j7ghkx4_.arc thread=1 sequence=1

archived log file name=/u01/app/oracle/flash_recovery_area/ORCL/archivelog/2009_11_18/o1_mf_1_2_5j7gkddx_.arc thread=1 sequence=2

archived log file name=/u01/app/oracle/flash_recovery_area/ORCL/archivelog/2009_11_18/o1_mf_1_3_5j7glm85_.arc thread=1 sequence=3

unable to find archived log

archived log thread=1 sequence=4

RMAN-00571: ===========================================================

RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============

RMAN-00571: ===========================================================

RMAN-03002: failure of recover command at 11/18/2009 14:48:07

RMAN-06054: media recovery requesting unknown archived log for thread 1 with sequence 4 and starting SCN of 593472

Please note that we got the error for the same sequence number 4 which was of our current log group. Oracle couldn’t find its archived file and the redo log surely got deleted by us only. So whatever we had inside it is lost and that’s what the point we mentioned that this would be an incomplete recovery. Let’s  open the database and check the number of the rows in the table.

SQL> alter database open resetlogs;

Database altered.

SQL> select count(*) from test_recovery;

COUNT(*)

———-

32

So indeed we got only what we could save via archive logs in our backup. And since we had lost the redo logs , with the Resetlogs , we got them back but only as blank with nothing available in them yet,

SQL> select * from V$log;

GROUP#    THREAD#  SEQUENCE#      BYTES    MEMBERS ARC STATUS

———- ———- ———- ———- ———- — —————-

FIRST_CHANGE# FIRST_TIM

————- ———

1          1          1   52428800          1 NO  CURRENT

593473 18-NOV-09

2          1          0   52428800          1 YES UNUSED

0

3          1          0   52428800          1 YES UNUSED

4>>Loss of control files with no backup available but redo logs are fine

Now this is a serious situation. You have ended up with loss of all of your control files and to put the last cherry on the cake, you have no backup either. This means that you can’t simply restore the control file from the backup. So you need to recreate the control file. Fortunately, oracle does gives us a way out to do so by its Create Controlfile command, using which we can recreate the control file. Let’s see this case as well.

To start with, we shall move the control file. Please note that in this case, we do need to have the redo logs fine. Using redo logs and archive logs, we shall be able to open the database. To simulate the loss of control files, let’s move them to some other location.

[oracle@dhcp-otp-phase2-floor-g-1-10-177-54-233 orcl]$ ls

control        control03.ctl  redo02.log    system01.dbf   users01.dbf

control01.ctl  example01.dbf  redo03.log    temp01.dbf

control02.ctl  redo01.log     sysaux01.dbf  undotbs01.dbf

[oracle@dhcp-otp-phase2-floor-g-1-10-177-54-233 orcl]$ mv *.ctl control/

As we have lost the control files, we need to immediately kill the instance with the Abort option.

SQL> shut abort

ORACLE instance shut down.

To create the control file again, we shall need all the information of the database that this file stores and this must match exactly with the database structure of ours. You can get the current strcuture and the create command by using the

Alter Database Backup Controlfile To Trace command. This would create the text based trace file in the User Dump Dest(10g) and Diagnostic Dest(11g) . If you have not done any changes in the database structure of yours, you can simply pick up this file and use it. This file contains two section , with Resetlogs and without Resetlogs. If you have shutdown your db with the Abort option( like we did), you must use the with resetlogs option only. Also, here as the control file is older than the rest of the database so you need to use the Recover Database Using Backup Controlfile option to tell oracle that an old file is being used. Fortunately all this is already there in the trace file that you would get from the Alter database option.

The trace natively contains lots of information which may not be needed. Its better to get rid of that and let only that content be there in the trace file which is needed by you. You need to run this script while being connected as the Sys user. It has all the commands to go ahead with the rest of the process. Let’s try to do so,

SQL> @/home/oracle/a.sql

ORACLE instance started.

Total System Global Area  171573248 bytes

Fixed Size                  1298668 bytes

Variable Size             138415892 bytes

Database Buffers           25165824 bytes

Redo Buffers                6692864 bytes

Control file created.

PL/SQL procedure successfully completed.

PL/SQL procedure successfully completed.

ORA-00279: change 593476 generated at 11/18/2009 14:48:31 needed for thread 1

ORA-00289: suggestion :

/u01/app/oracle/flash_recovery_area/ORCL/archivelog/2009_11_19/o1_mf_1_1_%u_.arcORA-00280: change 593476 for thread 1 is in sequence #1

ORA-00308: cannot open archived log ‘–’

ORA-27037: unable to obtain file status

Linux Error: 2: No such file or directory

Additional information: 3

ORA-00308: cannot open archived log ‘ALTER’

ORA-27037: unable to obtain file status

Linux Error: 2: No such file or directory

Additional information: 3

ORA-00308: cannot open archived log ‘–’

ORA-27037: unable to obtain file status

Linux Error: 2: No such file or directory

Additional information: 3

ORA-00308: cannot open archived log ‘–’

ORA-27037: unable to obtain file status

Linux Error: 2: No such file or directory

Additional information: 3

ORA-00308: cannot open archived log ‘–’

ORA-27037: unable to obtain file status

Linux Error: 2: No such file or directory

Additional information: 3

ORA-00308: cannot open archived log ‘ALTER’

ORA-27037: unable to obtain file status

Linux Error: 2: No such file or directory

Additional information: 3

ORA-00308: cannot open archived log ‘SIZE’

ORA-27037: unable to obtain file status

Linux Error: 2: No such file or directory

Additional information: 3

ORA-00308: cannot open archived log ‘–’

ORA-27037: unable to obtain file status

Linux Error: 2: No such file or directory

Additional information: 3

Specify log: {<RET>=suggested | filename | AUTO | CANCEL}

/u01/app/oracle/oradata/orcl/redo01.log

Log applied.

Media recovery complete.

SQL> alter database open resetlogs;

Database altered.

So we are done with the recovery!

In the creation of control file, there may be the cases when just going ahead with the default given file may not be sufficient for you. One of the case can be that you have added a new tablespace and then have lost the control file with no or old backup of it which doesn’t know about the newly added tablespace. In this case, its very important that the information of the newly added tablespace must be told to the previous control file and also to use the “using backup controlfile” option with the recover database. If you dont’ have a backup of the old controlfile stored some where already and you are using the option to create it from the scratch using the Trace option, again you need to take care that the path of all the datafiles and the names of all the tablespaces which were added new must be added in the listing of this control file.

To illustrate the point , let’s see the same case here. Following are the files and tablespaces that what we have in our database.

SQL> select name from V$datafile;

NAME

——————————————————————————–/u01/app/oracle/oradata/orcl/system01.dbf

/u01/app/oracle/oradata/orcl/sysaux01.dbf

/u01/app/oracle/oradata/orcl/undotbs01.dbf

/u01/app/oracle/oradata/orcl/users01.dbf

/u01/app/oracle/oradata/orcl/example01.dbf

SQL> select * from V$tablespace;

TS# NAME                           INC BIG FLA ENC

———- —————————— — — — —

0 SYSTEM                         YES NO  YES

1 SYSAUX                         YES NO  YES

2 UNDOTBS1                       YES NO  YES

4 USERS                          YES NO  YES

3 TEMP                           NO  NO  YES

6 EXAMPLE                        YES NO  YES

6 rows selected.

Let’s add into the list one more tablespace.

SQL> create tablespace new

2  datafile ‘/u01/app/oracle/oradata/orcl/new.dbf’ size 1m;

Tablespace created.

SQL> select count(*) from V$tablespace;

COUNT(*)

———-

7

So now we have got one more file in our database about which the backed up control file doesn’t know.

After getting our file lost,

RMAN> restore controlfile from  ‘/u01/app/oracle/flash_recovery_area/ORCL/backupset/2009_11_19/o1_mf_ncnnf_TAG20091119T083712_5j9fm60b_.bkp’;

Starting restore at 19-NOV-09

using channel ORA_DISK_1

channel ORA_DISK_1: restoring control file

channel ORA_DISK_1: restore complete, elapsed time: 00:00:01

output file name=/u01/app/oracle/oradata/orcl/control01.ctl

output file name=/u01/app/oracle/oradata/orcl/control02.ctl

output file name=/u01/app/oracle/oradata/orcl/control03.ctl

Finished restore at 19-NOV-09

SQL> alter database mount;

Database altered.

RMAN> recover database;

Starting recover at 19-NOV-09

released channel: ORA_DISK_1

Starting implicit crosscheck backup at 19-NOV-09

allocated channel: ORA_DISK_1

channel ORA_DISK_1: SID=152 device

allocated channel: ORA_DISK_2

channel ORA_DISK_2: SID=151 device

allocated channel: ORA_DISK_3

channel ORA_DISK_3: SID=150 device

allocated channel: ORA_DISK_4

channel ORA_DISK_4: SID=149 device

Crosschecked 4 objects

Finished implicit crosscheck backup at 19-NOV-09

Starting implicit crosscheck copy at 19-NOV-09

using channel ORA_DISK_1

using channel ORA_DISK_2

using channel ORA_DISK_3

using channel ORA_DISK_4

Finished implicit crosscheck copy at 19-NOV-09

searching for all files in the recovery area

cataloging files…

cataloging done

List of Cataloged Files

=======================

File Name: /u01/app/oracle/flash_recovery_area/ORCL/archivelog/2009_11_19/o1_mf_1_7_5j9fqchp_.arc

File Name: /u01/app/oracle/flash_recovery_area/ORCL/archivelog/2009_11_19/o1_mf_1_6_5j9fq1bs_.arc

File Name: /u01/app/oracle/flash_recovery_area/ORCL/backupset/2009_11_19/o1_mf_annnn_TAG20091119T083931_5j9fqdg7_.bkp

File Name: /u01/app/oracle/flash_recovery_area/ORCL/backupset/2009_11_19/o1_mf_annnn_TAG20091119T083931_5j9fqdrj_.bkp

File Name: /u01/app/oracle/flash_recovery_area/ORCL/backupset/2009_11_19/o1_mf_nnsnf_TAG20091119T083712_5j9fmf0n_.bkp

File Name: /u01/app/oracle/flash_recovery_area/ORCL/backupset/2009_11_19/o1_mf_nnndf_TAG20091119T084437_5j9g0xgk_.bkp

File Name: /u01/app/oracle/flash_recovery_area/ORCL/backupset/2009_11_19/o1_mf_nnndf_TAG20091119T083712_5j9fm1ny_.bkp

File Name: /u01/app/oracle/flash_recovery_area/ORCL/backupset/2009_11_19/o1_mf_annnn_TAG20091119T083921_5j9fq1s8_.bkp

File Name: /u01/app/oracle/flash_recovery_area/ORCL/backupset/2009_11_19/o1_mf_annnn_TAG20091119T083931_5j9fqdb0_.bkp

File Name: /u01/app/oracle/flash_recovery_area/ORCL/backupset/2009_11_19/o1_mf_nnndf_TAG20091119T083712_5j9fm2lr_.bkp

File Name: /u01/app/oracle/flash_recovery_area/ORCL/backupset/2009_11_19/o1_mf_ncnnf_TAG20091119T083712_5j9fm60b_.bkp

File Name: /u01/app/oracle/flash_recovery_area/ORCL/backupset/2009_11_19/o1_mf_annnn_TAG20091119T083931_5j9fqdo9_.bkp

File Name: /u01/app/oracle/flash_recovery_area/ORCL/backupset/2009_11_19/o1_mf_nnndf_TAG20091119T083712_5j9fm1sd_.bkp

using channel ORA_DISK_1

using channel ORA_DISK_2

using channel ORA_DISK_3

using channel ORA_DISK_4

RMAN-06900: WARNING: unable to generate V$RMAN_STATUS or V$RMAN_OUTPUT row

RMAN-06901: WARNING: disabling update of the V$RMAN_STATUS and V$RMAN_OUTPUT rows

ORACLE error from target database:

ORA-19922: there is no parent row with id 0 and level 1

starting media recovery

archived log for thread 1 with sequence 6 is already on disk as file /u01/app/oracle/flash_recovery_area/ORCL/archivelog/2009_11_19/o1_mf_1_6_5j9fq1bs_.arc

archived log for thread 1 with sequence 7 is already on disk as file /u01/app/oracle/flash_recovery_area/ORCL/archivelog/2009_11_19/o1_mf_1_7_5j9fqchp_.arc

archived log for thread 1 with sequence 8 is already on disk as file /u01/app/oracle/oradata/orcl/redo02.log

archived log file name=/u01/app/oracle/flash_recovery_area/ORCL/archivelog/2009_11_19/o1_mf_1_6_5j9fq1bs_.arc thread=1 sequence=6

archived log file name=/u01/app/oracle/flash_recovery_area/ORCL/archivelog/2009_11_19/o1_mf_1_7_5j9fqchp_.arc thread=1 sequence=7

archived log file name=/u01/app/oracle/oradata/orcl/redo02.log thread=1 sequence=8

creating datafile file number=6 name=/u01/app/oracle/oradata/orcl/new.dbf

RMAN-00571: ===========================================================

RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============

RMAN-00571: ===========================================================

RMAN-03002: failure of recover command at 11/19/2009 09:11:12

ORA-01119: error in creating database file ‘/u01/app/oracle/oradata/orcl/new.dbf’

ORA-27038: created file already exists

Additional information: 1

SQL> recover database using backup controlfile;

ORA-00283: recovery session canceled due to errors

ORA-01111: name for data file 6 is unknown – rename to correct file

ORA-01110: data file 6: ‘/u01/app/oracle/product/11.1.0/db_1/dbs/UNNAMED00006′

ORA-01157: cannot identify/lock data file 6 – see DBWR trace file

ORA-01111: name for data file 6 is unknown – rename to correct file

ORA-01110: data file 6: ‘/u01/app/oracle/product/11.1.0/db_1/dbs/UNNAMED00006′

SQL> alter database rename file ‘/u01/app/oracle/product/11.1.0/db_1/dbs/UNNAMED00006′ TO ‘/u01/app/oracle/oradata/orcl/new.dbf’;

Database altered.

RMAN> recover datafile 6;

Starting recover at 19-NOV-09

using target database control file instead of recovery catalog

allocated channel: ORA_DISK_1

channel ORA_DISK_1: SID=152 device

allocated channel: ORA_DISK_2

channel ORA_DISK_2: SID=150 device

allocated channel: ORA_DISK_3

channel ORA_DISK_3: SID=155 device

allocated channel: ORA_DISK_4

channel ORA_DISK_4: SID=149 device

RMAN-06900: WARNING: unable to generate V$RMAN_STATUS or V$RMAN_OUTPUT row

RMAN-06901: WARNING: disabling update of the V$RMAN_STATUS and V$RMAN_OUTPUT rows

ORACLE error from target database:

ORA-19922: there is no parent row with id 0 and level 1

RMAN-00571: ===========================================================

RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============

RMAN-00571: ===========================================================

RMAN-03002: failure of recover command at 11/19/2009 09:17:19

RMAN-06067: RECOVER DATABASE required with a backup or created control file

SQL>  recover database using backup controlfile;

ORA-00279: change 564920 generated at 11/19/2009 08:43:22 needed for thread 1

ORA-00289: suggestion :

/u01/app/oracle/flash_recovery_area/ORCL/archivelog/2009_11_19/o1_mf_1_8_%u_.arcORA-00280: change 564920 for thread 1 is in sequence #8

Specify log: {<RET>=suggested | filename | AUTO | CANCEL}

/u01/app/oracle/oradata/orcl/redo02.log

Log applied.

Media recovery complete.

SQL>

SQL> alter database open resetlogs;

Database altered.

SQL> select * from V$tablespace;

TS# NAME                           INC BIG FLA ENC

———- —————————— — — — —

0 SYSTEM                         YES NO  YES

1 SYSAUX                         YES NO  YES

2 UNDOTBS1                       YES NO  YES

4 USERS                          YES NO  YES

3 TEMP                           NO  NO  YES

6 EXAMPLE                        YES NO  YES

7 NEW                            YES NO  YES

7 rows selected.

P.S: Any misspelled stuff apologies.

Rgds…R

N

The exceptions can be disabled with the following steps:
   
    1. Login to the application as SYSADMIN
    2. Navigate to the responsibility Workflow Administrator Web Applications > Business Events
    3. Search for the Name: oracle.apps.fnd.system.exception
    4. Click Go
    5. Click on Update
    6. Change it to ‘Disable’
    7. Save the changes

/appl/oracle/product/1013/Apache/Apache/bin/apachectl startssl: execing httpd
/appl/oracle/product/1013/Apache/Apache/bin/httpd: error while loading shared libraries:
create the symbolic link

# ln -s /usr/lib/libgdbm.so.2.0.0 /usr/lib/libdb.so.2

chmod 755 /usr/lib/libgdbm.so.2.0.0

chmod 755 /usr/lib/libdb.so.2

As root and the retry the installation.