Archives For March 2012

The OS is prepared and everything is configured. Time to come to the first part of the installation: Oracle Grid Infrastructure

If you did not already transfer the source files to your Virtual Machine do it now. If your workstation runs on linux, here is an example:

scp -P 3000 p10404530_112030_Linux-x86-64_1of7.zip grid@localhost:/opt/oracle/stage
scp -P 3000 p10404530_112030_Linux-x86-64_2of7.zip grid@localhost:/opt/oracle/stage
scp -P 3000 p10404530_112030_Linux-x86-64_3of7.zip grid@localhost:/opt/oracle/stage
scp -P 3000 p13348650_112030_LINUX.zip grid@localhost:/opt/oracle/stage

Windows users may use tools like winscp, filezilla or cygwin to name a few of it.

As the Grid Infrastructure will be installed under the “grid” user, establish a ssh connection to your Virtual Machine and do the installation ( we will do a software only installation here, nothing will be configured for the moment ):

su - grid
cd /opt/oracle/stage
unzip p10404530_112030_Linux-x86-64_3of7.zip
cd grid
./runInstaller \
INVENTORY_LOCATION=/opt/oracle/oraInventory \
SELECTED_LANGUAGES=en \
oracle.install.option=CRS_SWONLY \
ORACLE_BASE=/opt/oracle/product/base \
ORACLE_HOME=/opt/oracle/product/crs/11.2.0.3 \
ORACLE_HOME_NAME=HCRS112030 \
UNIX_GROUP_NAME=oinstall \
oracle.install.asm.OSDBA=asmdba \
oracle.install.asm.OSASM=asmadmin \
FROM_LOCATION=../stage/products.xml \
DECLINE_SECURITY_UPDATES=true \
-ignoreSysPrereqs \
-ignorePrereq \
-waitforcompletion \
-silent
su -
/opt/oracle/oraInventory/orainstRoot.sh
/opt/oracle/product/crs/11.2.0.3/root.sh

That’s it for the software installation. Surprised ? The most important thing to remember if installing oracle software is: Read the installation guides and fulfill all the requirements. If you do this, the installation will be fast and easy.

Some explanations for the above commands:

  • runInstaller: this is oracle installer on linux/unix ( it is setup.exe on windows )
  • INVENTORY_LOCATION: the location where you want to store the central oracle inventory
  • SELECTED_LANGUAGES: the languages you want to install for the product
  • oracle.install.option: how do you want to install ( CRS_SWONLY is for installing the software, but do not configure anything )
  • ORACLE_BASE: the location you want to use for the $ORACLE_BASE
  • ORACLE_HOME: the location where you want to install the product, this is the $ORACLE_HOME
  • UNIX_GROUP_NAME: the OS group you want to own the installed software
  • oracle.install.asm.OSDBA: the OS group you want to use as asm dba
  • oracle.install.asm.OSASM: the OS group you want to use for sysasm
  • FROM_LOCATION: references the xml where all the products are defined
  • DECLINE_SECURITY_UPDATES: true means: do not try to connect to oracle support for downloading the latest patches
  • -ignoreSysPrereqs: ignore the system prerequisites
  • -ignorePrereq: ignore all the other prerequisites
  • -waitforcompletion: wait until the installation completes before returning to the shell
  • -silent: do a silent installation ( do not use the graphical mode )

Take a look at the sample response file if you want to learn about all the other options you can use ( /opt/oracle/stage/grid/response/grid_install.rsp ).

The root scripts:

  • /opt/oracle/oraInventory/orainstRoot.sh : Creates the pointer to the oraInventory and sets the correct permissions
  • /opt/oracle/product/crs/11.2.0.3/root.sh: Calls various other scripts to set up the installation

Lets see if if everything works as expected and configure the cluster stack for single node ( The Grid Infrastructure sofware is exactly the same for Cluster and Single Node installations. For single node installations, oracle calls this “Oracle Restart” ):

su -
/opt/oracle/product/crs/11.2.0.3/crs/install/roothas.pl

The output should like this:

[root@oracleplayground stage]# /opt/oracle/product/crs/11.2.0.3/crs/install/roothas.pl
Using configuration parameter file: /opt/oracle/product/crs/11.2.0.3/crs/install/crsconfig_params
Creating trace directory
User ignored Prerequisites during installation
LOCAL ADD MODE
Creating OCR keys for user 'grid', privgrp 'oinstall'..
Operation successful.
LOCAL ONLY MODE
Successfully accumulated necessary OCR keys.
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
CRS-4664: Node oracleplayground successfully pinned.
Adding Clusterware entries to inittab
oracleplayground 2012/03/23 09:18:15 /opt/oracle/product/crs/11.2.0.3/cdata/oracleplayground/backup_20120323_091815.olr
Successfully configured Oracle Grid Infrastructure for a Standalone Server

If you had troubles you can deconfigure the whole stack and try again:

su -
/opt/oracle/product/crs/11.2.0.3/crs/install/roothas.pl -deconfig -force
### check the logfiles ( $ORACLE_HOME/log/oracleplayground/* ) for the issue and try to configure again
/opt/oracle/product/crs/11.2.0.3/crs/install/roothas.pl

There is one important thing to note: Take a look at this line:

Using configuration parameter file: /opt/oracle/product/crs/11.2.0.3/crs/install/crsconfig_params

This is the one and only configuration file you will need to adjust if setting up the stack for a cluster ( If you want to configure for a cluster rootcrs.pl will be used instead of roothas.pl, but the configuration file keeps the same ).
In our case we do not need to modify it, as we will work with a single node.

What happened here:
The script created same trace directories, the olr ( oracle local registry ), pinned the node and added a call to inittab. Can verify this with:

cat /etc/inittab | grep ohas

If you take a look at your processes now, you’ll notice that several oracle processes started up:

ps -ef | grep ora

This are the basic cluster stack processes which will manage you oracle infrastructure.

  • ohasd.bin: the process resonsible for starting up all the cluster processes
  • oraagent.bin: will spawn agents to start resources
  • evmd.bin: the event volume manager daemon
  • evmlogger.bin: spawned by evmd to log events

To make a life a bit easier for us, lets define some aliases and environment variables for the grid user:

su - grid
echo "ORACLE_HOME=/opt/oracle/product/crs/11.2.0.3
ORACLE_BASE=/opt/oracle/base
ORACLE_SID=+ASM
PATH=$ORACLE_HOME/bin:$PATH
LD_LIBRARY_PATH=$ORACLE_HOME/lib:$LD_LIBRARY_PATH
PS1='\[33[1;34m\]${ORACLE_SID}\[33[1;33m\]@\[33[1;32m\]`uname -n`\[33[0m\] $LOGNAME:$PWD $ '
export ORACLE_HOME ORACLE_BASE ORACLE_SID PATH LD_LIBRARY_PATH PS1
alias crsstat='crsctl status resource -t'" >> ~/.bash_profile

It is a good idea to reboot your Virtual Machine now and to check if the cluster stack comes up again.

su -
reboot

Once the Virtual Machine is up and running again ( notice that it will take some time for the oracle processes to come up ), verify if you can interact with the stack ( we will use the alias defined above for this ):

su - grid
crsstat

If everything is fine, the output should like this:

--------------------------------------------------------------------------------
NAME TARGET STATE SERVER STATE_DETAILS
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.ons
OFFLINE OFFLINE oracleplayground
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.cssd
1 OFFLINE OFFLINE
ora.diskmon
1 OFFLINE OFFLINE
ora.evmd
1 ONLINE ONLINE oracleplayground

To summarize: We installed the Grid Infrastructure software without doing any configuration initially and executed the root scripts for setting up permissions, the oracle inventory and the grid infrastructure software. roothas.pl did the configuration of the cluster stack, added an entry to the inittab and generated the oracle local registry. We did a reboot to check if everything comes up again and defined some environment variables and aliases to make life a bit easier for us.

If you’ll ask yourself what the crsctl is for: We will come back to all the commands in next posts and introduce them when appropriate.

In the next post we will apply the first patch to the grid infrastructure and will learn about the oracle patch utility opatch.

Again, remember to take a snapshot of your Virtual Machine if you want to save your work or if you want to revert to this point. Delete the source files if you do not want to become your Virtual Machine too big before:

su - grid
cd /opt/oracle/stage/
rm -rf grid/ p10404530_112030_Linux-x86-64_3of7.zip

On your workstation create the snapshot:

vboxmanage snapshot oracleplayground take "after gi installation and configuration"

 

 

Advertisements

We did the initial operating system setup in the last post. Before we can install the oracle software we need to prepare the operating system for oracle. I will assume your Virtual Machine is up and running and you have a working ssh connection established to it ( as user root ).

Because it is a training environment turn off the firewall and selinux

chkconfig iptables off
service iptables stop
vi /etc/sysconfig/selinux
## SELINUX=disabled 

The first requirement we need to fulfill is to install all the software packages oracle requires to successfully install. A list of the packages can be found in the oracle documentation.

yum install -y binutils compat-libstdc++-33 elfutils-libelf elfutils-libelf-devel gcc gcc-c++ glibc glibc-common glibc-devel glibc-headers ksh libaio libaio-devel libgcc libstdc++ libstdc++-devel make sysstat libcap openssh-clients unzip numactl man-pages man

Documentation links:

Next we need to set the kernel parameters ( again, see the documentation for details ):

echo "fs.file-max = 6815744
fs.aio-max-nr = 1048576
net.core.wmem_max = 1048576
net.ipv4.ip_local_port_range = 1024 65000
net.core.wmem_default = 262144
net.core.wmem_max = 1048576
net.core.rmem_default = 262144
net.core.rmem_max = 4194304
kernel.sem = 250 32000 32 1024 " >> /etc/sysctl.conf
/sbin/sysctl -p

For an excellent description of the shared memory parameters take a look at orafaq

As oracle will run under it’s own user accounts and groups we need to define them ( again, see the documentation for details )

/usr/sbin/groupadd -g 501 oinstall
/usr/sbin/groupadd -g 502 dba
/usr/sbin/groupadd -g 503 oper
/usr/sbin/groupadd -g 504 asmadmin
/usr/sbin/groupadd -g 506 asmdba
/usr/sbin/groupadd -g 505 asmoper
/usr/sbin/useradd -u 502 -g oinstall -G asmadmin,asmdba,asmoper,dba -m -c "Grid Infratructure Software owner" grid
/usr/sbin/useradd -u 503 -g oinstall -G dba,oper,asmdba,asmadmin -m -c "Database Software owner" oracle
passwd oracle
passwd grid

Because Oracle will need resources from the OS, lets set the resource limits ( again, see the documentation for details )

echo "oracle soft nproc 2047
oracle hard nproc 16384
oracle soft nofile 1024
oracle hard nofile 65536
oracle soft stack 10240
grid soft nproc 2047
grid hard nproc 16384
grid soft nofile 1024
grid hard nofile 65536
grid soft stack 10240
grid hard stack 32768" >> /etc/security/limits.conf

Make sure we have a correct hosts entry:

echo "10.0.2.15   oracleplayground oracleplayground.fun" >> /etc/hosts

Almost finished. What’s still missing are the directories for the Oracle Inventory ( the central location where oracle records the products installed ) and the software locations. For a description see the documentation.
Note: As this will be a training environment we will not create separate mount points as you would do on a production system.

mkdir /opt/oracle
mkdir /opt/oracle/stage
mkdir /opt/oracle/product
mkdir /opt/oracle/oraInventory
mkdir /opt/oracle/product/base
mkdir /opt/oracle/product/crs
chown grid:oinstall /opt/oracle
chown grid:oinstall /opt/oracle/stage
chown grid:oinstall /opt/oracle/product
chown grid:oinstall /opt/oracle/oraInventory
chown grid:oinstall /opt/oracle/product/crs
chown oracle:oinstall /opt/oracle/product/base
chmod 770 /opt/oracle
chmod 770 /opt/oracle/stage
chmod 770 /opt/oracle/oraInventory
chmod 770 /opt/oracle/product
chmod 770 /opt/oracle/product/base

That’s it. The OS is ready for the oracle installation.

To be prepared for the next step, you’ll need to download the below files from oracle support. On the “Patches and Updates” tab search for patches 10404530 and 13348650.

  • p10404530_112030_Linux-x86-64_1of7.zip: Oracle Database 11gR2 11.2.0.3 Part 1 of 2
  • p10404530_112030_Linux-x86-64_2of7.zip: Oracle Database 11gR2 11.2.0.3 Part 2 of 2
  • p10404530_112030_Linux-x86-64_3of7.zip: Oracle Grid Infrastructure 11gR2 11.2.0.3 Part 1 of 1
  • p13348650_112030_LINUX.zip: Oracle Grid Infrastructure Patchset Update 11.2.0.3.1 ( includes database PSU )

If you do not have an oracle support account you can go with the base release ( 11.2.0.1.0 ) which can be downloaded from Oracle Technology Network. The procedure for installing the software remains the same but you will not be able apply the PSU ( Patch Set Update ) from above.

For your convenience or if you’d like to automate the tasks performed in this post, here is the complete script:

#!/bin/bash
if [ $(id -un) != "root" ]; then
echo "you must run this as root"
exit 0
fi
# install the required software
yum install -y binutils compat-libstdc++-33 elfutils-libelf elfutils-libelf-devel gcc gcc-c++ glibc glibc-common glibc-devel glibc-headers ksh libaio libaio-devel libgcc libstdc++ libstdc++-devel make sysstat libcap openssh-clients unzip numactl man-pages man
# set the kernel parameters
echo "fs.file-max = 6815744
fs.aio-max-nr = 1048576
net.core.wmem_max = 1048576
net.ipv4.ip_local_port_range = 1024 65000
net.core.wmem_default = 262144
net.core.wmem_max = 1048576
net.core.rmem_default = 262144
net.core.rmem_max = 4194304
kernel.sem = 250 32000 32 1024 " >> /etc/sysctl.conf
/sbin/sysctl -p
# add groups and users
/usr/sbin/groupadd -g 501 oinstall
/usr/sbin/groupadd -g 502 dba
/usr/sbin/groupadd -g 503 oper
/usr/sbin/groupadd -g 504 asmadmin
/usr/sbin/groupadd -g 506 asmdba
/usr/sbin/groupadd -g 505 asmoper
/usr/sbin/useradd -u 502 -g oinstall -G asmadmin,asmdba,asmoper,dba -m -c "Grid Infratructure Software owner" grid
/usr/sbin/useradd -u 503 -g oinstall -G dba,oper -m -c "Database Software owner" oracle
passwd oracle
passwd grid
# set the resource limits
echo "oracle soft nproc 2047
oracle hard nproc 16384
oracle soft nofile 1024
oracle hard nofile 65536
oracle soft stack 10240
grid soft nproc 2047
grid hard nproc 16384
grid soft nofile 1024
grid hard nofile 65536
grid soft stack 10240
grid hard stack 32768" >> /etc/security/limits.conf
# create the directories
mkdir /opt/oracle
mkdir /opt/oracle/stage
mkdir /opt/oracle/product
mkdir /opt/oracle/oraInventory
mkdir /opt/oracle/product/base
mkdir /opt/oracle/product/crs
chown grid:oinstall /opt/oracle
chown grid:oinstall /opt/oracle/stage
chown grid:oinstall /opt/oracle/product
chown grid:oinstall /opt/oracle/oraInventory
chown grid:oinstall /opt/oracle/product/crs
chown oracle:oinstall /opt/oracle/product/base
chmod 770 /opt/oracle
chmod 770 /opt/oracle/stage
chmod 770 /opt/oracle/oraInventory
chmod 770 /opt/oracle/product
chmod 770 /opt/oracle/product/base

Remember to take a snapshot of your Virtual Machine if you want to save your work or if you want to revert to this point:

vboxmanage snapshot oracleplayground take "after OS configuration"

… and as a dba you should have an understanding on how the operation works and what needs to be configured.

The first thing you need to do if you want to install oracle software is to prepare your operating system. For the little series I will use CentOS 5.8 x86_64 because it’s free and compatible to RedHat and Oracle Linux.

What you will need:

If you prefer VMWare or any other virtualization software you can do so. Just make sure to create a new Virtual Machine that matches the one created here.

Once you have installed the VirtualBox software you are ready to create new Virtual Machine. The first steps will show the command line usage for creating the Virtual Machine and mostly will go with the defaults options. For documenting the OS setup itself i decided to provide the screen-shots as automating the OS setup is not in scope of this post.

Note: Everything shown for the command line can be done with the GUI, too. I personally prefer the command line as it is easier to document and faster once you’re common with the syntax.

If you have any questions on the commands used below, please take a look at the Virtual Box documentation.

Create a new CentOS 64bit Virtual Machine and name it oracleplayground ( or whatever name you like, just make sure you replace oracleplayground with the name of your choice in the below commands ):

vboxmanage createvm --name oracleplayground --ostype redhat_64 --register

Modify the VM to use 1512 mb of memory:

vboxmanage modifyvm oracleplayground --memory 1512

Create a new virtual hard disk which will contain the OS and the oracle software ( you can specify any location you like for the –filename parameter ):

vboxmanage createhd --filename ~/VirtualBox\ VMs/oracleplayground/oracleplayground_dh1.vdi --size=30720

Create a new virtual storage controller:

vboxmanage storagectl oracleplayground --name "IDE Controller" --add ide --controller PIIX4

Attach the virtual hard disk to the newly created virtual controller:

vboxmanage storageattach oracleplayground --storagectl "IDE Controller" --port 1 --device 0 --type hdd --medium ~/VirtualBox\ VMs/oracleplayground/oracleplayground_dh1.vdi

Attach the Centos Linux ISO to the virtual machine:

vboxmanage storageattach oracleplayground --storagectl "IDE Controller" --port 1 --device 1 --type dvddrive --medium ~/Downloads/CentOS-5.8-x86_64-bin-DVD-1of2.iso

As we will use NAT for the Virtual Machine we need to configure port-forwarding. This enables us to connect to the VM using ssh once the operating system is installed. This command will forward all requests to your workstation on Port 3000 to the VMs Port 22 ( the default port the ssh daemon listens on ):

vboxmanage modifyvm "oracleplayground" --natpf1 "guestssh,tcp,,3000,,22"

Finally start the VM:

vboxmanage startvm oracleplayground

From this point onwards there will be some screen-shots to document the OS setup ( You would use tools like kickstart for installating a qualified server ).

Note: If your cursor is catched inside the Virtual Machine window you can use the “right CRTL” key to return to your workstation.






















This was the last screenshot and for all remaining task I assume you’re logged in as root and provide the command line usage to proceed:

From now on you should be able to login through ssh. If you workstation runs on Linux/Solaris/OS X this should get you the connection:

ssh -o Port=3000 root@localhost

If you are working on windows, here is the putty example:

If you want to bring your system up to date now, you can use yum to download and install the latest patches:

Do not forget to set the proxy, if you need to use one ( btw. you can permanently set the proxy for yum in /etc/yum.conf ):

export http_proxy=proxy.yourdomain.com:8080

Update the system to the latest patchlevel:

yum update

Now is the perfect time to take a snapshot of your Virtual Machine. This will be very helpful in case you want to come back to this point later or if you damage your VM during one of the next steps. Reverting a snapshot takes minutes, doing all the work above again takes hours.

On your workstation take a snapshot of the Virtual Machine:

vboxmanage snapshot oracleplayground take "after OS installation"

If you want to export the Virtual Machine as appliance, you can do this, too. Appliances are useful if you want to import the whole Virtual Machine to a new one or if you want to transfer it to a another workstation.

vboxmanage export "oracleplayground" --output oracleplayground.ovf

For importing it ( on the same or any other host ):

vboxmanage import "oracleplayground.ovf"

That’s it for the basic Operating System setup. The next post will be about the various configuration tasks that need to be done before we may start installing the oracle software.

… there are plenty of excellent oracle posts out there. why another one?

When I started working with oracle I had one big problem: There is so much information around that it’s hard to decide where to begin. I started as most of the people would do:

… had a question or issue -> go to an internet search engine -> search -> millions of different answers, some correct, some useful, a lot of them misleading or even wrong

What I always was missing was a clean guide on how to setup a training environment that can be used as a base for trying all the interesting things. This includes, for example:

  • Setting up the Operating System
  • Preparing the Operating for Oracle
  • Installing the Oracle Software
  • Patching the Oracle Software
  • Create an ASM Instance and configure the Grid Infrastructure
  • Create and configure the database instance
  • Start playing with it …. and provide some examples

As you can see from the above list, this are not advanced topics, so we are back to what this blog is about: It is for all those who are at the beginning of their oracle experience. You will see as less screenshots as possible because I believe that you’re not going to learn how things work if you trust the graphical tools. Another big point which still is mess if I look around: Documentation. If you put your work in a script you are documented. People using the graphical tools tend to not document their work and this can be very dangerous if others try to continue. In qualified environments it’s even strictly forbidden to do anything on the server or database without going through an approval process. And it may be even forbidden to use any of the graphical tools.

So let’s try the experiment: Feel free to comment, feel free to correct me, feel free to ask…. Hopefully this is useful to some of you

I will start this blog with a little series on how you may prepare your os, install the oracle software and do the configuration tasks you need to do for getting an oracle instance up and running.

… and please don’t blame me for my English :)