Currently I am thinking on how to setup an oracle lab environment with minimal efforts providing the same configuration and standards to all lab machines. There are several tools around which might help in solving this:
docker is much in the press currently. but docker is not designed for systems that use shared storage like the grid infrastructure and rac. this might be possible somehow but it seemed too much of a hack to me. the same with vagrant. as soon as you want to have shared storage it gets difficult.
having thought about all the pros and cons I decided to give cobbler a try. as I do not have a real server to play around the cobbler machine as well as the test clients will be virtual box machines using the host only network adapter.
the below script will setup a virtual box vm using the vboxmanage command. just copy it to your linux workstation and adjust the variables at the top of the script (the script should be easily portable to windows ):
#!/bin/bash
################## CONFIGURATION SECTION START ##############################
## The name for the VM in VirtualBox
VM_NAME="oel6_staging"
## The VMHome for VirtualBox, usually in $HOME
VM_HOME="/media/dwe/My Passport/vm/${VM_NAME}"
## The hard disk to use for the VM, can be anywhere but might
## use up to 60gb. We need plenty of space for hosting the
## the OEL 6.6 yum repository and any source files for the oracle
## database and gi installation zip files
VM_HARDDISK="/media/dwe/My Passport/vm/${VM_NAME}/oel_staging.vdi"
## The path to the OEL 6.6 iso we are installing from
## The ISO can be downloaded from oracle edelivery at
## https://edelivery.oracle.com/linux
BOOTIMAGEFILE="$HOME/Downloads/OracleLinux-R6-U6-Server-x86_64-dvd.iso"
################## CONFIGURATION SECTION END ##############################
################ NO NEED TO EDIT FROM HERE ON ############################X
## Clean up everything before starting the setup
echo ".. Clean up everything before starting the setup"
vboxmanage unregistervm ${VM_NAME} --delete >> /dev/null 2>&1
vboxmanage closemedium disk "${VM_HARDDISK}" --delete >> /dev/null 2>&1
rm -f "${VM_HOME}/*"
## Creating Virtual Box Maschine
echo ".. Creating Virtual Box Maschine"
vboxmanage createvm --name ${VM_NAME} --register --ostype Oracle_64
vboxmanage modifyvm ${VM_NAME} --boot1 disk
vboxmanage modifyvm ${VM_NAME} --boot2 dvd
## Creating Hard Disk for the Virtual Machine
echo "Creating Hard Disk for the Virtual Machine"
vboxmanage createhd --filename "${VM_HARDDISK}" --size 61440
## Creating a storage controller and attaching the Hard Disk to the VM
echo "Creating a storage controller and attaching the Hard Disk to the VM"
vboxmanage storagectl ${VM_NAME} --name ctl1 --add sata
vboxmanage storageattach ${VM_NAME} --storagectl ctl1 --type hdd --medium "${VM_HARDDISK}" --port 1
## Creating DVD Drive and attachinf the ISO
echo "Creating DVD Drive and attaching the ISO"
vboxmanage storagectl ${VM_NAME} --name ctl2 --add ide
vboxmanage storageattach ${VM_NAME} --storagectl ctl2 --type dvddrive --port 1 --device 1 --medium "${BOOTIMAGEFILE}"
## Setting VM parameters
echo "Setting VM parameters"
vboxmanage modifyvm ${VM_NAME} --memory 1024 --cpus 1 --nic1 nat --cableconnected1 on
vboxmanage modifyvm ${VM_NAME} --natpf1 "guestssh,tcp,,2222,,22"
vboxmanage modifyvm ${VM_NAME} --audio none
vboxmanage modifyvm ${VM_NAME} --usb off
echo "Booting and installing VM"
vboxmanage startvm ${VM_NAME} --type sdl
once you execute the script the vm should start with the oracle linux iso attached to it. basically just click through the installer using a minimal installation and you’re done. for your reference here is a pdf with some screenshots:
https://danielwestermann.files.wordpress.com/2014/12/setup_oel66_staging_env_vm.pdf
note: the machine will use the NAT adapter for now as I need to install some packges from the internet. this would not be possible using the host only adapter we will use later on.
As you can see from the last screenhot in the attached pdf we only have the loopback adapter up and running. To enable the network click into the VM Console window and adjust the
ONBOOT parameter to yes:
cat /etc/sysconfig/network-scripts/ifcfg-eth0 | grep ONBOOT
ONBOOT=no
Once done, restart the network and check if eth0 is up:
service network restart
...
ifconfig
eth0 Link encap:Ethernet HWaddr 08:00:27:21:56:79
inet addr:10.0.2.15 Bcast:10.0.2.255 Mask:255.255.255.0
inet6 addr: fe80::a00:27ff:fe21:5679/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:398 errors:0 dropped:0 overruns:0 frame:0
TX packets:252 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:38219 (37.3 KiB) TX bytes:41027 (40.0 KiB)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)
From now on you are able to connect from your workstation:
ssh -o Port=2222 root@localhost
The 2222 port comes from the NAT network we defined for the VM:
vboxmanage modifyvm ${VM_NAME} --natpf1 "guestssh,tcp,,2222,,22"
To avoid any issues with iptables and selinux turn it off and reboot the machine:
chkconfig iptables off
service iptables stop
note: as of rhel/centos/oel 6.6 (or even earlier) is not possible to turn of selinux by editing the /etc/sysconfig/selinux file. to permanently turn it of we’ll need to adjust the grub configuration. To do this just add “enforcing=0” to each kernel line in /etc/grub.cong and reboot the machine:
grep "enforcing=0" /etc/grub.conf
...
kernel /vmlinuz-3.8.13-55.1.1.el6uek.x86_64 ro root=/dev/mapper/vg_oelstaging-lv_root rd_NO_LUKS LANG=en_US.UTF-8 rd_LVM_LV=vg_oelstaging/lv_root rd_NO_MD SYSFONT=latarcyrheb-sun16 crashkernel=auto KEYBOARDTYPE=pc KEYTABLE=sg rd_NO_DM rd_LVM_LV=vg_oelstaging/lv_swap rhgb quiet enforcing=0
kernel /vmlinuz-2.6.32-504.1.3.el6.x86_64 ro root=/dev/mapper/vg_oelstaging-lv_root rd_NO_LUKS LANG=en_US.UTF-8 rd_LVM_LV=vg_oelstaging/lv_root rd_NO_MD SYSFONT=latarcyrheb-sun16 crashkernel=auto KEYBOARDTYPE=pc KEYTABLE=sg rd_NO_DM rd_LVM_LV=vg_oelstaging/lv_swap rhgb quiet enforcing=0
kernel /vmlinuz-3.8.13-44.1.1.el6uek.x86_64 ro root=/dev/mapper/vg_oelstaging-lv_root rd_NO_LUKS LANG=en_US.UTF-8 rd_LVM_LV=vg_oelstaging/lv_root rd_NO_MD SYSFONT=latarcyrheb-sun16 crashkernel=auto KEYBOARDTYPE=pc KEYTABLE=sg rd_NO_DM rd_LVM_LV=vg_oelstaging/lv_swap rhgb quiet enforcing=0
kernel /vmlinuz-2.6.32-504.el6.x86_64 ro root=/dev/mapper/vg_oelstaging-lv_root rd_NO_LUKS LANG=en_US.UTF-8 rd_LVM_LV=vg_oelstaging/lv_root rd_NO_MD SYSFONT=latarcyrheb-sun16 crashkernel=auto KEYBOARDTYPE=pc KEYTABLE=sg rd_NO_DM rd_LVM_LV=vg_oelstaging/lv_swap rhgb quiet enforcing=0
So our base installation is ready. Let’s get the latest updates and install all the packages we need for operating cobbler.
Note: If you are behind a proxy, now it is the time to set it for yum being able to connect to the internet:
export http_proxy=http://[proxy]:[port]
export https_proxy=http://[proxy]:[port]
Do an update of all the packages currently installed:
yum -y update
Cobbler is available through the epel repository which we need to add first:
rpm -Uvh http://download.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm
Install the required packages:
yum install -y openssh-clients wget cobbler cobbler-web syslinux pykickstart createrepo httpd mkisofs mod_wsgi mod_ssl python-cheetah python-netaddr python-simplejson python-urlgrabber PyYAML rsync syslinux tftp-server yum-utils Django dhcp
As everything we need is now installed and we do not need to connect to the internet anymore let’s switch the VM to the host only network.
Shutdown the machine:
shutdown -h 0
From your workstation adjust the VM configuration and start it up again:
vboxmanage modifyvm oel_staging --nic1 hostonly --hostonlyadapter1 vboxnet0
nohup vboxmanage startvm oel_staging --type sdl &
Click in the VM Console and check your current ip address (should be 192.168.56.101 if you did not change the virtual box’ host only network):
ifconfig
eth0 Link encap:Ethernet HWaddr 08:00:27:D6:B2:CE
inet addr:192.168.56.101 Bcast:192.168.56.255 Mask:255.255.255.0
inet6 addr: fe80::a00:27ff:fed6:b2ce/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:75 errors:0 dropped:0 overruns:0 frame:0
TX packets:50 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:13775 (13.4 KiB) TX bytes:10020 (9.7 KiB)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)
Connections from the workstation to the VM from now on are established like:
ssh root@192.168.56.101
To be sure this IP does not change we change the VM network confiruation for eth0 from dhcp to static and restart the network:
# adjust the ifcfg-eth0 file
cat /etc/sysconfig/network-scripts/ifcfg-eth0
DEVICE=eth0
TYPE=Ethernet
ONBOOT=yes
NM_CONTROLLED=yes
BOOTPROTO=static
IPADDR=192.168.56.101
# restart the network
service network restart
Time to make some basic cobbler configurations. The main cobbler configuration file is /etc/cobbler/settings.
For any new system that get’s generated by cobbler there needs to be a root password. Let’s create one and write it to the settings file (I’ll use admin123):
openssl passwd -1
Password:
Verifying - Password:
$1$5E2W.o99$UTCUZnZQrFFQuPpvgILHD.
Put the generated password to the settings file (if you do not want to use sed just edit the file with your favorite editor):
sed -i 's/default_password_crypted.*$/default_password_crypted\: \"\$1\$5E2W.o99\$UTCUZnZQrFFQuPpvgILHD\."/' /etc/cobbler/settings
Next we need to adjust the server settings:
sed -i 's/server\: 127.0.0.1/server\: 192.168.56.101/' /etc/cobbler/settings
sed -i 's/manage_dhcp\: 0/manage_dhcp\: 1/' /etc/cobbler/settings
This should result in the following:
egrep "server|manage_dhcp" /etc/cobbler/settings | grep -v "#"
build_reporting_smtp_server: "localhost"
default_name_servers: []
ldap_server: "ldap.example.com"
manage_dhcp: 1
next_server: 192.168.56.101
redhat_management_server: "xmlrpc.rhn.redhat.com"
server: 192.168.56.101
The next part is to modify the dhcp template (although we do not use it for the scope of this post). The “subnet” block should look like this:
subnet 192.168.56.0 netmask 255.255.255.0 {
option routers 192.168.56.1;
option domain-name-servers 192.168.56.1;
option subnet-mask 255.255.255.0;
range dynamic-bootp 192.168.56.200 192.168.56.254;
default-lease-time 21600;
max-lease-time 43200;
next-server $next_server;
class "pxeclients" {
match if substring (option vendor-class-identifier, 0, 9) = "PXEClient";
if option pxe-system-type = 00:02 {
filename "ia64/elilo.efi";
} else if option pxe-system-type = 00:06 {
filename "grub/grub-x86.efi";
} else if option pxe-system-type = 00:07 {
filename "grub/grub-x86_64.efi";
} else {
filename "pxelinux.0";
}
}
}
Now we are ready to enable and start the services:
chkconfig cobblerd on
chkconfig httpd on
service cobblerd start
service httpd start
Cobbler provides a check command which we now should use to see if we are fine with our configuration:
cobbler check
The following are potential configuration items that you may want to fix:
1 : SELinux is enabled. Please review the following wiki page for details on ensuring cobbler works correctly in your SELinux environment:
https://github.com/cobbler/cobbler/wiki/Selinux
2 : dhcpd is not installed
3 : some network boot-loaders are missing from /var/lib/cobbler/loaders, you may run 'cobbler get-loaders' to download them, or, if you only want to handle x86/x86_64 netbooting, you may ensure that you have installed a *recent* version of the syslinux package installed and can ignore this message entirely. Files in this directory, should you want to support all architectures, should include pxelinux.0, menu.c32, elilo.efi, and yaboot. The 'cobbler get-loaders' command is the easiest way to resolve these requirements.
4 : change 'disable' to 'no' in /etc/xinetd.d/rsync
5 : debmirror package is not installed, it will be required to manage debian deployments and repositories
6 : fencing tools were not found, and are required to use the (optional) power management features. install cman or fence-agents to use them
Restart cobblerd and then run 'cobbler sync' to apply changes.
We safely can ignore those messages for the moment.
Let’s do the first sync:
cobbler sync
...
task started: 2014-12-12_132415_sync
task started (id=Sync, time=Fri Dec 12 13:24:15 2014)
running pre-sync triggers
cleaning trees
removing: /var/lib/tftpboot/pxelinux.cfg/default
removing: /var/lib/tftpboot/grub/efidefault
removing: /var/lib/tftpboot/grub/images
removing: /var/lib/tftpboot/s390x/profile_list
copying bootloaders
copying: /usr/share/syslinux/pxelinux.0 -> /var/lib/tftpboot/pxelinux.0
copying: /usr/share/syslinux/menu.c32 -> /var/lib/tftpboot/menu.c32
copying: /usr/share/syslinux/memdisk -> /var/lib/tftpboot/memdisk
copying distros to tftpboot
copying images
generating PXE configuration files
generating PXE menu structure
rendering DHCP files
generating /etc/dhcp/dhcpd.conf
rendering TFTPD files
generating /etc/xinetd.d/tftp
cleaning link caches
running post-sync triggers
running python triggers from /var/lib/cobbler/triggers/sync/post/*
running python trigger cobbler.modules.sync_post_restart_services
running: dhcpd -t -q
received on stdout:
received on stderr:
running: service dhcpd restart
received on stdout: Starting dhcpd: [ OK ]
received on stderr:
running shell triggers from /var/lib/cobbler/triggers/sync/post/*
running python triggers from /var/lib/cobbler/triggers/change/*
running python trigger cobbler.modules.scm_track
running shell triggers from /var/lib/cobbler/triggers/change/*
Looks fine. Time to import our first distribution (which will be same ISO we used for setting up the VM). Go back to your workstation and copy the oel 6.6 iso to the vm:
scp OracleLinux-R6-U6-Server-x86_64-dvd.iso root@192.168.56.101:/var/tmp/
Back into the VM we’ll import this iso into cobbler:
mount -o loop,ro /var/tmp/OracleLinux-R6-U6-Server-x86_64-dvd.iso /mnt
cobbler import --name=oel66 --arch=x86_64 --path=/mnt
...
*** TASK COMPLETE ***
So, we have our first distro in cobbler:
cobbler distro list
oel66-x86_64
cobbler distro report --name=oel66-x86_64
Name : oel66-x86_64
Architecture : x86_64
TFTP Boot Files : {}
Breed : redhat
Comment :
Fetchable Files : {}
Initrd : /var/www/cobbler/ks_mirror/oel66-x86_64/images/pxeboot/initrd.img
Kernel : /var/www/cobbler/ks_mirror/oel66-x86_64/images/pxeboot/vmlinuz
Kernel Options : {}
Kernel Options (Post Install) : {}
Kickstart Metadata : {'tree': 'http://@@http_server@@/cblr/links/oel66-x86_64'}
Management Classes : []
OS Version : rhel6
Owners : ['admin']
Red Hat Management Key : <>
Red Hat Management Server : <>
Template Files : {}
Time to create a new system:
cobbler system add --name=test --profile=oel66-x86_64
cobbler system list
Let’s do some configuration for the newly created system:
cobbler system edit --name=test --interface=eth0 --ip-address=192.168.56.201 --netmask=255.255.255.0 --static=1 --dns-name=test.lab.ch
cobbler system edit --name=test --gateway=192.168.56.1 --hostname=test.lab.ch
cobbler system report test
Let’s build the iso:
mkdir /var/www/html/iso_store/
cobbler buildiso --systems="test" --iso=/var/www/html/iso_store/test.iso
Now we have an iso we can boot from and everything is retrieved from the cobbler server. Let’s try it on our workstation with the following script:
#!/bin/bash
################## CONFIGURATION SECTION START ##############################
## The name for the VM in VirtualBox
VM_NAME="oel6_test"
## The VMHome for VirtualBox, usually in $HOME
VM_HOME="/media/dwe/My Passport/vm/${VM_NAME}"
## The hard disk to use for the VM, can be anywhere but might
## use up to 60gb. We need plenty of space for hosting the
## the OEL 6.6 yum repository and any source files for the oracle
## database and gi installation zip files
VM_HARDDISK="/media/dwe/My Passport/vm/${VM_NAME}/oel_test.vdi"
BOOTIMAGEFILE="test.iso"
# we get the boot iso directly from the cobbler server
BOOTIMAGEURL="http://192.168.56.101/iso_store/${BOOTIMAGEFILE}"
################## CONFIGURATION SECTION END ##############################
################ NO NEED TO EDIT FROM HERE ON ############################X
## Clean up everything before starting the setup
echo ".. Clean up everything before starting the setup"
vboxmanage unregistervm ${VM_NAME} --delete >> /dev/null 2>&1
vboxmanage closemedium disk "${VM_HARDDISK}" --delete >> /dev/null 2>&1
rm -rf "${VM_HOME}"
mkdir -p "${VM_HOME}"
## Creating Virtual Box Maschine
echo ".. Creating Virtual Box Maschine"
vboxmanage createvm --name ${VM_NAME} --register --ostype Oracle_64
vboxmanage modifyvm ${VM_NAME} --boot1 disk
vboxmanage modifyvm ${VM_NAME} --boot2 dvd
## Creating Hard Disk for the Virtual Machine
echo "Creating Hard Disk for the Virtual Machine"
vboxmanage createhd --filename "${VM_HARDDISK}" --size 8192
## get the boot iso
cd "${VM_HOME}"
wget ${BOOTIMAGEURL}
## Creating a storage controller and attaching the Hard Disk to the VM
echo "Creating a storage controller and attaching the Hard Disk to the VM"
vboxmanage storagectl ${VM_NAME} --name ctl1 --add sata
vboxmanage storageattach ${VM_NAME} --storagectl ctl1 --type hdd --medium "${VM_HARDDISK}" --port 1
## Creating DVD Drive and attachinf the ISO
echo "Creating DVD Drive and attaching the ISO"
vboxmanage storagectl ${VM_NAME} --name ctl2 --add ide
vboxmanage storageattach ${VM_NAME} --storagectl ctl2 --type dvddrive --port 1 --device 1 --medium "${VM_HOME}/${BOOTIMAGEFILE}"
## Setting VM parameters
echo "Setting VM parameters"
vboxmanage modifyvm ${VM_NAME} --memory 1024 --cpus 1 --nic1 hostonly --cableconnected1 on
vboxmanage modifyvm ${VM_NAME} --audio none
vboxmanage modifyvm ${VM_NAME} --usb off
echo "Booting and installing VM"
vboxmanage startvm ${VM_NAME} --type sdl
Choose “test” in the boot screen, sit back and see how the vm gets installed :)
Time to close this first post on the topic. In the next part I’ll show how to create customer cobbler profile, attach a custom kickstart file to it and fire up a VM that has all the oracle required OS packages and users alreay installed and configured.
