Oracle: Unbreak my Linux Heart

If you do an installation of a Linux server over the network (called a "network install"), you'll save yourself a lot of time: you don't need to change CDs. You can also automate installation with a so-called "scripted install" using a kickstart file. This will get you an "unattended installation". A sample file anaconda-ks.cfg is saved in /root at the end of every manual install. You can use that as a starting point to create your own. When doing a network installation of Linux, you can tell Linux to use the kickstart file using a range of options . Please check the Red Hat docs for your favorite options. How do you do a network installation when booting from a CD (or PXE server)? Simple! Insert CD 1, or mount that image through DRAC or iLO, at boot the machine. At the first command line of the boot CD type: linux ks=http://server/path/kickstart_file.cfg url=http://server/path For instance, linux ks=ftp://myserver/default-ks.cfg url=ftp://myserv...

I tried to image our Linux servers, taking a "golden image" from one of our bare metal servers. I wanted to have a quick restore available in case I seriously cripple a serever during testing. In earlier runs, the restore worked but the Linux image had become unusable. GRUB wouldn't produce anything beyond its first 4 letters, so I created another server and imaged that. Same thing... Turns out I am using a Dell Poweredge 2950 with a local disk in RAID1 (mirror). I only use the local disk as swap area, creating a big LVM Volume Group (VG) on it called VGinternal. The actual boot disk is a LUN on the SAN. It also uses LVM and I have a VG in the LUN called VGsystem. VGsystem holds the actual system partitions /, /var and /tmp. When Altiris images this, it uses raw mode to capture the LVM partitions. Upon deploying the image containing LVM disk info, you must tell the DS to use the Linux environment . Not the WinPE or DOS modes... Testing it later...

The Register reports: "Oracle's assault on Linux looks to take the shape of a fork in the near future, according to Canonical founder and Ubuntu chief Mark Shuttleworth. 'They must be on track to fork soon,' he told us, during an interview here at OSCON. 'They are hiring too many people just to deliver patches. My assumption is that they are on track to fork and build their own distribution.'"

In order to use the Dell QLE2460 HBA to boot from SAN, you need to have the latest Dell BIOS (v.1.3.7 at the time of writing) and the HBA needs to be at firmware level v1.24. BTW, we are booting from an EMC DMX SAN beast. Your mileage may vary with other SANs. The qlogic 2460 firmware v1.24 can be found in their driver package v1.56, FYI.

If you, like me, love LVM for your Linux installations and LVM's ability to add disk space to partitions before they fill up without reboot or unmounting... then you'll be glad to know you can also use LVM if you have EMC PowerPath for your multi-path issues. In a bug fix from 2005, rhn.redhat.com | Red Hat Support , Red Hat lists that you can use Powerpath pseudo-devices with LVM. That way, LVM is unaware of a path failure and will get the fail-over path from PowerPath. Update: What you should do, though, is tell LVM to ignore local devices and only focus on emcpower devices. I distinguish between VMs and bare metal (BM) servers that also have local SAS disks. Here is what I've put in the lvm.conf file: # add EMCpower devices and sdb, reject all other SCSI disks and any CDs #BM#filter = [ "a|/dev/emcpower.*|","a|/dev/sdb|","r|/dev/sd.*|","r|/dev/cd.*|" ] # add any SCSI devices, reject all IDE disks and CDs #VM#filter...

I had another issue with Dell's Poweredge 2950 and the combination on Oracle Enterprise Linux (OEL) 4 update 5 (4U5), qlogic 2460 single port HBAs and an EMC DMX SAN. I was trying to install a Linux server to boot from SAN... It seems be caused by Dell's BIOS and Linux device enumeration, about which I've reported earlier. Dell uses DRAC to service its servers remotely. Using DRAC, you can mount virtual devices: a floppy and/or a CD (image). With these, you can install driver updates, install an OS or apply updates to a machine as if you're sitting directly with them. Very handy, but... These virtual media get recognized as USB/SCSI devices, or so it seems, and this causes Linux to assign device names to them, just like ordinary (SCSI) disks. In my case, the Dell Virtual Floppy gets enumerated as /dev/sda , which bumps up the internal PERC controller and my QLogic HBA. This results in a boot drive letter change from /dev/sda to /dev/sdb or sdb to sdc . Therefore, you...

The docs on Red Hat Network Installations can also be applied to Oracle Enterprise Linux (OEL)... Easily. Just make sure that you may have to edit the file TRANS.TBL in Enterprise/RPMS . You can also use Altiris to setup a Linux ftp deployment server. Just boot the OEL cd with the prompt: linux askmethod Then, the OEL CD image will ask you if you'd like to use the CD-ROM or opt for an NFS, ftp or http install source. Before you hit the OK key, make sure that the local CD image disk 1 is unmounted . If you don't, the network install will fail and you'll see the file product.img appearing in your logs. Try again, this unmount the boot CD image from which you started and then hit OK to start the network install. This fixed all my problems (after several hours).

I am having a tough time with up2date on Oracle Linux. The backward package manager has survived the Red Hat "fork" or spin-off, if you will. Sad, very sad. I have always hated the cumbersome up2date thingy. It's vague and opaque about how it does things. yum is better, but sadly yum won't work (yet, Juli 2007) with Oracle's update network ULN . So you can't use it. I am trying to find a way to strip a lot of packages off of an installed system in order to clean it up for an upcoming 'golden image' creation with Altiris. Altiris will be used to quickly provision new bare metal machines with a 'golden image' of OEL, containing all packages and configs. The only thing I can find, is system-config-packages in the Red Hat documentation on Removing Packages . Sadly, whenever I try to remove packages, it always whines and complains about missing libs, prompts to insert CDs or whatever. I can never simply wipe a system and remove unwanted packages o...

Many people don't know that Oracle, contrary to Red Hat, does give everyone free and unlimited access to the ISOs of Unbreakable Linux, or Oracle Enterprise Linux (OEL). Simply go to their Unbreakable Linux Network and follow the link. Only if you want their updates do you have to register and a support fee. But testing the platform, your application and infrastructure is free... as it should be!

Oracle Unveiled Oracle Database 11g last week.

"IBM guarantees its products will work with Red Hat's version of Linux; although, they are being very cautious in adopting support for Oracle Corporation's Unbreakable Linux, despite the fact that Oracle's distribution is a mere clone. Oracle Corporation's Unbreakable Linux became widely available in October of last year, and is based on the much popular Red Hat Enterprise Linux distribution. The only difference, according to Oracle, is the claim to provide cheaper support for their customers. On Friday, Matthew McMahon, a spokesman for IBM, stated they are not ready to guarantee their software to be compatible with Oracle's version of Linux. If any compatibility issue is raised between the two, it will be strictly up to Oracle to provide a fix." Source: Blorge.com :

Device mapper (dm) is the native Linux tool to create enterprise grade multipath and fail-over devices with that utilize SAN storage and other modern storage features from within Linux. The Device-mapper Resource Page is a good starting point. As we're using Dell hardware (poweredge 2950), I found the white paper on using dm for multipath I/O rather helpful. It lists some basic knowledge and concepts, for those not familiar with SAN, HBA cards and other devices. Interestingly, it also mentions that device mapper only became native to Linux with kernel 2.6.13 and backports have been ported to RHEL 4 update 2 as well as SLES 9 SP2. That makes me wonder whether device mapper is included standard in RHEL 4.4, or OEL 4U5 in our case...? Does a backport mean that you can safely add it to your existing installation if it older than update 2? And is it included by standard in any later version or update, or not?

We found an interesting issue on the same server (VM) when connection to terminal window in VNC and through ssh... When trying to list the limit for the number of open files, the terminal window in VNC showed the limit to be 1024. Very low indeed. Increasing the limit using ulimit failed. The same server but now through ssh (putty), showed the same parameter to be 131029. This confused us a bit until we realized that the VNC session was initialized by the VNC daemon and disconnecting from it, merely closed the session but didn't restart the main process. When we restarted VNC, a terminal showed a much better value for nolimit of 65536. However, that was still different from the value we got when using an ssh session. The difference was explained in this document: Increasing the number of open file descriptors . You must align a few more config files...

Although this site explains and shows Automated Installations of Red Hat Enterprise Linux , it can be easily adapted for Oracle Enterprise Linux (OEL). Or at least it should. Alternatively, you can automate Linux installations using Oracle's Enterprise Manager. That will be the new way to go anyway, I believe. Until then, the instructions on the site about post-installation scripts for Oracle, modifying kickstart files and performing network installations with sources on a web server are particularly handy. The Oracle site lists additional info on adding Oracle scripts to the post-installation section of the kickstart script.

Today, we noticed a CPU hungry gam_server process using 100% CPU in some (32 bit) VM running Oracle Enterprise Manager. Not knowing what is did, we did some research. gam_server belongs to the gamin file alteration monitor , a part of FAM . By default, it monitors all /mnt/* and /media/* mounts. We happen to have an NFS share for common configs, sources and ISOs and such. So it makes some sense. Searching Google turns up a surprising amount hits for the same problem , dating back as far as 2004. There were some memory leaks with versions older than 0.1.7 but most problems should be fixed starting with that version. So update gamin if you don't have that version. The next issue is the default config for gamin. Two seperate solutions ( 1 | 2 ) both outline the solution: adjust the /etc/gamin/.gaminrc and set appropriate poll intervals for the file systems of your choice. In our case, we'll adjust NFS polls to every 30 seconds, ext3 to 10 secs. fsset nfs poll 30 fsset ext3 poll 10...