Oracle: Unbreak my Linux Heart

HPC Community - Systems Management : "The Dell OpenManage Nagios Plug-in is a freely available, open source plug-in for the Nagios network monitor. By harnessing Dell OpenManage (OMSA), the plug-in provides the ability to monitor the overall health of supported Dell hardware models, all via the Nagios interface. Additionally, the Nagios notification mechanism may be used to provide alerts regarding system state based upon the information reported by the Dell OpenManage Nagios Plug-in."

Found an add-on for Nagios , the well-known open source monitoring tool based on SNMP. NagViz adds visualization to Nagios, in a way that - as far as I understand it - lets you create a kind of dashboards to show everything is up. For instance, all devices related to Email delivery or a web application farm or your extranet portal. Should help convince board members that Nagios is worthwhile as well as providing people with easy to use screens. Check out SourceForge.net .

I was just playing a test set-up for Nagios , to monitor some hosts, services and devices for me across the data center, when I ran into Splunk in a config file. Being naturally curious, I checked out Splunk... Splunk is basically your IT search repository. It has a search engine that collects everything you want to track from all your servers. Use Splunk to collect SNMP traps, mail queues, log from web servers, firewall or routers, transaction and database logs and more. It can index anything because it uses live tracking. What it doesn't know but occurs frequently, you can teach it. Best of all, use Splunk to trigger an SNMP alert for a host or service in Nagios, when say more than 10 HTTP 404 errors occur on your corporate website. Probably there has been an error that some other system may or not have tried to inform you about... Excellent! Some assembly required of course, as everyone has different logs, needs, wishes and demands...

There is a quick FAQ on Oracle VM , explaining the basics about Oracle new virtualization technology. For instance, basic OracleVM is free as anyone can download the binaries . Support is available on a per-socket basis, not per-core. There are also ready-made templates with Oracle Linux (OEL) 4 update 5 for both 32 bit and 64 bit VMs. You will manage VMs with Oracle VM Manager. Oracle supports their own database 10g and 11g inside OracleVM, as well as OEM, Siebel, Peoplesoft and E-business Suite. Support for Oracle RAC clusters will be available somewhere in 2008. Testing is in progress, but timing issues in RAC are very strict, so it must be thoroughly tested. The requirements for OracleVM as similar to Xen's requirements: a new/recent CPU with VTX virtualization extensions is required for running VMs with Windows. If you only run Linux, you can use older hardware as well (and a kernel supporting paravirtualization).

While waiting for HP Openview to complete and manage everything, we found the need to get a program to help maintain and organize the quickly growing pile of servers, firmware versions, hostnames and WWPN numbers. Luckily, SourceForge is a never ending treasure chest of open source software and OneCMDB seems to fit... will try!

We're using Oracle Enterprise Manager (OEM) to monitor and control all of our Oracle Linux servers. I know, Cacti , Zabbix or OpenQRM or any SNMP-based tool could also do the trick. But OEM comes integrated very well with alerts and triggers for the RDBMS, as well as the Linux platform. So it's less work to get up and running. Thus we need the OEM agent to be installed on all machines. Download the latest agent that matches your platform from Oracle's website. Careful! The Management Agent varies between 250 MB and 1000 MB in size, depending on your version and platform! Use a fast connection and a proper download manager like wget .

Today, we noticed a CPU hungry gam_server process using 100% CPU in some (32 bit) VM running Oracle Enterprise Manager. Not knowing what is did, we did some research. gam_server belongs to the gamin file alteration monitor , a part of FAM . By default, it monitors all /mnt/* and /media/* mounts. We happen to have an NFS share for common configs, sources and ISOs and such. So it makes some sense. Searching Google turns up a surprising amount hits for the same problem , dating back as far as 2004. There were some memory leaks with versions older than 0.1.7 but most problems should be fixed starting with that version. So update gamin if you don't have that version. The next issue is the default config for gamin. Two seperate solutions ( 1 | 2 ) both outline the solution: adjust the /etc/gamin/.gaminrc and set appropriate poll intervals for the file systems of your choice. In our case, we'll adjust NFS polls to every 30 seconds, ext3 to 10 secs. fsset nfs poll 30 fsset ext3 poll 10...

I attended a seminar at Oracle about their new management product: Oracle Enterprise Manager. It's a consolidated tool to create, provision, manage and update Oracle nodes on Linux. RAC, Grid, stand-alone. Everything. While not rocket science from a management viewpoint, it is a nicely integrated tool that you are sure to use in a Linux and Oracle environment. However, the current version kinda drops you off the deep end and you have to take care not drown in features. A upcoming update will shield functionality based on access rights and roles. Download the (powerpoint) presentation in PDF at Oracle .