Oracle: Unbreak my Linux Heart

Dell has released version 6.5 of their OpenManage repository with firmware and BIOS updates as well as Linux monitoring of hardware found in Dell servers.

There are at least 2 version of the new iDRAC, Dell's remote server management and monitoring vehicle. Documentation from Dell lists the differences between the two and the older BMC interface.

Where I had great difficulties trying to get OMSA to install let alone work in Oracle's Enterprise Linux 4 (u5 and above), I am happy to report that I got it working out-of-the-box after an install of my custom 5u3 image, following the instructions on the OMSA Repository Wiki . A simple http proxy setting, the wget boot strap line to configure yum and the dependencies and I was in business! Excellent work, Dell!

Red Hat released a new version of their Enterprise Linux OS. RHEL 5.4 features enterprise-grade KVM virtualization, optimizations for Intel® Xeon 5500 Series and AMD Istanbul platforms, network enhancements such as Generic Receive Offload (GRO) support the latest generation of NICs, and more.

I've been playing a bit with the nscd now and want to share some tips related to tuning the nscd.conf file. To see how the DNS cache is doing, use nscd -g. nscd configuration: 0 server debug level 26m 57s server runtime 5 current number of threads 32 maximum number of threads 0 number of times clients had to wait yes paranoia mode enabled 3600 restart internal passwd cache: no cache is enabled [other zero output removed] group cache: no cache is enabled [other zero output removed] hosts cache: yes cache is enabled yes cache is persistent yes cache is shared 211 suggested size 216064 total data pool size 1144 used data pool size 3600 seconds time to live for positive entries 20 seconds time to live for negative entries 66254 cache hi...

Got a handy little reference on the PowerEdge Linux mailing list over at Dell. Someone mentioned the location of the OMSA MIB files that define what can be monitored, measured, etc. on a Dell server using IPMI. I was wondering where these MIBs were located because I might be able to use them myself for other purposes on that hardware. Anyway, the MIBs for OMSA /opt/dell/srvadmin/sm/mibs/dcstorag.mib

Although I'm using Oracle's Linux in the data center at work, I still keep up with other distributions. Just to check out what's going on and keep up. Lately, I've been playing with a new MediaWiki virtual appliance (VA) at work - where we keep most rough documentation and work-in-progress before moving it over to MS Word offial reports - and I must say turnkeylinux.org builds some mean VAs using Ubuntu JeOS. A 180MB ISO file is all you need. You create a VM, mount the ISO, boot, enter some root passwords and your set. It's integrated with Webmin, so you can completely configure the VM using your browser. No need to ssh into anything. :) Kudos! Anyway, I was going to install VMwareTools into the VM and remembered VMware had opened an apt-style repository for VMwareTools for use with Debian-based Linux distributions. So I made a few changes to the VM and installed it using apt-get! Wow! I love Debian, did I mention that before. ;) Here is what I did: wget http://p...

Dell has released OMSA 6.0.2 for monitoring and remotely managing Dell PowerEdge servers that run Linux. Plugins for Nagios, MOM and others exist, so you can tie your Linux servers to most corporate tools.

Never seen so many IPMI messages flying around on the Dell Linux mailing list. It's an open standard to monitor information of basic health parameters of a server. Here is a very good from by someone from Dell that sums up the usage and support of IPMI under Linux on Dell PowerEdge servers. Basically, you only need ipmitool to issue power on/off the servers. If you want to do more, you need to install the IPMI packages (yum install OpenIPMI)

Wouldn't it be great if you could use the LCD on a Dell PowerEdge to display the names of the VMs running on it? You may never to know where a VM actually, but if you do, you usually need it badly. Well Nickapedia wrote the script and the HOWTO for you: Dell Server LCD Update Script . It uses Dell OMSA on ESX and IPMI to get access to the VM names and then outputs these on the LCD. Brilliant! See video on YouTube ! BTW: won't work on ESXi because OMSA is not supported there.

Been testing my new Oracle Linux stack based on RHEL 5.3. We have a lot of problems with sysops and admins editing files to fix problems. Only recently we started using subversion for release management so we can check for specific versions and verify the (unmodifed) integrity on files. Subversion will also let those sysops and admins commit their changes back into subversion, so changes get documented, verified, discussed and distributed through bug fix releases. However, OEL/RHEL 5 comes bundled with AIDE - Advanced Intrusion Detection Environment and I've started running tests on my test servers. When you initialize the AIDE database as part of the final post installation steps, you run aide -i . This creates base reference database - aka snapshot - of all files that you use later on for any modifications. Using aide -u you update that db. Small problem is that you may a lot of lgetfilecon_raw failed errors. In my case, this had to do SELinux being disabled and aide checking ...

HPC Community - Systems Management : "The Dell OpenManage Nagios Plug-in is a freely available, open source plug-in for the Nagios network monitor. By harnessing Dell OpenManage (OMSA), the plug-in provides the ability to monitor the overall health of supported Dell hardware models, all via the Nagios interface. Additionally, the Nagios notification mechanism may be used to provide alerts regarding system state based upon the information reported by the Dell OpenManage Nagios Plug-in."

Dell's OpenManage tools keep some logs under /var/log and do not clean them up. You also can't turn them off and they contain important info from the PERC controllers. Anyway, the mailing list mentions this often, so I thought I post a little solution using logrotate: Dell OpenManage and their TTY logs . /var/log/TTY_*.log { daily notifempty missingok rotate 7 # how many you want to save compress delaycompress sharedscripts postrotate /usr/sbin/invoke-rc.d dsm_om_shrsvc restart >/dev/null 2>&1 || true endscript } See also Planet Malaysia .

Someone on the Dell mailing list was kind enough to point that although OMSA (OpenManage) on Ubuntu is not supported by Dell, the SARA institute in the Netherlands has a couple of OMSA Debian packages for Ubuntu ready for use. Check their site !

Don MacAskill from Smugmug wrote an interesting article on a problem with Linux and single-app-boxes. I.e. servers that only run one application that consumes basically all available resources. It seems Linux will swap something sooner or later, even when it is swapping the only application out to disk. So your performance will plummet. But you can not disable the swap partition or go without one (according to their experiences, not mine). However, there is a simply sweet "solution" or work-around: create a small swapdisk on a RAM disk! Read more details and a howto in Don's article. Of course, the article is about MySQL. But Oracle RDBMS is also a single-use application. At least in our data center. And I have been getting reports of some Siebel apps showing weird performance problems. This may be it!

I have been struggling with Dell's OpenManage Server Management a bit lately. Mainly because other people asked me to try it and gave me wrong links to information. So I'll clarify it here, for those getting started as well, and hope it helps someone. Dell Server Management has a lot (!) of components and you don't really need all (or any) of them. But they can be helpful. We'll use it for Server Management , mainly to monitor our PowerEdge 2950 servers but also (hopefully) to unify BIOS settings and DRAC configurations remotely. That's why we need OpenManage Server Administration (OMSA) . OMSA consists of a number of things: - OMSA Server - OMSA Managed Node - OMSA ITA (IT Assistance) You'll only need one server (maybe two for fail-over and redundancy). The easiest way to install the server is add a yum/up2date repo from Dell. but I had trouble finding the "OMSA agent" or "OMSA client" that I'd install on all servers to be monitored by...

Just getting started using Dell's OpenManage Server Administrator (OMSA) on our Oracle Linux platform. There are some confusing instructions going around so it's not immediately clear what to do, hence my blogging here. :) There is a site on Dell - Hardware Monitoring , as well as a wiki with instruction on how to setup their OMSA tooling using yum or up2date. [update]My first update for their instructions: be sure your server has Internet access, as most servers will use a proxy or so. use export http_proxy=http://yourproxy.example.com:port to configure it just for the session, and setup up2date to use an HTTP proxy by editing the settings in /etc/sysconfig/rhn/up2date .

Found an add-on for Nagios , the well-known open source monitoring tool based on SNMP. NagViz adds visualization to Nagios, in a way that - as far as I understand it - lets you create a kind of dashboards to show everything is up. For instance, all devices related to Email delivery or a web application farm or your extranet portal. Should help convince board members that Nagios is worthwhile as well as providing people with easy to use screens. Check out SourceForge.net .

I was just playing a test set-up for Nagios , to monitor some hosts, services and devices for me across the data center, when I ran into Splunk in a config file. Being naturally curious, I checked out Splunk... Splunk is basically your IT search repository. It has a search engine that collects everything you want to track from all your servers. Use Splunk to collect SNMP traps, mail queues, log from web servers, firewall or routers, transaction and database logs and more. It can index anything because it uses live tracking. What it doesn't know but occurs frequently, you can teach it. Best of all, use Splunk to trigger an SNMP alert for a host or service in Nagios, when say more than 10 HTTP 404 errors occur on your corporate website. Probably there has been an error that some other system may or not have tried to inform you about... Excellent! Some assembly required of course, as everyone has different logs, needs, wishes and demands...

We're using Oracle Enterprise Manager (OEM) to monitor and control all of our Oracle Linux servers. I know, Cacti , Zabbix or OpenQRM or any SNMP-based tool could also do the trick. But OEM comes integrated very well with alerts and triggers for the RDBMS, as well as the Linux platform. So it's less work to get up and running. Thus we need the OEM agent to be installed on all machines. Download the latest agent that matches your platform from Oracle's website. Careful! The Management Agent varies between 250 MB and 1000 MB in size, depending on your version and platform! Use a fast connection and a proper download manager like wget .