Oracle: Unbreak my Linux Heart

Dell TechCenter posted a very easy "how to get started" guide for KVM Virtualization that you may find useful if you're just getting started and know little or nothing about virtualization (in Linux or elsewhere). It's good for the basics.

EMC (VMware) has a great technical webcast and PDF online about Performance and Provisioning for Virtualized Oracle . I highly recommend you listen to it or read the sheets. Very informative if you want to use virtualized Oracle databases in a production environment. Most set-ups of virtualizing Oracle databases on VMware fail because storage I/O becomes a bottleneck. It turns out, 50% of all surveyed set-up had storage configured wrong. The problems has to do with storage queues in the VM and subsequently on the ESX host. Additionally, vSphere 4.1 introduces features ( SIOC ) to optimize the I/O better to the SAN and para virtualized SAN drivers can further reduce latency and increase throughput. Then the webcast also mentions Oracle's "native" NFS client ( dNFS , PDF) inside the database kernel. This reduces the CPU overhead and latency normally observed on NFS client in the OS. By using Oracle's native NFS client, you can use shared storage in a grid computing envi...

At work we are in the process of upgrading our VMware infrastructure to vSphere with new Vmax storage from EMC. One of the new features is VAAI , vStorage APIs for Array Integration. This abstracts certain aspects of storage to the SAN layer, where e.g. cloning can be a mere functional call that is then handled transparantly for the layer above. Yellow Bricks has a great post on VAAI sweetness collected from twitter and gathered or our convenience. Awesome!

Dell Community : "RHEL 6 provides SR-IOV functionality on supported hardware which provides near native performance for virtualized guests. Single-Root I/O Virtualization (SR-IOV) specification, introduced by PCI-SIG details how a single PCIe device can be shared between various virtualization guests. Devices capable of SR-IOV functionality support multiple virtual functions on top of the physical function. Virtual Function is enabled in hardware as a light weight PCIe function. Operating System cannot discover this function as it does not respond to the PCI bus scan and requires support in the host’s driver. As in PCIe pass-through, a Virtual function of a SR-IOV capable card can be directly assigned to the guest operating system. A virtual function driver running in the guest manages this device."

Like many others, we are running into issues with Oracle licensing. Oracle charges a lot for its software and using state-of-the-art hardware you may even end up paying more, rather than less. The thing is, Oracle refuses to see VMware virtualization as "hard partitioned" but instead as "soft partitioned". That is, a single core VM may run on all cores that are available to it. So Oracle licensing requires you to factor in every core. Oracle's own Oracle VM (OVM) for x86 and SPARC (Xen-based) uses hard partitioning, but this comes at the cost of loss of overcommitment. That is, you cannot assign more cores than you have and increase the CPU utilization of your hosts. This is not necessarily bad, but means you can run less VMs on a given host. Reference: Oracle Storage Guy and Weinshenker.net

Oracle's Virtualization Blog informed me that they've expanded the use of Oracle VM (OVM) significantly by supporting visualization on the SPARC platform. "Oracle VM Server for SPARC, previously called Sun Logical Domains, leverages the built-in SPARC hypervisor to subdivide supported platforms' resources (CPUs, memory, network, and storage) by creating partitions called logical (or virtual) domains. Each logical domain can run an independent operating system. Oracle VM Server for SPARC provides the flexibility to deploy multiple Oracle Solaris operating systems simultaneously on a single platform. Oracle VM Server also allows you to create up to 128 virtual servers on one system to take advantage of the massive thread scale offered by the CMT architecture."

OraclePartnerNetwork has created a series of videos on YouTube about Oracle VM 2.2 . They vary from installing Oracle VM, creating a VM, to live migration, enabling HA, etc.

How does ordinary virtualization relate to virtualization used in a cloud computing environment? That was the topic of a talk at the Cloud Computing Expo in Santa Clara last week. Is there a difference? Or is it merely a logical or organizational distinction but the techniques used are still the same? Adam Hawley talked about this , slides linked to in the article, and how Oracle's products do or will fit into this paradigm. Database services in the cloud, Application Server services in the cloud, middleware provisioning in the cloud for yourself and your customers, isolation, privacy, security, replication, disaster management. Everything is still the same yet different.

Oracle's announced Oracle VM 2.2 , their latest Xen-based general purpose virtualization technology platform for their Red Hat-based Enterprise Linux version (OEL). Coupled with a new ocfs2 version and a few enhancements, you get ThinProvisioning for VMs and snapshots on cluster-aware files systems. That means they can span across data centers or locations. Update: wording changed as suggested by comment Update2: Oracle VM 2.2 CLI available too

Red Hat released a new version of their Enterprise Linux OS. RHEL 5.4 features enterprise-grade KVM virtualization, optimizations for Intel® Xeon 5500 Series and AMD Istanbul platforms, network enhancements such as Generic Receive Offload (GRO) support the latest generation of NICs, and more.

Oracle's virtualization solution, Oracle VM, now has a command line interface . This is mostly important to Linux administrators and those who'd like to be able write their own scripts and tools for working with Oracle VM (OVM). The OVM CLI works with Linux and Mac OSX (through a work-around). Read Wim Coekaerts Blog for details.

Red Hat Enterprise Linux 5.4 Bèta is now available. Most interesting improvements, apart from the usual bug fixes, performance benefits and better hardware support, are the introduction of KVM virtualization (in addition to regular Xen-based virtualization). New hardware support in virtual environments through SRIOV ( Single Root I/O for Virtualization ), utilities for management of FCoE cards over Internet, FIPS-140 security certification and a libvirt Perl interface for virtualization. Read the release notes for full details. Better yet is this announcement: " An important feature of any Red Hat Enterprise Linux update is that kernel and user APIs are unchanged, so that Red Hat Enterprise Linux 5 applications do not need to be rebuilt or re-certified. This situation extends to virtualized environments: with a fully integrated hypervisor, the application binary interface (ABI) consistency offered by Red Hat Enterprise Linux means that applications certified to run on Red Hat ...

A few days ago we released Oracle VM 2.1.5 which is available for download (registration free but required). This release contains the following : bugfixes on both Oracle VM Manager and Oracle VM Server side the new webservices interface which I wrote a blog entry about, a few weeks ago external data-collector cronjob/thread moved into the Oracle VM Manager instance minor cosmetic improvements on the Oracle VM Manager side This is a full CD release. So it is possible to do an install from scratch starting with 2.1.5, both the management and server side. Source: Oracle VM 2.1.5 (Wim Coekaerts Blog)

Although I'm using Oracle's Linux in the data center at work, I still keep up with other distributions. Just to check out what's going on and keep up. Lately, I've been playing with a new MediaWiki virtual appliance (VA) at work - where we keep most rough documentation and work-in-progress before moving it over to MS Word offial reports - and I must say turnkeylinux.org builds some mean VAs using Ubuntu JeOS. A 180MB ISO file is all you need. You create a VM, mount the ISO, boot, enter some root passwords and your set. It's integrated with Webmin, so you can completely configure the VM using your browser. No need to ssh into anything. :) Kudos! Anyway, I was going to install VMwareTools into the VM and remembered VMware had opened an apt-style repository for VMwareTools for use with Debian-based Linux distributions. So I made a few changes to the VM and installed it using apt-get! Wow! I love Debian, did I mention that before. ;) Here is what I did: wget http://p...

An interesting read from Slashdot mentioned a company YippieMove who switched from VMware to FreeBSD Jails when their infrastructure ran into so many problems doing I/O that something had to change. Interestingly, BSD Jails resembles Solaris' Zones and Linux vservers . Bottom-line is that, as the comments mention very well, you have to “know your workload before picking the technology.” Also, which hardware generation you run VMware on has a big impact. Newer hardware has a significantly better support for virtualization when compared to older generation hardware. We've had similar I/O problems with NFS shares exported to various bare metals (BMs) and VMs and we choose to move to an ECM NAS instead (based on the number and the volumes of the shares). I doubt installing vSphere 4 on our old PowerEdge 2950 would make any difference as the old hardware is simply already stretched to its limits.

There's a big discussion going on at my workplace about virtualizing Oracle databases. It used to be not done because of performance, but it offers many really useful features such as hardware independence, fail-over, VMotion and cloning, vmdk copies, etc. Oracle had previously told us they supported Oracle in VMs. When they announced Oracle VM Server based on Xen, they retracted this statement. We weren't impressed. Microsoft tried to do the same things while they were working on Hyper-V and so on. Eventually, anti-monopoly laws forced them to support competitors in the same way as they support their own virtualization flavor. So I expect Oracle to do the same. In the mean time... we're playing with Oracle Xen thingy as well as investigating virtual databases running on VMware 4.0 aka vSphere .

If you care about clustering and currently use VMware, you should read Wim Coekaerts' blog about vCenter Server Heartbeat and Oracle VM Manager with Oracle Clusterware ... I was gonna look into Oracle VM (and Xen) later this year anyway, but may move it up because of this.

As we run 700+ VMs in our data center, 40% of which Linux BTW, this news is pretty relevant to me: VMware decided to make their Advanced Technical Trainings (PDF) - mainly used for in-house support staff and engineers - available for free! Previous VCP certification is recommended, or at least thorough experience with VMware. Source: VMware - VMware Partner News

Here is a useful tip from the Dell Linux-PowerEdge mailing list : If you run RHEL3, perhaps because your software still won't support a higher release or your app was built on it, and when moving your physical servers over to VMware for better support (P2V?), you notice that the network won't come up, try removing the VLAN ID from the ifcfg file, i.e. /etc/sysconfig/network-scripts/ifcfg-eth0 .

I have been getting some weird errors in some Linux VMs running on VMware ESX 3.0.x (and perhaps ESX 3.5, as we're migrating to that). The symptoms are always that some weird errors occur for users or some maintenance task, and when you check the VM's consoles, you discover that the file system is suddenly read-only. My dear friend MrVanes discovered there is a known issue with the VMware LSI driver mptscsi in VMware guests running Debian . "...one could get SCSI timeouts when there is massive workload on the host system. Some kernel versions these will get the file systems remounted read-only, which probably makes sense for real hardware, but doesn't make sense for emulated hardware. Instead it should just wait a bit longer." A savvy user called TuxyTurvy has thoroughly investigated this issue. He was using Red Hat RHEL 4 and 5 on Dell 1850 servers with a Clarion AX150i el-cheapo SAN. When I/O load got high, running under VMware, due to storage contention, SCSI t...