Skip to main content

Posts

Showing posts from July, 2008

Strange I/O errors with SAN storage

We've found some really strange I/O errors (Qlogic qle2460, firmware 1.24) using LUNs on our DMX-3 SAN. One HBA was faulty so we replaced it. However upon restoring the OS and reinstalling it, more problems appeared. The new HBA would not boot at all using the existing disks. So we disabled it in the BIOS and booted from the other (original) HBA. Both HBAs have the same firmware, same settings. Upon booting anything involving the disks (we boot from SAN and have data disks there as well) is extremely sluggish. Letting the server do its thing, I got a ton of I/O errors first during disk discovery, then again during mounting of file systems. ERROR: ddf1: reading /dev/sdb[Input/output error] ERROR: hpt37x: reading /dev/sdb[Input/output error] ERROR: pdc: reading /dev/sdb[Input/output error] ERROR: pdc: reading /dev/sdb[Input/output error] ERROR: pdc: reading /dev/sdb[Input/output error] ERROR: pdc: reading /dev/sdb[Input/output error] ERROR: pdc: reading /dev/sdb[Input/output error

VLAN tagging (802.11q) in Linux

Just as a pointer for everyone looking to do the same, here is an article on the Dell PowerEdge mailing list about VLAN tagging on Red Hat Linux . It should also work for Oracle Enterprise Linux (OEL). The main tip was not to setup IP addressing for the native interface (eth2 in the example), but to add it for the sub interfaces eth2.vlan1 and the2.vlan2 instead.

Red Hat issues Enterprise Linux 4.7

Red Hat has released the seventh revision of Red Hat Enterprise Linux 4, RHEL4 for short, which is aimed at corporate customers. The release contains large number of bug fixes and a lot of small improvements. In line with RHEL 5.2, RHEL 4.7 now uses Firefox 3. RHEL 4.7 also provides "para-virtualized" drivers. These drivers allow RHEL 4.7, when running as a guest in a paravirtualized environment as such as Xen, to access the virtualised hardware without any emulation. This in turn increases I/O throughput and reduces the load on the CPU. Red Hat has cautiously extended the packages that were originally supplied with RHEL 4, so there are no obvious version number changes. The kernel is based on the now rather old Linux version 2.6.9, but Red Hat developers have back-ported many improvements from later kernels in order to support newer hardware. This means that RHEL 4.7 kernel can now handle Intel's I10 or AMD's new motherboard chipsets. Alongside updates for newer netw

Debugging sudo and sudoers

Often, when getting started using sudo and /etc/sudoers on a Linux system to authorize users to do something somewhere, you would like to debug sudoers and see why things fail when you think it should work. First, try sudo -l on any system as any user and get a list of things (if any) that you are allowed to do. Check and see if anything is missing. The sudo mailing list also hinted to a way to parse the sudoers file to see if anything is malformed. This can verify at the sudoers does not contain any typos or syntax errors. Finally, and most importantly, know this : "When multiple entries match for a user, they are applied in order. Where there are multiple matches, the last match is used (which is not necessarily the most specific match)." So order matters!!! Start with general rules and become most specific towards the end of the sudoers file. If you grant something at the top and deny it at the bottom, sudo won't allow it for the reason above. In that case, change th

Subversion Tools, Contributions, Hooks and Scripts

Finally seem to have gotten a stable design for our subversion server in place. Started to import my own scripts, setting properties on files, adding keywords to all my source files, and so on. I am now debating a way for setting and enforcing access control, if any is really needed. I already took a look at the hook templates and checked the subversion mailing list for useful tips and ready-to-use hooks, but instead I found Tigris' own Subversion Tools and Contrib section. This leads to many scripts and hooks, but my problem is that I don't have enough (svn admin) experience to decide which scripts are useful and which aren't. So instead, I'm blogging this here, hoping to return to it later on.

Restoring server images

While backups are usually often considered when designing high-availability server parks, doing an actual restore is often overlooked. You simply don't want to pull a plug and rather wait for the actual disaster to happen. Having a full file system image of a server, while space and time consuming can be the fastest way to get a server back online after a failure. If you have a SAN available, it can clone LUNs for your transparently and you can swap them back in when a machine fails or burn down. However, not everyone has one. Using clonezilla , you can easily pull images off of Linux or even Windows servers and restore them (even using multicast!) in minutes. As it only copies used blocks, that 300 GB local disk can result in a 5 GB image. For a restore, Clonezilla restores the appropriate image based on MAC address.

Integrit - file verification system

While reading an article to get you started on using a configuration deployment system that keeps all your servers configured in the same way, I read about the integrit file verification system . It's a simple tool that fingerprints your file system when you yourself have finished with it and let's you know when and where changes where made. Either by a user with too many rights, or by a hacker, or by a runaway process in a badly coded script or any combination of the above. Bottom-line, you can use integrit to help detect intrusions, hackers or your own admins doing something they probably shouldn't (at least not without your knowledge!)

Subversion, SCM and tools

Now that I've been reading up on the use of Subversion (SVN) for SCM and version control, after testing CollabNet's subversion , their CEE integrated SCM toolbox as well the virtual appliance SFEE , I decided to go back to plain Subversion and set that up properly. That is, as far I can without much experience in daily administration, maintenance, etc. Getting ahead of myself, I briefly checked out Warehouse and Lighthouse which add resp. add a visual subversion browser and issue tracker. However, ViewVC does that and is bundled by CollabNet. Lighthouse is for later. However, I think I'll install svnLogBrowser , which let's you visually browse the SVN logs and inspect or monitor change logs. It's much handier than the command line tool and may help getting newbies up to speed.