BOFHJournal - November 2005

1999 | 2000 | 2001 | 2002 | 2003 | 2004 | 2005 | 2006 | 2007 | 2008 | 2009 | 2010 | 2011 | 2012 | 2013 | 2014 | 2015 | 2016 | 2017 | 2018 | 2019 | 2020 | 2021 | 2022 | 2023 | 2024

October's Journal
December's Journal

30/11/2005 [01/12/2005 - 10:30] Another one of those so busy days when you think there's tons of time for writing your journal entry and it turns out it's the end of the day. Most of the day was spent working on little bits of this and that before recieving an email which required instant attention. It was lucky in a way that the replacement Sun V20z I'd been waiting for didn't turn up (like I was promised it would) as I wouldn't have been able to do anything with it after all. The email was to do with the fact that I run a number of web servers in SSL mode only and this was causing some problems for people who were attempting connections via non-SSL means and then logging numerous stupid helpdesk calls when their connections didn't work. A quick fiddle with some VirtualHost directives and some consultation with some friends online and I'm pretty sure I solved the problem to everyone's satisfaction. I now need to port the changes to the other web servers on the machine so that everything is nice and generic and similar across the services. I'll do that tomorrow. Climbing this evening.

29/11/2005 [12:40] As Elaine also has today off from work she stayed up quite late last night chatting to friends online. I shouldn't have been as bothered as I was by it, but I guess I just wanted to know she was getting enough sleep, etc. In the end she toddled in to bed at around 04:00 which woke me up (I'm a very light sleeper most of the time) and meant I lay awake for an hour or so listening to her breathe peacefully, which was the only positive aspect of the experience, but a very nice one. Consequently I wasn't particularly awake when I got up this morning. I've counteracted this with a small bit of chocolate and an early lunch to bolster me for giving blood this afternoon. Everyone else in my office is out today as well as many of the DBAs. They've gone to see NetApp with a view to buying one of the biggies in a little while. Hopefully I'll get to go down too at some point to investigate NFS and Samba connectivity. For the moment though it's just me holding the fort. Today I am determined to give blood after missing the last two opportunities because of work commitments or over full sessions. This means I'll be out from mid afternoon until the end of the day. So see you tomorrow.

28/11/2005 [13:50] A rather lazy weekend all things considered. Saturday I managed to finish off HalfLife 2 and start on one of the third-party mods available on Steam by someone I know. Elaine spent some time doing some revision for the MCP she's working on at the moment. The house is really quite cold at the moment, I think it's something to do with the entire rear wall of the house being an outside wall (obviously) and thus getting very cold. It's almost like there's no cavity wall insulation. James came over some time in the late afternoon and took us shopping for some essentials at the big (huge) Tesco a few miles away. As compensation we fed him an evening meal. Sunday morning was fun. We stayed in bed for a while and kept warm. Once we'd ventured out I headed to the pub to meet people for an intitial photo shoot for a project someone's working on (more on that in time) before coming home again. I helped (hindered) Elaine prepare a wonderful roast dinner. During the afternoon Dunk and Ritu rang up to say they'd not in fact made it to the continent but were in fact in town and did we fancy meeting up for a few drinks? Of course we would, so following the delicious meal Elaine and were visited by said two plus Keith. We walked to the same pub I'd been to six hours earlier and drank and chatted and tried to play board games with most of the pieces missing until throwing out time. Very slightly the worse for drink we then wandered home and ended up watching some questionable content on DVD and eventually from my laptop through the television. Who said that kind of media isn't for group consumption? Around one in the morning everyone else went home and Elaine and I retired, slightly tipsy, to bed to a very pleasant night. Of course Elaine and everyone else last night didn't need to get up for work this morning so I was the only one groaning and cycling in to work feeling a little dulled around the edges. Luckily there's not much on my plate this morning so I've been trying to track down someone who can tell me when I'm doing downtime on one of the main finance servers and waiting for Elaine's second bit of Christmas present to be delivered... which it just has been as I was typing this.

25/11/2005 [15:55] Cold-caller baiting by my friend (I):

CC: "Can I speak to the person who deals with your telephone service please?" I: "I'm afraid we don't have such a person." CC: "Ah, is that outsourced then?" I: "No, we just don't have telephones." CC: "... Are you not speaking on a telephone now?" I: "No, no I'm not." CC: "... Okay thanks for your time, bye." I: *click*

Nuts. Completely. In other news, I helped Elaine get up and out of bed this morning at some awful hour and then dozed for a while listening (as I do every morning) to The Today Programme on Radio 4. Once up, about and in to work I spent some time working through work emails before getting up to do the backup tapes change for the week, at which point I found that one of the DBAs had started off a 200GB+ backup of some of the Oracle 11i databases. I therefore sat back down again and spent some time flirting (well, it is a Friday) and sending Elaine SMSes. It snowed a few times while I was waiting. Nothing settled, they were just flurries to be honest. I doubt anything decent will happen before tonight, or even the weekend. I fully intend to spend some time updating my wishlist this weekend, watching the new DVDs which turned up today (Farscape and Mr & Mrs Smith) with Elaine, snuggled on the sofa keeping warm, and probably some other stuff which'll come to us at some point. Probably some washing too. For the remainder of the day here I'm going to listen to some music, catch up on the week's happenings and then go to the gym before heading home to put my feet up.

24/11/2005 [16:50] I feel tired and drained. I don't really know why although I am feeling a bit down at the moment. I think my mood'll pick up as soon as I get a better handle on what's going on everywhere. Sometimes it would be nice if my life were just a bit more certain and I didn't feel quote so adrift and powerless to affect my direction, as happens every now and then. I'm sure everyone feels like this and I'm nothing special, or odd (that would be nice, actually) but sometimes I get hit hard and the problem is that it affects the people around me too, which I desperately hate (as I'm sure they do too). I think I will just go home and prepare for this evening's assemblage.

23/11/2005 [14:45] Luckily today has been far less exciting than yesterday. I found out why my drive had given up the ghost; it was a Maxtor. There's a reason they're so cheap. Never Buy A Maxtor, I don't. So I was here until very late last night getting everything copied back onto the machine. Initially I tried to installed RHEL 4 but that had some odd X issues which I really couldn't be arsed to deal with given the time constraints so I went back to RHEL 3 and the configuration I'd had before. While I'd been able to copy off all the essential stuff I only got a third of my MP3s. This isn't particularly irksome as I have them all at home too and a quick portable USB2 hard drive operation later and they'll be back up to full strength again. I've got the camera up and running, my cron jobs, handy little shortcuts and all the other small things that make a workstation your own. It's _almost_ like nothing ever happened but you know what it's like when you reinstall a machine you've used for ages; there's always something you missed or a feeling that something's different even when you can't find it. Most of this morning I've spent preparing detailed idiot instructions on how to bring down one of our more important live servers for LOM, system board and OS patching. I really don't want to mess it up given that if I do Bad Things will happen. Suffice it to say a few simple operations have now been dissected to a command-by-command script which I should be able to do with at least one eye closed. In other news it looks like our purchase of a NetApp is going to be preceeded by myself and one of the other sysadmins going to NetApp HQ in Uxbridge at some point, which should be fun. I need to get together a plan of things to do to stress test whatever they give us to play with... [16:45] Gah! There I was, very little to do and actually told to have fun with the new V20z we had delivered a few days ago. So I open the box and pull out the thing. Straight away I can hear a rattle. Not something you expect from Sun kit so I open the chassis and eventually shake out a small metal washer. This gives me the chance to notice that not only are the CPU heatsink fins a little bent, but that the Management RJ-45 sockets riser doesn't project properly through the back of the chassis but is instead angled up slightly. I pop it into place gingerly and install the rack kit in the rack. With some help from the Networks guy we get the server in the rack, at which point I noticethat the chassis has a marge impact scratch on the front of one of the drive caddies and that there's a dent substantial enough to stop the optical drive opening on the top edge too. At this point I decided enough was enough and even if the box came up OK I wouldn't be happy with it long term. It's now sitting in my office and I've emailed the Operations Manager asking him to get the company who sold it to us to send a replacement (incidently that I've installed the rack kit and don't really want to have to take it out just to install an identical one). I think I'll go home and get ready for climbing.

22/11/2005 [13:10] Crap, I get in this morning and first of all I find that the hard drive in my workstation is dying and that I have a two hour meeting at the same time when I would really like to be copying all my data off my drive somewhere safe. So between 10:00 and 12:30 I've been helping to sort out the purchase of the new hardware and software for the HR project we're undertaking, but I would rather have been trying to get my system sorted out. I'm trying to save as much of the configuration as possible but it's going to be a real arse getting back to where I was before hand. And I'd just got it the way I liked it, too. [20:35] Got my machine something approaching reinstalled. I think I need to go home now.

21/11/2005 [13:00] So I got in this morning and the clock on one of our live DB servers reckoned it was November 3rd. Note that today is November 21st. This had caused all kinds of problems with the Oracle DB and the application servers that talk to it. We downed the application and the DB and the DBAs started repairing the damage/restoring from backup. The sysadmins (me and Co.) reset the clock and started looking in to why it'd happened. Initially we thought it was an NTP fault, but no other machines had been affected. The next time I looked at the date on the server in question is was exactly an hour behind where it should have been, while still saying GMT. We finally tracked it down to this bug: http://sunsolve.sun.com/searchproxy/document.do?assetkey=1-26-56680-1 which is patched (through a fairly long OBSOLETED chain) by this patch: http://sunsolve.sun.com/search/document.do?assetkey=urn:cds:docid:1-21-114525-05-1. Basically, if your server has been powered on for more than 528 days (reboots don't count, we're talking about a hard power cycle/loss of power to the machine) and your LOM firmware isn't patched past this bug then eventually it's going to interfere with your system clock and there's nothing NTP can do about it. Nice.

18/11/2005 [16:35] Looks like they want to put all the sysadmins in with the DBA people in some kind of new open-plan office in the near future. I think this would be a bad idea in terms of work and noise. I'll not go into details except to say that one group does one job and the other does another. Mixing them up in the same room would be counterproductive. We've spent some of the afternoon looking at variations that would mean we got to keep our office. Not a lot else happened today except for me doing a whole load of throwing out of old documents, manuals and software installation media which hasn't been used in over three years. As a result I now need some bookshelves for my desk. I'm off to see Norwich play at home on Saturday and hopefully spend the rest of the weekend relaxing and having fun with Elaine.

17/11/2005 [17:50] Nice quiet day. Got up and cycled in the cold over the crunchy salt on the roads. Need to remember to start oiling my bike a bit more in this weather. Spent the day looking at SCSI errors on an SRC/P Raid Controller on one of our Sun E450 machines. Talked to Sun and found a solution in the end, which was nice. Some of the afternoon was spent trying to fix someone's computer. Seems that every time they ran a game the PC froze, the screen went blank and the OS hung. I tracked it down to whenever anything 3D happened (but not the stuff in dxdiag (this is running Windows XP). I think there's just some problem with the part of the card that does 3D stuff as the desktop operation was perfectly fine. I couldn't solve the problem or even update the OS as they were running with a warez license key so I gave it back to them with a stern telling off and a recommendation to try getting another graphics card. I'm running a bit low on work to do here at the moment, but tomorrow should be fun as I get to go through a cupboard full of things the previous sysadmin hoarded away for no readily apparent reason, so don't expect to see me on camera much. I should go home really, before the temperature drops below freezing.

16/11/2005 [11:45] Got up horribly early again this morning so I could get in in time to do an application restart at 08:00. That went OK. I fiddled for a little while helping one of the Windows administrators track down a Travan tape drive for a legacy server that isn't going to die any time soon (or if it does it needs resurrecting). I got a CD-RW out of the deal, which was nice. I'm off into town at 13:00 to watch a Sun engineer change a disk in one of our servers and then to a technical seminar for the rest of the afternoon. Hopefully climbing this evening. Between now and leaving for the hardware thingy I'll be looking at RT again a bit and streamlining the installation process for next time.

15/11/2005 [16/11/2005 11:15] Got up at the same time as Elaine so I could get to my contact lense checkup on time. That didn't take more than twenty five minutes as the person doing it was extremely professional and a nice man at the same time. Once that was done I pottered around HMV for a little while, payed in a cheque and then cycled to the station to attempt to lock up my bike. All I can say is that getting to the station ten minutes early is essential if you're going to find somewhere to lock your bike up securely and make the train you want to catch. I caught my train. I waited at the far end for my family to arrive, which they duly did eventually. We then went to get a visiting present before travelling on to my cousin's house. It was very lovely in nice grounds I won't say any more than that. We chatted, had lunch, admired the house, the cars, the dogs, the multiple plasma TVs, the kitchen (and I admired the TVs again for good measure). A lovely time was had by all I think. Eventually we left and I took the train home. I met Elaine at the station and we cycled home through the cold and dark streets.

14/11/2005 [12:10] Exceedingly fun weekend. Friday, right after work I took a train to Peterborough with Keith to meet up with Andrea, Andy and James. After a few drinks and some food in the Drapers Arms we headed next door to play more pool than I have in a long time. Through a sequence of lucky shots and accidents by other players James and I managed to stay on the table for most of the evening, which was nice as I got to bring my cuemanship back up to somewhere near to where it used to be during my University days. Around 11:30 we headed home (everyone except Keith and I going in James' car) where we started watching Team America until we decided sleep was a more attractive option. While James left unsociably early in the morning to go to work and Andy left a few hours later for the same reason (work, on a Saturday?) the rest of us had bacon (a Saturday requirement when away from home or having guests) and watched the whole of Team America before heading home. Elaine and I wandered around town for a bit where she bought some new gloves and a fleece and I vacillitated over some new gloves myself. In the end I paid for some nice food from Marks & Spencer instead. We slept late on Sunday and spent most of the afternoon either lazing or playing computer games. Probably should have done more, but hey. This morning we were up bright and early to a major frost (first of the season) which meant that I cycled to work with an aching throat due to the coldness of the air. Still, the fact that I have to cycle into town at lunch time to do an errand is hugely offset by the fact that I appear to have won some kind of DVD player for going to the IT Exhibition last Wednesday and filling in the form for the free prize draw. I don't know how good it is yet, or even what kind of player it is, but still, free stuff is always cool. At the moment though I'm installing another RHEL 4 AS box for the purpose of testing RT for our group. We'll see how that goes. Hopefully I'll head off the gym this evening, although it'll depend how large this DVD player is whether I go home first or not. [17:30] DVD player turned out to be a Phillips DVP3005, which isn't too shabby. Was a bit of an arse getting it back to the office from town, so I'll be leaving it here until Wednesday when I'll bring a rucksack big enough to fit it to get it home. The reason I won't be doing it tomorrow is that I'm not in as I'm going for my contact lense checkup (only 25 months late) and then with my parents to visit a long-lost relative for the day.

11/11/2005 [16:35] A quiet day. Mostly configuration of RRDtool for one of the Networks people and some setting up of automatic share mapping from linux clients to Windows servers. Lots of new knowledge about /etc/X11/gdm/{Pre|Post}Session/ areas assimilated and far too many Fruit Pastilles. Off to see Andy, Andrea and others to play pool for the evening and eat bacon in the morning now.

10/11/2005 [17:25] I've been given a Wyse 3320SE and NCD ThinSTAR 200 thin clients. While I've been able to find something called xrdp which is very much beta software (it's still at version 0.1) - which allows a linux box to act as an RDP server over the top of Xvnc - neither of the clients seem to be able to connect properly (one of the crashes, the other says the connection was terminated). I have tried tracking down XDMP/X client software for both of them, but haven't had much luck as yet. If anyone has any helpful information on either of the two bits of kit that'd be super.

09/11/2005 [15:10] Spent the morning polishing the script and submitting it to the Big Brother scripts site. I've already had one download. Hopefully the people who do use it like it, or tell me what the don't like about it so that I can make it better. I'm off to an IT Supplier exhibition in a few minutes, and then going climbing. It's been a pretty useful day as things go. Now to have some fun poking fun at the Dell guy about AMD chips.

08/11/2005 [17:00] Woohoo! I've finished the script. It's only taken two days but I think it not only deals with our trivial RAID controller setups, but also much more complicated ones that might exist in the world. This means I have far fewer qualms about releasing it onto the Big Brother scripts contribution site for other people to use. It does a great job of reporting the controller, battery, virtual and physical disk statuses, but will limit what it shows if there aren't any problems. Much of yesterday was spent trawling through Dell MIB files for SNMP states that the various components might give in varying circumstances. I'm pretty sure I've captured everything I need to, but I'll have another look tomorrow with a fresh brain and eyes and maybe add some extra settings for "yellow" alerts. After that we can push it out to all of the servers here and get on with the next job. For now I'm going to head to the gym and then go home.

07/11/2005 [08/11/2005 16:55] We had a stunning weekend what with Andy and Andrea coming over to see us for the fireworks. The display was excellent and the drinking afterwards gave plenty of time for chat and watching some DVDs. I flaked out (again) well before everyone else (around 01:30) yet Elaine was up at the same time as me on Saturday morning. Saturday morning was spent making cooked breakfasts for all concerned before heading out to a retail park for some retail therapy. I ordered a chest of drawers and Andrea bought a DECT phone. Real grown-up things. Andy and I couldn't resist going in to PC World for a laugh though. Sunday I spent lots of time playing Half Life 2. The entirety of today was spent working on the Big Brother monitoring script for the Dell PowerEdge PERC device when OpenManage Server Administrator is installed. I had a damned good go at it and nearly gave up a few times before getting some help from a co-worker. By the end of the day and after a few complete rewrites I had something approaching a solution. Naturally, even though I arrived at the blood donation forty-five minutes before it closed they had too many people to deal with and sent me away. This meant I'd missed the gym for nothing. Not great. Still, Cormac came over and cooked a jambalya.

04/11/2005 [16:25] Today has been a much more prductive day than yesterday. Although I spent a fruitless few hours struggling to get awk to do what I want, in the end I decided not to go with it and take the easy way out by doing a simpler method. Of course, it took someone else to figure out why the output of the script I was writing wasn't getting dealt with by Big Brother. Still, it works now and 'all' I have to do is work out all the possible states that hardware I'm monitoring might get in to. It's a RAID controller so this means I've been manually trawling through MIB files looking at SNMP states. Not something I've had to do since I was running Castle Rock's SNMPc application and looking after a few racks of Cisco kit. I've had a fairly productive and certainly very informative day today and with luck I may be able to do some funky stuff on Monday with battery and other hardware monitoring too. Elaine's coming to the gym this evening for an induction, which should be interesting. She's not going to do the whole yearly pay thing, but I expect she'll come on an occasional basis if all goes well. I expect to play a fair amount of Half Life 2 this weekend, just as a break from using a computer for work. Plus it looks really good on my widescreen LCD monitor...

03/11/2005 [04/11/2005 16:15] I near had a fight with someone on a bike this morning. Not only did they cut me up, but they also went through a red light at the same time. I cycled up next to them afterwards and mentioned that this might be dangerous from a health point of view and was subjected to a tirade, a veritable torrent of abuse. She looked to be about fifty-five, or a very ugly and wizened younger woman. In the end I just started laughing at her as she exhorted me to "join Blair's Nanny State." By the end I was still a little shaken as at one point she'd mentioned that if she wanted me killed she'd get off her bike and do it herself. It took a lot of effort to remain civil and not resort to pointing out her numerous mental and physical flaws. That was the start of what turned out to be an annoying and unproductive day. Everything I did ended up either not working or not being finished. I spent about two hours tweaking a script until it worked and still wasn't happy with the result. By the time the end of the day had arrived I just wanted to go home so I was pretty peeved when, two minutes out of the office, both of my bike lights died. Cue a slow walking-pace cycle home on the footway (I know, I know).

02/11/2005 [17:20] A morning of trawling truss outputs to try and get a new Trent install to work and an afternoon of looking at RHEL 4 and wondering why bits of it fail to work in new and exciting ways such as hotplugging throwing a fit, our installation of Procallator giving strange and wonderful new errors and Sendmail being odd, garnished with an hour or so of pondering whether CSV imports to databases which have gone down in size for the first time ever will cause data loss or not. Lather, rinse, repeat. Serve with a cycle home in the pouring rain and add wet shoes to taste. I'm going home. Don't know if I'll go climbing or not. At this rate I'd rather just install Steam, download HalfLife 2 and have a bowl of carrot and coriander soup and some crusty white bread.

01/11/2005 [14:10] I've spent the morning playing with firmware updates for the web team's set of servers. Now they're all up to the most recent revisions of BIOS and RAID hardware and Dell's OMSA seems to be 100% happy. This means my co-worker can get on with the unified Big Brother RAID monitoring script on his return from the hell that is a project management course. I've also been reinstallaing a server here with RHEL 4 AS. It's going to be our first 'live' RHEL 4 box so I've managed to get myself a week's worth of time to play with it and see how it differs from RHEL 3. I've already found out that Dell's OMSA - while seeming to be working OK - generates some odd warnings on startup, also that the old Raidmon stuff just doesn't work any more with megaraid_mm and megaraid_mbox drivers as they don't put anything in /proc/megaraid. This is annoying, but means we'll definitely have to start using OMSA when we move any other boxes to RHEL 4. Other than that I'm off on a course to learn about Trent this afternoon. It's only a basic technical overview (I'm going to be supporting the box, not the applications per se) so I won't be back before the end of the day. After that I'm off shopping with Elaine and then home.