BOFHJournal - April 2005

1999 | 2000 | 2001 | 2002 | 2003 | 2004 | 2005 | 2006 | 2007 | 2008 | 2009 | 2010 | 2011 | 2012 | 2013 | 2014 | 2015 | 2016 | 2017 | 2018 | 2019 | 2020 | 2021 | 2022 | 2023 | 2024

March's Journal
May's Journal

29/04/2005 [16:45] Extremely tired this morning. It's not as if we stayed up particularly late last night either. Watching Supersize Me on the Channel 4 was an extremely entertaining and thought-provoking experience though. I don't know how many diets/food plans/changes of direction/exercise regimes have been started as a result of watching it, but I can testify to the fact that I'm going to make further efforts to get back into the kind of shape I was in around three years ago based on having watched it. Today I've been dealing with very little, to tell you the truth. The guy from MGE who was dealing with our replacement network card has been on voicemail all day (yeah right). There's this issue where the date on the card can be NTP synchronised. This is working. It lists the date in ISO format too, which is even nicer (YYYY/MM/DD, yeah I know it should be -, but never mind), however when you go to the email notification settings page there's a dropdown for which day of the month you want notifications to be sent out. This dropdown runs from 1 to 31. Obviously days of the month, right? That's what I thought too. So I decide that as it's the 29th today, I'll select 30. When I do this a popup box appears saying (remember, it's 2005/04/29 according to the time page) that "30/5/29 is not a valid date." Damned right it's not, but as I can't specify more than a day of the month this is your fault, not mine. Gah, helpful eh? All I want is a firmware that works, is that too much to ask? We went for lunch at a great little pub a few minutes drive from here. Mexican food. I was pleasantly surprised to find that a chicken salad (heavy on the salad) filled me up nicely. Luckily there was a decent amount of chicken in there too, but it was good quality and with no skin, etc. That, coupled with the exercise I did this morning (every morning now) means I don't feel too bad about the delicious Lindor chocolates I've been accepting all day from my co-worker who has to give them to someone else as he's on a diet (fiance-instituted). I think that's it for today. Off to Mel's "I'm not going to be 40 tomorrow!" party and drink for a little while. I have no idea what to do this weekend or on Monday (UK bank holiday, no journal), but I'm sure it'll involve being outside and happy.

28/04/2005 [17:05] Not quite as good a day as yesterday unfortunately. While the morning went fairly well with my taxi-transporting of two servers to another machine room going to plan and after a bit of worry, getting them configured to use the terminal server there, I had the horror of finding that some essential Oracle installation scripts had vanished into the ether in the disk lost episode on Monday. Luckily I'd had the foresight to squirrel a copy away on another test server and was able to recover all the files bar one (the response file for the Oracle installer). I'm going to build that again before I go home. The fun thing of today was discovering the new network card for the MGE UPS. It looks like they've sent us a Network Management Card rather than an XML-Web Card so there's no SSL/HTTPS any more. However all the clients were still trying to talk to it (once I'd configured it with the same IP as the old card) via HTTPS so they managed to lock up the card in exactly the same way as the old one. Cue me ssh'ing to all thirty-six machines to disable SSL certification and restart the clients. Stupid thing. Now we're not using SSL I imagine the old card would work too, but the menuing system on the new card is much nicer so I think we'll stick with this. The firewall will keep out people wanting to shut down our servers anyway. My only woe now is about the email reminders thing. Don't get me started on endianness of dates. I think all in all I just want to go home. I forgot to have lunch today so I'm tired, hungry and missing Elaine.

27/04/2005 [14:55] Today has been pretty good thus far. Setting aside the fact I had to cycle with it in a rucksack on my back, I took the eighteen kilograms of 1p, 2p, 5p, 10p and 20p coins to the only bank that opened at 09:00 (HSBC) and was gratified to learn that they were willing to take it all. After Elaine and I counted and recounted it last night we were only fifty pence off what we thought we had. In the end Phil had managed to amass £248.60 in his loft over who knows how may years. This should go very well towards providing a good meal for the six people who were in the room at the time the idea was mooted. Once I'd relieved myself of that load I headed to a vendor-initiated seminar on Sun's new ZFS filesystem, Veritas NetBackup 6.0, Opteron servers and some other NDA things. All things considered it wasn't quite as good as the Esteem one I went to a few weeks ago, but there was the bonus that I won a brand new iPod (20GB, like the one I have, but a more recent generation) in a prize draw. This means that Elaine can have my old one if she wants it and she doesn't have to buy one herself, which is always good. I'm in work now and while the replacement XML-Web card for the UPS hasn't arrived everything else seems to be working fine at the moment. If they continue to for another two hours I think I'll be able to go home and climbing having had a very good day. Must remember to leave in time to get home for that and to email Phil and Co. with the loose change total.

26/04/2005 [17:00] Urgh! I need more sleep. After dashing home in time for Cormac to pick us both up, we got everything we thought we'd need for the evening together and were only a few seconds behind when he turned up. After going to Phil's house to collect him and Helen we then left them behind while Cormac dropped the two of us off at the place we were to fly kites and eat the picnic. Of course, the weather began to close in ominously as we lay on the grass at the side of the car park and chatted about the day. Eventually Cormac returned with Phil and Helen and we set off up the hill to put out blankets, try flying the kites in the dead air and eat some food/drink wine. The weather deteriorated as more people arrived and we ended up wearing waterproofs while playing with an Areobie. As the light faded to a point where it was impossible to see the (spinning, fast-moving) flying disk until it was inches from your face we made a move towards the bowling alley where we were told that there wasn't any chance of a lane (let alone two) until well past ten o'clock. With it only being nine, we left rapidly for Phil's house where Elaine unpacked her cheesecake, covered it in raspberry coulis and served it to the extremely impressed assembled masses. As the night wore on we drank a lot more, found out some extremely weird things about our host (like the fact he has bags of pocket change in his loft because it's "too hard to spend", and two antlers which he'd really like to get rid of because they're both lefthand side ones) and played human Buckaroo with Helen once she'd fallen asleep. This is just like the game with the little plastic mule, only you use a sleeping human and cover them in anything to hand. We managed to get skiing goggles on her and lit candles on her shoulders without her waking up, not to mention plates, glasses and other lounge ornaments. In the end it just got too late for everyone, especially the person who'd have to drive home after driving us home so we left in good humour early in the morning. I ended up taking about 30kg of loose change away with us to count, take to the bank and then (on the orders of Phil) take everyone out for a meal with the proceeds. Getting up a few hours later was a real trial, but luckily all my work is still in other peoples' courts at the moment. I went to the other two machine rooms over lunch to make sure there was enough power for the machine I'm moving soon, prodded the people in Web Services to get their bit done and wandered round all the linux boxen with SSH and up2date getting kernel updates in place for their next scheduled reboots. The backups have been vaguely shafted because of the problems we had over the weekend and on Monday. Developers and DBAs with the root password on the staging server are causing minor problems which means my colleague is tearing her hair out. Hopefully words in the right ears have stopped that for the moment. Personally I have a monster headache which has come out of nowhere so I'm going to go home.

25/04/2005 [13:05] Already today we've had an LTO2 drive (Sun L25) jam, a disk in a concatenated stripset go down taking lots with it and further disk failures in the SAN. Sun are in and out of here like regular employees at the moment. I've been in initial consultation with MGE about our XML-Web card and have been doing some kernel upgrades on our RHEL3 boxes. Rather than wait for the phone to ring this afternoon with progress on the MGE issue I'm going to a lecture/seminar/calming-down session on the pay and grading exercise that is going to be taking place here in the next few months. I have a few questions I want to ask. Cormac's birthday today, hopefully if the weather stays fine we'll be off kiting this evening. Elaine made some wonderful cake and cheesecake over the weekend so I'm looking forward to eating them too. On Saturday we were at a picnic with Dunk and Alyzande and assorted partners, children and food. A great time was had by all, frisbees were thrown, balls lobbed and caught and birds watched. All in all, a great time. Must head off and find this presentation thingy.

22/04/2005 [15:25] I've run out of useful things to do today. I have people prodding our UPS vendor to deal with the network connectivity issues we're having, I have multiple developers working on testing suites for the new machine's applications, other machine builds in other people's courts, Scrubs (Season 1) ordered on DVD, run out of things to eat and am generally at a loose end. I have a picnic planned this weekend in the hopes that it won't rain. We'll have to see how that turns out. Otherwise we may go climbing in London again on Sunday (the Castle this time I think).

21/04/2005 [17:40] OK. So creating a set of statically NATted addresses, pointing them at the internal IPs and then creating a whole new set of firewall rules has meant that most - if not all - of my projects on there can now no-longer talk to the Oracle databases they need to. I have a large number of firewall rule change requests stacked up to give to the extremely helpful network admin people tomorrow morning. On the plus side I did go into town today to pick up my brand spanking new SSL certificate and ended up having an extended pub lunch with Bob, Mel, Charlotte, Jo and Paul. That was most enjoyable and a wonderful chance to catch up with what everyone else has been doing. I may see if I can make it there every Thursday to see other people and get away from the keyboard during the day. Speaking of which, I should head off and do all the things I need to do this evening that meant I cancelled the regular Thursday get-together I host.

20/04/2005 [17:20] Unless I'm very much mistaken I'm just a hair's breadth, or an SSH tunnel test, away from being able to say that all the applications (bar one that's being re-written) are up and running on the new server. I spent the entire morning working my way through some extremely nasty crontab stuff that was visible from one part of the file system but not from another inside a chroot. This is fairly obvious until you factor in the fact that the reverse was true for other crontabs. Suffice it to say that with some help from the original sysadmin I've been able to sort it out. Lunch with Shaun and Jenny was a chance to talk about the Lib Dem's local income tax with regard to firemen and nurses (see online) and the cost of sending your children to University these days. By the time I have children I hope things are a little easier for parents. During the afternoon I was tied up with talks with my line manager to get IPs allocted to the server in the static NATting range, emails to the developers to come up with testing strategies for the new services and a sudden announcement from one of the developers that about now was a good time to completely rewrite one of the applications to use a centralised authentication mechanism. This is great as it means masses of unmaintainable perl get thrown out of the window... I just wish he'd make a start on it some time soon. Finally today I've done my first real SSL certification signing request. This needs to be fulfilled before the middle of next week so I'm hoping it'll be processed soon. In the meantime I've created a thirty day self-signed certificate so the httpd will actually run in HTTPS mode. We're planning on setting a deadline for getting the new server online soon. Frankly I can't wait because the OS level and general disrepair of the current live server gives me the screaming willies sometimes. Climbing again this evening. Hopefully I'll remember my rope this time and will be able to do some leading again.

19/04/2005 [17:20] Wahey! A productiveish day. For most of the morning I had a go at categorising all the stuff in the live instance of the web application I'm still fighting with. This went fairly well until I started hitting bits that made no sense whatsoever. Happily at that moment the Sun engineer turned up and we spent about half an hour replacing disks in an E250R and using lots of meta* commands. Once he'd gone I handed some RHEL4 CDs to someone and settled down to see what I could do with the remainder of the files in the live instance. Luckily, one of the two people I was hoping to talk to walked by the office and I was able to grab them, sit them down in front of the computer and get rid of a lot of crap from the setup. This has made my job a whole lot easier. What helped even more was one of the actual developers of the application happening by to do something else. Quick as a flash he was snared and forced to work through the entire directory structure to point out what was necessary and what was cruft. This helped enormously. I honestly think I have a chance at cracking this thing tomorrow. As for now, I'm going home before my run of good luck fails.

18/04/2005 [17:15] We made it to Westway on Sunday. The weather was excellent, which was helpful as it took us a little time to find the place. In fact we got off the tube a stop early initially and wandered around bits of London we'd never been to before for about half an hour until a call to someone with a web connection got us sorted out. The wall was interesting. A huge selection of routes (mostly to lead, rather than top-rope). The grading system confused us and a few other new climbers for a while until we realised it was French grading and consequently the 6A climbs I'd been leading weren't actually that impressive. Owing to one thing and another we didn't last more than about two hours on the wall before we needed to go and get some food inside of us. A quick couple of tube rides took us to Tokyo Diner in Leicester Square and some much needed calories. Next time you're there try the Teriyaki Chicken, it's gorgeous. Saturday was a half-day on account of not actually getting up until gone 12:00. We didn't do much other than relax and get the house in order. Most of today has been spent exploring the largest project on that server I'm trying to consign to hell. Well, that and the UPS interface card which refuses to respond even once you've actually taken it out. I'm worried that a power failure here is going to result in some really unpredictable happenings. I've been thinking about how I'm going to vote in the upcoming election and find that I have very little feeling about the whole thing. No matter how energised I try to get about it, all I can think of is the sense that the Liberal Democrats seem to make on most things and how for me it's just an open and shut case. Replacing the council tax with a local income tax, their policies on immigration and their scrapping of University tuition fees just seem to make a lot of sense to me. I can't honestly find anything to discuss with people unless they're rabidly opposed to those ideas. Does this make me apathetic? I hope not, but I don't care (!) if they do. I'll be voting, I just don't really feel any need to get into politics any more than that. If you think there's something else I should be doing let me know. I'll happily chat, discuss or otherwise argue with you if you really want, but besides that I really can't think of anything to do other than vote. I'm not politically active, and don't really want to be. I don't think that leaves much else. Something I am thinking of doing again soon is writing stories. It's been a while since I did, and I think it's time I had another go.

15/04/2005 [17:00] I should have gone home half an hour ago but I was determined to get something working before I did. After a meeting I thought was going to be on Monday I had another meeting with my line manager to discuss the moving and test period of the services on the old server to the new server. This turned into an extended discussion of IPs, DNS, static NATting, the number of network ports and power sockets at the target site and when things could be done by. After that I was pleased to be able to sit down and get some scripts working. At least I thought they were working. Turns out I knew less about RedHat's rc chain than I thought and while scripts beginning S.. work perfectly well in places like /etc/rc.d/rc3.d/, you can't expect K.. scripts to work in the same way unless you've got a matching file in /var/locl/subsys/ for it to check against before running your script with a 'stop' argument. This took me a fair while to work out. Still, now that I have chkconfig doing my bidding with some well chosen additions to my script I think we're looking good. Or at least better. In a further attempt to be healthy and have some fun we will be going to London this weekend to climb, probably at Westway rather than the Castle, to see what it's like. Other than that I plan to sleep late, kick back and try and relax.

14/04/2005 [15:50] I've managed to move another installation from one machine to another. While I will have no problem creating a CSR for the new site I'm working on, there's one site I'm waiting for someone else to deal with (Monday) and one that is just so damned complicated that I don't really have any idea how to get it sorted and tidied at the same time. In fact I'm exremely tired right now. I could do with going home and getting a bit of sleep. I think we both could. Thursday tonight, so just some DVDs to watch and some friends around.

13/04/2005 [16:30] Still ill today, and keeping Elaine awake during the night isn't helping her either. I spent the morning at a meeting trying to get people to agree simply to split one project off from another and not digress into four or five other topics of interest. About an hour and a half later I was pretty sure that I had my answer and asked people to go ahead and do what they needed to do, as I was doing what I needed to. We 'lost' the root password for one of our servers so a colleague and I worked through breaking the mirror of the root filesystem disk, resetting the password and recreating the metadevice information. Certainly very interesting and a worthwhile procedure to remember for the future the next time I have a Solaris misshap like that. Other than that I've been feeling terrible all day because I'm not really up to being in work. I really only came in for the meeting but thought I could get some other stuff done. I think I did. Don't think we're climbing this evening as the usual climbing wall is closed. We may go to London, but I think everyone could do with saving money for once.

12/04/2005 [13/04/2005 - 09:25] Ill today. Well not actually 'ill', more an extremely sore throat, coughing and generally feeling a bit tired. I guess that's ill in one sense of the word. Anyway, cue lots of relaxing, eating too much food and trying not to speak. With no-one else being around this was fairly easy. I did go climbing at someone's home wall last night though. That was interesting. Some very hard-core climbers there, far better than me. I may go back next week, I think it'll be good for me. I'm also intending to join the local climbing club in a month or so. I've held off for a while for various reasons but now I think it's time I do what I enjoy.

11/04/2005 [14:10] It was my birthday on Sunday. Saturday was pretty good. We got up late and didn't do much (this is good). Elaine went out to get some shopping while I spent some serious time relaxing from the week. Shortly after she came home people started turning up and after a while we wandered down to the curry house via Keith's house to pick him up. The food was good, the company good and afterwards the conversation on the way back interesting. We drank wine and watched films until the small hours when everyone else went home. Sunday was good until Elaine's father (still on Australian time) rang us indecently early and wished me happy birthday. Energised, Elaine dashed off and returned with a bucketload of presents and cards which I opened in bed. I don't think I've had a better birthday morning in the last ten years. I was so pleased with how things were going I didn't even mind cooking my own birthday breakfast. We had previously decided to go to London to climb after breakfast but the opportunity to be outside in the sun rather than on a train and then indoors meant that we grabbed out bikes and headed out to cycle downriver a few miles. As the sun shone we looked at the wildlife, laughed and joked and watched dogs jumping in the water. We found an excellent pub (with non-excellent drinks prices) to stop at for a while before going far enough to watch topper/laser racing. Heading back Elaine cut the grass while I started tidying the house. A couple of houseguests later and we were feeling quite tired so I ordered some Chinese. I have to admit that we watched Air Force One before falling asleep on the sofa. This morning I seem to have a really sore throat. I'm hoping it's not whatever some other people had last week. I'm going to try and climb this evening at someone's personal climbing wall. [16:20] Whee. One and half months after I left my old work place, about one and a half months before they got around to getting someone permanent to replace me, it looks like they were hacked. What with one thing and another they're going to have to clear the machines up substantially, reset everyone's passwords and generally cause hassle for a number of people through no fault of their own. [16:35] And on cue the Dynarod guys appear to have made the smell of urine permeate the entire corridor, and my office. I'm going home before I begin to smell of it myself.

08/04/2005 [14:45] Right. I don't think there's anything I can reasonably do today. I've been twiddling my thumbs for most of the day, emailing developers to ask them to look at the live areas of the projects they're responsible for. I've got a few things done - two projects moved across for a start - but the other three are going to need some care an attention to ensure I don't miss anything important. It so happens that it's my birthday this weekend. I intend to have a good time with those people who're able to turn up on the Saturday (curry at some point) and hopefully go climbing in London at some point. I'm trying to ignore the fact that this will mean I'm entering my thirtieth year on the planet. Still, never mind. I'm not sure what else I can usefully do today. I may head off at 15:00.

07/04/2005 [17:35] I've made some stonking progress with the new live server, and also fixed some silently failing holes on the development box that we (the developer in question and I) didn't think to check previously. As a matter of fact I I've moved two of the five project areas across already. The others are going to be a real bitch, but I think it's going to happen. Once that's done all I have to do is come up with some kind of rsync thing to deal with the live data and we'll be sorted. Yeah, right. Nice to dream. I left for work this morning in a really strange mood. Wasn't sure about all kinds of things. Happily, as the day went on, I was pleased to find that my mood actually perked up on its own. How about that? Not even listening to the non-stop disturbations that are Chris Morris' Bluejam season 1 recordings managed to make a dent in what has turned out to be a fairly positively-mooded day. I really have to say that this job is so much better than my last one for making me think, work and generally improve myself.

06/04/2005 [16:50] I ran in this morning. The headwind aside I think I did less well than normal. I think it was something to do with lack of sleep again. Still, I'll be doing it again so I don't doubt I'll have some kind of improvement over the coming weeks. I sent out an email about my birthday this year and next year. Hopefully I and a few other people will be going to the south coast of Spain at Easter next year. I had planned on Canada but I think that was a bit rich for everyone's blood at the moment. I know I'll go one day, just not in the next two years. I've been doing reinstalls of what will be the new live server and have got the basic setup complete. Now comes the hard bit of getting the extra stuff on and configured in such a way that it looks like the live service, only tidied up behind the scenes. I think for the moment I'm just going to write some scripts and see what happens. I won't do another install until tomorrow afternoon at this rate. But that should be fine. Climbing tonight, so I better head off.

05/04/2005 [17:15] This morning was spent joyously updating the BIOS, RAID, backplane and PSU-handling firmware on the server I've just taken out of commission. Of course the RAID scrub is now taking up most of the rest of the day. I've also taken the opportunity to open the case and have a look at what I'm working with inside. I've removed one of the SCSI cards (the other two being on the motherboard) and turned the other one off. The box now boots in something less than half the time it used to. I was actually able to get into the chassis by dint of going via a server room of ours on the other side of town on the way into work this morning. Luckily the sister machine did indeed have keys dangling from it and I have appropriated one for this one. The remainder of my time has been spent fighting firewall rules and messy cabling in an effort to get the box connected to the serial terminal console. This one doesn't have BIOS serial redirection so you can't see what it's doing remotely until the grub bootloader comes up. But that should be OK. I didn't go climbing last night on account of things needing dealing with at home. Not that we got everything sorted, but some of it. Tonight we'll have to make a decision on one thing, or it'll happen without us. I've also thought about where to go for my 30th birthday next year and decided that Canada was just a bit too far to expect people to come with me so I've begun to plan on going to the south coast of Spain instead.

04/04/2005 [17:30] I've developed a mild hatred for RedHat in the last few hours. I got in this morning and twiddled my thumbs waiting for any one of a number of developers to get back to me with new requests for the builds I've been doing. When one of them did it turned out that they needed a whole load of perl modules installed. Weeding out the ones they actually didn't was easy enough, but getting one installed was a complete pain in the arse. Let me tell you a little story about Apache::libapreq. This little module wants a version of mod_perl less than version 1.99 but greater than 1. This was annoying as my box was running 1.99 according to the output of rpm -q. No worries, a bit of ferreting around found me a version of libapreq2 which (while development-level) would probably work. After fettling for a bit I got to compile it by hand. Lo and behold it required a version of mod_perl greater than 1.9915. Naturally I turn out to be running 1.9909. Thanks RedHat. In desperation I decided to upgrade the version of mod_perl. On attempting this I'm informed that mod_perl2 requires Apache 2.0.47. Of course, I'm running 2.0.46. RedHat really need to catch up a bit with their packages. I don't believe that there are (m)any functional changes between 2.0.46 and 2.0.53, let alone 2.0.47. And as for mod_perl, 1.9909 is extremely alpha as far as I can make out. Happily in the end I was able to convince the developer in question that CGI.pm was a simple solution to this "for want of a nail" problem and went away happy. In fact doubly happy as I took a detour by the last developer on the old development box and he confirmed that pretty much everything on the new box was working as expected and he'd be emailing his 'official' sign-off on the old box shortly. As a result I've emailed all the developers involved with a deadline of 17:00 today to get anything they forgot about off the old box before I wipe it. The weekend was pretty good. After an extremely pleasant morning, Elaine and I drove with Cormac and Steph to High Wycombe for a party. After walking from the house we were sleeping at to the party house we were fairly amazed at what we saw. Long story short; it's a house that belongs to a gambler, it's full of cool toys. We drank, played, danced and had a great time before wandering home at around three in the morning. Sunday, once awake, we went climbing in Amersham and I for one had an excellent time. I think we were all a bit tired by the end of the session so drive home. I'm tempted to go climbing this evening. But probably won't as Elaine and I need to discuss weekends away.

01/04/2005 [11:15] I don't know about you, but I don't really have time for the April Fool web pages today. I've got far too much on at the moment. On the plus side if I can get everything I want to do done today I'll have an absolutely great weekend of relaxation to look forward to with something enjoyable on Monday. I have managed to get an Oracle 10g client (for linux) installation wrapper script completed which I think will work perfectly (fingers crossed). I've just started what I hope will be the last rebuild of the first of the new web servers for one set of developers (including a nifty fiddle with a perl module to get around our firewalling issues here and a silent installation of the UPS client software as well), been told I can start moving the last developer from one of the ageing machines to the new development server (freeing up the ageing server for a rebuild as the new live box) and hopefully will be having a nice lof-fat lunch as well. It can't last. For the moment though I'm going to ignore the bad night's sleep I had last night and just go with the flow. [17:55] Well damn. I've only managed to get the final and hairiest project moved over from the old development server to the new one. Not only that but I've managed to decipher some really odd logging variables to make the logs go to somewhere sensible too! Fear me! Or don't, I don't really mind. Either way, that coupled with the pretty good Oracle installation wrapper script I did this morning and the fact that I get to go home and be with Elaine in a few minutes knowing there wasn't anything else I could have done today means I'm feeling pretty damned good. This weekend we should be going down south to a party, we'll have to see how that goes.