BOFHJournal - November 2011

1999 | 2000 | 2001 | 2002 | 2003 | 2004 | 2005 | 2006 | 2007 | 2008 | 2009 | 2010 | 2011 | 2012 | 2013 | 2014 | 2015 | 2016 | 2017 | 2018 | 2019 | 2020 | 2021 | 2022 | 2023 | 2024

October's Journal
December's Journal

29/11/2011 [17:10] I resigned today. This morning to be exact. I am now officially in my 3 months notice period. This would be because my visa interview date has come through (actually it did a few weeks ago, but don't worry about that). Yes, I'm aware that I shouldn't make life changing decisions before I actually have the visa in my passport, but work have very kindly said that if for some strange reason I don't get my visa they'll be happy to tear up my letter of resignation until I do get it. Wait. Did I not mention that I was going to be moving to America? I'm pretty sure I did. It might have been a while ago and we all just forgot. Anyway, yes I'm moving to America as we weren't able to find a job for my good lady wife in this country. We've shipped pretty much all of our worldly belongings already, I've sold the house (to my ex-girlfriend's ex-landlady who's very happy for me to rent it from her until the day I leave, so that's good) and am selling all the furniture and electrical items which won't work on America's stupid voltage to her when I go (only I haven't sold my amplifier and speakers yet, which is a little concerning) and I'm living in a pretty sparsely-decorated house right now and feeling... well, rather OK about it. Minimalism is something I could get used to I think. So yes, the visa interview is just before Christmas down in London at the US Embassy (actually the day we were supposed to go North to my parents' for Christmas so we're actually traveling separately now (given we'd already bought train tickets... I had to buy another one up from London instead), there's a medical thingy too, but that shouldn't be any kind of problem. I think that's the big news of the day. Yesterday's return to exercise (6K run and gym in the evening) left me feeling a little light headed at the end of the gym session and a tad achey today. The latter can only be a good thing. The former I'll work on by staying a bit more hydrated I think. I was tempted to go to a running club this evening, but I honestly think that this week more than most I need to have days where I don't do proper exercise in between the ones when I do. Maybe I'll go next week instead. I'm running and gymming tomorrow, so it'll be fine. Other than that... that seems to be all the news I can remember from today. Really quiet for work in my team at the moment. Or it's not but my team leader is hiding everything from me and/or doing it all himself. That'd be mildly frustrating if so. I think I will go home and have some Thanksgiving leftovers for dinner.

28/11/2011 [16:50] Wotcha. I'm back. Kris has been dispatched back to the States again to do her thing while I do mine here. We've had a wonderful time for the last ten day or so. After she spent the day in London at a conference on Saturday last we've spent five days in Polesworth in an abbey gatehouse which had been superbly renovated into a perfect hideaway for two people, been to the Christmas Markets in Birmingham, walked around the countryside close to the Abbey, visited Stratford-uopn-Avon (Americans love Shakespeare) and, after returning home on our one year wedding anniversary hosted a Thanksgiving dinner for seventeen people (sixteen plus small children) which has left me with enough delicious leftovers to see me fed until Kris returns for Christmas on the 15th. It was really, truly fabulous to have Kris back again. I can't wait until it's a permanent state of affairs again (us being together). More on that in the next few days. Kris was on the 03:00 coach to Heathrow this morning. As usual, waking up at 02:00 was unpleasant and put a bit of a downer on the day. It also involving Kris leaving didn't help. Morning exercises did restore some zip to the day though, which was goodish. Ten or so days without them was definitely beginning to be noticable. Another noticable effect of ten days without hard exercise was that the lunchtime run was much less pleasant than its usual slight unpleasantness. With luck a few days of getting back in the groove will see me right. It's good that there's a mass of food (although not as much as you might expect) left over from Thanksgiving. It means I don't have to think about meals besides reheating and adding a few things for the next few days. That'll be good tonight when I get back from my first gym session in ten days, too. Hopefully Kris will make it back to the apartment safely some time in the next four or five hours and I'll get a text, phone or Skype call from her saying so. I'd rather not go to bed until I know she's back and safe. As for me, the daily grind restarted today, although with a brief period of learning about this here place's "values" and how we can work smarter/harder/whatever-er to embrace them. Or something like that. It all felt a bit pointless to be perfectly frank. I engaged in the discussions more to stay awake than because I thought it would do any good. Apparently our comments (those written down at any rate) will be passed up to director level unvarnished and unedited. I can't see it causing any massive changes.

18/11/2011 [16:35] A slighty less poor, but still not awesome night's sleep. Of course, there was a highlight in that Kris was there, but being tired and generally half-asleep wasn't great in the early morning. Hopefully as I get used to her being back with me things will settle down and start working correctly. That would be nice. Work was again quiet today. Just a few general systems administration tasks to do like creating SVN repositories, setting up users for Redmine to interact with SVN, cleaning up Cacti warnings and generally keeping things ticking over. Went out at lunch time with Kris to see another bank person about money. While he was practically useless in every way he did highlight some interesting issues we'll need to address. Anyway, enough about that. I'm off for the weekend to compete in a (rowing) race with old guys on Saturday while Kris is in London giving a talk, spent all of next week in an abbey gatehouse with Kris celebrating our first year of being married and then hold a Thanksgiving meal for 18 people (17 + three small children). It's going to be brilliant. However, for tonight: gym and dinner! Have a good weekend, week, weekend again and I'll see you bright and early on the 28th, having just said goodbye to Kris at 03:00, again.

17/11/2011 [14:50] Patched ten machines this morning, one of them a DB box, all of them the live "flagship" service for here. And, aside from having to wait an age for updatedb to complete on them, everything went perfectly. What wasn't perfect was having to leave Kris at home at 06:30 so I could get it done in good time. Still, she was shattered from the travel and we both slept terribly last night so it was good for her to sleep in. Given she usually gets up at 05:00 where ever she is, sleeping in until 10:00 definitely meant she needed it. We went out for a curry last night as a celebration for her being back. The local (excellent) place opened at 17:30 so we went there and had the place to ourselves, which was nice. Having walked to the house from town it we had big appetites, which were excellently dealt with by starters and main courses. We met up at lunch time today to see a bank person about money things and I'm about to head back into town to meet her again (early arrival means early departure from work) to sit in a bookshop cafe and read and have nice things while she finishes her talk that she'll be giving on Saturday in London while I'm rowing. We'll head to the pub this evening to see people for a little while and then go home and collapse a bit more. Thank goodness it's Friday tomorrow.

16/11/2011 [14:15] This morning would have gone a whole lot more smoothly if I'd actually followed the instructions I'd painstakingly written over the last few years of doing the exact same patching routine I did this morning. Forgetting to comment out mount points, not rebooting in single user mode, not them unmounting semi-failed mount points before attempting RDAC driver and initrd installation. These are all schoolboy errors I could have avoided if I'd just read my own instructions. Still, it all finished properly and I had the machines and service back up right on the dot of 08:00, an hour before it was supposed to be. So that was OK. In contrast to that, last night's two hour driving refresher lesson went really quite well. For the first eighty minutes or so I drove around town and didn't stall once. There were a few reminders to check mirrors at obvious junctures (and junctions) and some advice about changing gear more in anticipation of the road conditions ahead, but otherwise it was pretty good. I did get a few stalls in later on due to clutch control issues, and my parallel parking needs to be a whole lot slower, but I think on the whole I'm safe to be in the car with (or in a car on the same road as). Which is nice. Of course this doesn't mean I'll be any good in a car where people drive on the other side of the road, or in an automatic. But knowing how to drive a manual should mean that driving an automatic just means fewer things to think about at important moments, right? Speaking of important moments, Kris is even now approaching town in a coach from Heathrow. I'm due to get out of here at 15:00 which is pretty much when her coach arrives, so I'm going to go and meet her at the coach stop and we'll walk home in the bright, clear, cold day. Given I arrived at work in darkness and recently have been leaving work in darkness this should be a nice change. Also: wife! A web developer seems to have dropped a (sort of) live database from the live web database server. He did this trying to install some new software and believing that the database on there was... well, I don't really know what caused him to delete it, but he did. He also installed a new version of the softare as he couldn't configure the version I'd installed as part of the build process for the servers I'd created (not my fault). So all morning we've been trying to work out what database has gone missing considering the one we restored (from verified-as-working backup and hotbackup scripts) doesn't seem to a) have any data in it and, b) hasn't been altered since August. It's all a bit confusing given the person who installed it all in the first place doesn't seem to understand what's going on either and is part of the reason why we normally do the application installation so we know what the hell is going on. I'll be glad to get out of here in just under forty-five minutes and make it someone else's problem. At least for the rest of today. Tomorrow morning I'm taking down all of the live instance of our flagship service (web, app and DB layers) for RHEL patching. All being well all should be well but a) I'm having to leave Kris earlier than I would like and, b) it's a live service so I'm sure something will go wrong. I think we have enough servers that if one of the tiers goes tits-up we should be OK, but I'm reasonably sure it won't.

15/11/2011 [16:50] Tonight I am having my driving refresher lessons. 17:30-19:30. It will be interesting given I haven't driven since I had my test (which I passed) in 2002. I imagine I will have next to no clutch control and could even find myself failing to look at my mirrors enough, signaling too late and potentially hitting curbs. I may need more than one two hour lesson. Still, it's about time I dusted off my skills so that Kris can have a break from driving sometimes. Especially from the middle of 2012 onwards. Went for a run at lunch time today. After last night's just-under-8K erg doing a flat-out 6K run was a bit painful, but I think I did pretty well. It is a wonderful feeling to know that the discomfort I feel in the first kilometre or so dies away and I'm left simply feeling like it's hard work rather than painful. Work has consisted of preparing for tomorrow's patching at the other server room (FC detachment necessary during one of the two reboots I'll be doing), doing lots of package downloads, and learning a bit more about configuration file parsing with Augeas. I've even discovered a bug in it, which is nice. As I write this Kris is doing her last class and then heading to the apartment and thence to the Metro and the airport. In approximately 22 hours she'll be here again! Woohoo! It's going to be wonderful! I should do the washing, and the washing up, and probably shave. I'm glad tomorrow morning is an early start as I doubt I'd be able to sleep in, and I get to leave early too! Right, time to go home and find both bits of my driving license.

14/11/2011 [16:25] Had a pretty good weekend. Broke my 5K PB substantially (19:07!) which means I have a new target for the end of the year (oh dear). Dashed home to shower and then out to the train station to get to a friend's birthday meal in London. Walked from the station to Camden, which was fun, had food, felt pretty ropey and have put it down to the effort I put into the run in the morning. Came home at the end of the day after a pub visit and didn't really eat anything much for dinner. Slept terribly but got up feeling OK for rowing with some very good rowers in a 4+. Came back from that and was at Tesco before it opened (must remember it doesn't open before 11:00 on a Sunday). As well as chatting to Kris and setting the world to rights, spent the rest of the day cleaning the house and generally trying to get some relaxation in in the midst of two loads of washing. The weather this weekend was so mild I had all the windows and doors open and the washing out in the garden. Today has been quiet, which is always nice for a Monday. I've fixed a service check I had pending which was giving sort-of-not-quite false positives and now gives real positives. I've also been looking into another service check which may prove to be extremely difficult/impossible to automate with curl and artfully constructed data arguments. Silly Oracle for not using the same authentication scheme across all of their application stacks. If anyone knows how to provide the password to an Oracle Discoverer viewer login page, do get in touch. I was going to be having a driving refresher lesson this evening, but that's now tomorrow evening instead, so I'm off to the gym for a 30 minute erg and a little of bit of weights instead with Tom.

11/11/2011 [16:20] Big news today! Well, last night, but I only found out this morning so it's more today than yesterday. Anyway, I can't tell you quite what it is yet, except that it's big and it marks the final phase of something which has been taking place for the last few months. As soon as I can tell you what it is I will. I doubt you'll be surprised, but for various reasons it makes sense not to tell you, yet. Anyway, thinking about that, doing logistics and spending a bit of money all happened this morning. Making telephone calls and putting dates in the calendar happened this afternoon. I think it's all going to work out. No exercise at lunchtime today after yesterday's rather hard run. That meant I was able to get some good stuff done on setting up a web server for someone and answer questions about modules, Redmine, Phusion Passenger and things like that. Not much else has been happening at the moment. I do have one bit of work I've been putting off (a service check for our Oracle Discoverer service which I set up the servers and load balancing infrastructure for a few weeks ago). I'll get that done on Monday, I imagine. This weekend I'm running (parkrun), heading to London for a friend's birthday lunch, spending some time in London afterwards, rowing on the Sunday morning (probably) and then going to Tesco. Oh, and I should probably dash around the house and do a clean (and tidy, although there's not that much to tidy these days).

10/11/2011 [16:30] Phew! With the help of my team leader I've managed to save the Cacti installation I just accidentally upgraded. Stupid cacti-plugin-architecture and updates to /cacti/include/config.php. The run today was my attempt to do a flat out 6K. I think I did pretty well. Without a watch I can't say it was better than my 'personal best' of 24:35 (unofficial) but it certainly felt quick. I'm seriously looking forward to ordering a pizza for dinner. I know I shouldn't as I've already eaten out once this week (Vietnamese with Kate) but dammit, I'm an adult with some disposable income and I deserve it. Plus I forgot to get anything out to defrost this morning. The most exciting part of today (other than the run) has been a discussion (online) about the merits of postfix vs sendmail, RHEL6 and puppet. I really do think IRC has become a great tool for me in learning more about Solaris and linux. Plus sometimes I actually know what someone's asking about and can even provide answers. Which is nice.

09/11/2011 [15:30 - 10/11/2011] Ooops, completely forgot to do this on the day. So, let me think back on what happened... Almost nothing at all. Work happened, things were done, I ended up leaving work past 17:30 due to just trying a few things out. I did go to the gym after work and after seriously considering not going at all, did a monster workout and even a bit of an erg on top of that. I think I came quite close to doing too much as I felt quite lightheaded at one point. Luckily I appear to know my limits and kept things together. I definitely think I might be owed a pizza because of the effort I put in.

08/11/2011 [15:00] The last of the consecutive early mornings, this morning. And it all went pretty much flawlessly. Which was nice. There were some temporary worries about Debian guests rebooting and a minor problem here and there with incorrect interpreters and typos in startup and shutdown scripts, but otherwise it was great. This was nice as after two 30 minute ergs last night (I was supposed to do 40 minutes with Tom, but he didn't turn up until I'd just done 30 minutes and stopped, so I did another 30 minutes with him) which tired me out fairly comprehensively. I was actually tired this morning and slept all the way through to my alarm at 06:00. The lunchtime run was good. 8K, but pretty slow. I combatted this somewhat by running loops on the 'hills' to let Hazel and her running friend catch up. I'm off now to do a psychology experiment for two hours and get some money for it... which should pay my half of tonight's bill at the Vietnamese restaurant Kate and I will be visiting.

07/11/2011 [16:00] Another Monday, another 06:40-in-work-for-patching start to it. This time things went well again (aside from an Oracle database not wanting to shut down properly and an NFS mount hanging on machines being rebooted) but I'm more concerned about tomorrow when I'm doing another early morning but patching an entire service all in one go. This time it's not just a series of Xen hosts and MySQL servers, but the Xen guests are Debian. Not that I'm patching the Debian boxes, but I know very little about how they hang together and how the services come up, so if they don't I'm going to have to start calling people fairly rapidly. After that I'm not due to have another early morning until the 16th. Over the weekend I failed to get a PB at parkrun (only a second off my current one though), went to a pretty good Guy Fawkes Night fireworks display which I viewed from the boathouse with friends, went for a curry and had a "Nurpuri" which turned out to be extremely tasty and probably the hottest curry I'd go for and still enjoy utterly, failed to sleep in on Sunday morning, went for a 22K run (although with some stops to coach a crew from my boat club and rig another boat at the boathouse), chatted to Kris, went to Tesco, and watched a whole lot of television I'd fallen behind on. That's about it really. As I said, this morning's patching went OK. Since then I've done some work on the logistics for a get-together we're planning, called some people, reverted someone from x11vnc to VNCserver again (stupid crashy X servers), booked some refresher driving lessons for next Monday evening and spoken to Kris on Skype (I really think I should buy shares). I didn't run today as I figured a day off from that might be a plan, although I am about to go and do a 40 minute erg with Tom at the gym in about half an hour.

03/11/2011 [15:45] Well, that's now a complete working week of getting in before 06:45 and performing operating system updates without anything going even moderately wrong. I find this gratifying. Unfortunately the rest of the days have been remarkably empty relatively speaking. There've been the usual bits of troubleshooting for users and bits of things to read about and learn but for the most part it hasn't been the busiest of weeks. Being a good "self-starter" is helpful, but at the end of the day it would be really nice to be handed a pile of work and told to make my way through it either as quickly as possible (we like deadlines) or in whatever order I deem to prioritise the work. Maybe next week. I ran out of bread this morning too (forgot to defrost some last night) so ended up with a sandwich from one of the vans. Despite it having bacon in it I really don't think it was as nice as the ones (with salad in) I make. Still, it was good to have a change. Exciting reading, huh? Right! I've definitely run out of things to say about this week so I'll do the usual weekend in prospect thing (gym tonight, parkrun, Tesco, chatting with Kris on Saturday (plus Guy Fawkes Night at the boathouse), and probably a run on Sunday plus a psychology experiment for a bit of spending money over the lunch time bit when I'd normally already be embedded in the sofa for the afternoon) and sign off.

03/11/2011 [16:00] Patching went well. Although I did forget one step and had to go back and reboot the box a third time to make things 'stick'. It didn't really matter as I was still well within my downtime window. Most of today I've been preparing the eighteen servers I'll be patching tomorrow. Reconciling patch lists, getting all of the servers to the same level and fixing issues thrown up by the developers having asked me for software three years ago and then never using it (the self-same software now being responsible for blocking other updates from now happening). Gym last night with Tom went pretty well. Having to wait between sets meant more recovery time so I opted to put the weights up a notch or two. This resulted in me feeling pretty pumped afterwards and quite leaden on my run this lunch time. I'm glad I'm not doing anything else other than cycling home and washing up today. Said hello to Kris just after lunch too. Like me she's really beginning to feel the being-apartedness. I really do think that 40-50 days is the maximum we can be apart without it getting unbearable. I'm so pleased that she was able to find the space in her schedule to make it over at the half way point 40-odd days ago. She's wonderful. And in other news, I may be entering some duathlons, triathlons (non-traditional) and quadathlons over the next few months. It'll be nice to do something other than parkrun on a weekend.

02/11/2011 [16:00] This morning's patching also went well. Betterer than yesterday in fact as the machines being patched really weren't doing anything that was impacted by anything other than the glibc, kernel and multipath drivers. In fact we think we might have solved some problems by moving up about three revisions and two new versions of the multipath stuff. Two more machines patched this afternoon (part of the backup suite of machines) resulted in a bit of a problem with one of the attached tape drives, but we're used to that happening every time we reboot that box. I think it's all fixed now, anyway. Tomorrow morning is just one machine, but it requires me to stand over it and disconnect and reconnect FC cables between reboots while I get RDAC drivers compiled into the newly installed kernel. Mildly annoying, but worth it for the reduced boot times (touching /fastboot also considered helpful) and getting this particular machine (one of four, the others also having been scheduled for the near future) out of the way for another six to eight months or so. Naturally, having been in early again I'm off early to head to the gym and meet up with Tom. We'll see how that goes. Otherwise... I think that's it for today.

01/11/2011 [15:00] Another day, another early morning start. This time I was firmware upgrading two servers and patching three more (two of which were Xen guests on a third). The firmware updates went almost without a hitch other than the fact that one of the servers didn't boot its BIOS after being updated and it took two CMOS resets and a cold power cycle or two before it decided that life was worth living. The software patching seemed to go perfectly except that the two Xen guests didn't boot due to the Xen host not remounting its NFS share and the blacklisting on its SAN stuff being wrong (despite it having worked when we put it in place). Anyway, everything got sorted, including a piece of badly-written software having to be restarted a few times before it would accept logins. So yes, everything was working by around 09:30, which isn't too bad, really. The rest of today has been spent checking on all the little updates which may have sneaked in under the radar during the major patching that took place, reading up on some SSH and KVM stuff, learning more about multipath, going for a 6K run and making an appointment to get my hair cut. I think the run felt a little harder than normal due to the fact that I did my first 30 minute erg in a few months last night with Tom (an AR friend). I'm glad I'm not going to the gym this evening. Chatting to Kris is much more restful. Tomorrow is another early morning start. Only five machines and two of them are in the afternoon.