BOFHJournal - October 2010

1999 | 2000 | 2001 | 2002 | 2003 | 2004 | 2005 | 2006 | 2007 | 2008 | 2009 | 2010 | 2011 | 2012 | 2013 | 2014 | 2015 | 2016 | 2017 | 2018 | 2019 | 2020 | 2021 | 2022 | 2023 | 2024

September's Journal
November's Journal

29/10/2010 [16:45] Time has just flown by today, which is nice. I managed to forget my clean clothes for after my run so ended up making my run to home and back to collect some. Turns out to be around about 6km round trip, I think. Work today has been replacing the chassis (everything but the HDD) of the machine we display the Hobbit/Big Brother monitoring on after some kind of thermal event over night, rebuilding some initrds for machines which we're thinking of moving over to RHEL SCSI RDAC stuff rather than, well, not using RDAC and some further tidying up of the decomissioning stuff. Last night was fun with some sitting around catching up with people and then a late night burger and chips with Kris. We were both pretty pooped by the day so were in bed pretty soon after getting home. I've not got any rowing planned for tomorrow but aren't sure if I want to do parkrun tomorrow so soon after getting back into running following my knee issues. We'll see. Maybe I will go and do an honest-to-goodness erg instead to give Kris some time alone on a Saturday morning. Sunday I should be in a boat. I have my evaluation on Monday, apparently. I'm pretty pleased I've been keeping a logbook these last few months as it'll give me more to talk about than the last one or two I've had. In other news, we've paid for the flowers, the remainder of the wedding venue and food and hopefully put off the pushy kitchen saleswoman/designer until the new year. Oh, and hurrah! Kris has a job teaching at Dartmouth for a semester next year, which means I'll get to visit her and see where she got her degree. That should be all kinds of fun. Don't forget the clocks go back on Sunday if you're in the UK.

28/10/2010 [17:15] Well the socks seemed to do the job. In fact either the weather was particularly warm or they're even better than I thought. Either way we had a rather nice outing in the pitch blackness with me back from the other end of the boat to the bow (on bow side) again. Working on decomissioning a box today. We've finally created a checklist of all the things we're supposed to do to remove one from all the systems here now and followed it through. Have also watched an Oracle Serengeti install, patched, patched patched and read some interesting documents on deployment recommendations for Oracle Server. Now it's time to go and eat crepes, but not too many as I'm also going to have dinner with Kris at GBK too once she gets back from London.

27/10/2010 [17:10] I wimped out of going to erg last night. Theoretically I was looking after a very tired Kris who appeared to have been wiped out by the previous day's travel to and from Oxford for the stuff she was doing. In the end we spent some time together, ordered Chinese food and I took an antihistamine to help with the itching bite on my neck. It's the latter which probably lead to me going out like a light at 22:15 and feeling especially dozy this morning. We still managed to do a little bit of wedding planning before I went to work as well as putting the bins out and generally being a couple in surburbia. Today I have mostly been going to a meeting to discuss RPM updates I'm allowed to apply to six servers, the RPMs I'm allowed to remove and when I was allowed to do it (turns out it was right then). Other than that there were the usual firewall rule updates and general things that I do to make the lights stay on (or off, depending on circumstances). Definitely rowing tonight. I even have new socks.

26/10/2010 [16:55] Mostly tidy up stuff today. Emails, thoughts, plans, patching a load balancer here and there and adding new rules to them too. Oh, I also stopped monitoring a database which had been left around for no reason and was told that the new service we're adopting isn't going to be configured the way I was told it was going to be... but that the work I'd done probably wasn't a waste of time, which was nice. I appear to have been stung or bitten right on the hollow of my throat as I went for my run at lunchtime yesterday. It still itches now, but I don't seem to be dying or swelling up in any major way so I guess I just have to wait for it to go away. I just wish I knew what it was that I'd blundered into. I'm trying to decide if I want to go and erg this evening. I really should. But it's raining and Kris could do with some looking after. In fact we both could. I think a bit of time to ourselves would do us the world of good right now.

25/10/2010 [14:45] Shaun and Fran's wedding was fun. Plenty to be interested in and to think about with regard to our own wedding (in a month's time). The bride and groom looked very happy and we were very happy to be there. Over the weekend I rowed twice. Once in a 4+ and once in an 8+. Both outings weren't too bad (the Sunday 4+ being the better of the two) and neither involved rain. In fact the Sunday outing was in brilliant autumnal sunshine (so cold). The rest of both weekend days were spent either cycling to kitchen and food shops or at home doing housework and generally catching up on stuff. Kris is in Oxford today (or by now is on the bus home) and I should be going into town shortly to install a machine having done its partner this morning and fixed a few issues with the kickstart procedure at the same time. I might erg tonight, I might not. The lunchtime run was good so I might just leave it at that for today.

22/10/2010 [11:50] Half day today on account of Shaun getting married. Amazingly, last night we managed to not only get me a shirt and tie for today's wedding but also a shirt and three matching ties for myself and my two groomsmen for our wedding. I'm not proud enough to say that TJ Maxx really is good for cheap deals on previously expensive things. I'm a little concerned about the shape of the collars on the shirt for the wedding but I'm reliably informed that this is what collars are doing these days. I'm off at 12:30 today to get my hair cut, then home for lunch and to wait for a taxi which should already have a Cormac and Steph in it. Hopefully the hair thing will go OK as it's the last one before I appear in a bazillion and one wedding photos with a girl who always looks as pretty as a picture.

21/10/2010 [16:55] So last night I went rowing in an 8+. Not only was I rowing on stroke side rather than bow side but rather than being at bow I was actually stroke for the boat. On top of that it was a night outing. Practically pitch dark. Despite all of those handicaps the outing went really quite well, even with an artificially low stroke rate of 12-16 strokes per minute. I quite enjoyed myself, truth be told. I wasn't even terribly cold either, which was nice. Part of this was probably down to not getting splashed at all (being 'in front' of everyone). Anyway, today I actually seem to have managed to arrange a meeting (technical) with someone who's lead sysadmin for the service we adopted recently. This should allow me to get their servers patched (or at least tidied up a bit) within the next two weeks or so. I hope. Much of the rest of the day has been spend discussing RHEL6 with people online and rolling out an improved LSI RAID monitoring script to all the servers with that style of hardware in it. It all seems to have gone well so I'm just waiting for a server to break to test the error conditions. Off to get a shirt and tie for a wedding tomorrow (not mine). Then home for housework and sausages.

20/10/2010 [16:55] It's going to be cold in the boat tonight. Cold, and if I sit behind the wrong people, wet. That won't be good. I think this will be the first outing of the season where I consider not wearing shorts (but instead put on something longer rather than nothing at all, of course) and possibly even wear something heavier on my top half too. It'll be nice to get out in an 8+ again. I just hope that we manage to get a nice set boat or it's going to be a painful and annoying outing. Today I have mainly been setting up a whole new kickstart structure for the new project I'm about to be installing machines for. It's an Oracle-based thing and should be rather interesting. I mean, not so much for me, I'll only be doing the install and then kibitzing over the shoulders of the guy doing the install of the userland software, but it's something else I get to learn about and I've also got the opportunity to roll out some streamlined kickstart and monitoring stuff which I rarely get to innovate on because of how infrequently we reprep machines. I'm pretty sure everything's ready to go. So all I need to do is a test install or two to make sure there aren't any typos or bugs and away we go. I was late in to work this morning on account of the guy we've had out to do repairs to our bathroom, who we selected as a potential kitchen fitter, turning up just after 08:00 without confirming he'd be coming the day before. A small oversight and not really anything to worry about in the grand scheme of things but it did mean we were able to get through a good few bits of planning and discussion as well as ascertaining that the woman who'd come out to measure up had got a few measurements completely wrong. Cue him going away to talk to her some more. Although before he went I was able to show him the grout in the bathroom which has been on my mind for the last few months and get confirmation (finally!) that I was in fact right and it is failing. He's decided to rip out all of the grout on the wall the failing bits are on and completely redo it. This is awesome. Otherwise today has been about catching up on some technology developments, patching boxes (which seems to be all I do sometimes), arranging meetings where I'll discuss more patching and going for a pretty good 6km+ run at lunch time with pretty much zero discomfort anywhere (even the top of my foot). I certainly don't think I'm completely healed. Nor do I think there's no chance of the injury ever coming back again. I would like to think that I've taken a good few steps (no pun intended) towards changing my running style to lessen the chance of the same injury reoccuring, though. Right, I think that's it for now. Nothing more to say other than there's nothing quite like being able to decide on the spur of the moment not to cook one evening and instead have Chinese food delivered and have it on your plate before you've even finished doing the home made (20 minute) rice.

19/10/2010 [16:55] Mostly today I have been patching tomcat5 and hoping for the best with regard to what did and didn't get overwritten. It turns out I needn't have worried as most of the machines I was patching tomcat5 on are either not running it or are running tomcat6 instead. I wish someone had told me earlier so I hadn't held off doing it for so long. Anyway, I now have a list of what machines are running what. Normally I'd know this kind of thing but I was bypassed when it came to the installation of tomcat(5 and 6) and no-one ever got around to telling me what the final layout of services was. Still, now I know. Otherwise today has gone far too slowly as far as I'm concerned. Hopefully Kris and I will have a lovely evening at home tonight (I may or may not go to the boat house depending on how things go when I get home). Oh, and I managed to get my work Skype's sound going again, which is useful. I should head home and see if Kris' new mobile is as shiny as it looked in the shop yesterday.

18/10/2010 [16:50] Hurrah for awesome stag weekends! For lo, it came to pass that on Saturday friends descended on my home town and took me to a place where there was quad biking and clay pigeon shooting and outdoor track karting. And there was much excitment and excellence and I discovered a surprising talent for shooting things with a shotgun (missing only one of ten targets in the mini competition at the end of the session) and other people proved to be rather impressive in karts (although I was only a second slower than the fastest person's lap). And following said activities there was relaxation and showering to remove mud before going to a Chinese restaurant where the food was also the toppermost of notches. And the saki, it did flow quite freely (even the 82% stuff). I'm afraid that even on my stag night I didn't stay out much past 23:30 and was in bed (with my lovely fiancee) before 01:00. Others did stay out a good bit longer, I'm told. Sunday was more about going for walks in the sun, copious amounts of tea, bacon and chatting. Once everyone had gone home I settled in to have a quiet and relaxing afternoon with a film or two while I waited for Kris to come back from her second day of yoga teacher training. By the time she got home I had had time to realise that the weekend hand indeed been truly fantastic and that I was extremely grateful to have brilliant friends who were able to join me in celebrating the end of my batchelor life. We even got a much-missed addition to the wedding guest list! Although I was a little the worse for wear on Sunday morning I felt fine by this morning and aside from a bruise on my right knee from the steering column of my kart I appear to have come through the weekend unscathed. In fact I felt good enough to bump up my lunchtime kilometreage to my usual distance today with pretty much no ill effects (other than that new niggle in the tendons on the top of my right foot). Today's work has been mainly trying to diagnose why one of the main DB boxes decided to reboot mid-morning, getting some more web applications moved off a box which should have been decomissioned over two years ago, installing new Thawte SSL certificates (and remembering to add in the intermediate certificates straight away too this time) and getting a few more users' CVS repoisitories reconfigured correctly. Kris dropped her phone this morning and it broke so we're off to Tesco after work to get her a new one. Speaking of which I should probably finish up and get out of here so I can meet her on time.

15/10/2010 [16:05] Mostly today has been quiet and uneventful. Even the changing over of the DIMMs this morning and my 4km run at lunchtime went off without any kind of hitch or highlight (other than my knee still not hurting which means I'm up to 6km runs next week!). I eventually figured out why the javascript .el file for someone's emacs wasn't working and got that fixed and did a few minor patches to a few machines just to keep from faling asleep. Apparently there's some fun and games planned for this weekend. I'm sure I'll have something fun to tell you about on Monday, if I survive. Shame about the rowing on Sunday morning, but I think it's for the best. Kris has the first two sessions of her yoga teacher training on Saturday and Sunday so I look forward to hearing how they go. She's in London again today giving a talk at SOAS so I hope that went OK. Right, I'd best be off to Tesco before it rains and the light goes completely. Have a good one and I'll see you on Monday.

14/10/2010 [16:40] Another day, another previous evening where I didn't manage to make it to the boat house after all. I feel bad, but obviously not bad enough to have gone. There's always tomorrow. Anyway, today I was in work at a moderately early hour to bring down a server and reseat DIMMs. As the ECC errors had already gone away (at least for the moment) it seemed like a bit of a waste of time but as I'd arranged the downtime I thought I should go through with it anyway. Happily, both the system_stop.sh and system_start.sh scripts worked almost perfectly and I didn't seem to need to give a blood sacrifice to get things done or anything. Which was nice. By the time I got up to the office from the server room the RAM I'd forced out of our vendor had arrived so I've got to come in at the same time tomorrow to do the same thing, only with two new sticks of RAM. Happily I know which ones I need to replace so it's all good. Otherwise today has been a day of patching machines, patching tomcat5 on sacrificial machines in preparation for being allowed to do it on the live boxes, discovering further issues with commandline clients of my CVS/SVN service which I thought would be a nice easy transition and is proving not to be unless the clients are Windows based and GUIed and tracking down disk filling issues (turned out to be a runaway tomcat (that I hadn't patched yet) process). We're all (those who're coming) out for a Thursday this week. Some drinkies, some food, some more drinkies and then a cycle home. I hope the weather doesn't get too nasty. Also, hmmm, must remember to go and buy lots of bacon and other breakfast (meat) products for the weekend. That may put paid to going to the boat house tomorrow evening too.

13/10/2010 [18:00] I learned something very useful today. You can install herd (Sun's diagnostics swiss army knife) on a box after you've had ECC errors and then feed it the physical addresses coughed up by the OS's EDAC modules and it'll give you which DIMM pairs are at fault. Very useful when Sun's sending you out new DIMMs and you aren't sure which ones you have to replace. As it is the box is going down for a DIMM reseating tomorrow morning before 09:00 but I'll be asking for more downtime to do the RAM swap on the Friday morning at this rate. Otherwise today I made my calves hurt a bit more by going for another run. No pain in my knee again (which is good) but there's a weird ache in the tendons across the top of my right foot (you know the ones which attach to your toes) which is probably down to the probably slightly unnatural way I'm running at the moment as I try to learn a new technique which doesn't cripple me in the long term. The CVS/SVN switchover didn't go exactly according to plan today as I didn't factor in a few changes of user (running the services) and the fact that CVS stores a lot of path information in the checked out/sandbox copies of its repositories. Cue me going around a number of users' machines and manually frobbing CVS/Root multiple times. Hopefully I won't have to do it more than once per person (and there aren't that many people). Subversion appears to be fine because of the way it's run out of an apache configuration with directory aliases. Otherwise I've been patching boxes, sending emails about email bounce addresses and generally Getting Stuff Done. I should head home now as it's late and perhaps even go and do a bit of an erg as I haven't been to the boathouse this week and this evening's outing was cancelled.

12/10/2010 [17:10] It all got really busy towards the end of the day today. Patching, bug fixing, configuration file typo checking and helping other sysadmins track down missing files which had been backed up (we hope) and had then vanished. Honestly, I'm about ready to go home. Kris came back from London completely shattered yesterday. The cold she has has really knocked the stuffing out of her. That's why I'm here though, to take care of her just as much as I can. As a result (at her urging) we had delivery pizza last night (and therefore for lunch today) and watched fun things on television before catching and early bed. I should go home and see how she's doing really. Before I go though I should mention that we're starting to think about the whole kitchen redoing thing, have had almost everyone pay for their hotel rooms for the wedding and got another chunk of Kris' stuff put away. So, things are reasonably good right now. I just have to work out if I'm going to go and erg this evening.

11/10/2010 [17:00] Another one of those busy weekends. Oddly though it was almost 100% good from beginning to end. It started with going to Tesco on Friday which is always good as it means we don't have to do it over the weekend. Ran into Jenny while we were in there which was a surprise (in both the fact it has taken this long and that it happened). Still, there we go. Anyway, both weekend mornings were mainly about rowing. Saturday morning I was out in a 4+. We had a reasonable time but I don't think we work too well as a crew. Perhaps with a few more outings in that particular selection of people, but I don't think it would work. Sunday morning I was in an 8+ with a good few high quality veterans and the boat felt pretty sweet and balanced. Kris and I made excellent progress on getting things squared away at home. When Kris wasn't yoga'ing she was making delicious soup, clearing up stuff both in the house and in the garage (which we tidied significantly) and getting together piles of things to take to the charity shops. I'm pleased to say that the two Ikea Benno DVD towers were built and despite one of them being damanged (not my fault this time!) they were both attached to the sides of their respective Pax wardrobes where they look pretty dammed good if I do say so myself. We also went for a bit of a cycle, went to the hardware store for some screws, organised a replacement wedding 'cake', dealt with a few logistical details for this and that and even sat on the sofa for a few minutes here and there over the course of the weekend and caught up on some television. Today Kris has been in London doing some things connected with an installation at the Tate, I have been mostly doing bug patching (by rolling out a new version) of the application I upgraded last week, turning off unnecessary tomcat5 and postgres services on servers and the usual Monday 'stuff'. Oh! Important news in that I went for my first run since I injured my right knee at the London Rat Race two weekends ago. It can't have been more than 4km but I managed it at a gentle pace without any pain, twinges or even tickles. I also tried to run on my mid-sole as much as I can which I figure is a) going to be much better for my knee(s) and, b) give me calves the size of houses within a very short period of time. That or shin splints. I'm not entirely sure...

08/10/2010 [16:30] Today I have been mostly patching machines, doing more bug-finding on the application I just rolled out an upgrade for, relishing my last weekday of not running (in that I can't wait to start running again next week when I hope I don't find my injury hasn't healed yet), shutting down unused postgresql/tomcat5 instances (and then having to start them back up again) and joining an E.ON thinktank kind of thing and working my way up the leaderboard. I'm off to Tesco with Kris after work as we've both got fairly full weekends ahead of us. Last night was a bit odd for various reasons when we thought no-one was going to come over and then they did and we were in the middle of other stuff and couldn't host. This has shown that it's probably time I rethought how to advertise Thursdays. I'll think about it over the weekend while I'm rowing and cycling and such.

07/10/2010 [17:10] Today I have been mostly doing not quite so much. Initially in the morning there was a bit of a scramble to get some downtime on the application I updated yesterday. There were some issues with the database modification SQL scripts (not my fault) so they had to be fixed, some tables dropped and the SQL re-run. That actually didn't take more than five minutes but it was good to get it out of the way cleanly. Since then I've been... honestly I have no idea what I've been doing otherwise today. I know I've been doing something but I seem to have neglected to write it down in my notebook for once. I know I did some rsync stuff for the move of the CVS/SVN services next week, but everything else seems to be a blank. Maybe it's all the sugar I'm consuming at the moment left over from the snack attack. Bookshelf building tonight. It's time.

06/10/2010 [16:50] Well we have a new item of furniture (as yet unbuilt). Fun and games getting to Ikea, and me, the bookcase and Kris home again with two bikes. We did it though. Teamwork came into play with Kris getting the Burger King evening repast for the train and making sure she was on the same train as I was going to catch. Pretty amazing of her, really. Perhaps not the most dateful date ever, but I'm certain we'll have a better one very soon. Maybe even this weekend. Dates are important, I think. Today I have mainly been troubleshooting MySQL configurations, setting up rysnc cronjobs (which I must remember to disable next Wednesday) and struggling to deal with problematic SQL scripts which work fine on test databases but not on live ones when they're run in conjunction with application upgrades. The rest of the upgrade went flawlessly (apart from some bad programming on the developers' part which they will need to address some time soon) so getting in for 07:45 this morning wasn't in any way a waste of time. Speaking of time, it's almost time for me to go home and spend precisely two hours blitzing the house tidy with Kris before we collapse for the evening and perchance find ourselves watching a whole pile of quality TV with Italian sausage-based pasta. I might even build a bookcase.

05/10/2010 [15:50] Today I have been mostly been working on updates, RVM, replacing the RAM in a server and trying not to eat any of the huge amount of snacks which my team leader brought in to the office. I'm sure he's just trying to make everyone the same size. Having them right in front of my desk really doesn't help. I think I'm going to move them right after I publish this entry. Right after I have another rocky road cluster... or two. All going according to plan I intend to go home at 16:00 today, change bikes, pick up my wallet, railcard, a book and the Ikea discount card and take a train to an Ikea which has the bookshelf we want still in stock (apparently they're being discontinued in the veneer we want). This should see me home some time around about 21:30 this evening. Hopefully, that is. Kris and I are also going to try for a dangerous mid-travel meeting so she can help me home with the thing I'm getting. All being well we'll have a new item of furniture before the end of the day!

04/10/2010 [17:25] Rather than cause myself more injury I bowed out of both doing parkrun and rowing this weekend. Well on Saturday anyway. On Sunday morning I did go and do a 10km erg which seemed to indicate that the motion of rowing doesn't (or at least doesn't now at this stage of my recovery) cause me any pain or discomfort. Anyway, as Kris was in Oxford the whole day on Saturday I spent most of the morning cleaning and tidying what I could of the house. I felt quite a lot better for it afterwards. Adding two loads of washing and going to Tesco on top of that simply made me feel pretty pleased about the day. What time I didn't spend doing stuff I spent watching a little bit of television and cleaning my bike. Just before Kris came home I did get to make the unfortunate confirmation that we need to mostly dismantle the wardrobe in our room to move it across the floor about nine inches so as to fit a Benno DVD tower down the side of it. It'll require taking the doors off, the door rails and then removing the fastenings between each frame that makes up the unit as a whole. Not to mention taking everything out of each frame (although we might leave the shelves in). Still, I think it'll all be OK as we've plenty of cross-bracing in now to stop a repeat of the big issue we had with the other one. We just need a reason to do it now. Probably the day we get the stuff from Ikea. Sunday, after the erging I came home and happily spent much of the afternoon and evening watching TV with Kris. Of course I managed to precipitate an argument (100% my fault for bringing it up) but awesomeness on both sides solved the problem before bed time. Today has been pretty good as Mondays go. Woke up to find Kris beside me (always amazing), got to work and dried out quickly (raining again), got a rapid response from Sun after the server which wasn't memory erroring any more started doing so again, worked out which server I was going to migrate our CVS/Subversion repositories to and probably did 99% of the work required to move it all over. All I have to do now is send out the email telling people when the move will happen. Turn off the services on the old box, rsync all the repositories across and then turn on the services on the new one. That and find out when I'll be allowed to do it. Once I have it's just a short two week wait after that until I can turn off another RHEL3 server! Oh, and I think I've managed to solve my RVM issues too. Turns out that two days after installing it I'm helping the author troubleshoot and bugfix it. And now I think I have a working solution I can install on the right server so that the developer who needs it can use a more recent version of Ruby to write his code. Not that I don't think that RPM'ing off the version on RHEL5 and putting on a new version that way wouldn't work just as well. I just think it might be more sensible to leave the system one in place. Right, time to go. Perhaps erg training tonight, perhaps not.

01/10/2010 [16:45] Today I have mostly been battling with a potentially really useful bit of software called RVM. It's for managing multiple copies of Ruby on the same server (along with their attendant gems). It could be really, really good if I can work out all of its little foibles. Being on IRC with the slightly distracted author of said utility has been fairly good, but I can't help thinking he's just a bit too busy with his day job to be fully focused on the issues that people are innudating him with (like me). In other news Kris and I have had the first rough plans of the new kitchen sent to us, a few more RSVPs have arrived, I've tried wearing my wedding ring in the evening at home to start getting used to it and it's still raining. Oh, and I went to the doctor about my knee and should be getting a physio referral (NHS) in a few weeks. By that time it might have fixed itself and I could have paid £50 for private physio, but I think if I can save the money and use the advice the GP gave me in the meantime it might be a money saved. As a result, this weekend I won't be rowing, attending parkrun or aything else like that. However I will be trying to remember to do some strengthening exercises and generally trying not to eat too much. If all seems well enough I might do some long cycle rides to keep from going totally insane. Kris is off in Oxford on Saturday and involved in yoga all of Sunday so I'll be a brief batchelor again.