Menu Members Guests LTQ
BOFHcam navigation bar

BOFHJournal

1999 | 2000 | 2001 | 2002 | 2003 | 2004 | 2005 | 2006 | 2007 | 2008 | 2009 | 2010 | 2011 | 2012 | 2013 | 2014 | 2015 | 2016 | 2017 | 2018 | 2019 | 2020

January's Journal



12/02/2020 [17:00] Another useful day, without doing anything too rash the day before a holiday. I got in at a sensible time, then forced myself to go straight over to the remote server room and - amazingly quickly - got four physical servers' operating systems installed. I came back, did the NIC teaming, and got the iSCSI stuff set up all well before lunch time. That left me quite a while to get some other useful things done. Despite being peppered with annoying phone calls from users I still managed to get even more useful stuff done, patched and rebooted the Satellite server (potentially a bad idea), and even handed over a server I'd just installed to one of the other sysadmins to check to see if his SAN scripts work under iSCSI as well as fibre channel. The afternoon has mainly been about tidying small things where I can, without messing anything up as much as possible. I'm determined to get out of here at 17:01.

I'm now off home, from where tomorrow I'll head to Rachel's parents after a run in the morning, then over the weekend we'll be in Glasgow watching some indoor athletics, running in what could be horrendous weather on the West Highland Way (a simple out-and-back for three hours from Milngavie), and then be home again on Sunday evening. Monday's a speed session on the path by the river, then Tuesday Rachel and I meet at St. Pancras International to get various trains to Austria for a few days of cross-country skiing, snow-shoeing, saunas, food, and relaxation. Then home via various other trains, a night in Paris, and the Eurostar again on the Monday 24th. Tuesday 25th I'm back in the office for an early start and patching, again. Nearly done though.

First though, a much-needed holiday.

11/02/2020 [17:50] For such a poor start (I had a terrible night's sleep), I've had a fairly amazing day in terms of achievement. I got to work nice and early and every single one of the servers (physical, virtual) patched and installed their firmwares without issue (other than a known issue that I was ready for). Then all the services on them came back up without a hitch (as far as I know given that two sets of servers were running live/production services, so I left them to the people running them so as to ensure everything went as it should), then I got on with a load of other useful stuff until I was sure everything was going well, then I learned more about iSCSI than I thought possible, and even solved a problem with my particular setup which wasn't mentioned in the (very helpful!) documentation I found, and was able to replicate it to two more servers (and will hopefully replicate it to four more tomorrow), then I wrote a document about everything I'd done with iSCSI so I remember it when I come back from vacation (and everyone else knows about it too), then I had a productive meeting with my team leader, then I went for a track session run at lunch time and even with high winds managed to do fairly OK at keeping to my times (hint: I didn't hit them, but I'd have to have been Superman to do so), then I had an afternoon of building VMs, replying to email, and printing out important things for Rachel and my trip to Glasgow this weekend. Frankly, there's not much more I could have done with today.

So I'm going home. Tomorrow I'll be out of the office for at least the whole morning getting four servers installed. Then I'll come back and do the iSCSI and other stuff configuration from the comfort of my desk. And then I'll go home and won't be back in the office until the 25th. I definitely need this break.

10/02/2020 [18:05] Busy weekend, and a busy day, and it's not even Tuesday, which is going to be really busy. So, the weekend... Rachel was around for much of it, when she wasn't back in Croydon playing hockey. There was running both days, although the Sunday run was adjusted as the storm winds meant that I couldn't do quite what I wanted to in terms of intervals. We'll have to see how things go tomorrow on the track if the weather's still bad. Otherwise we had plenty of time together, Rachel did a wonderful amount of cooking and baking, and we watched a bit of television, too.

This morning she headed back on the train again to London and I got on with getting into work and then heading over to one of the server rooms to get four servers installed with operating systems. I nearly had a showstopping issue with network cards before I vaguely remembered something about an issue I'd had previously. The solution was to simply unplug all of the DAC cables that I wasn't using for the install. I'm very glad I remembered! One server though refused to install because its network interfaces were configured differently. It's a low-to-no priority server though, so I'll get around to it another time. Once the other three were installed and I'd cycled back in the rain to my desk (I picked the one time to go over to the server room without my waterproof, so got a bit damp on the way back) I got on with configuring them for use. These are my first dual-connected, multipath-via-iSCSI servers, so I'm feeling my way somewhat. However, I have some documentation which is sort of applicable, so I'm hoping to make more progress before the end of tomorrow.

For now though it's late, I'm going home, and I'll see you very early tomorrow morning for lots of patching of various servers.

07/02/2020 [14:10] Rachel arrived last night, earlier than usual, so there was a modicum more rejoicing than usual. We both got a bit peeved by a few things, but settled down again fairly quickly, and enjoyed a slightly longer evening together than we usually get during the week, which was great. What wasn't great was another poor night's sleep for me. Not as terrible as the last few nights, but still not great. I had an aching jaw when I got up in the night, and then for good this morning.

Happily though the cycle to work and the morning's patching went perfectly. That meant I was able to head over to a server room to configure a few more servers' lights out management stuff. That, and the fact that Networks have configured all the ports I need now, means that next week, if I have time, I can start installing operating systems on some of the 8-9 servers I have waiting for me. I'm not sure I'll have time to do them all before I go on holiday, but maybe I can get a few done. Otherwise, I've not done too much else today as I've been trying to ensure I get to go home early after all the extended working days I've done this week. I appear to have succeeded, other than attending a rather frustrating meeting which thankfully ended after an hour. I've had my lunch, had a quick shower as I didn't have time for one at home this morning, and now I'm going home to read on the sofa while Rachel works.

This weekend there'll be two runs, food shopping while Rachel's back in London playing hockey and... whatever else we decide to do.

06/02/2020 [17:05] A proper early morning again today. Luckily the last one this week as I had a rotten night's sleep after 04:00. I alternated between being awake and being half-dozed in a way which didn't really leave me feeling any more rested. Still, I managed to get out of bed at 06:30, strip the bedding, get dressed, breakfast, make lunch, and be out of the house by 06:50. That meant I was in work in good time and had a pretty much flawless patching cycle. Which was nice. That left me with a morning to catch up on other things rather than concerning myself with cleaning up after anything which might have otherwise gone wrong.

I ran at lunch time. It was supposed to be a track session, but when I got there there were some people in track suits which said EA Sports on the back, and they were using the full width of the finish straight, so I had to go to the tarmac foot path next to the track and run my 800m sprintervals there instead. I don't know if I got the distance exactly right, but it's possible I was running further than that. That's what I'm choosing to believe in the hopes that'll mean my intervals which were slower than 2:58 were in fact not slower than 2:58 after all. Back in the office a whole slew of actual proper work came in, so I ate lunch around doing some of that. Just as I was about to leave early, something came in which I had to at least make a start on, so I've ended up still being here gone 17:00. I'll make it up to myself tomorrow by leaving soon after lunch, I think. That has the added advantage of my getting to spend more time with Rachel, who's arriving tonight and working from home tomorrow. I will have to not bother her while she works, but I think I can manage that. Although I will be on drink-making duty for her, I hope.

So, one more semi-early morning tomorrow to patch some more servers, and then a morning of building VMs, I think. Then home just after lunch time. Unless I go to a server room and set up some lights out management cards for some servers which just got racked up this afternoon. I might do that. But anyway, home now to get the house ready for Rachel, and a relaxing evening together, if her train journey is unproblematic.

05/02/2020 [17:35] In semi-early this morning, necessitating a semi-early rise. Everything went off as it was supposed to, thankfully. Over the course of the morning I then had to deal with some of the software restart fallout from that. I did managed to schedule a few more patch/reboot dates for some more servers, and even got a few more done over lunchtime and this afternoon today. While I was able to ascertain that a pair of servers weren't doing anywhere near as much as they used to, and could therefore be rebooted during the day, the one which I'd converted to a VM, and which therefore should have rebooted the quickest out of the two turned out to want to fsck its rather large filesystem, and consequently still hasn't come back even though I started its reboot about three hours ago. I've decided to stay in the office until it's done. Hopefully within the next half an hour or so.

I'm in properly early again tomorrow morning. Hopefully the servers being dealt with will all behave nicely and I'll be able to have a relaxing morning so as to be able to give a bit more to my track session at lunch time, not to mention going home early so that I can go to Tesco for dinner and then get the house in order for Rachel's arrival in the evening.

04/02/2020 [18:00] Early start this morning. The first of four (although only this morning and Thursday morning are truly early). The patching went pretty smoothly, if I'm honest. Everything happened as it was supposed to, apart from a few things due to DBA ineptitude which I had to fix. After that though, although I didn't realise for a few hours, something was epically broken and it wasn't until I'd done a lot of troubleshooting and been down a few blind alleys that the DBA responsible for the issue worked out what was wrong and fixed things on his end. We have a much more resilient solution in place now, but it's still a bit poor that we didn't check out what he'd done sooner to see if it was a working change he'd made. After that I had about a third of the morning left to do lots and lots of other things, which meant that I didn't get out for my run until around 13:00. A fifty minute run meant that I had to go straight from a rapid shower into a 14:00 meeting (which I was a few minutes late for) without having my lunch. This made the next hour a bit frustrating as I really wanted to eat (and not be in the meeting). Once I escaped, it was an afternoon of trying to get fiddly things to work, and preparing for tomorrow morning's patching, which I hope will go at least as well as this morning. It's all one service, and none of it is proper live stuff, but I won't tempt fate by saying it'll be fine. For now though I'm off home to eat a sensible dinner and then try not to pig out on unsensible things like Christmas cake.

03/02/2020 [17:30] I spent the weekend at Rachel's where a good time was had by all, I think. Saturday morning I ran to, around (in the mud) and back from parkrun, Rachel marshaled and then went to play hockey. In the afternoon there was relaxation. Sunday morning we went our separate ways to do long runs. Mine was around Croydon, Rachel's went into central London where she then took a train home again so she could collect her bike from the train station. In the afternoon we did a bit of logistics for our holiday, and did more relaxing. It's very important, this relaxing. When you're running as much as we are, you have to take downtime seriously. Both of us are still suffering from the tail ends of our colds so I don't think either of us were at our best. I certainly didn't feel like I was.

Rachel packed me off with a mega-lunch this morning. My train home was fine, as was the cycle into work from the station. At work, the most important things I did today were to go across to one of the server rooms and get four servers wired up for Network, ILOM, and iSCSI connectivity, as well as configure their BIOSes. Now I have to wait for Networks to put all the ports on the right VLANs for me. That could take a few days. I may have to nudge someone. The other useful thing I did was to get a load more servers scheduled for patching and rebooting. The downside of that is that I'm now in early or very early every remaining day this week. Although it does mean I can leave early every day because of that. I might even get to go home in daylight.

I'm running tomorrow lunch time, so I very much hope tomorrow's patching goes without any major issues. For now though I'm off home to do grocery shopping for the week and have a relaxing evening, hopefully.