[17:10] And so we reach the end of March. Next month is going to be quite an
interesting one. Especially if this strange ache in my left knee doesn't pack
up before the marathon. It was noticable during this morning's run (only 1h15m
rather than 1h30m), and has been noticable for the rest of the day (but in a
different part to that which it did during the run). Seeing as I'm into my
taper I won't be running too much further this week (24km on Saturday, 10km on
Sunday) so that should help. Anyway, this morning's run was nice enough,
especially as the weather was good. Not only that but the bike is behaving
itself, and more importantly Rachel is here this evening (which makes the day
the best of the week so far).
Work-wise I had a few meetings about stuff, including the first vacation cover
for systems I haven't encountered before but are being thrust my way as part of
my new team leader's territory. He's the one going on vacation for a week and a
bit. When I go away there'll be two people who know my systems (even if
neither of them are theoretically still responsible for them), so that should
Right, I should head off, get bits for dinner, and then be home to wait for
[17:00] Early start this morning for one of my last early morning runs. I did
the full banana, the full 1.5 hour run, probably for the last time. Having then
done that, come into work, and spent the morning reading many many articles on
tapering (provided kindly by Rachel) I am trying desperately to embrace the
'taper'. The part of the training schedule where less is very much more, in
terms of allowing your body to prepare for a race pace held over 26.2 miles. As
Rachel says, I've trained really hard for this marathon, and it would be a
criminal shame for me to mess things up now by going up to the start line
without adequate rest and preparation. Therefore tomorrow morning's run will be
shorter (if not a little bit slower, too). Likewise the weekend runs will be
shorter, as will the following week's. I'm looking at 'only' 80km this week
(down from 100-104km), then 60km, then perhaps as low as 40km in the final week.
We'll see how that pans out.
I collected my commuter bike last night. They've upgraded the disc rotors to
heavy-duty ones, replaced the pads, and made sure the pistons actually retract
when you stop squeezing the brake levers. Hopefully this'll make things a
little better. In other news work has been nice and quiet today, with just a
few maintenance tasks, some emails, and the usual daily stuff to do. It is,
however, home time now. So I'm off.
[17:00] Easter is over! I had a really wonderful time. On Friday I spent a
good chunk of the day helping Cormac clear stuff out of the house he's inherited
and is in the process of doing up, then got a free lunch out of the deal, which
was great (venison and blueberry burger). Then in the afternoon I got a few
things done at home whilest waiting for Rachel to arrive. Oddly, I developed a
stomach ache (something I haven't had in months), which lasted into the evening
and then overnight. This probably contributed somehow to the really poor
running I did with Rachel the following morning, only just managing 34km slower
than I'd done that particular route before. I recovered reasonably well though,
and we put it down to just having an 'off' day. After heading to Tesco for the
regular shop I cycled over to the bike shop with my commuter bike to get the
brakes sorted once and for all/again. I pushed another bike at the same time,
which meant I could head home fairly quickly afterwards. On Sunday we went for
a much shorter and significantly faster run together which I was somewhat
happier with, thankfully. My mind has gone completely blank as to what else we
did with the day, though. I'm sure it'll come to me eventually. On Monday
though we had a very lazy morning followed by home-made pancakes, maple syrup,
blueberries, and bacon, before cycling into town to select guide books, baked
goods, browse outdoor shops, buy second hand books, and lunch at a Patisserie
Valerie, before coming home again. Such a late lunch lead to a late dinner,
which was preceeded by me doing a nice quick 10K in the lovely spring weather,
and Rachel catching up with a bit of work whilest I was gone. This morning I
was absolutely shattered, even though I have officially started my taper for the
marathon. That didn't stop me from staying in bed until 08:00. I came into
work on my best bike today, but will be going via the bike shop on the way home
to pay pretty much £100 for new rotors, pads, and labour. I hope things
are fixed properly this time. Then I've a mass of washing up to do, before
settling in for a quiet evening. Tomorrow morning is the first of the week's
early mornings for running. Happily, only 1 hour and fifteen minutes, rather
than 1 hour and thirty minutes. The taper continues!
[17:10] It turns out I ran, pretty much to the meter, this morning what I ran
yesterday morning. I think this morning's run felt a little easier though,
even if the heart rate track doesn't seem to quite support that feeling.
Anyway, no major niggles/injuries to report, and a nice hot shower afterwards.
Then I cycled to work and spent the day doing useful things for the most part.
I even built a VMWare guest for someone in a few minutes so they don't bother me
just before I head off to do the marathon in a few weeks. Much better to get it
out of the way now. I also showed my new team leader around the server
architecture I have to look after day-to-day. He's a huge fan of moving things
to VMWare over physical servers. I guess I can get behind that, it just feels a
bit... odd to be dispensing with so much tin in the future.
It's Easter weekend here, so I'm off tomorrow and Monday (four day weekend!).
Tomorrow I'll be helping Cormac do some stuff at his house (filling a skip).
Rachel will arrive in the late evening just in time to eat Indian delivery food
with me. Saturday we'll go for a long run, I hope. Sunday a shorter run.
Monday... I'm not sure yet, but I think Rachel may be heading off at some point
to see other people. I'll be doing a shorter run again (as the plan dictates).
Then it's back to work on Tuesday. Have a good few days and I'll see you next
[17:00] This morning's run was a strange combination of hard and easy. Well, I
say easy, it was when I settled into my stride/rhythm. Hard because, well,
running is hard work when it's over medium to long distances. I'll do the same
again tomorrow and be happy with whatever I can do for 1.5 hours first thing in
the morning. Hopefully it won't be raining. I did run with my new running belt
with two gels in it, to get used to the feeling. Other than (I think) it
causing a blocked pore I didn't notice it all during the run, which bodes well
for the marathon.
At work I spent the entire morning patching every single server and workstation
we have, by hand. This is because of various letting-downs by my organisation
as well as Red Hat and Oracle. I won't go into it, but suffice it to say I now
have all my OS installations up to date, at least for the next day or so. And
other than the servers I'm still not allowed to update, yet.
Thankfully the afternoon has been pretty quiet, with me being able to catch up
on email, logistics of various things, and generally reading stuff on the
internet. I'm going home very shortly for a nice quiet evening, and then
tomorrow morning's another run and then a day of work.
Oh, I figured out what was roughly wrong with my bike's hydraulic disc brakes
and hopefully the bike shop will be able to fix over the weekend, when I leave
the bike with them.
[17:00] I'm actually really happy that today is a rest day. It has, I think,
allowed my right achilles a bit of time to recover from the abuse I've been
subjecting it (and the rest of my legs) to these last few months. That, coupled
with not having to get up early this morning for patching meant that I could
wave Rachel off and start my day fairly gently relatively speaking. At work it
was odd to come in and see my old team leader's desk completely empty (and now
cleaned by the cleaning staff). The dynamic for the period of time we're all
still in this part of the building is going to be very different, I think. Even
with someone else moving into the space just this afternoon. He's a lot
quieter, and much more like my ex-team colleague (who's now this new person's
team leader). Anyway, over lunch I cycled out to my favourite running shop and
bought myself a minimalist running belt for gels for the marathon, as well as
two gels from the brand that I usually use, only with caffeine in, to see if
they make any difference on my last few long runs, of which I have two proper
ones left before I begin to taper for the marathon itself. This afternoon I've
mainly been looking over the results of the all-systems script I ran this
morning and am pleased with them, as well as installing a test VM to see if
COBOL (don't laugh) will install and run on RHEL6. More on that as the week
goes on, I imagine. For now I'm off home to keep myself occupied without
Rachel, and then to be up early tomorrow morning for my usual Wednesday morning
pre-work run. I hope the weather remains as nice as it has been today.
[17:25] Rachel came back! She arrived back from Africa all present and correct
and, following a day of working (Saturday!) as well as a run, came to see me. I
filled up my morning doing a long run and house work, before settling down to
fix my media PC, that I'd managed to break the sound on in a very interesting
way by fiddling with the Registry a bit too much. Anyway, she arrived on
Saturday evening after I'd been to Tesco and much happiness was had. We slept
in on Sunday morning. Something neither of us has had the opportunity to do in
quite some time. We weren't very good at it, but gave it a good stab
nevertheless. This left us with time to do a nice 15km run before Rachel had to
head off across town for a meeting for one of her microfinance groups. I
fiddled with things and caught up on email until she got back. We then had a
quick look at a few things for America and Iceland, then settled down with
dinner and a DVD of Inspector Morse, which we're working through the whole set
of episodes. This morning was then a normal Monday with Rachel here, in that
she had to get up early for a train. Not back to London though, but to Bristol,
to meet some potential donors to her other microfiance group. Not only that,
but she's back again this evening, which is great. In fact if I don't get home
soon she'll start on the washing up before I have a chance to.
Work-wise I have some new hard drives to fit into a server that's running low on
disk space, I helped move my old team leader of my old team into his new office,
tidied up a few things in and around his old work space (I hope the cleaning
staff vacuum around there before tomorrow's working day), and discovered that
many of my Red Hat installations don't appear to be reporting their
out-of-datedness to Red Hat Network (Classic), so I'll be doing a manual resync
tomorrow morning by using some of our internal infrastructure. Just as soon as
I'm reminded how it works by one of my colleagues. Home time now though. So I
can clean the kitchen up before Rachel gets home.
[16:40] A fairly pleasing day in general. I chatted to Rachel via text and made
sure she'd gotten home safely from Tanzania. Hopefully she had a bit of a nap
over the course of today, rather than just working solidly off the back of an
overnight flight. I got a good few emails sorted, dealt with, replied to, and
filed. Some small progress was made on my birthday celebrations. I'm still not
sure if anything other than an evening meal is going to happen, but we'll see.
I went for a nice gentle run at lunch time. Wonderfully short in the grand
scheme of things, and the weather played ball in terms of only being cold rather
than cold, windy and/or wet.
Home time now. Pizza for dinner, along with some entertaining television. The
weekend should be pretty great, with a long run tomorrow morning, then Rachel
arriving whenever she has the energy and inclination to get on two trains (I'm
so lucky she's happy to come to me so much). There'll be a Tesco trip in there
somewhere, too. Sunday will be a much shorter run, followed by lots and lots of
doing nothing at all (although Rachel has to go to a meeting at 16:00) for a few
hours. Then it'll be Monday again. And a week where currently I have nothing
in particular on the calendar. I'm sure that won't remain the case for long.
[17:15] I'm pleased to say that this morning's run went very well. Not a PB for
the time, but a good run. My right achilles attachment point ached for a bit,
but then stopped (mostly). I took some ibuprofen anyway, just to give my body a
better chance of healing. The weather was lovely though. Cold, and quite
foggy. Always nice for mentally drifting off into the rhythm of the run. After
the run I had a lovely hot shower and then pootled into work.
Today work has mainly been about getting a VMWare guest to PXEboot and run an
autoyast.xml file of my own hacking, which I committed to an unfamiliar build
system via CVS. Amazingly, everything worked perfectly first time and I now
have a SLES SP4 server to hand over to the developer to see if he can restore
stuff to it to make it into one of his incredibly complexedly configured things.
I'll hear more about that tomorrow, or more likely next week.
Rachel is in transit back from Tanzania at the moment. I don't know quite where
she is right now, but she should be back in the UK early tomorrow morning, I
think. She's - I hope - going straight home to get some sleep/rest, and then
after doing a much-needed long run on Saturday, will come and see me. I'd go to
her, but she has a meeting here on Sunday afternoon, so it makes much more sense
for her to come here, rather than us both bouncing back and forth on trains.
Speaking of bouncing, I'm getting out of here. It's definitely time I went
home, and ate more.
[17:25] After last night I decided that it was sensible not to run this morning
after all. So when I woke at 04:00 all I did was turn the alarm to the second
setting (07:00) and try and get some more sleep. I think I managed to doze a
bit, and while it wasn't proper sleep, it was better than getting up at 05:20
and running for 20km. I'll do it tomorrow, definitely.
At work a lot of the morning was spent deracking old servers and disk arrays and
doing a tape run for the backup systems. The remainder of the day was mostly
about sorting out the massive amount of spare hardware I have on the shelves
behind me, working on autoyast stuff, and fielding emails about all kinds of
things. I'm going to go home now, having not exercised all day, do the washing
up and watch lots of television before bed. Then tomorrow I will definitely get
up stupidly early and do my run. I got my email from the Boston Marathon today,
telling me my bib number, start wave and corral. 32 days to go. Training is
going OK, I think. I do wish some of my minor niggles were a bit more minor if
I'm honest. I have this feeling that I'm right on the edge of something
problematic with my right achilles and my left shin, but we'll just have to
[16:30] I had a pretty hectic start to the day, today. I got up as if it was
Wednesday or Thursday (which I'll be doing tomorrow and the next day) and went
for a shortish run, then got back and had a shower and a long-overdue shave so
that the haircut at lunch time wouldn't look silly when my beginning-to-become
bushy proto-sideburns were removed. After that I then got in to work for around
07:30 to start on the patching and rebooting jobs I had set for today. Other
than two odd cases of virtual servers not doing anything when booted into a new
kernel (so I power cycled them and set them to boot older kernels, which worked)
and two physical servers which took a long time to boot new kernels too,
everything seemed to go pretty smoothly. I now have only sixteen servers which
aren't fully patched and rebooted. Hopefully I can get permission to do them
before I go to the Boston Marathon, but I doubt it. The remainder of the
morning was spent dealing with non-critical fallout from the patching, and
getting access to a CVS/build system for a server I'm having to reprep as SLES
rather than RHEL/OL as the outcome of the meeting I had yesterday made it plain
that to avoid massive amounts of pain it was just going to be easier that way.
We'll wait for the new version of the application the servers I'm building for
it starts development before I poke my head above the parapet on that score.
Over lunch I had my hair cut, finally, then came back to spend the afternoon
learning CVS and SLES' autoyast syntax. I'll have a first go at building a
(virtual) box via PXE boot and autoyast tomorrow, I think.
I'm off shortly for a physio appointment, then home for another very short run
before dinner, stretching, and putting my feet up before an early night. Then
again, all I seem to get is early nights these days.
[17:25] Pretty good weekend all things considered. I managed a reasonably good
long run on Saturday, even if I cut it short by about 4km. Turns out I ran it
faster than the marathon pace for Manchester last year, which was good. Of
course, the reason I shortened it was because I was feeling pretty tired, but at
the point at which I shortened it I still had 5km to run to get back, so I still
managed to keep the pace up. Satisfying, anyway. Part of the reason I
shortened the run was so that I could get home in time to shower and eat
something before heading in to two to attend two talks on running and endurance
training. Both were interesting, and have left me with some more crystalised
thoughts as to how to finish training for the Boston Marathon, and carry on for
the ultra (and marathons and ultras between Boston and the target ultra). After
the talks I just cycled home slowly and went to Tesco before I ran out of
energy completely. Following that all that was left to do was to have a nice
quiet late afternoon and evening. Sunday morning I decided to go for a run with
the local triathlon club, so cycled 10km out to some nearby hills, did the 19km
or so run, had a drink and cake (bacon roll) with them and then cycled home
again. It was nice to run in company, and slower than I might otherwise have
done my Sunday run. The remainder of the day was definitely spent doing very
little other than lamenting the fact I should mow the front lawn and trim the
bushes. Maybe I'll do that on one of the other evenings this week.
This morning came around all too quickly, and with it the usual Monday morning
stuff to do. I did manage to squeeze in a nice run with Max at lunch time
before going into a meeting with my new team leader and a developer we now have
to support, the outcome of which was some face-palming, some giving in to
inevitablity, and me ending up with a pile of new things to learn before I can
give the developer what he wants. Luckily all of the new things can be learnt
and done from my desk, so I can still get on with everything else on my plate at
the same time.
Rachel also popped up on Skype briefly from Tanzania, so we got to check in with
each other and say how we were doing. That was the highlight of my day. Not
hard for a Monday, but still rather nice. I'm debating doing weights this
evening or just going home. I think I'll just go home to be honest with you as
I'm upping my running a tiny bit this week, so I should lay off cross training.
At least until I settle things down a little. Maybe I will mow the lawn
[16:50] I got to sleep in this morning. Or at least lie in bed until 07:30,
which was pretty great. Morning exercises followed, breakfast, and then a cycle
in freezing fog to work. I've had a pretty good day updating all but 16 of the
servers in my purview. This is rather great. I've also got a load of emails
which were clogging up two servers (although more to follow would be good)
starting to be sent to the people who should have been dealing with the contents
of those email for the last three years or so. We'll have to see if that deluge
of emails stimulates them into doing something about the problems they detail.
The weather outside has been sunny and clear all day, which has raised the
temperature inside the building, even as I think the air temperature outside
hasn't risen much into double digits centigrade. I'm going to head home shortly
via the boat house and collect Max so he can come and root through my clothing
for thermal things to take to Scotland.
This weekend I'll be running long on Saturday, then attending some psychology
talks (if I can be bothered), then running shorter on Sunday, perhaps with a
local triathlon club. We'll see.
[17:15] Somewhat of a better day than yesterday, thankfully. It started off
with a reasonable run in dry weather (although the path by the river was flooded
in places, so I had to go over a bridge rather than under it, twice). Then I
headed into work and sent some useful emails before the main workload kicked in
for the day. Then I got to reply to some emails from Rachel, which is always
good. She's in Kenya at the moment (did I mention that?). The afternoon was
started off in a meeting about decommissioning some old servers I've been trying
to get rid of for years. Progress was made, but I don't think anything concete
will happen for at least another week. At this point another week won't really
matter all that much, I hope. Then I took a look at the RAID subsystems for all
of the servers I currently manage, and noticed that at least three of them had
non-critical issues with their physical disks. Just like that stupid server
last week that caused me so many problems. I've asked if I can replace the
problematic disks in two of them, and decommission a third, so we'll see how
that goes. If I do replace disks I'm definitely going to find out if there's a
way to flash the activity light on the ones I want to replace this time
around. Other than that I've made it to the end of the day alive, so I'm going
to go home and have two large ostritch steaks, rare, with broccoli, baked beans,
and potato cakes. Last night I went to Tesco on the way home to get some more
squash, and ended up getting two Indian curry main courses, and eating them back
to back. I really am very hungry this week. And yet despite all the running
I'm doing, I still feel like I'm not losing that last bit of excess around my
waist. First to arrive, the very very last to leave.
[17:05] I am so incredibly ready to get out of here, today. The day started at
about 03:40 when I woke up to the sound of high winds and torrential rain, and
couldn't get back to sleep. At 05:20 when my alarm went off I quickly got out
of bed and started my morning routine so as not to cry off my run. Cycling to
the boat house was a bit of a drag in the wet, but I persevered, all the way out
and onto my run. Which felt grim in places. I was wrapped up against the
elements in the lightweight waterproof I'd bought (or got as a present) many
years ago to do adventure racing in, so that wasn't too bad, but I think the
pasta and carbonara sauce I had last night didn't sit too well, so I didn't feel
all that great. I think that, plus yesterday's run, plus the marathon on the
weekend all came together. Then there was the headwind to contend with.
Suffice it to say I was about 1km off my best distance for the time. In any
event, I survived it, had a nice hot shower, and got to work. Where the problem
with IP addresses from last night was fixed, but the whole thing about
kickstarting, change management, and configuration management reared its head
and has left me worrying that there's going to be a mass of new things I'm going
to have to learn about in very short order. Then my old team leader (who's
highly knowledgeable, and holds a lot of important stuff in his head) announced
that he's passed his CV to a friend who has some very attractive roles at his
company. So he could be out of here very soon, which is both good, but mainly
quite bad as lots of responsibility for a lot more stuff (some of which I know
nothing about) will come crashing down on my head. On the bright side I'll be
learning a lot of new stuff and my CV will explode with awesome new skills in
the event I want another job, on the down side my life could get a whole lot
busier in all kinds of horrible, stressful, ways. We're basically starting to
lose all the good people here as the management continue to make incredibly bad
decisions based on incomplete, or even false information (some of which they
seem to be making up themselves). I fear for the future, even more so than when
the Networks guy left. I'm going home to eat a lot and lose myself in
downloaded television for one evening.
[17:55] In early this morning to do some patching and rebooting again. Nothing
too hard this time around. However, I did have issues with another server I
was working with which refused to do what the technician at the company who
made it said it would do. I had to send out a colleague to start the
diagnostics I'd been planning for it whilest I finished off the other work I
had to do. Once I had, I took over for him at the data center and got things
back on track. Annoying to say the least. For the rest of the day I've
managed to get to a point where I've been able to make a start on something I
should have done three weeks ago. Admittedly it has gone 17:00 and I should
have gone home, but this really needed doing, and now that I've got the first
part of it out of the way the rest of it should go a bit more quickly. It's
basically building a server on the VMware cluster I've recently been given
access to. Whether or not I remember to keep it updated, or even if it has
the right configuration (there are some weird-ass partition/LVM decisions I've
had to go with) remains to be seen. But, there it is, installed and booted.
I'll do a bit more work on it before I head home (not to the boat house to erg,
I ran this, rather than yesterday, lunchtime). Sadly, the IP details I have
been given seem to be already being used by something else, so I can't do
anything network-related with it for the time being. I think this is the
perfect time to call it a day.
[17:35] I have struggled mightily with not running today. The sane and sensible
decision has been to go with the struggle. This is because I ran a marathon
yesterday with Rachel. Neither of us pushed it, really, as it was specifically
a training run for us for Boston. But let me go back a few days...
Rachel arrived on Thursday night and worked from (my) home on Friday. I left
work early on Friday to buy her some spare hockey socks, collect the bike lock
she'd left at the station, have tea with Sarah, and then headed home in time to
catch Rachel as she was finishing her last Skype call of the day. We had a nice
relaxing evening together and then a moderately early bed. This was so she
could get back on a train early in the morning to play her last hockey game of
the season. Unfortunately, she got all the way to the station on her bike
before she discovered she'd left her bank card somewhere in my house. She
texted me to look for it (thankfully I decided to get out of bed and track down
my phone when I heard it beep), and I was able to get dressed and find it before
she got back to collect it. As I was already out of the door and heading
towards her I cycled back to the station with her where she got the train with
minutes to spare. This allowed me to say hello to a few people from my rowing
club who were also heading to London for the WeHORR. Once I was home I went for
a nice gentle run of about 16km to keep me from going too stir-crazy, then did
the Tesco thing before settling down for the rest of the day to be ready for the
following day. Rachel arrived towards the end of it and we had a carb-laden
meal before bed. In the morning we cycled over to the same site my workplace is
on for the race. A half off-/half on-road marathon without any traffic
diversions. We decided to run together, and not to push too hard at any point.
In the end it was a pretty great run with only a little bit of wishing it was
over in the second half. We got exactly the same times for every single split,
but she was placed ahead of me in the results, even though we crossed the line
holding hands. She was also first woman, which was pretty cool. 3:27:38 for
the marathon distance, which I think is quite respectable for a training run.
We decided to shower at my workplace, but discovered that there's no hot water
on the weekend, so had to head home to shower there. There was a bit of a rush
as Rachel needed to be on a train to Heathrow (she's in Africa for two weeks
now) within about 40 minutes of us getting back. As everything else was packed
and ready and stuff this was achieved. I didn't envy her the muscle aches she
had sitting on trains and then the plane as I stretched in my house for the rest
of Sunday aftenoon and evening.
Getting up this morning I didn't feel too bad at all, so I packed my running kit
and clothes for doing weights this evening at the boat house. On the way to
work I stopped off at Tesco to get a stiff brush to get the thick mud off my
running shoes from yesterday. However, although I almost got out of my chair
twice today to go running, and will have a hard time not stopping off at the
boat house this evening to do weights, I have managed not to exercise today. I
can't promise I won't do something tomorrow though.
Also tomorrow is an earlyish start to patch another batch of servers for the
GLIBC vulnerability. Far fewer than last Tuesday morning, so it should all be a
little less fraught. For now though, as ever, I'm off home for a quiet evening
without computers or thoughts of work.
[15:30] After yesterday's debacle it was a relief to get home last night and
welcome Rachel into the house. After a wonderfully relaxing evening and a good
night's sleep, getting in the morning I was able to head over to the data centre
and get the work I needed to do done. Sure, there was a lot of waiting around,
but I was definite that the disks I was going to use would be defect free.
After building and syncing the new RAID1, and verifying the disks as well, I
kickstarted the server installation and after it was finished came back to the
office and did the remaining bits of stuff. After a few minutes of panic when
the application I'd restored wouldn't connect to the database I tracked down the
reason why and fixed it. That made me quite smug. I think I deserve a bit of
smug after the week I've had. I'm now going home via some high tea with Sarah,
a trip to the sports shop for socks for Rachel's away hockey game, and to
collect her lock from the train station where she left it last night. Have a
great weekend, I intend to run gently tomorrow and then do a marathon distance
race on Sunday (definitely not at race pace).
[18:05] Things have not gone well in the latter half of today. It started OK
though. I got up early having fallen asleep on the sofa repeated last night
whilest watching television before 21:00, and therefore dragged myself to bed
around 21:20. Down to the boat house and running stuff on for the Thursday 1.5
hour run. Whilest I made sure not to push too hard, and to generally take care
of myself (as I was feeling not very up for it) I still managed my second best
distance for the time (second only to yesterday morning). This was pleasing.
In work is where things started to go wrong. Around 11:00 I noticed that a disk
that'd had a few bad blocks on it had suddenly reported a large amount more. I
decided it was time to swap it out with one of the many spares that we have. I
cycled over to the data centre and believed Sun when the front panel on the
server said which disk was disk 0 (of two, 0 and 1). Sadly, the cables from
disk backplane to RAID card were reversed, so I'd taken out the good disk. I
slotted it back in and settled in to wait for the resynchronisation to take
place. Over the course of the next 50 minutes I watched with growing panic as
the bad disk (that I'd meant to take out) generated more and more errors. If
the disk failed whilest resynching, we'd lose the server entirely. It was at
this point that I checked to make sure we were backing up the right things on
the server... and found that we weren't backing it up at all! Thankfully, even
with a massive amount of disk errors (so yes, probably data corruption) the
resync finished and I was able to pull out the bad disk and slot in a new one.
I then headed back to the office and took a look at the new resync's progress...
only to find that the original good disk that I'd just brought back into sync
with the bad one was now showing massive amounts of errors as the new good disk
I'd just slotted in synced with it. I got backup client software installed and
got off 16GB of something from the server. Whilest watching the RAID
repeatedly try to resync itself (failing, restarting, failing, etc.) I tried
restoring the backup to another server (first of all having to install packages,
Oracle client software, create users, and more)... only to remember as the
software started up cleanly that the installation is tied to the IP of the
original server, and I can't give that IP to this other server for a variety of
technical reasons. So, right now I've been told to go home and come back in the
morning. Then we'll see if the original server has finished syncing its RAID.
If it has we'll try another disk swap, swapping out the remaining original disk
for another one. If it hasn't, or if it has fallen over in a mess, I'll go over
there with two different replacement disks and rebuild it from scratch, then try
restoring the software directory and seeing what happens... For now though I'm
so thoroughly done with the day. I'm going to go home, have a shower
and a shave and wait for Rachel to arrive. Hopefully she can calm me down a
little. Oh, and in other news another of my exes is getting married. That just
leaves Kris to get remarried and all of them will be. That occupied my brain a
bit on the run this morning, annoyingly.
[16:15] In keeping with the vein of yesterday's entry, I prepared for this
morning's 1.5 hour run feeling pretty tired and unenthusiastic... only to pull a
new "PB" out of the bag for that time (outside of race conditions). Surprising.
And now I honestly don't feel all that much tireder than I did before the run,
either. I am, however, grateful that I don't have to anything else today other
than go to the physio and then home. I'm running again tomorrow, but it's the
last run before the weekend (when I'll probably do a shortish slow run on
Saturday before the marathon distance on the Sunday), so it might be a good bit
slower than this morning's effort.
The stuff I've been putting off doing at work (because I've been so busy with
other useful and essential things) finally made itself known again today. I've
managed to do some of the preliminary stuff, and the rest of it should fall out
fairly easily (if I make time for it) over the rest of this week. I hope. It's
still a bloody stupid setup I have to replicate, but at least I can do it all
from my desk rather than having to go and stand in a server room with a monitor
and keyboard plugged into a chassis. Anyway, if I have the time I'll see about
making a proper stab at getting things sorted, or at least a start on getting
things sorted tomorrow.
For now it's clearing down my inbox until 16:30, then I'm out of here for physio
and a cycle home.
[16:45] I'm not sure I quite managed an hour of weights last night, but it
should have been something close to that. I was a bit tired, so I figured to
err on the side of going home, putting my feet up and eating a large amount of
pasta and peas. Also some chocolate.
This morning was the one I hadn't been looking forward to from as far back as a
week ago. The patching and rebooting of 72 servers (virtual and physical).
Something of a mammoth task and fairly prone to something going wrong
considering the amount of packages being updated, firmware being installed, and
services both documented and sadly undocumented which might or might not start
up owing to the fact that the developers who put them in place might not have
sorted that stuff out. In the end, having arrived at around 06:35, I was
finished apart from a bit of tidying up as people complained about certain
applications and web sites not working, by 08:50. A full ten minutes before the
end of the vulnerability period. Only one server gave a bit of a scare, but
that was nothing compared to the work I did last night at 20:30 in getting KVM
disk images back onto the server I'd rebuilt the RAID5 volume for. Firing up
that guest resulted in a black screen for many minutes before the kernel boot
scroll began to happen. Thankfully, further reboots weren't quite so lengthy.
Not sure what was happening there, but it hopefully won't do it again.
I'm going to leave shortly for a 20 min erg at the boat house and then home for
food and relaxation. Rachel and I have entered a marathon length race on Sunday
(purely as a fun training run, not as a race race), so I'm going to take it a
little easier on the Wednesday and Thursday morning runs, as well as swapping
Saturday and Sunday round in terms of distance.