[16:05] I want to eliminate the person who thought that marking a RAID array
status line as "Degraded" when the firmware didn't match a specific base value
was a good overloading of a diagnostics field. When I do I will feel a shiver
of pleasure as literally one or two other people don't have to go through the
confusion of trying to work out in what way the array is degraded. Yes, today
I've been working with Dell's OMSA suite and finding that, aside from one or
two really odd foibles it seems to be a fairly cool bit of kit, if you just
install the diagnostics and storage bits, rather than the whole shebang. We're
looking at using it to replace and augment the Big Brother monitoring we have
for the servers by unifying the RAID monitoring script for PERC 3
(aacraid/Adaptec) and PERC 4 (megaraid/LSI) RAID solutions and adding
temperature monitoring. We have a sacrificial PowerEdge 1850 I'm allowed to
blow the configuration and operating system away on, so that's going to be
probed within an inch of its life over the next few days in ways it'll never
have been probed before. Baby.
Quiet weekend, coming down with a cold. Elaine spent most of it recording from
minidisc to MP3 with the soundcard I got her, the optical cable from my PS2 and
some transcoding software I found on the web called WavePad and RecorderPad, I
think. Free and uncrippled which was a surprise, but then she doesn't need to
do much with it.
I'm off to the gym in an hour or so then we'll probably go to see the new Zorro
film with Cormac and Steph and try not to swap too many germs with them.
[16:25] Actually I didn't do much today. Looked at the requirements for RT on
linux, did some work, had some lunch and read far too much Penny Arcade. I did
have my first two ever Krispy Kreme doughnuts and spent the sugar high cleaning
whiteboards with some kind of freakish manic intensity, but I'm over that now.
As pennance I'm off to the gym now. Have a nice weekend.
[17:20] I got a new keyboard today. It's a little different to my last one so
all my finger macros are a little bit out of alignment. I expect I'll settle
down after a few days. In the meantime I'm having to use the backspace key a
lot more than normal. Still, the angle this keyboard is at means that I'm
sitting up a lot straighter now, so that'll help my posture if nothing else.
I didn't actually get to attach the new keyboard until well into lunch time on
account of being busy with new versions of Ruby (the previous version I
installed being one minor-minor version out of the values that work with Rails
properly. I ended up having to use a Fedora Core 3 source RPM and rebuilding
it. At the same time as that was going on I had to fight my way through a
configure of TOra (a TOAD-alike for linux). Cue lots of fiddling with
environmental variables (especially LD_LIBRARY_PATH) and a ton of help from
mobbsy. Thanks for that. It all worked in the end and the user for whom I was
doing it is now happy.
My team leader has asked me to look into some issue tracking within our group.
Where I install it is going to be my first challenge. In the meantime I've
been having fun getting gkrellm to work on Solaris 9. It was fairly easy in
the end. Just the Common GNU package, gkrellm and (after a bit of thought) the
Gnome 2 Desktop installation. Now we have to decide which machines really need
it on, or not.
I ran my gym today to ask if Elaine could come and just use it for a session.
Turns out she'd have to pay the £10 required for an induction that'd
probably only take five minutes and then the cost of a single session on top of
that. I don't think she'll be coming. At least this time.
I think I'm going to go home and fit a soundcard with Elaine so that she can do
her minidisk to MP3 stuff this weekend when she brings home a minidisk deck for
[12:30] Well, the Thai food I made last night was an unqualified reasonable
attempt at a success. Maybe. The Tamarind water was made with paste rather
than pulp, wasn't steeped for 24 hours, the rice wasn't fried as I suddenly
realised I didn't have a fryer for that kind of thing and there were another
few things that I didn't manage to get right. With Elaine's help once or twice
I was able to cobble something which didn't taste too bad together and had the
whole thing with coconut rice. All in all, perfectly edible. Next time I will
learn to read ahead and prepare a bit more.
I was in work for 07:30 this morning to make sure that everything was ready for
the application restart at 08:00. While everything went pretty much according
to plan it turned out that there were no .pid files for any of my Resin servlet
containers. This was a bit annoying. In the end I had to kill the perl
processes which start them and restart them, which seemed to fix everything.
This afternoon I'm off to the second half of a talk on LDAP and its adoption by
the Institution as a whole. Climbing this evening. Tomorrow's the first day I
go to the gym as a member. Hopefully Elaine'll come along too.
[17:05] Hard time getting out of bed this morning. Managed it though,
obviously, and got into work to reboot six machines. No-one complained and it
allowed me to install the last of the IPMI monitoring stuff on them. Now all I
have to do is work out what all the sensors relate to and how to munge them
into something useful to give to BigBrother.
I left work for a 'late lunch' at around 14:00 to go and have my gym induction
session. It was fairly standard, although I think the woman who inducted me
wanted to see my arse as she demonstrated and then required me to try some odd
exercises on a gym-ball. She seemed fairly keen to see how I was on the leg
extension and bicep curl machines too... Still, I'm a member now, I've paid my
year's worth of membership fees (easier and, in the long run, cheaper than a
monthly direct debit) and all I need to do now is get some non-iPod headphones
and I can exercise and watch Sky One while I do.
I'm in early for the third Wednesday in a run tomorrow to install a new set of
Java servlet applications (requires burping the functionality) and make sure
everything's OK. Hopefully this'll mean I can head off early for once.
Unless we go to the cinema tonight (and perhaps, even if we do) I'm cooking the
house's first attempt at 'authentic Thai' tonight. Well, so long as Elaine
brings home some tamarind paste. I should head off and get the minced pork out
of the freezer (if it's not actually in the fridge already).
[17:00] It's been a quiet weekend really. Elaine and I took a good long look
at the storage situation in the bedrooms and tried to find something to store
our smalls in to make the built-in wardrobe a bit easier to work with. We
found somewhere but weren't able to buy it as it's not available to order
online but wasn't in the shop we wanted to get it from. I'll probably order it
sometime this week and hope we get it within a few weeks after that. Either
way we're making some progress on getting the house in order again.
Saw the new Wallace and Gromit film last night. Most excellent it was too.
Especially all the jokes-for-adults that were sprinkled throughout the plot,
scenery and script.
Today has been a bitty day. I've installed the IPMI stuff on all of the
servers I directly administer now. This is helpful only if I get some joy from
Dell on what the multiple identically named sensors map to within each of the
different kinds of chassis we have here. Currently I'm in contact with a
senior software engineer who isn't holding out much hope. If nothing else I'll
still be able to see if something goes wrong and take it from there.
Either way today has been a day of a little bit of work in lots of places. Not
what you really want for a Monday when you're feeling logey and could do with
a day off. As a way of waking up I'm intending to cycle to a gym this evening
and take a look at it with a view to joining. It's not a swankypants gym, just
one in a leisure centre-type place that'll allow me to do some cardio-vascular
work without high impact on my knees to keep me fit and unfat, and some weight
work to keep me in shape for climbing.
[16:40] Nothing happened today.
It was that interesting.
Oh, I went to town and bought some soap.
[17:35] The import of some fairly useful information is still failing to work
due to data which doesn't seem to be coming out of the previous application in
a 100% sane manner. I've been tasked with running the import procedure and
have been reporting back on what has been going wrong. With luck something
internally consistent should be available by tomorrow. Either way I've got
more work to do on this before it becomes something automatic. Luckily the
import tool is fairly smart and won't import 'bad' data which means that we
don't have some nasty DB/application interactions which will result in lost
That's kind of been going on all day. In the mean time I've installed k3b on
someone's workstation and successfully written a CD (no DVDs to hand), used
that CD to do an Ubuntu install on a spare machine into which I put a spare
Dell PERC 2/DC for fun and been for another punishing run with Brett, the
ex-Army captain who is far, far fitter than me. I think my right knee is
going to play up again when I stand up to go home. But that's the price you
pay for being unfit.
I made an abortive attempt to tidy my workspace today. Thus far I've collected
all the papers together and stapled those things which are of the same subject
together. Tomorrow is filing day. Whether it's in the bin or in a ring binder
I haven't decided yet.
I think I'm going to cycle home and have a long relaxing evening.
[16:55] Elaine still a bit ill, but on the mend I think. I left her in bed
this morning while I dashed in to be in time for the service switch over
vulnerable period. This came and went quickly and effectively and we were back
up and running with only about six minutes downtime. From there I spent some
exasperating but ultimate successful hours getting IPMI, openipmi and ipmitool
to work on some of our Dell PowerEdge servers. I can now monitor (Big Brother
scripts still to be finalised) the CPU, chassis and riser temperatures as well
as the fan speeds. The only thing I can't do is tell precisely which
sensor is where. I'm hoping someone'll be able to enlighten me on one of the
mailing lists as to how they tie up with the readings I'm getting. Of course
the experience wouldn't have been complete without getting to email the author
of one of the scripts to tell them about a few typos/bugs in the RedHat one and
showing them the corrections. It's always good to give something back.
At lunchtime we had the director of the other part of the institution's IT
area come in to tell us about what his parts is doing, how it'll intergrate
with ours and some other stuff. To cut a long story short the man's a twit, he
insulted his staff and ours in an offhand manner, talked too fast,
stacked/nested his statements and went off on tangets way down the stack/inside
the nest and never popped things back off it again. Confused, annoyed, and
worried is basically how many of came away from the presentation. I consoled
myself by writing up the key points and emailing them to someone.
Cormac's not climbing this evening so Elaine and I have press-ganged someone
else into taking us (and Brett) instead. Unfortunately, ringing Brett has
meant that he's convinced me to go for a run tomorrow. He's far better than I
am and it's only going to make me feel like I want to die when I finish the
run. Plus he'll make me do pressups at the end too.
[16:10] Lots done today. Not much time as off to see Night Watch shortly.
Installed a network monitoring machine for someone. Bemoaned the build quality
of Dell PowerEdge 1850 parts/rails/etc. Spent some time talking to people
online and hopefully arranging a visit some time soon. With luck that should
be a whole lot of fun.
Still feeling tired but not terribly snotty. Elaine home ill.
[15:15] What a weekend! After dashing home from work on Friday, Elaine helped
me with my final bits of packing (including lunches for Saturday and Sunday).
She then accompanied me for the mile or so walk to my pickup point. Owing to
one thing and another my ride didn't turn up for about fifty minutes so we got
to chat a bit before I headed off for the entire weekend.
The ride up to the Roaches was uneventful apart from some terrible traffic jams
on the outskirts of town. Other than that we arrived at the Don Whillans
Memorial Hut - following a fish and chips supper - some time around 21:00.
There are plenty of pictures of the place in the gallery collection for this
weekend (entitled The Roaches)
so I won't describe it much except to say it's half built into the rocks of the
lower tier of the Roaches themselves, consists of a kitchen, entrance hall,
drying room (below ground), a communal room, a shower room/toilet, spiral
staircase and two bed rooms, one with bunks.
When we arrived there was already someone there with a DVD player and LCD
projector so we ended up watching Hard Grit before heading to bed at around
midnight. I elected to sleep in the large room on the top level of the bunks
and was rewarded with one of the worst night's sleep of my life. Snoring, heat,
smell and shuffling combined to leave me feeling terrible on Saturday morning.
All that went away after a shower, a cooked breakfast and a trip out to do some
early morning bouldering on the rocks. Although the rocks were wet and the
air very misty there was a good chance the weather was going to perk up so I
paired up with a new addition to the club called Mat and we headed out with
two other people to Hen Cloud nearby. For the remainder of the day Mat lead up
some reasonably hard routes while I seconded and removed gear. I took part in
my first multi-pitch ascent as well as some of the hardest climbing I've ever
done on a climb called Delstree. By mid afternoon we were fingersore and tired,
had eaten our food and were very much in favour of heading back. That wasn't
the end of the day's climbing though as we headed off just past the hut to some
of the lower tier bouldering problems. While the weather had started off misty
and overcast, the rocks went from wet and treacherous to rather grippy by the
end of the day. Which was nice.
Dinner was a communal effort chilli con carne which filled the rather large hole
left by a day's climbing and while other people stayed up to drink many of us
got an early night. This was mainly because Sunday was all about showing the
40 or so new climbers who were brought in by coach at 10:30 how to climb,
boulder and abseil. I spent the day teaching groups of five people how to do
basic and not so basic bouldering moves on a variety of different problems.
Although we were on the upper tier of the Roaches we were protected from the
wind and received only the gradually warming sunshine. Of course, the walk off
down to the hut meant enduring the high knot winds with a bouldering mat (read:
sail) and trying not to take off. The whole day was extremely interesting for
me as a climber showing new (and not so new) climbers how to do certain moves
and watching their own interpretations of same.
I'd elected to leave in the 'early car' needing to head back to get things
ready for this morning so I found myself heading down to the car at 17:30 and
performing a push start on a small red Volkswagen Polo before an uneventful
drive home to be dropped almost at my door.
Elaine and I both appear to have come down with different kinds of lurgy over
the course of the weekend, although I seem to be doing better at the moment.
This morning neither of us were in particularly good fettle and as a result,
after doing the usual Monday morning's worth of work, working through lunch and
realising we weren't going to do much with the rest of the day feeling as we
are we're both heading home in a little while.
Oh yes, a present I bought for Elaine turned up this morning so I'll be taking
that home for here in about 15 minutes and hoping she gets some fun out of it
in the next few weeks/months.
[16:25] Elaine's physio turned out to be more of a chiropractor in the end.
They clicked her neck and back a fair bit and put some pressure on the muscles
which seems to have helped a bit, but not enough. I think next time she'll
find someone who'll massage the muscles a bit more than trying to pull off
both shoulderblades and twist off her head.
I on the other hand am feeling pretty good after the 10.5km run I did yesterday
with an ex-Army captain who's far, far fitter than I am. We did it in about
fifty minutes with me puffing away and him having merely broken a gentle sweat.
I swear I'm going to improve my cardiovascular fitness again to the level I
used to have. It's embarassing.
Today has been a bit problematic at work. We lost some network connectivity
overnight and it caused a few cron jobs to hang, coupled with some firewall
issues, and general network slowdown. They were all fixed (I think) some time
this afternoon while I was out bringing home the last of the old Dell PowerEdge
4400 beasts. We're on 1850 or 2850s throughout now and boy does it feel good.
I'm going to spend Monday thrashing the bits off this new one, which is going
to have all the services I failed off its predecessor allocated to it on
Wednesday morning. Other than that I've done some application installs, set up
NetBackup on a few machines and had a curry for lunch. All that remains now is
to go home dead on 16:30, pack for a weekend of climbing and head on up to the
Roaches in the Peak District (just about) for some fun. It just needs not to
rain much (if at all).
[10:05] Elaine's off work again today with the same tension headache she had
yesterday and the day before. This time she's found someone in the city who
practices physiotherapy and has an appointment free this morning. Hopefully
she'll let me know how things go and will be able to greet me with a smile that
doesn't contain discomfort this evening.
[12:30] I was in very early this morning (before 07:30) in preparation for the
move of a large number of services from one machine to another. Nevertheless I
was beaten in by both of my colleagues who were dealing with what was thought
to be a catastrophic SAN failure. There were two Sun engineers on site hard at
work and they'd been here since around 07:00. Luckily everything seemed to
come back OK and we were back in service (with my flawless service transfer) by
08:30. Since then I've been playing with gkrellm
and being amused by simple things.
This afternoon I'm off to a technical seminar on the Institution's new LDAP
service which may prove to be quite interesting if the Director doesn't stick
his well-meaning but cackhanded paws into it. Climbing tonight, too.
Elaine's off work today probably with whatever lurgy my parents brought when
they visited. Hopefully I won't catch it too. In the meantime I hope she
stays warm and comfortable.
[15:55] Just had a bit of a flap trying to get one of the tools that've always
worked to work just when we needed it. I even ended up ringing one of the
program's authors to find out what the problem was. In the end it was finger
trouble on my end.
I have finally removed the last non-RedHat Enterprise Linux linux box from the
network as of this morning. In doing so I've also emptied the Institution's
DMZ as well (two birds with one Ctrl-Alt-Del) which is almost as good. I've
also been able to go out in the great weather we're still having here too.
I put in the replacement NAT box for home last night. Obviously this meant
that the NTL connection went completely wrong for a few hours so I couldn't
test the new iptables rules (moving from the easy-to-understand ipchains and
ipmasqadm the RH6.2 2.x kernel used). Instead I had to wait until this morning
when I realised that none of my port-forwarding rules were having any effect.
I think I've fixed them now but it's a suboptimal fix as far as I'm concerned.
Either way the new machine is in, it works but it makes an awful noise compared
to the old one (which had passive cooling). I'm going to have to see what I
can do about a quieter colling solution for a P-III 866MHz Coppermine in a
Socket 370 ZIF.
[11/10/2005 - 15:00] Mostly a day of trying to stay awake. After getting in
at a sensible time I and my co-worker decided to pop out for an early morning
bacon and sausage sarnie and a free coffee. Not having those would probably
have resulted in two unconsious sysadmins by around 10:00. The weekend was, as
usual, pretty good. We saw Serenity on Friday night. It was pretty much
excellent. Despite being made in Hollywood it had enough rough edges and odd
feel to it to make it distinctly un-Hollywood and therefore just a bit better
than expected. Either way, I liked it a lot and will be adding it to my DVD
collection eventually, along with the truncated TV series.
My parents turned up on Sunday evening as I was doing the last of the copying
of data from the house's old NAT/firewall/Apache/Samba/fileserver to the new
one. The old (still perfectly usable) box is a P166 with 32MB of RAM. It runs
RedHat 6.2, which is the reason I've finally decided to replace it with
something a little more recent. The new box is a P-III 866 with 256MB of RAM.
Additionally it's got a RAID1 / filesystem which gives it a bit more resilience.
Both boxes have an identical 160GB of storage for us to stick things on. I
made us all dinner before (it being Sunday night) we all went to bed.
Most of the rest of today has been spent preparing for Wednesday.
[17:40] I went to change the backup tapes this morning and only took five tapes
out of the robot, not six. When I went to get the new tapes I realised this
and brought a sixth tape back. This means I have an extra live tape on my
desk which I really can't be bothered to take over to the fire safe. I'll put
it somewhere safe for the weekend and take it on Monday or give it to the
person doing the tapes on Monday to take over.
In an effort to get myself out of the doldrums I find myself in at the moment I
went for a 10km run at lunch time. It gave me time to think and to realise
that the things that're on my mind should be solvable by simply changing my
expectations and making the most of what I have. They're probably unreasonable
anyway. There's plenty of scope for the improvements I would like, it's just a
case of allowing them to happen rather than forcing the issues all the time.
In the meantime there's plenty I can do to keep myself occupied. In fact I'm
very slowly upping the amount of things I do in the evenings and on weekends
again after a rather long hiatus. I should try and get things in some sort of
order and up on the calendar at home so I can see where there's room for all
the things I need to fit in.
Today's been a bit of a non-entity as far as work goes. There's nothing to do
except prepare for the service switch next Wednesday. To stave off complete
boredom and to try and stay awake after completely shattering myself after
running what was close to a 45min 10.5km lap I delved into Ruby on Rails in the
folorn hope that RedHat's Ruby RPMs would put all the Ruby stuff in a sensible
(read: standard) place such that RubyGem would install happily and provide a
nice mechanism for getting Rails installed. Alas, it was not to be. As a
result I'm going to leave it all alone until Monday at which point I'll talk to
the people who want the installation and probably come to the conclusion that
running "rpm --erase (list of ruby RPMs)" followed by a download of a recent
Ruby tarball and the Gem tarball is the way to go. I really wish RedHat
wouldn't make sticking to a supported/vanilla install so bloody difficult. Why
oh why oh why do they insist of putting stuff in their own places?
I'm off to see Serenity (the Joss Whedon
finishes-up-the-abortive-Firefly-series film) this evening. I won't post any
spoilers except to say that even though the posters don't mention it, all the
nine major characters are in there, which was my primary worry. Whether they
can tie up all the characters' backplots in one film... well I have no idea,
that's why I'm going to see it, see?
If nothing else, there's always someone online who'll perk you up with some
banter over the electronic communication medium that is MSN (with Kopete, not
[17:30] Two song lyrics/samples I listened to today which struck home in
various different ways. The first is "What a man had to go through for a
piece of ass in the modern age was highly ridiculous." I won't comment on that
one for fear of retribution. The second was (to my shame) from an Avril
Lavigne tune and was "I'd rather be anything than ordinary, please." There's
not much I can say about that one either, without sounding trite or weird.
Suffice it to say that it struck a chord today. I'm trying to be more than
ordinary these days; to do more with who and what I am. I'm not sure how much
chance I have of achieving anything other than shining mediocrity, but it's
better to attempt than to simply wish it. Part of my attempts are volunteering
to help with showing people the ropes (literally) with the climbing club I'm in
as well as some larger things which may or may not happen, but if they do could
be really rather impressive.
I think I should head off for the day. I've done some, but not much today.
What I have done has been useful though. Fighting for a morning with RHEL3's
sendmail version proved to be more than even my team leader could deal with so
I don't feel so bad that I wasn't able to get it to look at the /etc/aliases
file when sending to root. In the end I just frobbed the actual application
(arpwatch) itself to send to a valid external email address.
[17:00] I went to the dentist this morning. Turns out it's been more than
seventeen months since my last visit. It really didn't feel that long. I
think the dentist took my tardiness out on me in the form of an x-ray and some
enamel to fix a minor chip in one of my front teeth. The whole deal cost
£23.50 and I've been told to make an appointment with the hygenist (at a
cost of another £30) to have my teeth cleaned/tartar removed, etc. When
you get down to it, fifty-odd quid to keep your teeth in working order isn't
that bad a deal, given how long you want to hold on to them for.
Work today has been sporadic but achieveful. I've helped the web team migrate
off their last bit of desktop-hardware-being-a-server (their second oldest box
after the one running RedHat 4 a few months ago), rewritten a few more scripts
now that the database updates have to go through a man-in-the-middle machine
and finally helped one of the network guys get their network monitoring RHEL3
box send them email properly.
I'm also feeling a bit low at the moment, could do with some attention and am
generally in need of a pick-me-up. It's probably all the hyphens I've used in
this entry. They can really take it out of you.
Climbing tonight. Must go home and do some washing up.
[18:10] Went out this morning pretty much as soon as I got in to decomission
two old machines in two different locations across the city. I got the bus
into town, removed on machine, lugged it across town to the other place and
unplugged that one too. Happily there were only Dell GX270 desktops rather
than anything larger and I was glad to be getting rid of them as servers. You
shouldn't run services on PC hardware. Anyway, as I was lugging the two of
them out of the building my bus went by so there I was running down the road
through tourists and the like with two moderately heavy boxes under my arms.
Of course I miss the bus, but that meant I could sit around on top of them
watching fresher students wander by looking lost and lonely.
Back at work I settled down just long enough to do some documentation on the
Wiki before my team leader dragged me off for a Thai lunch before our meeting
with a sysadmin at the other end of town about how he stores his terabytes of
data on NetApps. We're in the market for a new SAN-like solution and are
scouting out possibilities. That took most of the afternoon so by the time I
got back here there was only enough time to do some quick scripting for the
move of a database and it's associated website from one server to another
before it was ten past six and time to go home. Like now!
[17:50] First days back after time away are always a bit fraught. Today was no
exception. Aside from a mildy disturbed night (check out my bags at the
moment) I got in to fid that Big Brother was complaining about RAID and backup
issues on two servers. As I'm trying to deal with that my line manager comes
in to ask about a cronjob which seemed to have stopped working a few months
ago and is just now requiring being looked at. I then fire up RedHat Network
and notice that RHEL3-u6 has come out. Cue me spending most of the day
upgrading machines, doing testing, etc.
In the middle of all of that I got a helpdesk call about some users - some
ragingly anti-techno users - who needed access to a Samba drive I didn't even
know we provided on one of our live database servers. After dealing with one
person whose clipped-to-the-point-of-rudeness tones let me know exactly what
she thought of techie people I managed to get the whole thing sorted such that
I could close the ticket and get back to emailing a person of such uncureable
uncluedness that I'm not sure what (short of beating them to death with their
own spleen) would penetrate their head with regard to acceptable email
I'm now deciding whether to go home or not before heading off for a curry with
the climbing club I've been a member of for a while. We're expecting new
people to be joining soon so it'll be time to teach again as well as new
outdoor weekend trips.