BOFHJournal - January 2005

1999 | 2000 | 2001 | 2002 | 2003 | 2004 | 2005 | 2006 | 2007 | 2008 | 2009 | 2010 | 2011 | 2012 | 2013 | 2014 | 2015 | 2016 | 2017 | 2018 | 2019 | 2020 | 2021 | 2022 | 2023 | 2024

December's Journal
February's Journal

31/01/2005 [01/02/2005 - 09:00] Spent the entire morning working on documenting the build process I've been doing since I started here. Seems to be a fair amount of work considering how out of my depth I feel sometimes. The afternoon was spent at the first of the two sessions of more advanced perl teaching. The second one's tomorrow. I introduced Elaine to A.C.C.'s 2010 which was showing on TV when she got back from shopping with Cormac and I finally got home after going back to work after the course. Following that we watched the first few episodes of Scrubs' series two.

28/01/2005 [09:55] I have my meeting in five minutes. It's the first one I've called and I'm a bit nervous in case I'm asked to lead it and/or it turns out there's tons to do and I don't know how to. Of course, this project's been stalled for a good few months so there's no real rush, but still... I kickstarted the one spare target machine I have with the right specifications as soon as I got in this morning. The procedure went perfectly, which was a big weight off my mind. Now all I have to do is add in whatever else is required and help with the data migration and the backup procedures. [17:30] The meeting went really well. I didn't really lead it, which was fine, and I had the opportunity to ask all the questions I needed and get all the answers that were required. I've spent the rest of the day tweaking the install scripts to deal with the different exim configurations (learning a bit more about sed in the process) as well as making sure that the kickstart IPs have been updated to deal with the fact that this place is VLANed to hell and back. Of course, some of the really funky stuff broke when I reimaged a machine on the new VLAN, but some fettling with sendmail and trying some manual SMTP to the organisation's mail switch/hub seemed to work in the end and now things are back on track. I really do like centralised scripting, it makes things a hell of a lot easier when you're setting up multiple machines. As it's Friday I think it's time to go home now. With luck Elaine and I will be going to Reading this weekend to help Dunk with some interior decorating ideas. I know nothing about this kind of thing, but Elaine goes nuts for it. If I don't come in on Monday it's because Elaine killed me for that comment.

27/01/2005 [28/01/2005 - 09:55] A rather successful day all told. Completely rewrote the kickstart stuff to take advantage of the Jumpstart method of adding a script to the rc chain and running it once on boot then having it delete itself. I even managed to work with RedHat Network's activation keys system to subscribe the target machines to a subchannel. It's all gone rather swimmingly thus far and all I need to do now is have the meeting to talk about what else the developers want from the build before they get it. Apologies for these entries being very work-related at the moment, but it's exciting for me to be doing something challenging at work finally.

26/01/2005 [17:20] I'm rapidly going mad listening to the same tunes over and over again, but the SSH server I need to be up to get to my MP3s is still down. With luck I'll be able to get it working tomorrow. I've been able to get some of the last few wrinkles out of the way with my kickstarts to a point where I've arranged a meeting with the developers who need the service the machines will provide on Friday morning. I expect that the remainder of Friday will be spent changing and fiddling with my setup to conform to whatever extra or different things they require. I'm not too fussed though (at present) as this is What My Job Is. Other than that my money for the website I was doing at the tail end of last year has come through (not a moment too soon as I was down to very little owing to one thing and another recently). It's enabled me to make some purchases without going into my overdraft (something I hate doing). Off climbing this evening.

25/01/2005 [13:20] Slowly but surely I'm getting to a point where the Kickstart stuff is complete, fully-featured and gives me exactly the type of machine installation that won't need any tweaking from me once installed. The storing of certain config scripts on the KS server means I get a whole load of extra configurabilty for no extra work. The camera went down this morning, not sure why. If it doesn't look like it's updating you're more than welcome to email and tell me. It's a bit more supportable these days since I moved from RH5.2 to RHEL3. I'm off to the second part of the perl course this afternoon. In fact, I'll be heading off in about twenty minutes or so.

24/01/2005 [25/01/2005 - 08:40] Busy day, again. I may just start leaving that bit out, because it's almost always true. The morning was spent working out why the kickstart disk I had didn't boot the machine I had. This turned out to be because the machine I had was my old Windows desktop box which is now surplus to requirements on account of the Windows 2003 Terminal Services prototype we have here and the wonders of Rdesktop. Anyway, having been a Windows box it was on a completely different VLAN and behind who knows how many firewalls compared to the linux workstation I now sit at. The IPs I had to play with didn't work on that machine so I simply created a kickstart .cfg file for its IP and everything was 'fine'. I say 'fine' because I still need to do a lot of work on the actual kickstart itself. However, now I have a first approximation of something that works I can go about refining the install as well as having a meeting with the developers this box is destined for to find out what they want on top of the base install. This afternoon is Part 1 of a Perl course, so I won't be here.

21/01/2005 [17:10] Every day is just so busy here. It's great. The rest of the day, after a pleasant lunch with Shaun and Cormac, was spent updating the post-install section of the kickstart setup I'm working on. I've got everything except the NTP stuff sorted (I think) now. Still to do on Monday is an updated kickstart installation point (RHEL 3 AS update 4), a few more little things and then I try it on the test box I have. That'll be the moment of truth. As for now, I'm off home then to Ian's birthday curry. This weekend should be all about more relaxation, potentially some serious reading of books for pleasure and some late nights and even later mornings in bed. I'm really enjoying work at the moment, although I can't help thinking that I'm going to be confronted by some extremely difficult tasks fairly shortly. Thus far things seem to be going my way. Apart from some freaky screenshot keyboard combination I seem to be hitting in Gnome, the arrival of my PS2 keyboard and mouse to USB dongle has allowed me to start using my natural keyboard again and my wrists have stopped complaining. However, I've done a good day's work today and it's time to be off.

20/01/2005 [17:10] So today I embarked on a trip through RedHat RPM dependencies hell. Luckily it seems that it's improved since I last did anything serious with it. This was in an effort to get Dell's more recent AFA management tools installed on the test build of the three-server setup I'm working on. Although I eventually worked my way through the twisty maze of packages, all alike, I still had to update the RAID card's firmware (and hence the BIOS too) before I could run the applications in case something 'unsupported' happened. This turned out to be the easiest part of the day. Three floppies written, a few minutes of running executables from DOS and we were happy. I've kept the disks (apparently gold-dust around here) for the other two machines. Of course, as one of them was given to another project we're getting a new one to replace it. That'll probably have up to date BIOS and other firmware already. With the help of some helpful people on the web I also have some scripts which should check the status of the array and even allow me to assign the hot spare without any kind of action on my part. I'm still swaying between RAID1 and 5 for the machines. I was treated to lunch by someone from another department who was interested in my Windows/linux integration skills as he's been given a vaguely blank slate to create a resaonable system where he works. Over food I helped answer some of his more Windows-related questions and we sketched out some rough ideas of where to go with Samba, OpenSSH and the like. More RHEL kernel updates got pushed out this morning so I did a few more kernel compilations (ostensibly for modules only, but the practice is good for getting the sequence of events straight in my mind) and pushed out the updates to the growing list of machines I have some control over. Given my requirement for burning .iso images to CD for installation purposes I spent a semi rage-inducing half an hour or so installing k3b (and associated cdrdao stuff) and getting annoyed that the setup program (once installed) wouldn't run unless I was actually running KDE as the window manager. Even though I don't think I've been pushing myself particularly hard as yet, I feel absolutely knackered at the moment although this only seems to manifest itself as yawning rather than droopy eyes etc., except when I get home and crash. Speaking of which, I think it's time to do so. 19/01/2005 [15:55] Far too impatient to wait for the order I put in for a USB expansion backplane thingy for my PS2 natural keyboard, and because my wrist is already starting to play up a smidgen I splashed out £10 on one from an online store and hope it arrives before the end of the week. It'll probably beat Dell and their delivery time. I've done another two kernel compilations (for the purpose of module installations today owing to Redhat's release of a new kernel for Enterprise Linux. I've also put a fair bit of research into communicating with Dell's onboard RAID controllers. Of course, being part of RHN now any extra RPMs I need I can get from there, but it takes a fair while to have them installed. I can wait until tomorrow I think. Then it's all hands to the pumps and some interesting statistics to be pulled out. I think I can get the SNMP stuff running too and have emails being sent with a bit of work. Elaine and I went to see Team America: World Police last night. Amusing, childish and extremely coarse in places. Typical Parker and Stone fayre to be honest, although some of the songs were quite catchy. Climbing tonight, with a chance of kebabs afterwards. I bought some mango chutney especially. When I say I, what I mean is I whinged enough times that Elaine got some the next time she went to Tesco. 18/01/2005 [16:45] Quick entry as I'm off to the cinema. Have had monster success with machine configurations, getting things added on to other people's work queues (to give me results to work with) and generally getting things to work on my machine (like sound, should have plugged in headphones sooner!). Another reasonably successful day over with I'm going to go to the cinema to see Team America: World Police with Elaine. It's our first 'date' ever. Does going to Tesco for thw weekly shop afterwards invalidate it? 17/01/2005 [19:00] So today I decided that it might be worth working out whether or not it was possible to add new modules to a running kernel on my machine. This turned out to be a bit harder than I'd envisaged. Suffice it to say that I need to remember the command 'make oldconfig' rather than 'make dep' in the grand scheme of things. A happy by-product of this kernel frottage is that I now have BOFHcam I back up and running (as you can see if you take a look at the main menu). I now consider myself to be far more clued into how this whole kernel thing goes about happening (even if I'm a bit annoyed at RHEL3 and its shoddy misuse of source files. As a result I'm now going home happy, rather than angry. 14/01/2005 [23:55] Talk about a long day. Sorry, I'm doing this from home before bed. I was in reasonably early due to Elaine getting up to ensure she worked her hours while taking some time to go to the gym during the day (when it's free for her). Once in I started on some more RedHat Network stuff, trying to make some sense out of the concepts I have to deal with. While I had most machines up to date there were still some inconsistencies in the values for licenses, subscriptions etc. I took a break to go and find my new linux PC and managed to get a very helpful guy from PC Support to bring it and a monitor down with it in short order. Of course it was shy half a gig of memory and the PS/2 expansion card I'd need to run my natural keyboard (Dell GX280) but these were both expected. I fully intended to install it over lunch but ended up talking to Ian, who'd been in the building for training and was due a tour of the machine room. After lunch I installed the box and took the opportunity to install one of the BTTV capture cards for BOFHcam to go back on the air. I also cleared its operation with my two line managers, just in case. So long as it only points at me and I let people know it's there, we're sorted. Of course, once I had the machine installed I decided to register it with RHN. No dice. A frustrating number of hours later and I'd discovered a new level of complexity to the situation. Not only do you have licenses and entitlements. You also have channel subscriptions, which also cost money. This is... annoying. Anyway, tearing my hair out I finally discovered how to activate a subscription. Of course, it being Friday the subscription didn't activate until about ten minutes ago. This meant that I sent a fairly involved email to RHN Technical Support and got to speak to the lovely lady at RHN-EMEA who clued me in to a few few things and left me feeling that, should I have any more problems, she'd be a very useful person to speak to again. By that time it was about 18:30 so I went home, made some food and watched five episodes of Scrubs back-to-back with Elaine. A reasonable end to the week. All I need to do on Monday is work out why my monitor is not doing the resolution I want, and why we have ten machines registered with RHN but eleven subscriptions active... 13/01/2005 [17:20] It's great, for the first time in a long time I'm consistently too busy to do Journal updates during the day. I love it. Anyway, just before I go home here's a bit of a run-down on what I've been doing today. As I left my bike here last night (on account of being too busy to get home in time to be picked up) I ran in this morning in almost exactly twenty minutes. I felt exceedingly unfit so there's every chance of doing the route faster as I get fitter. Anyway, when I got in I immediately got down to getting machines logged, entitled and up to date with RedHat Networks. It's been a labour of love sorting out who has what, what machines are 'off the map' in terms of being known to RHN and not being more holey than some of those cheeses. As things stand, at the end of the day, all machines bar one are fully patched and happy. From here I can extend my domain out to the other machines which have yet to feel my benevolent hand upon them. I will bring peace and secure workstations and servers to this place during my tenure here, and it will be good. Power-madness aside it's been a good day all things considered (especially with the climbing last night which helped regrow some thicker skin on my hands) and I'm actually looking forward to coming in tomorrow. Of course, a day in sysadmining wouldn't be complete without rebooting a box for the first time and not having it come back up again. Luckily I'd checked (twice) whether it was used for anything (it wasn't) and after a few minutes of searching in the server room I found it attached to a terminal server and was able to fsck it and send it on its way. 12/01/2005 [12:40] I went climbing in London last night. After five minutes it was obvious how much skill I'd lost over Christmas and how less fit I was. Luckily I was climbign with two friends I made at the wall who like to go in for 'interesting' problems, so I spent the evening regrowing calluses and getting my balance back. Getting home through a head wind by bike from the station was tiring so I was ready for bed when I got in some time around 23:20. Of course, even though I couldn't get to sleep quickly I had to be up for 07:00 this morning to get to work in time for the 08:30 meeting I had with some of the important heads of various departments here. After actually finding the place the meeting was going to happen I settled down to try and keep track of what was going on and not to end up in a situation where I didn't know what was happening or what would be expected from me. I have to say that I think I got a fairly good handle on the proceedings and am fairly sure I know what I'm going to need to do in the coming weeks. Luckily though there's a good concept of "of you don't know, tell someone, rather than suffering in silence and buggering something up" so I'll be relying on that if I need to do something outside of my current skill set. [18:10] I now know very much more about RedHat Network and licensing than I thought I ever would (this is going to be a stock phrase for a while, I think). There's all kinds of confusion because we I've been assured we bought a certain number of 'management' entitlements and yet we appear to have a whole load more 'update' entitlements that no-one knows anything about. According to the guy who used to be in my post before me this is an artifact of RedHat Network such that if you connect a machine incorrectly in some fashion it registers as a purchased-for-nothing update entitlement (presumably to allow RedHat to prod you about it at a later date or something). So those 'update' entitlements don't actually exist and will expire in a year from their start date. In conjunction with that little mess there's the issue of the actual 'license keys' we have which don't amount to the same number as those machines we have registered. I think this just comes down to not registering them via a new system so as to preserve their start dates in terms of expiry. If I understand things correctly we have a good few server and workstation licenses left and things should be OK. Maybe. For now I'm going climbing in an effort to wind down. 11/01/2005 [14:35] Still no linux box for me to work on as yet. It's a bit annoying to have to open PuTTY every time I want to connect to one of the other boxes, but it gets the job done. I've also been generating points for discussion (I hope) for tomorrow's early morning meeting. I mean 08:30! What kind of time is that? In an effort to inject some seriousness into my demeanour at that time in the morning I will, for the first time in a few years, be wearing trousers to work. Don't all laugh at once. So anyway, the rest of the day is consisting of getting more understanding of what I'm going to be required to do here in the next few months, putting up some of my posters, crib sheets and printouts from the last job (to make it feel a bit more homely here) and finally been given some wallplugs and screwed my whiteboard to the wall. Things are starting to come together here, at least in my corner of the office. I stayed a bit later last night working through a promising line of reasoning with Shaun about mysterious banner printing with our Oracle applications. That didn't come to anything but did prove to be an interesting journey into the depths of Oracle Forms and may yet be useful in the future when other printer problems appear. I've already received two emails from BOFHcam readers with some helpful tips for my new tasks (thanks guys). All I need to do now is have a meeting, get access to the boxes and start hacking about with things. One of the mindset changes I'm going to have to get used to is that I'm now just the person who looks after the machines. Someone else installed the applications, someone else does stuff with the box. All I do is install and maintain. You'd think this would probably leave my left empty of meaning. But no... [16:50] We've just spent about fifty minutes trying to work out one of the more complicated chrooted environments during a supposedly routine certificate replacement (needed to bounce the server processes). For the first time I felt vaguely useful as I tracked down where the startup and shutdown scripts were found and called from. Unfortunately I don't have the experience to get my explanations across to other people in the right way yet so things took a little longer while other people came to the same conclusions as me. However we're all on the same page now so things are sorting themselves out. 10/01/2005 [15:15] Despite watching two episodes of Desperate Housewives (not a porn show) on E4 last night, which didn't end until midnight, I still managed to get up for a reasonable time this morning. Of course on getting in to work I came to realise a little more the full impact of what I've let myself in for here. I had a meeting with the head of operations whom together with my input drew a rather complicated network/service diagram embracing three different sites, two firewalls, and numerous different live, staging and development servers. It seems I'm responsible for working out a coherent kickstart scheme, reimaging the machines, maintaining them and handing them off to the developers once they're up and running. Coupled with a fairly complex set of chroot environments and we're looking at something that, once I get on top of it all, will be fairly funky. I just need to get on top of it all. Still no linux box or sensible-sized monitor as yet; working on a Windows machine with a 17" screen which I've convinced to run at 1152x864. Slowly but surely I'm getting a grip on what's required here, learning where all the documentation my predecessor left is and filling in more gaps in my self-taught education of linux and Solaris. All I need to do is just keep up. 07/01/2005 [16:10] Today has been pretty good all things considered. We've been working without our line manager today and I've been learning some good back-end stuff. I don't think I could run the place on my own yet, but I'm answering questions and making sense more than 90% of the time. Printing issues have been cropping up and I've had to stick my nose in to more PostScript and .PPD files that I ever thought I had to before now. Our line manager is someone who works overhard some times so she's done a lot of the sticky stuff. This means that we're playing catchup on what she's achieved but also that we're keeping well up with the things we need to be doing. I've been learning about NetBackup, more about lsof than I already knew (not much!) and about ssh tunnels. Those are especially useful. I think for now, with a recent change in place which should fix a good few of the printing problems we'vwe been seeing, I'm going to go home and reflect upon the week. With luck and a following wind, Elaine, Cormac, Steph and I will be going climbing this weekend somewhere at some point, followed by some kind of communal cooking effort at the place Steph is house sitting. 06/01/2005 [09:45] I went home at lunch time yesterday to take a shift waiting for the package to be delivered. Elaine headed off to work and got there at a reasonable time. Naturally as soon as I started on something CityLink arrived and practically threw the thing at me when I opened the door. I made him wait for a signature while I opened the box to make sure everything was there. Given that this was only my second day in the new job I decided that it might be a good idea to head back to work and see what I could do to lighten the load on my colleagues. It's odd being a PFY again but there's a certain lightness of being in not being the one giving the orders, at least at the moment. This morning I'm learning about the printing system associated with some of our applications. It's certainly far more involved that the Windows printing system but vastly more powerful. I look forward to getting my hands dirtier soon. Cormac popped round last night to return my portable hard drive. I can now get my Firefox bookmarks et al off it and onto my machine (soon to be machines when the other one finally gets delivered) here. We chatted for a while and played far too much Tekken before he headed off. This morning in the spirit of just trying to get on with things I curtailed some of my online activity. [14:50] I think I've solved my first problem. Well not so much problem as issue which turns out to be widely recognised and currently under investigation by minds vastly more superior to my own. Oh, and I had my first delivery to work, which is cool. Elaine should be vaguely pleased this evening. I've also just been shown the helpdesk software. Joy of joys. 05/01/2005 [13:00] I spent the morning in a very interesting meeting to do with authentication methods for systems that're being developed elsewhere in the building. Not really anything to do with my area of expertise but as I need some kind of entrance into the way things work here it seemed like a sensible thing to be in on, notwithstanding the fact that the team I've joined is going to have to support whatever hardware and OS is required to make the systems run. I'm pretty certain I kept up with what was happening and perhaps even made a reasonable contribution at one or two points. Either way it's a completely different working environment to the place I came from, and that's a good thing in pretty much every way. I still feel as though I'm not contributing to much yet (second day blues?). I think this'll change as I get more on-board with how things run here. It's all a bit strange and new, but my colleagues are great and, time permitting, are very willing to help. Having Shaun here helps too as a friendly face. As far as I can tell, Elaine's still at home waiting for the CityLink delivery we were told would arrive before 12:00. Naturally it hasn't yet. 04/01/2005 [13:30] Well, I'm here in my new job. Thus far it's not been too bad. There aren't any PCs for me yet, the phone line hasn't been connected and I don't have any work to do. Nevertheless I feel like I should be learning already and trying to help with things. Not that there seems to be much happening that isn't under control. I've rearranged where my desk is so that I don't have my back to the door. All that's on it at the moment is my keyboard and mouse from my old workplace, a notepad which is due to be filled with things I need to remember/learn/keep track of and a list of all the machines in the machine room and what they do. Compared to the last place there are... a lot more. Still, things seem to be fairly straightforward so I'm sure, with a little work, I should get a handle on things within a few weeks. This afternoon I'm going to be adding a bit of depth to my shell scripting while I wait for my PCs to turn up. This could be more than a few days though. [17:40] OK, I now have a PC (running Windows), a phone with a number, a desk behind which I can sit and some idea of what I need to do when I come in tomorrow, I think. I think it's time to go home and collapse. 03/01/2005 [22:30] Today has mostly been about preparing the house for the new working year (week), and getting some more things out of the garage and house and into the loft. As well as consolidating a lot of the bedding (throwing out a whole load of old stuff) we got to bin some of the less decent pillows, duvets and sheets. Elaine submerged herself in The Sims 2 for the rest of the day while I made some progress in getting involved a bit more in one of the local climbing clubs as part of my decision to do a lot more outdoor stuff this year, no matter the weather. Hopefully I'll be able to drag Elaine along too after a while. 02/01/2005 [03/01/2005 22:25] By the time we got up there was an SMS from Bob asking if we wanted to see House of Flying Daggers at the cinema. Owing to the success of Hero in our opinion we opted for an early afternoon showing so we could head back and make a decent Sunday meal with the last of our Christmas food. This was a duck and, after returning from the film we started work, only to find we were a few cloves or garlic short. Naturally the one place we expected to be open was closed. Nevertheless, the duck cooked off very well and we have a wonderful final Christmas-period dinner before lying back on the sofa to watch the last of Elaine's Red Dwarf (series 5) DVD episodes. 01/01/2005 [03/01/2005 22:15] Shaun and Linda woke us from our vaguely alcohol and late night induced sleep with the offer of a walk around a nearby country park to shake out the cobwebs a bit. After some mad dashing around and showering we were ready with seconds to spare. Trailing a very active young boy around and some cold winds is both fun and tiring. Elaine and I took some time to birdwatch near one of the man-made lakes while Thomas played on the swings and such before I took a turn with Thomas in the back papoose as we explored some more of the area. Heading back to Shaun and Linda's house we were treated to some wonderful hot crumpet action before settling down to quietly watch a film and relax. As time went on we got a lift back home in time to greet Andrea who was on her way back from a local football match. Tea and mince pies followed before we were turned green with envy by Andrea's tales of her experiences in Australia recently. We have to go there one day, all it'll take is money. Lots of it.