I found a box today in a squirreled away hosting provider that I’ve been paying the bill for mindlessly. I’ve found I can probably move the thing to s3 since the pages are essentially static. Amazingly enough, this may be one of highest uptime servers ever (maybe the most rooted server ever too but I don’t see any evidence yet). This host has amazing uptime. I think my company has experienced about 4 datacenter outages in the time this server has been up.
Now for something less business and profound and a little more practical.
Sometimes going through not just the day, but the week, I start really to sympathize with Leonard from Memento. If you haven’t seen the film I highly recommend it. As many of you know it’s about a guy who’s wife was killed, and he’s searching for the killer, but the catch is his short term memory is around 30 minutes max.
He creates a system to remember things, tattoos, notes, a polaroid camera and a sharpie.
“(For) just for day-to-day stuff, notes are really useful. Sammy Jenkis had the same problem but he really had no system. He wrote himself a ridiculous amount of notes but he’d get them all mixed up. You really do need a system if…you’re gonna make it work.”
For just managing yourself, it’s not hard. For a sysadmin, you’ve got Getting Things Done, Time Management for System Administrators. These are good systems, but I’ve always had a hell of a time making them work. This isn’t to say don’t read these books. If you’re looking at ideas for productivity, read them. Just don’t expect them to be the end all be all of task and project management. For me with weekly status reports I create, these methods don’t quite fit. Though the systems I use are heavily inspired from these systems.
The systems tend to have days compartmentalized onto separate files. It becomes a pain to transfer tasks from one day to the next, to the next. Eventually it’s as if nothing gets done because I get very little done that I planned on that day. Quite demoralizing. Also since each day is separate, context from previous days needs to be transferred daily. That is the flow of the week is on each file, so I’m flipping between different notes for requests and things I did previous days.
Instead I have a list I just keep in Evernote that keeps track of the entire week. What do I want to do that week? What do I want my team to do that week? Then I have below the days of the week what I want to do, and what I did. Hey most of my status is one one page now, I can just format it. Taking the week in chunks, I’ve found I can get more things done. Realistically tasks will probably get done in a day or two, sometimes three. However a week as a nice even chunk of time to keep track of the stuff I’m doing. I have to report on the week anyhow, so it fits.
Of course you still need some sort of project management. For this I’ve been using Asana which so far is about the least fuss project management app I’ve been able to find. It’s free to boot. Again I’ve been able to track which projects are occurring, who’s doing them (even if its me). It doesn’t do anything fancy like gantt charts, but I’ve found I’ve been able to live without them. The hardest part is just following up on missed dates, and getting people to update it with their current projects.
That’s really what’s necessary for groups that are doing projects. They need to know who’s doing what, and who’s on point for which task. I’ve had a very hard time making project management files say MS Project working, because not everyone knows what’s going on realtime. You want to communicate to folks realtime with each update you make? You won’t get much done.
The last tool is simply for customer requests, Request Tracker. Though I’m careful this is regulated only to customer requests. It’s not a wish list, it’s not a task list, it’s certainly not used to track projects. You can probably make it do these things, but in my experience this is a bad idea, and not very easy. I try and make this very title apropos since its strong point is being a liaison where a documented conversation between customer and fulfiller can exist.
Some fun notes about monitoring and uptime.
I’ll be giving some shout-outs to some products which have really helped me out. I’m not getting anything from them, (in fact my company is paying them money). However they’re making my life a bit easier.
So in world not so long ago, our monitoring situation was pure Nagios, and a physical pager. The pager was a bit like the hot potato in that it marked that you were on call. May as well have been an albatross for all it was worth. The service was cheap enough, but of course we had to come up with creative ways to create the on call schedule. Distribute it to folks that needed it, etc. Now there are probably cool ways to page with Nagios, and I’m sure people are using them, however we decided to outsource it to PagerDuty. It’s one of those companies that makes every sysadmin “Gah, why didn’t I think of that?!”.
PagerDuty takes care of the oncall schedule, rotations, being able to easily tweak the rotation because your on call guy wants to take vacation that week, or is otherwise unavailable, as well as the alerting you parts. It can call you, SMS you, e-mail you, all three at once. You get to pick. On top of that their rates are very reasonable. If you’re using a real pager, take a serious look at PagerDuty. I’ve got about 35 folks on it, multiple on-call rotations, and it’s working quite nicely.
Second shout out is for Keynote. You may be using Nagios (or Icinga) to monitor your uptime, and hey they have nifty performance metrics too! If you’re monitoring this stuff from the same datacenter that your website is on, you’re doing it wrong if you’re trying to figure out what your availability is.
Consider this (real) scenario. You’re tasked with providing marketing or sales, or whomever with you how awesome your site uptime is. Maybe it’s 100% and you don’t even do any outside monitoring, in which case you’re lucky or a liar. One day the power goes out in your datacenter, the whole place is dark for a bit, and of course it takes you a while to get your systems back up. Your monitoring system is blissfully unaware that there’s any site downtime because it’s down too. You go to pull up your reports and you still have 100% (or close to it) uptime.
The biggest issue is that Nagios is great for introspection. Making sure the gears in your machine are operating normally. It’s pretty crappy for monitoring what your customers on the internet are experiencing, and that’s what Keynote is good at. Other nice thing that Keynote does is dig through multiple clicks of a website with a simulated or real browser. Again emulating the customer experience as close as possible.
Hopefully this helps some folks out there. I know this took me longer than it should have to get right.
- You’re still engaging your brain, and doing nothing for your eyestrain, but now you’re doing something else.
- You’re annoying your boss and peers by such behavior. They think you’re just goofing off, and admit it. You are.
- There’s no rapid deployment system like Debian’s fai, or Redhat’s kickstart.
- You still have compile all your ports.
- An upgrade requires you to recompile the entire OS, and they rarely go smoothly.
- A port management add-on community has sprung up because they refuse to integrate a better native package management system.
- It’s still lagging behind for cloud applications.
- Desktop applications are quaint because everyone knows the multimedia support is lagging.
- Allow me to back up my music someplace besides a few devices at home so I’m not wasting disk space which would ultimately be useless in the case of theft or fire.
- Allow my various systems at work, and home to play music synced from, or based on a single authoritative source.
- The pen and paper is cheap
- No batteries required
- Easier to draw things on it.
- Using pen and paper is just more of a classic way to capture things.
- Information is not stored on my server (privacy issues)
- Batteries *are* required
- Something new to learn
- Something new to spend money on (yes Evernote is free but I ended up spending money on it anyhow).
- It provides an easily syncable space you can use to capture notes, photos, or voice notes (great if you’re on the go and have an idea).
- Very hard to lose your notes as it syncs with your phone, and your laptop. Also accessible via a web interface.