You really need a system if you’re going to make it work

Sometimes going through not just the day, but the week, I start really to sympathize with Leonard from Memento. If you haven’t seen the film I highly recommend it. As many of you know it’s about a guy who’s wife was killed, and he’s searching for the killer, but the catch is his short term memory is around 30 minutes max.


He creates a system to remember things, tattoos, notes, a polaroid camera and a sharpie.

“(For) just for day-to-day stuff, notes are really useful. Sammy Jenkis had the same problem but he really had no system. He wrote himself a ridiculous amount of notes but he’d get them all mixed up. You really do need a system if…you’re gonna make it work.”


For just managing yourself, it’s not hard. For a sysadmin, you’ve got Getting Things Done, Time Management for System Administrators. These are good systems, but I’ve always had a hell of a time making them work. This isn’t to say don’t read these books. If you’re looking at ideas for productivity, read them. Just don’t expect them to be the end all be all of task and project management. For me with weekly status reports I create, these methods don’t quite fit. Though the systems I use are heavily inspired from these systems. 


The systems tend to have days compartmentalized onto separate files. It becomes a pain to transfer tasks from one day to the next, to the next. Eventually it’s as if nothing gets done because I get very little done that I planned on that day. Quite demoralizing. Also since each day is separate, context from previous days needs to be transferred daily. That is the flow of the week is on each file, so I’m flipping between different notes for requests and things I did previous days.


Instead I have a list I just keep in Evernote that keeps track of the entire week. What do I want to do that week? What do I want my team to do that week? Then I have below the days of the week what I want to do, and what I did. Hey most of my status is one one page now, I can just format it. Taking the week in chunks, I’ve found I can get more things done. Realistically tasks will probably get done in a day or two, sometimes three. However a week as a nice even chunk of time to keep track of the stuff I’m doing. I have to report on the week anyhow, so it fits.


Of course you still need some sort of project management. For this I’ve been using Asana which so far is about the least fuss project management app I’ve been able to find. It’s free to boot. Again I’ve been able to track which projects are occurring, who’s doing them (even if its me). It doesn’t do anything fancy like gantt charts, but I’ve found I’ve been able to live without them. The hardest part is just following up on missed dates, and getting people to update it with their current projects.


That’s really what’s necessary for groups that are doing projects. They need to know who’s doing what, and who’s on point for which task. I’ve had a very hard time making project management files say MS Project working, because not everyone knows what’s going on realtime. You want to communicate to folks realtime with each update you make? You won’t get much done.


The last tool is simply for customer requests, Request Tracker. Though I’m careful this is regulated only to customer requests. It’s not a wish list, it’s not a task list, it’s certainly not used to track projects. You can probably make it do these things, but in my experience this is a bad idea, and not very easy. I try and make this very title apropos since its strong point is being a liaison where a documented conversation between customer and fulfiller can exist.

Monitoring and Uptime

Some fun notes about monitoring and uptime.

I’ll be giving some shout-outs to some products which have really helped me out. I’m not getting anything from them, (in fact my company is paying them money). However they’re making my life a bit easier.

So in world not so long ago, our monitoring situation was pure Nagios, and a physical pager. The pager was a bit like the hot potato in that it marked that you were on call. May as well have been an albatross for all it was worth. The service was cheap enough, but of course we had to come up with creative ways to create the on call schedule. Distribute it to folks that needed it, etc. Now there are probably cool ways to page with Nagios, and I’m sure people are using them, however we decided to outsource it to PagerDuty. It’s one of those companies that makes every sysadmin “Gah, why didn’t I think of that?!”.

PagerDuty takes care of the oncall schedule, rotations, being able to easily tweak the rotation because your on call guy wants to take vacation that week, or is otherwise unavailable, as well as the alerting you parts. It can call you, SMS you, e-mail you, all three at once. You get to pick. On top of that their rates are very reasonable. If you’re using a real pager, take a serious look at PagerDuty. I’ve got about 35 folks on it, multiple on-call rotations, and it’s working quite nicely.

Second shout out is for Keynote. You may be using Nagios (or Icinga) to monitor your uptime, and hey they have nifty performance metrics too! If you’re monitoring this stuff from the same datacenter that your website is on, you’re doing it wrong if you’re trying to figure out what your availability is.

Consider this (real) scenario. You’re tasked with providing marketing or sales, or whomever with you how awesome your site uptime is. Maybe it’s 100% and you don’t even do any outside monitoring, in which case you’re lucky or a liar. One day the power goes out in your datacenter, the whole place is dark for a bit, and of course it takes you a while to get your systems back up. Your monitoring system is blissfully unaware that there’s any site downtime because it’s down too. You go to pull up your reports and you still have 100% (or close to it) uptime.

The biggest issue is that Nagios is great for introspection. Making sure the gears in your machine are operating normally. It’s pretty crappy for monitoring what your customers on the internet are experiencing, and that’s what Keynote is good at. Other nice thing that Keynote does is dig through multiple clicks of a website with a simulated or real browser. Again emulating the customer experience as close as possible.

Hopefully this helps some folks out there. I know this took me longer than it should have to get right.

Using zen-like focus for minimizing distractions

For those of us in the world of tech, we tend to be busy folks these days. Some of us more than others, but at least in my work there is a lot going on all the time. The key is to know what deserves your attention, and what doesn’t. Every distraction is seemingly benign. A spam here, browsing a small site there, doing a little this and that. It adds up before you know it and pretty soon your day is over, and you’re wondering “What did I do all day? I know, I was just so *busy*”. Yes you were busy, but unfortunately you were busy much of the day doing the wrong things.

There have been several developments out there that call to attention this problem. Lifehacker has some tools to minimize distractions.  OS X Lion now has a full screen mode for its apps to remove create a distraction free environment to read or create. While tech tools are just one step in helping out, they’ll only get you most of the way there. So what to to do?

The tech tools are the easiest to implement. The hard part about minimizing distractions is that this isn’t a 5 minute, or even 1 hour activity. Instead, this is a continuous process. When, for example, you’re reading your e-mail, really decide if any of these are a waste of time. A lot of us tech guys get many of these and yes it’s easy to just hit the delete key, but if you’re doing this daily, or hourly, and this isn’t adding real benefit, you’re doing it wrong. Remember the sysadmin axiom that if you’re doing anything repetitive more than once, you should consider automating it.

The next step is the toughest. You’ll need to train yourself to focus, really hard, one one thing at at time. Make this your mantra. Unitasking is something you should be proud of. There have already been many studies that humans are horrible at multitasking. Multitasking simply doesn’t work, and if you have a boss that is convinced otherwise, you’ll need to disavow them of this notion. You need to focus on unitasking.

This means to focus on a single task till completion for as long as possible. If you can do this for 90 mins at a time, you’re likely well ahead of many.  You may even consider using a task timer to show yourself how well you’re unitasking. If you’re aware during this task that you’re finding yourself wandering towards other things, this is normal. Steer yourself back to the task at hand if the others can wait. In fact, this sort of unitasking is not unlike zen style meditation. Clearing your mind. Focusing on a single thing deliberately.

Another key to this style of unitasking is to schedule when you’ll be doing things to increase your mental discipline. Yes, you do need to take mental breaks during your long tasks. Again this can vary from person to person. Some folks can do 30 mins at a time, others much longer. Those mental breaks are a great opportunity to get up, stretch, focus your eyes elsewhere. Some folks will take a mental break by doing some social networking, and a little minesweeper, or tetris but this doesn’t work for a couple reasons.

• You’re still engaging your brain, and doing nothing for your eyestrain, but now you’re doing something else.
• You’re annoying your boss and peers by such behavior. They think you’re just goofing off, and admit it. You are.

Schedule when you will be doing these fun things however. Everyone wants to know what’s going on with their friends, and in the world. Nothing wrong with that, but just regulate it to certain times of the day. Likewise for the rest of your tasks, schedule them out so you know what you’ll be doing. The more structure you’re able to put here, the more chances for succeeding in what you sought to do that day you’ll have. I highly recommend Getting Things Done and Time Management for System Administrators to help with creating systems for task lists.

Don’t forget that you will still need to make time to go through your e-mail, build up your spam lists. Create tasks to reduce those distraction creators. With discipline, you’ll be able to further increase your productivity, and show yourself at the end of the day that you really were busy doing the right things.

Why BSD is dying, or How I learned to stop worrying and love Linux

I’ve been a fan of FreeBSD for most of my adult life. Ever since I first had to install it back at my first job in the bay area. It was the mid 90’s. FreeBSD was version 2.2.1.

Now before then I’d messed with the venerable Slackware, and a little of Redhat. Slackware was a nightmare to install back then (maybe it still is). You didn’t download anything because well, how far are you going to get with a 33.6kbps modem anyhow? I fell in love with FreeBSD. It was easy to install. Package dependancies were easy, ports just worked them out. It was stable as hell. The community wasn’t bad either.

Something happened between then and the 2000’s. Some important folks have since left the project, such as Jordan Hubbard and Matt Dillion. Likely for their own reasons. The project itself isn’t dead, they’re still doing a lot. They’re just focusing on the wrong things.

Section6 no longer runs on FreeBSD. It was much due to the fact that I’ve migrated from my own hosted server to EC2. AMI’s for FreeBSD are out there, but it’s not considered stable or widely supported. Ubuntu, and Redhat will just work. This is a reoccurring pattern with the project. FreeBSD is still stuck in the 90’s for a few reasons:

• There’s no rapid deployment system like Debian’s fai, or Redhat’s kickstart.
• You still have compile all your ports.
• An upgrade requires you to recompile the entire OS, and they rarely go smoothly.
• A port management add-on community has sprung up because they refuse to integrate a better native package management system.
• It’s still lagging behind for cloud applications.
• Desktop applications are quaint because everyone knows the multimedia support is lagging.

For a pragmatic sysadmin such as myself, it’s become increasingly hard to justify running it. Especially since my day job consists of all Linux distributions. It just takes so long to do anything. That is FreeBSD’s greatest sin. An operating system that forces you to waste so much time, between common tasks for hundreds of servers these days, compiles, installs, for such little benefit, isn’t doing anyone a service. The only thing FreeBSD has going for it these days is zfs and jails. Even then, Linux has xen, openvz, and vserver. Soon btrfs will be production worthy which has pretty much feature parity with zfs. Even then you still can get native zfs on Ubuntu and other Linux.

So I’ve gone with Ubuntu. It took about 2 weeks to migrate everything. I can save that for another post, but let’s just say that I could have made it easier had I used something like cfengine from the start. I’ll miss the ole daemon in favor of the currently mascot-less Ubuntu. The community certainly is just as good if not better. I’ll still keep the old tutorials around, but I won’t be writing anymore for FreeBSD. It’s obvious there’s no tutorials needed for Ubuntu. There’s quite a tutorial writing community out there.

Music in the Cloud? Solutions pour forth.

For those of us with large music collections, many of us have been aching for a solution to make life just a little easier. A few requirements:

• Allow me to back up my music someplace besides a few devices at home so I’m not wasting disk space which would ultimately be useless in the case of theft or fire.
• Allow my various systems at work, and home to play music synced from, or based on a single authoritative source.

Right now I have 84GB of music “backed up” on various devices (laptops, ipod, etc). The issue is that I have to sync all these devices from the master. Of course not all the devices are always reachable all the time. Being an Apple user, I have an rsync script that just rsyncs ~/Music/iTunes/ which runs into its own problems from time to time. I’ve been looking for quite some time to find a better solution for syncing my music. A few companies have entered the fray to help users with this sort of issue, though at the moment, there is no perfect solution.

Google Music:

So I tried Google Music (beta). To start, you install Google’s uploader app, Music Manger, which scans through your music and figures out what’s going on. If you have iTunes, it will just import everything from there. It also doesn’t matter what OS you’re using. Mac, Linux, PC. You can upload with any of them. Music manager also has some nice features, such as types of rate limiting so your internet connection isn’t dead slow during the process. Once you’re done, you’ll find it’s imported all your iTunes music and playlists exactly. There’s little after the fact mucking around with fixing things it didn’t import right. They also support the more popular formats, along with less popular onessuch as OGG and FLAC. Finally there’s a 20k song limit which doesn’t affect my 13k song library yet.

As for the player, the mobile web version works well on iphone.  They of course have an Android app. For everyone else there’s the browser which is very nice considering it’s just a web browser for control. Maybe not as nice as iTunes, but I’ve seen a lot worse native players for OS X.

The dealbreaker with Google Music is that their system is a one way backup. Once you’ze put all your stuff onto Google Music, then great, you’ve got your music there. The theoretical fire or theft happens, or you simply want to “cash out” your music and take your files somewhere else. Too bad. It’s stuck there. Sure you can play your music anytime you want, but for your music, Google Music is the Hotel California of music storage. The only upside is that (for now) it’s free. Google satisfies my requirement #2, but certainly not #1. One option would be backing my music up to something like Amazon S3, which would be nearly $12/mo for my 84GB.

Amazon Cloud Player

A very similar model to Google though it’s not free. It’s also not a bad as Apple claims. If you compare Apple’s competitor pricing to that of what Amazon claims. It’s a ways off. Also, Amazon now stores your music for free, technically it’s an annual cost of $20 since that’s the lowest paid plan with the free music option. Still, that’s much better than Apple’s claim. In fact today, Amazon’s Cloud Service is cheaper than what Apple is announcing since it’ll be $20/year vs $24.99/year. It seems they’re taking a preemptive strike against Apple and Google.  Like Google, you download an uploader app. In this case it’s based on Adobe AIR.

Like Google’s Music Manager, it’ll take all your iTunes playlist and replicate them with a great amount of accuracy. However, there are somethings the uploader is lacking. First, it won’t let you upload tracks over 100MB. Ironically, you can buy these tracks from Amazon, and store them on your cloud drive. The cloud uploader just filters them. If you have anything that’s not MP3 or AAC, you’ll need to convert these. The Amazon uploader won’t let you rate limit your uploads. It’s either on full bore, or off. Finally, you can only use a Mac or PC for uploading. Sorry Linux users.

For “cashing out”. Amazon does make it possible to download your music. It’s not quite the same as  s3, but they’ve released an Amazon Downloader app (to compliment their uploader app). So cashing out of the Amazon ecosystem is almost as easy to get out of as it is to get in. Selecting all tracks to download isn’t exactly easy. Likely the best way is to make a playlist based on adding each genre you have, and then downloading the whole playlist. Better than nothing I suppose.

The player is similar to the Amazon player. They also have an Android player. Cloud Player “works” on the iPhone. Meaning you can play from it, but as of this writing it’s not mobile optimized. Strange that they’re courting OS X users, but not providing a decent iPhone option.

Web Players Lowdown

Like Apple claims, yes it takes a very long time to upload my music. Really I don’t care that much. I’ll let my system happily upload music all day long. I just want it to work, and I’ve been trying to devise my own solutions for years, so what’s a few weeks?  I admit I also like the fact it’s all browser based, so it works on anything. Message to Apple: “Being forced to use iTunes is a negative, not necessarily a positive.” One other thing with these services. If you’ve painstakingly populated a bunch of fields in your tracks, well, they’re not really visible in these services. It’s just simple track number, title, artist. Nothing as fancy as what iTunes provides.

The Dark Horse: iTunes Match

iTunes Match is touted as a service which will store your music on it’s cloud (named unsurprisingly, iCloud). They’ll upconvert your music to their standard of 256kbps AAC. Mostly by scanning your drive with itunes, and then tagging that song as yours in their collection (that’s the match part). There’s no uploading here. Recently used songs are stored on your system as a caching measure. It sounds like a utopia of music sync if you’re an iTunes user, right?

As of now, most assumptions about how iTunes match works are speculation. Their site is somewhat vague with how the service will actually work. It seems that iTunes Match will satisfy my requirement #2, so long as I’m using iTunes. Though I wonder if I’ll be able to cash out and download all my music back to my own storage if I want to. On the surface, it seems like Apple is using iTunes Match to provide a much demanded service to their customers. I do wonder if Apple is trying to lock it’s customers into it’s iTunes ecosystem, or if they’re just relying on this feature to make themselves more successful, and they don’t care what people do after the upconvert. We’ll find out this fall.

Evernote: A Reflection

At work I’ve been pretty busy, and have gone through a few ad hoc capturing tools. First it was the steno notebook. Followed by the hipster pda. That worked pretty well except when I forgot it or inevitably lost it, it would wreck havoc on the rest of my day since half my brain was in there. The last time I left it at home, I decided to look for alternatives.

Enter Evernote. Now the reason I had not originally gone this route was for the following reasons:

• The pen and paper is cheap
• No batteries required
• Easier to draw things on it.
• Using pen and paper is just more of a classic way to capture things.

I also had some hestation in not using Evernote including:

• Information is not stored on my server (privacy issues)
• Batteries *are* required
• Something new to learn
• Something new to spend money on (yes Evernote is free but I ended up spending money on it anyhow).

I still wish I could store Evernote stuff on my server, but that aside, using Evernote over the hipster PDA has been a great success. I recently found my disassembled hipster PDA, noticing I hadn’t written in it in months. I can’t say I really miss it. A few additional benefits I’ve found with Evernote.

• It provides an easily syncable space you can use to capture notes, photos, or voice notes (great if you’re on the go and have an idea).
• Very hard to lose your notes as it syncs with your phone, and your laptop. Also accessible via a web interface.

I have to say it really has made my life better. That’s not something I’d typically say about the latest new shiny thing. To anyone who’s been on the fence about Evernote, I’d say it’s worth a try.