bash history forever

In what seems eerily like a burst of structured procrastination I recently got around to a task I'd been meaning to do for ages: infinite, global bash history across all of my shells.
I started by reading over how the history mechanism in bash is supposed to work, and then a co-worker and I set about writing the simplest hack that could actually record all my command lines and aggregate them for me.
Step 1 is to make sure that every typed command is recorded somehow. Simply extending my PROMPT_COMMAND to be a function that would call other functions is a good starting place:

export PROMPT_COMMAND='process_prompt'

function process_prompt {
  write_last_command
  write_global_history
  read_global_history
  xsettitle $HN $(shortenpath $PWD)
}

I'm going to ignore the titlebar setting stuff as it's not related to this post. As you can see, the basic structure at the top level is to record the last command typed in the current shell; recompute the global history; and then load the global history into this shell. Inefficient? Horribly so. Can I tell? Not on today's monster workstations. Now, to record the commands of the current shell requires a file to write to:

COMMANDFILE=~/.histories/$HN.$$
trap "rm -f $COMMANDFILE" INT TERM EXIT

where in the above snippet $HN is my hostname after processing. If you're actually following along you will also want a some lines like this:

export HN=$(hostname)
HN=${HN%%.*}

to set the hostname. The commands from each file, then, are to be written to my home directory in a file named .histories/hostname.processid that will get deleted when the current shell exits. The actual writing of each command line is, amusingly enough, accomplished via the history bash built-in:

function write_last_command {
  echo $(date +%s).$(history 1 | sed -e 's/^[ 0-9]*//') >> $COMMANDFILE
}

Then, the next step is to merge all the files in ~/.histories into a master file:

function dedup_history {
  sort -nr | sort -t\. -k 2 -s -u | sort -n
}
function write_global_history {
  cat $(find ~/.histories -type f) | dedup_history |
     sort -t\. -k 2 -s -u | sort -n > ~/.histories/global.$$
  mv ~/.histories/global.$$ ~/.histories/global
}

And finally, load the history into RAM:

function read_global_history {
  sed -e 's/[0-9]*\.//' < ~/.histories/global > $HISTFILE
  echo '-------------------' >> $HISTFILE
  cat $COMMANDFILE | dedup_history |
    sed -e 's/[0-9]*\.//' | tail -10 >> $HISTFILE
  history -c
  history -r
}

I personally did all this so that I'd have all my commandlines, forever, across all of my computers. So, clearly the default history size was far too small:

export HISTCONTROL="erasedups"
export HISTFILESIZE=10000
export HISTSIZE=10000

And that's a wrap! A simple script to copy the global history from each workstation up to a central repository and pull the workstation specific histories back to each workstation makes it cross-machine. For example, you can periodically copy ~/.histories/global to a central location as global.$HN, and copying global.* back into your history directory will get all the command lines from other workstations merged into your current environment.
I've had this scheme running for a few days, and love it! The idea that I now have a persistent store of all useful command lines may even change how I choose to write those command lines, making me deliberately include extra but useful information in them. Enjoy.

twitter's tweets

So I signed up for twitter this weekend, and I have to say the jury is really out on this one. I mean, I can barely understand the huge appeal of blogging, RSS, and non-job social networking (myspace and facebook) — so why would I understand twitter?

In case you haven't heard of it, twitter.com lets you set up an account for broadcasting your mood. It's like IM status, without the IMs. All status all the time; all one way communication. Blogging for the severely ADD. Instead of getting a feed reader, you just set it up to text all your incoming streams to your phone, because you need more interruptions in your life to make you more efficient, right?

So perhaps I'm sounding so negative on it that there would appear to be no point to me even signing up. Except that's not so: what's interesting to me about twitter is that it creates a new kind of socializing; a new way to be connected with friends and family and strangers. Online personae are becoming something that more and more people have — it's not just a handful of geeks on IRC, or MOO, or tinyMUD, but rather a full-blown epidemic of artsy types finding ways to express themselves to a broader audience. In addition to the social implications, there's the possibility that an intelligent blogger in India could now earn a decent living from Google ads alone. Soon, this blogger may also need to twitter regularly, to remind her readers that she can be interesting in 140 characters or less as well as in longer articles. Twitter is becoming big enough to be part of an online persona that may someday be as important as having a nice shirt or a decent haircut: so it's time to sign up, and see how it works.

How it seems to work varies a lot according to who you follow. Some people twitter whatever silly thing crosses their mind. You won't follow them for long. Some people put so much thought and effort into deciding what clever thing they should tweet every 12 hours that you can spend minutes looking for the hidden meanings behind what they're saying. But I think most are just trying to be a little cool, and stay connected. And that's what the internet is all about.

twitter.com/osric

Truly on track again

It's so good to be truly back on track - it's been going pretty well since my last post.

I did another closet cleanout. It was amazing how many shirts I had that just no longer fit - they're all too big. I wound up with a huge pile of large, xl, and even some mediums that had to go. It's so bizarre to clean out my closet and find nothing that's too small!

Finally, this time, a sweater that my daughter had picked out for me more than 5 years ago that has always been too small actually fit! It's a bit form-fitting, and last time I tried it on it fit but too well ... the man-boobs were just not tolerable ... but now I'm small enough that it's not really noticable and the sweater actually fits! Excitment.