blog.josephhall.com

Using the find command

2010-04-15T20:50:00.000-06:00

Today I was working on a problem with a coworker, and I saw him start to type the following:


find . | grep

I said, "no, I'm not going to let you do that. You need to do it the right way." (Yes, I do that sort of thing. Don't act like you didn't expect it.)

Don't get me wrong, there's nothing wrong with grep. It's a fine tool, and I encourage people to constantly try to become better with it. But it has its place, and this was not it. Using the find command properly in this case would result in less typing, and would spawn one less process. It may not be important for a one-line command, but in a larger script it might be more significant. Better to get in the right habit now, so that when you do find yourself working on that big script, you do it right the first time.

The find command is extremely powerful. Unlike locate, which uses a pre-built database of files and paths on your system, find searches your filesystem in real time, paying more attention to the individual files, and their properties. It may be slower than locate, but it's more accurate and far more flexible.

I see people use find mostly to search by filename, but it has plenty of other options. Let's start with filename and build from there. There are two relevant options:


-name
-iname

They are identical, except that -iname is case insensitive. Since files on a *nix system are traditionally all lowercase, you might want to save yourself a few processor cycles and just go with -name. If there's a chance that case may be an issue, use -iname instead.


find -name 'myfile.txt'

Using quotes is not strictly necessary with most filenames, but it's a good habit to get into. Keep in mind that by default, find searches by exact filenames. If you're not sure what the extension is, or you want to look anywhere in the filename, you can use globs:


find -iname '*myfile*'

The above commands will process files recursively, inside the current working directory. If you want to search a different directory, you need to specify it before any other options:


find /etc -name passwd

There are a few subtleties of find that you will encounter. They're not usually a big deal, but they can be annoying sometimes. For instance, find does not sort its results. If I expect a lot of results, I generally pipe it through the sort command. It also isn't very good at searching its own results, which is where grep can come in handy:


find / -name 'Net' | sort | grep -i perl

Now that you have the basics of find, let's explore some of the other options. Two that I use extensively are:


-ok
-exec

Again, these options are identical in purpose, but there is an important difference in how they behave. Both of them will execute a command on each file found, but -ok will ask for permission first (for each file) while -exec will just do it.


find /etc -name '*conf' -exec mv {} {}.orig \;

First off, everything between -exec (or -ok) and \; is the command that you want to run. Make sure you escape that semi-colon at the end with a backslash, or you'll be sorry. The {} is a placeholder for the filename that was found by find. In this case, we're actually going to be performing a series of commands that looks like this:


mv /etc/httpd/conf/httpd.conf /etc/httpd/conf/httpd.conf.orig
mv /etc/httpd/conf.d/ssl.conf /etc/httpd/conf.d/ssl.conf.orig
mv /etc/httpd/conf.d/perl.conf /etc/httpd/conf.d/perl.conf.orig
...SNIP...

You're not limited to just searching filenames by glob. The find command does actually have support for regular expressions, using the following:


-regex
-iregex

I don't think I need to tell you that -iregex is the case-insensitive version of -regex. If you already know how to use grep, this isn't much of a stretch:


find -regex ".*deskto."

The find command also supports boolean logic:


-and (or -a)
-or  (or -o)
-not

Let's combine these with couple more options from the man page:


touch /tmp/jayceweb.tar.bz2
find / -user jayce -and -group apache -exec tar --remove-files -jrf /tmp/jayceweb.tar.bz2 {} \;

This is the sort of command a person might run if they found a user on their system that they didn't trust, and wanted to quaratine all of their web files. First we make an empty tar file, then we add the suspicious files to it, removing them once they've been archived. It assumes that the user that owns the files is jayce, and the group that owns the files is apache. You could also make use of:


-uid
-gid
-nouser
-nogroup

That's probably enough of a primer to get you started. Now would be a good time to check the man page for some of the other myriad options that you can use to check by date stamp(s), file size, file type and even permissions. A little practice with this powerful command will save you time and energy, and increase your productivity like you won't believe.

Getting Started with Cassandra

2010-03-13T06:34:00.002-07:00

12 days ago, I read an article by Matt Asay which briefly mentioned Cassandra, Facebook's NoSQL offering. I had heard of it before, but hadn't really looked into it. For some reason, Matt's article caused me to look into it again. Within a couple of hours, I was evangelising it to a few friends. Matt's article pointed out that Facebook, Digg and Twitter had all started using it, and as I researched it, it seemed that Digg's and Twitter's migrations to it had taken only a few days. Last night, one of the people that I had been hyping it to sent me a thing from Reddit, posted only 11 days after Matt's article, talking about how they had just finished a 10-day migration to Cassandra.

What is Cassadra?

In order to understand Cassandra, it would first make sense to talk about exactly what this NoSQL movement is.

What is NoSQL?

NoSQL is a nickname that arose sometime last year to describe a series of increeasingly popular database management systems (DBMS) that do not use SQL as an interface, as has been common in database servers for at least a couple of decades. Indeed, SQL is hardly the issue here at all. If you were to migrate from MySQL to PostgreSQL, chances are you would still have to update several of your queries in order to be compatible with the new DBMS. It's almost like changing languages anyway, except that it's more like the differences between Canadian French and Hatian Creole: both French, but not pure French, and there are enough differences to matter.

Switching from SQL to NoSQL is a little more like switching from English to Japanese. Both are languages which accomplish the same goal (interperson communication), but both take very different approaches. They have different keywords, different grammars, and some would argue that Japanese is a much more precise and efficient language. One might even bring scalability into the discussion, as both languages have had the opportunity to grow. One might argue that English has done so sloppily, borrowing from odd places, whereas Japanese has done so with a little less mess, for instance, adding an entire syllabic infrastructure called katakana in order to handle foreign words, among other things.

Both SQL and NoSQL are DBMSs. They both hold data admirably, but whereas most SQL servers were originally built before the idea of database clusters was common, NoSQL servers were introduced around the time that database clusters were becoming a necessity in many infrastructures. This provided NoSQL servers with the ability to consider this concern during the design stages, rather than having to patch it in later. Some of the names that you will see in the NoSQL world are BigTable, Dynamo, HBase, Hadoop, CouchDB, and Cassandra.

So, What is Cassandra?

Cassandra is a NoSQL DBMS written by Facebook. It was open sourced in 2008, and added to the Apache Project in 2009. It is fault-tolerant, decentralized, and "eventually consistent", meaning that when data is added to the database, there is a propagation period before that data is available to all of the nodes in the cluster. A more famous database model that is also eventually consistent is DNS: zone records are updated higher up in the DNS tree, and then trickle down to relevant servers in an organized fashion. This used to take 72 hours or more in DNS, but these days takes closer to an hour. With Cassandra, it is more likely to take a few seconds. This means that your applications must be written with this consideration in mind,

Rather than using SQL, Cassandra uses a system of key/value pairs. This is not a new concept to most programmers, whether they refer to them as libraries, associative arrays or hashes. The concept should be immediately familiar to any Perl programmer, and possibly even more comfortable to anyone who has ever worked with JSON. One major difference is that each name/value pair is also timestamped. So a column, as it were, in Cassandra is comprised of a name/value/timestamp set. For example:


{
    name: "email",
    value: "test@test.com",
    timestamp: 1259991135887
}

Cassandra also has what's called a SuperColumn, which is a grouping of columns, much like a hash of hashes in Perl. For example:


{
    name: "person",
    value: {
        realname: { name: "realname", value: "Billy Bob Test", timestamp: 1259991135887 },
        email: { name: "email", value: "test@test.com", timestamp: 1259991135887 },
        ircnick: { name: "ircnick", value: "billybobtest", timestamp: 1259991135887 }
    }
}

That's all the technical detail that I'm going to go into at the moment, largely because there's already so many great articles out there to get you started, but also because I'm new, and still know just enough to be dangerous (mostly to myself). But I am going to link to a few of those articles for you, if you're interested enough now to check them out.

Bearing in mind that I'm a Perl guy, here are the links that I've already sent out to a couple of friends in email, which may or may not cover your language of choice.

For information about Cassandra and some theories behind it, you'll want to take a look at these links:

http://incubator.apache.org/cassandra/
http://www.allthingsdistributed.com/2008/12/eventually_consistent.html
http://bryanpendleton.blogspot.com/2010/03/following-links-to-cassandra.html
http://blog.evanweaver.com/articles/2009/07/06/up-and-running-with-cassandra/ (Ruby examples included)

When you're ready to install it and start playing with it, you'll want to read these, in roughly this order.

http://dustyreagan.com/installing-cassandra-on-ubuntu-linux/
http://wiki.apache.org/cassandra/CassandraCli
http://arin.me/blog/wtf-is-a-supercolumn-cassandra-data-model
http://search.cpan.org/~lbrocard/Net-Cassandra-0.35/lib/Net/Cassandra.pm (For the Perl guys)

As I continue to explore and learn Cassandra, you may see an article here and there about it on my blog. I'm pretty excited about it, and while I see no reason to completely abandon SQL databases (they all have their uses, many of which Cassandra is likely not well-suited for), I think that I'm likely to use Cassandra as a major component on an upcoming project.

Interpreting Recipe Input

2010-03-12T18:12:00.002-07:00

As some of you know, I've been working on recipe software off and on for a few years. It keeps finding its way to the backburner, mostly because other things have always taken priority, but also because doing it The Right Way (TM) is pretty intimidating.

A few weeks ago I started The Latest Attempt. I started simple, and had few eyeballs look at it. A couple of days ago, I went back to the beginning of my blog and started transcribing recipes into the simple interface that I had. I ended up finding several issues, most of which I don't think most of my testers ever encountered. I thought I'd lay them out here, and let them percolate in my brain.

I'm going to use an example that wasn't in my early archives, but that I've been playing with lately. The original is here. I blogged about a version of mine here. Hopefully Food Network won't mind if I reprint the original here, because I'm going to tweak it a little of the sake of demonstration.


6 squares unsweetened chocolate
3/4 cup unsalted butter
2 cups sugar
3 eggs
1 teaspoon pure vanilla extract
1 cup unbleached allpurpose flour
1 cup chopped nuts (optional)

Words like "pure" and "unbleached" sound pretty specific. But it starts with "6 squares unsweetened chocolate". What size is a "square"? The experienced baker will tell you that unsweetened chocolate is often measured in one-ounce squares. But it presents the first problem what I've encountered, and that at least one tester came across too:

Non-Standard Measurements

"two sticks butter". "one can olives". "one package spinach". These are all arbitrary sizes, that don't necessarily mean what you think they mean. Okay, so in America, butter comes in 4 oz sticks. That's something that we can rely upon. But one can of olives? Are we talking about the little 4 oz cans of sliced or chopped olives? Or are we talking about a 15 oz can of whole olives? How about the spinach? I've seen fresh spinach come in packages ranging from a few ounces to a couple of pounds, and who's to say we're not talking about frozen spinach? This actually leads into the next issue:

Inspecific Ingredients
What kind of butter? Salted or unsalted? A professional chef would never cook with salted butter. Joe Q. America, who knows? And I've already brought up the issues with olives and spinach. But the issue that I found here was actually in trying to categorize the food items, with minimal effort on the user's behalf. I've been using the USDA SR22 this time around, along with some auto-suggest AJAX code, to try and link up what the user has been looking for with something that the user can use for things like nutritional charts. The SQL looks something like this:


select NDB_No, Shrt_Desc from ABBREV where Shrt_Desc like '%butter%' limit 10;

Offhand, it seems reasonable that this would give us results like "unsalted butter", "salted butter", etc. But the SR22 isn't in an order that is condusive that that kind of thing. Instead, we get a result like this:


+--------+------------------------------------------------------------+
| NDB_No | Shrt_Desc                                                  |
+--------+------------------------------------------------------------+
| 42291  | PEANUT BUTTER,RED NA                                       | 
| 42307  | MARGARINE-LIKE,BUTTER-MARGARINE BLEND,80% FAT,STK,WO/ SALT | 
| 42309  | MARGARINE-LIKE,VEG OIL-BUTTER SPRD,RED CAL,TUB,W/ SALT     | 
| 43214  | BUTTER REPLCMNT,WO/FAT,PDR                                 | 
| 11866  | SQUASH,WNTR,BUTTERNUT,CKD,BKD,W/SALT                       | 
| 11867  | SQUASH,WNTR,BUTTERNUT,FRZ,CKD,BLD,W/SALT                   | 
| 11372  | POTATOES,SCALLPD,HOME-PREPARED W/BUTTER                    | 
| 11373  | POTATOES,AU GRATIN,HOME-PREPARED FROM RECIPE USING BUTTER  | 
| 11381  | POTATOES,MSHD,DEHYD,PREP FR GRNLS WO/MILK,WHL MILK&BUTTER  | 
| 11385  | POTATOES,AU GRATIN,DRY MIX,PREP W/H2O,WHL MILK&BUTTER      | 
+--------+------------------------------------------------------------+

Only one of those even remotely resembles what I'm looking for, and honestly, I don't want it. This problem is easily, if tediously solved: all I need to do is create a new database, of commonly used ingredients, and query it first, and then supplement it with the second database. Oog. Let's move onto the really tricky stuff.

Storing Measurements

Let's go back to our brownie recipe. One of the ingredients is 3/4 cup butter. Databases don't store fractions in any mathematically-usable format. Do we store it as a VARCHAR to maintain integrity with what the user entered, and then convert it later? Or do we store it as a decimal, and then store a flag saying whether it was entered as a fraction or a decimal? Or do we store it as a decimal, and assume that it will always be displayed to ther user as a fraction (which is usually what the user wants)? For the sake of argument, let's go with decimals as the storage mechanism, and not worry about anything else for now. In MySQL, we might have something that looks like this:


...SNIP...
amount DECIMAL(10,2),
unit VARCHAR(10),
...SNIP...

Measurement Ranges

Brownie recipe again. One of those ingredients, the nuts, is optional. It's technically a garnish, if an internal one. It's already subjective (walnuts? peanuts? pecans?), which means we can fudge a little on the amount too. Depending on how much you like your nut of choice, let's say you might opt for anywhere from 3/4 cup to 1 1/2 cups. This presents another problem with database management, at the very least. Maybe we can solve it by breaking the amount into two different fields?


...SNIP...
amount_min DECIMAL(10,2),
amount_max DECIMAL(10,2),
unit VARCHAR(10),
...SNIP...

Intermixed Measurement Units

Let's play with the sugar a little. A lot of people (like me) like to swap out some of the white sugar with brown sugar. Let's say that after careful testing and tweaking, I come up with the following measurements:


1 cup + 2 Tbsp white sugar
3/4 cup + 2 Tbsp brown sugar

Oh man. How do you store that? We could convert everything down to the lowest common denominator, Tbsp in this case, and then upscale when we display it back to the user. In the database, we would have something like:


18 Tbsp white sugar
14 Tbsp brown sugar

Of course, now we have to write code to scale this back to a reasonable measurement, since 18 Tbsp of anything is just weird, even just for behind-the-scenes storage. I think I would rather convert everything to a common unit, store it, and then convert it back. And I don't think any (non-metric) unit of measurement is more versatile than the ounce. Now we can store the sugar as:


9 oz white sugar
7 oz brown sugar

Of course, this leads us to the next problem:

Weight vs Volume

In the metric world, this isn't an issue, because everything is stored in either some version of grams or some version of litres. But in the Imperial system favored in America, an ounce could mean weight or it could mean volume. With some ingredients, this isn't a big deal. "A pint's a pound, the whole world round", right? Makes sense, since a pint is 16 ounces by volume and a pound is 16 ounces by weight. Well, not exactly (1 pint == 1.043 pounds), but close enough for the home cook.

In the above example, we know that we're referring to volume, if only because we know that we started with cups. But if I were looking at the recipe without any context, I personally would start off by thinking that it was a weight measurement. Some would assume it was volume. Even for the home cook, this is kind of significant, since a cup of sugar weighs a little over 7 ounces, not 8 ounces. In the aforementioned example, we could just store a flag that states whether this unit is by weight, volume, count, etc. If a user specified ounce, without any context, we'd have to store it as "unspecified", until it became important to the user (say, for nutritional data).

Of course, this is all backend stuff. Let's talk about another major problem:

Dealing With Users

If you can force the user to use your own forms, custom-tailored to suit your database, you can force them to do everything properly. And any UI designer worth his or her salt can tell you, when you start forcing users to what you want, you start losing users to your competitor. Your competitor's software may or may not do an inferior job, but if it makes your users feel better, that's what your users will use.

I've heard a lot of people complain about recipe software. From my limited experience, when a user gets tired of their recipe software (as most inevitably will), they will switch back to using a word processor, or possibly a spreadsheet. So it seems to me that the best way to get somebody to use your recipe software is to make it feel as much as possible like using a word processor. That means using a lot of fuzzy logic to convert what your users type into something that your software can use.

As you can see, writing recipe software The Right Way (TM) presents itself with a lot of issues. I haven't figured out how to handle most of them, and one of the biggest problems seems to be that some issues are dependent upon other issues. Well, I'll get it figured out.

Homemade Server Rack

2010-03-12T12:28:00.002-07:00

I actually built about a year ago, and never bothered to post it. I mentioned it to a couple of people this week, and thought I would post it for them:

I had a couple of 2U servers laying around that I was planning on installing, but that's kind of a weird form factor for setting up around the home. Fortunately, we had some laminate boards laying around, left over from a water-damaged built-it-yourself stand-alone closet that was in the basement when we moved in. I tossed the water damaged parts, but held onto the rest of the boards, just for something like this.

I used 4 shelves, and built a box about the right form factor. I already had some casters (not shown in the photo, because they're underneath) so I attached them to the bottom to easily move the box around. The only parts that I ended up buying were the mounting racks for the servers. It's just not a part that I had laying around the house. It's also not a part you can find at Lowes, but it's easy to find online. I bought mine from Star Case. I figured that a 10U set should last me for a while.

Primary Keys in MySQL

2010-02-18T07:17:00.004-07:00

I was reading an AJAX tutorial the other day, and something that the author said caught my eye:

"All we really need is the title, but I always provide a primary key for any table that I create."

Why add it if you know for a fact that you don't need it? In this case, as with every case I've seen so far, the primary key in question was an auto_increment integer. I have long maintained that while this is necessary in most tables, it does not necessarily belong in every table. What you really need is a unique identifier. Without this, all you have is a jumble of data that really doesn't make sense to be stored in a database.

But that unique identifier doesn't always need to be a counter. Let's take a look at a couple of examples. Consider the following hypothetical user table:


CREATE TABLE users (
    username  VARCHAR(50) NOT NULL,
    realname  VARCHAR(50) NOT NULL,
    password  VARCHAR(50) NOT NULL,
    PRIMARY KEY (username)
);

It makes sense for a username to be unique, because the username/password (or other token) combination will be required for access to this application. If you need to refer to this table from another table, you can just refer to the username, because it is unique and identifiable. I might note that MySQL itself does not use an auto_increment field for its own user table.

There are potential problems with this, however. When you change a username in this table, you need to change it in any tables that reference it as well. Additionally, you are taking up a little extra (if negligible) space in referring tables. An auto_increment uses an integer, which takes up less space, and will not change under normal circumstances. It should be noted that Unix-style operating systems use a UID to identify not only users, but file and process ownership. The username itself is rarely used by the system for anything other than making things more human-readable.

Let me show you a table structure that I've been working on this morning. I have a need to store recipes in a database, but because a recipe contains varaiable numbers of ingredients and directions, it doesn't make sense to try and store each in its own field in a single recipe table. Instead, I have broken out my structure into three separate tables:


CREATE TABLE recipes (
    recipe_id    INT(11) UNSIGNED NOT NULL AUTO_INCREMENT,
    name         VARCHAR(255) NOT NULL,
    source       VARCHAR(255) NOT NULL,
    preheat_qty  INT(4),
    preheat_unit ENUM('F','C'),
    yield_qty    INT(4),
    yield_unit   INT(11) NOT NULL,
    PRIMARY KEY (recipe_id)
);

CREATE TABLE recipe_ingredients (
    recipe_id      INT(11) UNSIGNED NOT NULL,
    rank           INT(4),
    name           VARCHAR(255) NOT NULL,
    qty            INT(4) NOT NULL,
    unit           INT(11) NOT NULL,
    PRIMARY KEY (recipe_id, rank)
);

CREATE TABLE recipe_directions (
    recipe_id      INT(11) UNSIGNED NOT NULL,
    rank           INT(4),
    direction      TEXT,
    PRIMARY KEY (recipe_id, rank)
);

The recipe itself is referenced by a unique counter, which is usual. INT(11) contains far more unique identifiers as I expect to need for that table. But there will be multiple ingredients per recipe, and multiple directions per recipe. Even if I only store recipes with no more than 4 ingredients and 4 steps (which is unlikely), I need 4 times as many unique identifiers per table.

I already have to store the recipe_id, to keep from orphaning the data. It's important to store what order each step is in, because if they got out of order, the recipe would quickly becaome confusing. It's nice to store the order of ingredients in the order in which they'll be used too, and many people write recipes with this in mind. I've called this field "rank". Since I already have those two fields, and they already uniquely identify the rows, why not officially make them primary keys together? MySQL allows it, and I'm going to make use of it.

I propose that whenever you create a table, you take a moment to consider whether or not you actually need a counter. Most of the time you will, but not always. Get out of the habit of doing things because that's the way you've always done them, and get into the habit of doing things because you've thought about them and have made an informed decision specific to the situation at hand.

Buzz off social networks!

2010-02-12T05:47:00.002-07:00

Buzz off social networks!

Posted using ShareThis

Dedicated to eightyeight.

Google Reader: Disable Sharing

2010-02-11T20:42:00.003-07:00

Don't get me wrong, I like Google Reader. Every other RSS reader that I've tried to use is comprised of at least 90% suck, whereas Reader only seems to be about 10% suck. Most of that suck has to do with its social networking "features".

I was surprised yesterday to find a follow invite from my brother, in Google Reader. This is because a) he hates social networking at least twice as much as I do, and b) I was entirely certain (and later confirmed) that he's never opened Google Reader. As I poked around, I discovered that I was somehow following 27 people. Four of them had apparently accepted an invite from me, and the rest had been sent invites that had not yet been responded to.

Let me be clear: I have never sent a follow request in Google Reader. Google automatically sending invites to people on my behalf feel a little like when I hear people say things like, "Oh sure, Joseph would love to come to your <INSERT EVENT HERE>." Would it kill you to ask me first? One of the people who had been sent an invite was a person who had sent me a business proposition, which I respectfully declined. That was to be the end of my contact with her, but Google was so kind as to send her a Google Reader invite for me. Gee, thanks.

Those who have tried to unfollow have probably found yourself frustrated with the fact that there is no interface inside of Reader to unfollow a person, or to turn off sharing altogether. Worry not, I figured it out.

From inside Google Reader, click the "Settings" drop-down in the top-right corner.

Select "Google Account Settings" (not Reader settings). This will open a page where you can edit your profile.

Click on "Edit Profile". There's nothing useful on this page, so click "Save" or "Cancel" at the bottom (intuitive, isn't it?).

The next page has a link that says, "<YOUR NAME> has X followers" and another link that says, "<YOUR NAME> is following X". You can't stop people from following you, but you can click that second link and stop following people. These are the ones that Google sent invites to without your permission.

Of course, it would be nice for Google to give you the option to disable sharing altogether, so that you could just use Reader as a regular RSS reader. One day I'll find one of those that doesn't suck.

Google Buzz

2010-02-10T20:02:00.002-07:00

People are probably sick of me complaining about stuff like this, so I'll keep it short.

Google Buzz showed up in my Gmail account this morning. I tried it a little bit, read some feeds, gave it a shot. At the moment I find it more annoying than Twitter, so I've turned it off. You can too!

Clicky!

Maybe I'll give it another shot later. For now I've shoved it back into the depths from whence it came. It can share rent with its other dejected roommate, Google Wave.

My Drive Array

2010-01-30T11:15:00.002-07:00

A couple of years ago I got ahold of an old ATX computer that I intended to use as a file server. Unfortunately, there were a few problems with it. The biggest problem was that the drive cage for the smaller drives was missing. Smaller problems like an underpowered power supply and limited onboard IDE adapters were fixed with things like a new 600W power supply, and extra IDE expansion cards. As it turns out, Linux had no problem with two onboard adapters and two more cards (two adapters per card). With IDE's master/slave setup, that brought me up to two DVD burners (/dev/scd0 to /dev/scd1) and six hard drives (/dev/hda to /dev/hdf). But the drive cage, that was a problem.

Fortunately, like many American bakers and pastry chefs, I spent a lot of time at the hardware store. And believe it or not, the roofing section at Lowes carries a simple solution: roofing ties. Not TILES, but TIES (no "L"). Behold, the drive array, now connected to my Thinkpad (click to embiggen):

Yes, dear readers, the file server is dead. It seems to have developed memory issues in its old age, leading to its untimely demise. Not Alzheimers, but some form of dementia. A close-up on the array itself:

Sadly, my Thinkpad does not have an external IDE adapter of any kind. But USB to IDE adapters are relatively cheap and easy to find. I can't access every drive at once, but that's not a big deal at the moment.

Lowes has several different sizes of roofing ties, and several different styles. I used two different sizes, both completely flat, but with rows of holes exactly the same width as a standard 3.5" internal drive. One is five holes high and one is three holes high.

If you're going to do this, you'll be buying them in sets of two each. And while you may be tempted to stack three drives in one 3-high roofing tie set, fight the urge! You need airflow between the drives, or they will overheat. I speak from experience. Limit yourself to three drives for the 5-high ties, and only use the 3-high ties for connecting sets of 5-high ties. Look back at photo #2 to see what I mean.

Speaking of heat issues, it's not a bad idea to point a fan at these if you're going to have them all on at once. It's not a big deal with one or two USB-connected drives, but with all six drives that I have in my array, I always had a fan going.

Importing the USDA SR22 into MySQL

2010-01-28T07:11:00.002-07:00

Some of you who were interested in my post on importing SR21 may have been waiting for this one. I know it's been a few months since SR22 came out, but I didn't have a need to import it until just now. There are changes to this new version, but they are minimal, and you may have already patched the code yourself.

Specifically, two new fields were added to the ABBREVS table: Vit_D_mcg and Vit_D_IU. This brings the total column count in that table from 51 to 53. That number is the only change in the Perl file, and those two columns were the only additions to the SQL file. With those in place, I was able to import the new database without a problem. For the lazy and/or efficient, here are the new versions of the files:

import_sr22.pl
sr22.sql

Based on the reponse to my last post, I expect more troubleshooting questions on this post. For those who know what you're doing, you can stop reading now. Everyone else, check here before asking.

You need to have at least Perl 5.8.6 installed.

You need to have the Perl DBI installed, and DBD::mysql.

In RHEL/CentOS/Fedora, these packages should be called perl-DBI and perl-DBD-MySQL.

In Ubuntu/Debian, these packages should be called libdbi-perl and libdbd-mysql-perl.

When you download the files, make sure you save the Perl script as import_sr22.pl, not import_sr22.txt.

This script assumes you've downloaded the abbreviated file. It is a separate download from the full version, so make sure you don't miss it.

I think that covers all the questions I was asked previously. For those who are interested, The Eloquent Geek posted a non-Perl way of doing this on the last post. I haven't tried it myself, but for your reference:

For those who do not want to use the perl here is how you import the data from the command line client:
load data infile '/file_path/TABLE_NAME.txt' into table TABLE_NAME fields terminated by '^' optionally enclosed by '~' ;

Just substitute the table name per table.

iPad

2010-01-27T19:39:00.002-07:00

Well, Apple's new tablet was finally announced today. Aside from a slew of jokes concerning the name, it would seem that the majority of the reviews can be summed up in one word:

"Seriously?"

Not all reviews were negative, of course. One that I read can be summed up in a few more words: "Look! It's a giant iPhone! Isn't that so cool?" Let me put this into perspective. A cute little baby, at the size that you would expect a cute little baby to be, is cute. The same baby, but the size of a house, is no longer cute. It is scary.

I'm not going to bother linking to any reviews of Apple's attempt to make Google look good in comparison, just like I'm not going to bother buying one when it comes out. That's all on you.

Okay, tell you what. Let me Google that for you.

Ice Cream Nomenclature

2010-01-26T06:08:00.002-07:00

Rather than responding in the comments area to Hollie's comment on Ice Cream Machine, I thought I would just make a new post. Her question was, "isn't that a sorbet?" No, Hollie, that wasn't a sorbet or even a sherbet. There are a few terms tossed around for frozen, churned desserts, and there are differences. Some of it has to do with dairy content, but not all.

Sorbet: This dessert generally contains fruit puree or juice, but can contain other things, such as coffee or chocolate. The important thing here is that there is no dairy content. So Hollie, with all the half and half in my recipe, it's definitely not sorbet.

Sherbet/Sherbert: You'd think my recipe would fall under this category, because sherbet can have dairy in it. But as it turns out, sherbet only contains a small amount of dairy, < 3%. The terms sherbet and sorbet are often used interchangeably, and I guess I'm not going to stop that with my little post. But as far as I'm concerned, they are different.

Ice Cream: Once you get above 3% dairy, your dessert becomes "ice cream". This is a pretty generic term that gets tossed around, and is applied to everything from sorbet to gelato. You can have large amounts of fruit puree like I had, or you can just keep it as simple as frozen, churned, sweetened milk or cream. I have a friend that is a big fan of unflavored ice cream: not even vanilla gets in the way of the taste of milk. One day I will have to try it.

Frozen Custard/Premium Ice Cream: Once you add egg to the mix, ice cream technically becomes a frozen custard. But most people still just call it ice cream. I have yet to see a premium ice cream that is not actually a frozen custard, but I'm sure one exists somewhere. From a technical standpoint, there is a definite advantage to using egg, which will help the ice cream set up a little more easily in the churn. It also adds a nice creaminess.

Frozen Yogurt: There's not really a whole lot of difference between ice cream and frozen yogurt, other than the former using cream (or at least half and half) and the latter using yogurt. Because of the yogurt, it's often a little more tart, and generally lower in fat.

Granita: This Italian dessert is kind of like sorbet, but with much larger ice crystals. This is due to the preparation, which is more of a shaved ice technique than a churning technique. The method that I see most often involves pouring flavorful liquid (usually coffee, but sometimes fruit juice) into a cookie sheet and putting it in the freezer, scraping with a fork every couple of hours or so.

Gelato: This is an Italian variety of ice cream who's name I have often seen mis-used and abused in America, so let's set the record straight. One big difference is the low dairy content: almost as low as sherbet. I have seen some gelatos with no diary at all. Gelato also has a higher sugar content than ice cream, and usually involves egg. I have also heard Italians mention some sort of mysterious stabilizer, which I have experimented with before. It's still unclear to me what it is, and if it's actually a requirement. Gelato is churned like ice cream, but has much less air incorporated into it, and is meant to be served fresh, the same day that it is made. While not necessary, gelato generally has a pretty high fruit content.

I have often heard the terms "gelato" and "spumoni" used interchangeably. Let me be clear on this: spumoni is a type of gelato. Not all gelato is spumoni. Spumoni is a layers ice cream, kind of like Neapolitan ice cream in America, but containing things like fruits and nuts.

Kulfi: My favorite dessert from India, this concoction differs from ice cream largely in that it is not churned. It generally contains flavors indigenous to India, like cardamom or pistachios. Americans be warned: it would seem that India loves their desserts sweet, as is evidenced in pretty much every kulfi I have ever eaten. It's not too sweet for me, but it's close.

I hope that clears things up a little bit for some of you. Obviously I haven't hit every type of frozen dessert, but there are a few important ones.

I Want Proof

2010-01-19T20:48:00.003-07:00

There is a fad that's been circulating around the Internets for years now, and I'm sure is even older than that: the idea of so-called "negative calorie foods". The basic premise is that some foods require more calories to digest than they actually provide. For instance, a food that provides 5 calories, but requires 10 calories worth of energy for your body to process it, is considered to have negative calories.

It's an interesting concept, and it would be awesome if it were true. Unfortunately, I have been unable to find any concrete proof either way. We know how to determine a food's caloric content, but I wonder if we know how to determine how much energy it takes to process it? In trying to find answers online, I found several people who claimed to be knowledgeable, but who were obvious idiots, and/or didn't bother checking their facts. For instance, on this page, I found this comment:

"Celery has almost zero calories, it's so minuscule we round down to 0."

I've never heard of anyone making this claim. If we consult the USDA Standard Reference, we discover that an 8-inch stalk of celery contains about 6 calories. This is not a miniscule amount (unless compared to a Big Mac), and I certainly wouldn't round it down to 0.

I found several other references to celery containing anywhere from 5 to 20 calories per serving (though the serving size was never stated), and guesses that eating a single serving would burn anywhere from 5 to 20 calories. Even Snopes, which I lose more and more faith in every time I read anything there, claims the "negative calorie" concept is true, but offers absolutely no evidence or proof.

Can anyone tell me definitively how many calories are burned by eating a single serving (say, 40g, the approximate weight of an 8-inch stalk) of celery? I want a number, and I want to know how that number was obtained.

My next complaint involves cooking alcohol out of food. There are plenty of people that will tell you, "don't worry, the alcohol burns out". In my experience, these are people that either think you're silly for caring, or are reassuring themselves because they want it to be true. Other people will tell you that you can never burn it all out. The most outspoken of these that I've heard is Alton Brown, followed by his good buddy Ted Allen.

Both Alton and Ted have discussed this on their shows, Good Eats and Food Detectives, respectively. Food Detectives is kind of a culinary Myth Busters, but is far more scripted. They frequently perform experiments to prove or disprove myths, but in the case of the alcohol, they did a food demo that proved nothing, and then stated their "fact" as gospel.

Alton Brown has stated repeatedly that alcohol never cooks out completely, but has never offered proof. Some years ago I did some research and found a report on the USDA's website that seemed to imply that after 2 1/2 hours of oven roasting, the level of alcohol left in foods is 0% (which I'm guessing is actually < 0.5%). Unfortunately, in more recent visits, this report seems to have been removed. I have been unable to find it for years.

So, it begs the question: does alcohol really cook out, or not? Does anyone have any proof? Or can anyone at least point me to a report or study somewhere that even suggests something either way?

I'm not convinced on the negative calorie thing, or the alcohol thing. And I'm sick of people making claims with nothing to back them up. I want proof.

No More Fast Food

2010-01-16T11:12:00.002-07:00

You may have heard that McDonalds is now providing free wireless Internet access at their restaurants. For those of you who eat at McDonalds, this is great news! For myself, I haven't eaten McDonalds in several years. And recently, I decided to abandon fast food in general.

Not eating at McDonalds was an easy choice, back when I made it. I still remember my first Big Mac. I was 12 years old, and my mom gave me $5 and told me I could eat lunch whereever was within walking distance of where we were. Finally, my big chance! I could finally try that Big Mac that I'd seen so many commercials about. Sadly, it did not live up to the hype. It was "okay", but nothing special. Since then, I had never eaten at McDonalds and thought afterwards, "wow, I'm glad I ate there!" So now I don't eat there anymore.

Since then, I've run into disappointment at every fast food establishment I've ever been to. Carl's Jr has one (just one) item on their menu that I like (the Western Bacon), and I always feel like crap after eating it. Wendy's also only has only one menu item I can stand (spicy chicken sandwich), and it's not worth the sheer incompetence that they tend to hire to sell it to me. Even Subway is on my black list, with their selection of styrofoam-inspired breads.

The problems with individual restaurants are just the tip of the iceberg. Fast food is famous for its unhealthiness. Granted, most restaurants now offer healthy options, but I have a hard time paying even a dollar (sometimes two or three) for a bag of apple slices, when I could just plan ahead and bring a $0.33 apple from the grocery store with me instead.

Fast food menus are not designed for healthy eating. What's interesting to me is how many aspects of this were pioneered by McDonalds. When Ray Kroc first found McDonalds, he was surprised and impressed at the manufacturing-line techniques that were used to churn out fast, cheap burgers. Decades later, McDonalds not only began selling value meals, but actually assigned numbers to them, for convenience. And let's not forget the famed "Supersize" option which everyone copied again. While they don't ask anymore if you'd like to supersize, I'm told you can still ask for it. The most classic example is a Big Mac with large fries and a large Coke. Tasty, no? Not for me. And just between you and me, I've never been a fan of McDonald's fries either.

I think my biggest problem with fast food has been using it as a crutch. When I don't bring lunch with me to work, fast food is there to keep me from being hungry. When I forget breakfast in the morning, there are plenty of places I can stop by on the way to work. And when I'm feeling just a little too tired to make my family a delicious and healthy meal, I can always pick up a bag-o-burgers on the way home.

Why do I find myself in these situations? I think that it's been a misguided set of priorities, coupled with poor planning. I could get up early and spend a few extra minutes making pancakes to show a little love to my family, or I could stop by McLazy's on the way into work and leave my family to fend for themselves. I could make a little extra for dinner one night so that I can have leftovers to bring into work the next day, or I could buy a bucket of chicken so as not to lose precious TV time.

I'm not saying that all restaurants are bad, of course. I'm still okay with casual dining restaurants. I will still go to diners. Fine dining, when it can be afforded, is a fine thing indeed. Even delis are okay with me, in moderation. If I could afford it, I would love to take my family out to eat once every week or two. Eating out is a treat, and a way to experience new foods and keep your palate from getting board. But when any treat becomes habit, it starts to lose meaning and spoil us. I'm not okay with that.

Note: I've already been asked this once or twice, so I'd better say it here. As far as I'm concerned, carry-out or delivery pizza is also fast food. I love Pizza Hut, but I don't love the expense, or the idea of using it as a crutch. I can make my own pizza at home, which may take more time and planning, but which will taste 10 times better, and cost less than a half as much.

Ice Cream Machines

2010-01-03T16:53:00.002-07:00

I got an ice cream freezer for Christmas. It represents the latest in a growing line of equipment that I've had the opportunity to make ice cream with, each progressively better.

You see, I started with a frozen core model. This includes a container which has a special solution built into it, which must be frozen for 24 hours before use, and an electric motor. This model was barely serviceable, because it never got cold enough to effectively freeze the ice cream, and it was only good for a 30-minute session. It held somewhere around a quart.

My next model was, aside from the fact that it also had an electric motor, a little more old fashioned. You actually had to add alternating layers of ice and salt, which was surprisingly more effective than the frozen core model. I could go for as long as 40 minutes before the ice got melty enough to raise in temperature again. It could also make somewhere around 3 quarts pretty effectively. The most major drawback was the freezer full of ice that you needed to keep around. The second most major drawback was disposing of the salt water. The third biggest was the water that would condensate on the side, and then melt into a puddle around the churn.

The first model was a joke. I would never recommend a frozen core to anyone. The second model was about the same price, and while it did have its drawbacks, it at least worked. I figured it would be the model I would use until I got rich and could afford a model with a built-in freezer.

Well, that model is what I got for Christmas. It technically holds two quarts, but I have yet to get more than a quart and a half out of it. But that's not the fault of the machine itself. The motor will run until it can't run no more, and then it will stop on its own. Since the built-in freezer won't shut off with the motor, you could probably just add ice cream mixture, turn it on and go shopping, and come back home to ice cream fully ready for consumption.

This brings up an important point. The first two models make soft-serve ice cream, which must be quickly moved into containers, and into the freezer, before it is ready to be served. While you can do that with this model, my first batch was actually hard-frozen. I wasn't used to the machine yet, and I ended up letting the motor run until the ice cream was hard.

This brings up something else important. I have discovered that, with all of my practice batches so far, I need a minimum of 45 minutes to get a good churn (and sometimes longer), something that my old freezers fell short of. I have also discovered that I no longer need to use egg yolk to get a decent freeze. Before, I always used egg-based recipes, because frozen custard is easier to churn. With one exception, I have yet to use anything egg-based in this freezer.

The one exception is eggnog. It's a little late now to do this, but it's something to keep in mind for next year. Commercial eggnog is little more than spiced, unfrozen ice cream. My favorite brand for the past few years has been Southern Comfort's Vanilla Spice Eggnog. They also have a "regular" Southern Comfort Eggnog. Both are alcohol-free (you're supposed to add the Southern Comfort yourself), and I've frozen several batches of the Vanilla Spice version, both for ourselves and for family and friends. For those of you that don't like eggnog, well, I'm guessing you don't like drinking melted ice cream either. And that's fine. It's okay to be wrong sometimes.

I will give you a recipe that I've been playing with. It's not perfect yet, but it's still pretty good. And it's totally egg-free:

Strawberry Ice Cream (beta version)

1 pint half and half
1 pound frozen strawberries
1/2 cup sugar
1 pinch salt

Combine all ingredients in a sauce pan and bring to a low simmer, just long enough to dissolve the sugar and thaw the strawberries. Use an immersion blender to puree the strawberries and homogenize the mixture. Cool and refrigerate overnight before freezing, as per your ice cream freezer's instructions. Makes a little over a quart.

Amano Chocolate: Dos Rios

2009-12-23T13:43:00.003-07:00

I can't even tell you how mad I am right now at Art Pollard, at Amano Chocolate. After my glowing review of his totally excellent Guayas chocolate, he rewarded me by completely failing to mention that he had another chocolate also on the way: Dos Rios. I would have had no idea if I hadn't seen it on the shelf at Pirate-O's today.

I bought a bar, along with my sandwich for lunch, and headed back to the office. After a few bites of sandwich, I decided that I couldn't wait to try the chocolate. So I put the sandwich down and broke off a piece of chocolate. The second I put it in my mouth, I knew I had a problem. Not only did I not want to finish my sandwich, for fear of losing the flavor that was suddenly in my mouth. In fact, I don't know if I can ever eat another kind of chocolate again. I have officially been ruined.

The box that this chocolate comes in describes it as tasting like bergamot oranges, cloves and cinnamon. They're not kidding. The orange punched me in the mouth immediately, and was complimented by an amazing set of spices. I used to like those cheap chocolate oranges that you can find everywhere in America around Christmas time. They are officially crap. This trumps that any day.

There is a bitterness that you expect from dark chocolate, but it's not an unpleasant bitterness. I broke off a piece and gave it to Harleypig, and told him that he had to try it. The look on his face was classic. He finally said, "I do not like dark chocolate. But I like this." The bitterness is one of the things he mentioned. It's not the dark bitterness of overly dark chocolate, but the pleasant bitterness of an orange that isn't too sweet.

You have to try this. If you're in the south part of the Salt Lake valley, go down to Pirate-O's right now and buy a bar. If you're closer to downtown, go to Caputo's and get it. If you're too far from either, order online. This stuff is effing amazing.

Disclaimer: Although he's apparently not much of a friend right now, I do know Art Pollard. I don't believe this to have biased my review of the chocolate itself, but that's your call. And maybe if Art starts telling me about new flavors again, I'll acknowledge him as a friend again.

Update: Art called, and we talked shop until I had to go change a diaper. We're friends again.

Fun with sshd and strace

2009-12-18T14:26:00.002-07:00

I suppose this would be a much bigger concern if you could pull it off as an unprivileged user, but you do have to have root access on a server to pull this off. And really, once somebody has root, all bets are off anyway. Still, it's an interesting excercise.

In one window, log into a Linux server (RHEL 5.3 in my case) as root. In another window, use ssh to log from a remote machine into the server:


jhall@bourdain ~$ sftp guest@myserver
Connecting to myserver...
guest@myserver's password:

When it prompts for the password, hop back over to the server and run ps to figure out which process is handling the connection:


[root@myserver ~]# ps auxf | grep ssh
root   28705  0.0  0.0  60672  1184 ?  Ss  12:32  0:00 /usr/sbin/sshd
root   29361  0.0  0.0  86856  3116 ?  Ss  14:36  0:00  \_ sshd: guest [priv]
sshd   29362  0.0  0.0  62016  1384 ?  S   14:36  0:00      \_ sshd: guest [net]

What you need from this is the PID of the sshd process with [priv] next to it. In this case, 29361. Use strace to hop in and monitor this process (redirecting STDERR to a file, for later reference):


[root@myserver ~]# strace -p 29361 2> strace.log

Go back over to the remote system and type in the password. Go back to the server, cancel the strace, and then take a look at the log file. On my system, the 3rd line down had the payload:


Process 29361 attached - interrupt to quit
read(6, "\0\0\0\f", 4)                  = 4
read(6, "\v\0\0\0\7inmelet", 12)        = 12
getuid()                                = 0
open("/etc/passwd", O_RDONLY)           = 4

The text that we're looking for here is "inmelet", which is our sample password. In the clear.

Of course, this was a very manual process. But plenty of techniques exist would would allow us to monitor sshd, and launch strace automagically every time a user logged in. Of course, if you're using ssh keys, then there would be no password to see in the clear anyway. I haven't tested to see if you could steal the ssh key though. That might be a fun excercise too.

Create a Custom MDA with Postfix and Perl

2009-12-17T12:41:00.004-07:00

Wow, this was a fun one. We have an internal "project manager" that we use at work, instead of my prefered program, RT. The other day, my boss asked me to set up this program so that they could email tasks to it, instead of having to pull up the site to create a new task. The easy part was building the queue into our system. But the fun part was setting up Postfix to receive and parse the emails.

My first thought was to set up Procmail to send the messages to my parsing script. I'd never used it before, and I'd heard horror stories about writing "recipes" in it. What I had not heard was how difficult it could be to get it to play right. The mail server that I was using was not one that I had set up, and it had some weirdness about it that I wasn't familiar with. After fighting with Postfix and Procmail for a while, I managed to learn enough about Postfix configuration to realize that I might as well just skip Procmail, and write my own MDA.

Now, when I say MDA, it's a bit of a misnomer. It receives and parses emails, but rather than filtering and delivering emails to a specific mailbox, it dumps a few fields into a database. To avoid confounding the issue too much, I will try and keep this post to the bare minimum. My setup uses virtual mailboxes, but I won't go into the steps to set that up. I also won't cover the DBI code that I wrote. If I get enough requests, maybe those can go into other posts.

First things first. You need to edit the mail.cf file to set up some transports. There were two specific lines that I needed to add to mine:


virtual_transport = virtual
transport_maps = hash:/etc/postfix/transport

This allows me to only send messages sent to specific email addresses to my MDA. So the next step is to add the addresses to /etc/postfix/transport that you want forwarded to your MDA:


tasks@mytaskmanager.com mymda
projects@mytaskmanager.com mymda

Make sure to hash the file once you've edited it:


postmap /etc/postfix/transport

You've probably guessed that "mymda" is what you're going to call your MDA. This doesn't have to be the name of your script, it's just a pointer to the lines that you're about to add to your master.cf file:


mymda unix - n n - - pipe
   flags=R user=vmail argv=/usr/local/bin/mymdascript.pl USER=${user} EXTENSION=${extension}

You can see in here where we actually define the name of your script, in this case "/usr/local/bin/mymdascript.pl". Now that we're done with the Postfix configuration (remember to restart postfix for it to take effect), we can go ahead and set up that script. It's going to look something like this:


#!/usr/bin/perl

use Mail::Internet;

my @rfc2822 = <STDIN>;
my $email = Mail::Internet->new( [ @rfc2822 ] );

my $from    = $email->head->get("From");
my $date    = $email->head->get("Date");
my $subject = $email->head->get("Subject");
my $body    = $email->body();
$body       = join( '', @$body );
...snip...

This is a very, very basic script. It will receive the email from Postfix using STDIN, and to save you the trouble of parsing it out manually, I just ran it through Mail::Internet (part of the MailTools package).

Keep in mind that each line that you pull out of the message will have a newline in it, so $from, $date, $subject, etc. may need to be chomped, depending on your needs. Also, the date is hopefully in RFC2822 format, so in my case, I had to run it throught DateTime::Format::Mail to get it ready for MySQL.

Like I said, this isn't a full MDA. But it can be used for accepting things like commands, preformatted data, etc. from email and processing them, without having to deal with the overhead of something like Procmail. And if you want to use it to write a full-featured MDA, by all means feel free. And really, now that you know that the script is going to pull the message from STDIN, you're free to use C, Python, even Bash if you want.

Travel Tips: Dealing with the TSA

2009-12-14T07:36:00.002-07:00

While reading today's post at the XKCD blag, I found a reference to a comic from a couple of months ago about the TSA. I knew that the TSA had been challenged on said comic [citation needed], but I didn't know that they had responded.

In the time that I worked for Guru Labs, I spent a lot of time at airports. I learned a lot about air travel, and especially about the TSA. I thought I'd share some of the experience that I've picked up, to hopefully make some body's life a little easier.

The vast majority of TSA agents that I've had to deal with seem to be kids that got beat up in high school a lot, and are now getting their revenge. Not all TSA agents are like this, of course. There are plenty that are helpful, friendly, and generally on the ball. Most of the good ones that I have found seem to be in some kind of supervisor position, but not all. The ones that got beat up a lot in high school are the ones you need to understand.

You see, they seem to think that now that they are in a position of power and authority, that they can throw their weight around and intimidate people that remind them of high school bullies. It's not so much that they're power hungry, they're just looking for vindication. But they look in the easy places. And fortunately for them, the easy places correspond with the profiling that their job requires. The TSA may claim that they don't profile, but they'd be stupid not to. They just don't profile people the way we think they do.

Contrary to popular belief, they're not looking for turbans. They don't care what color your skin is. They're looking for (among other things) nervous people who look like they have something to hide. This is good from a security standpoint. But the high school kid in them also seems to be looking for somebody to push around. Hey, if you had to sit around all day watching X-rays, you'd get bored too.

It took a few trips, but I eventually formed the persona that I would use at airport security. Polite, respectful, occasionally friendly, but for the most part dismissive. Oh, and efficient. When you get to the metal detectors, you should already have your pockets emptied into an easily accessible part of your luggage. You should have your shoes off (or at least untied), and your laptop ready to be taken out of your bag. Everything should be ready to go on the belt, and you should be ready to pass through the detector without causing the agent any problems.

Occasionally, your luggage will contain something that the X-ray agent thinks is worth taking a look at. I'll get to some of my own stories in a moment. If this happens to you, politely and respectfully comply with the TSA agent. In my case, I add in a little bit of boredom that says, "seen it, done it, nothing new". When you start challenging the TSA agent, you risk looking like that bully that beat them up in high school a lot. And really, when it comes down to it, you knew the rules before you showed up. If you didn't bother looking them up, then you're an idiot.

Many TSA agents will just finish their job and move on. But I have seen some visibly deflated by not being able to play their game of retribution on me. The disappointment of not being able to push that high school bully back was visible and obvious. It's time for them to let you get to your gate, and they have a job to do anyway. Rules are rules, and you haven't broken any of them.

Speaking of rules, these are pretty important. Yes, a good number of the rules are asinine and should be challenged. The securitty checkpoint is not the place to challenge these. The agents have NO AUTHORITY to change the rules. The rules aren't meant to be flexible, they're meant to be followed. Yes, they realize that your 5 oz tube of lotion may obviously contain only an ounce or two. But the rule isn't "a container with 3.4 oz or less", it's "a container than can hold no more than 3.4 oz". If you really want to bring that lotion with you, find it a smaller container. Most grocery and drug stores in my area have a travel section of their pharmacy that can help you out.

So follow the rules while they are in place, and if you want to get them changed, write your congressperson or something. Treat the TSA employees with the same amount of respect and politeness that you should treat anybody else with, and they will generally do the same for you. And if they don't, you have the right to demand to talk to their supervisor. Personally, I've never had to do that. They tend to leave me alone. But I have had some small encounters.

Salami, Salami, Balony

When I grew up, I remember my dad being a big fan of dry salami. He'd frequently buy these tubes of it covered in some white stuff (mold, as I later discovered) that he would peel off before slicing off a few pieces. This style of salami has being increasingly difficult to find in Utah. One day in the San Jose area, I found a big ol' stick of the stuff, and bought it to bring home to him. Little did I realize that to the X-ray, it probably looked like a club of some sort, strictly forbidden by TSA rules.

Of course I was stopped at the X-ray. It was in my carry-on luggage because I don't check my luggage (there's only two types of luggage: carry-on and lost). My bag was full of clothes and computer books, but the salami was on top. The TSA agent had already told me I could put on my shoes, so I did so while almost completely ignoring him looking through my bag. Occasionally I would glance over, and I saw that he found the salami almost immediately, and gave a look that said "that's probably what they were worried about". After a layer or two of books, and the realization that I was just letting him do his job and not even remotely worried about what he might find, he gave up, without even making it halfway through my bag. He looked a little deflated. He gave my bag back and wished me a good flight.

"But It's Just Candy"

One day in Phoenix, I bought a tube of marzipan. I didn't get a change to open it beforehand, so I just stuffed in in my bag. Of course the X-rays identified it as a paste (almond paste, to be exact), so they pulled it out. I told them, "but it's just almond candy", but it fell on deaf ears, and I didn't want to push the issue.

I was told that I had three options: I could check my bag with the airline, or I could walk down to the airport post office and ship it home. With either of these options, I would get a TSA escort, so that I wouldn't have to wait in line when I came back. My last option was to throw it away, which is what I did. The TSA agent was confused, and reiterated that with either of the other options, I could keep my marzipan. My reasoning was this: I was flying on Skywest, and they are the principle reason why I don't check bags. And mailing my marzipan back home would have cost more than just buying another tube when I got home. The TSA agent was perplexed, but he let me chuck the marzipan and gave me no more trouble.

Darts

I have a travel kit of tech tools that I never leave home without. I keep them in a bag in my trunk, and when I fly, I take a TSA-safe version with me. One of the tools I used to have was a mini-screwdriver, with removable bits. I had taken it with me on dozens of trips with no incident. But one day I arrived at security while it was particularly slow (about 3 or 4 agents per passenger). A bored agent with way too much time on his hands saw my screwdrivers (I had two of them with me) and thought they looked like darts. And while the pocket that I kept them in was easy to access and obvious to me, it took the agent several minutes to find them. And as per TSA rules, I wasn't able to help. Once he found them, he realized his mistake, told me he thought they were darts, and allowed me to move on.

Just Chocolate

When I was in Montreal, I found some excellent French chocolate, and I bought plenty of it. On the way back, going through customs, I declared that I was bringing food over the border with me. When I went through security, I was stopped at the X-ray. I was asked if I was bringing back anything with me that I didn't bring with me. I said, "yes, chocolate". They asked if that was the food that I declared, and I responded affirmatively. As they inspected my bag, chocolate kept falling out of various pockets. Towards the end of the screening, they were more amused than anything. I was sent back to America with a smile.

Cinnamin Craisin Muffins

2009-12-13T07:48:00.002-07:00

Okay, so I have a thing for silicone baking molds. And I know the shapes weren't quite what I wanted. So sue me.

Baking molds aside, these muffins are the perfect way to start off a cold, wintery day. As with most muffins, it takes longer for the oven to preheat before than it takes to mix everything together, so set your oven to 375F a good 10 minutes before you start mixing.

1 1/2 cups flour
1 1/2 tsp baking powder
1/2 tsp salt
1/2 tsp nutmeg
1 Tbsp cinnamon
1/2 cup melted butter
1 cup packed brown sugar
1 whole chicken egg
1/2 cup milk
2 oz craisins

Being muffins, we use the muffin method: whisk together the dry stuff (flour, baking powder, salt, nutmeg and cinnamon) in one bowl, whisk together the wet stuff (melted butter, brown sugar, egg and milk) in another bowl, then combine and mix together with a spatula (trust me it's easier to combine wet and dry with that than with a whisk). Fold in the craisins, pour into prepared muffin tins, and bake for 20 to 25 minutes at 375F.

On a diet? Or maybe just looking for a way to add a little extra flavor? I'm told that with the muffin method, you can swap out the liquid fat with apple sauce, cup for cup. I didn't try it with this recipe, but I've done it before and it's worked well. And you can still spread on butter after it bakes, so don't worry about losing that goodness.

Android Kitchen Timer: SupaCount

2009-12-11T10:15:00.000-07:00

I've been feeling handicapped when I cook, and especially when I bake. When I used to bake professionally, we used the oven timers to know when to check on our cookies. It wasn't long before my internal timer was telling me 30 seconds before the oven timer went off, that I needed to check on the cookies. But that didn't help with cakes, pies, breads, pastries, you get the idea. One of our bakeries had multiple digital timers, to help out with production.

At home, my oven doesn't have a timer. My microwave does, but that's also the only working clock in the kitchen, and even on that floor of that house. And if I need to nuke something while something else is in the oven, well that's a problem. And I occasionally have enough going that two ore more timers would really come in handy.

When I got my G1, one of the first types of apps that I looked for was a kitchen timer. I found one (and only one) that didn't complete suck; it only half sucked. It could time for more than 90 minutes, immediately putting it ahead of all the other timers (my microwave timer only goes to 99 minutes, not nearly long enough for proofing bread dough). This particular timer had a pretty graphic of a kitchen timer, and rather than typing in numbers, you had to use the touch screen to rotate it. It was clunky, and difficult to get the exact time that I wanted. And when the timer went off... well, I don't know what the crappy music was that started playing, but I hated it and you couldn't change it.

That app was removed from my phone this very morning, when I came across a timer to end all timers. It's not complex, it's not pretty, but it does the one thing that it needs to do: it works. I can set as many timers as I want, I can type in exactly the time I want (down to the second), and I can change the alarm sound.

This app isn't available on the Android Market. You just need to head over to synic's site and download it. Kudos to synic for putting together such an awesome app. This is one of the few that I will be adding to my desktop on my phone.

Periodic Tables of Food

2009-12-07T06:00:00.002-07:00

A couple of days ago, I stumbled upon a poster at AllPosters of a Periodic Table of Vegetables. It's an interesting concept, to be sure. The Periodic Table of the Elements is a mapping of a particular type of data, organized by groups (columns) and periods (rows). Why not use the same style as a visual representation of something tastier?

Unfortunately, the image is a little too small to make out most of the veggies clearly, but it got me wondering what other kinds of periodic tables of food exist. The search was, and still is, on. I found several interesting tables of food, a couple of which I had even seen before. I also found non-food tables, my favorite being of game controllers. Here's what I've found so far:

The table that I've found myself looking at, and wanting to hang up on my wall the most, is the Periodic Table of Dessert. It breaks down its categorization into separate ingredients, which is important, because that's pretty much what the elemental table does. It makes an excellent effort to categorize things clearly, and assigns one- or two-letter symbols to each item. However, it does contain far fewer columns than the original, and uses symbols that I don't necessarily agree with (why P for peanut butter, instead of PB? How does M signify honey?). I would love to send this table through another revision. On the bright side, it is accompanied by a thermal spectrum (which doesn't make a whole lot of sense to me) and what appears to be the crystaline structures for several compounds (which is just awesome). These are all available together as a single poster.

Next up is the Table of Condiments that Periodically Go Bad. Unlike the dessert table, this table is numbered. Unfortunately, the numbers really only make sense paradoically (is that a word?). Again, elements are given symbols, and any that crossover to the dessert table actually seem to match. This kills me. I don't think that salt should be S, I think it should be Sl or Sa. Ideally, we would break into a three-letter designation, and use Sal. <End Rant> The most important part of this table is the designation for each condiment of how long you have before it goes bad. Very nice, in terms of food safety.

The Periodic Table of Produce is similar, except that it feels much more serious to me. Really, it's a table of food storage of fresh produce, including storage suggestions and timelines to when a particular veg will go bad. I would love to find a higher-quality version of this, and put it on my refrigerator door.

Growing more complex, we have a Periodic Table of Cheeses, complete with full merchandizing on t-shirts, mugs, mouse pads and, of course, posters. According to the site, this table was created by the blind Russian cook Anatoli Grigor Konchalovsky, apparently in 1865. I don't know how true that is, but if it is, that probably makes it the first periodic parody of food. Some thought has clearly gone into its organization, but I'm not entirely sure yet what each color means, or how some of the groupings fit. I love the "Noble Cheeses" classification, though.

I found the Periodic Breakfast Table interesting, though I haven't yet found a copy that looks to be complete. Offhand, it seems to be sorted visually, rather than by type of grain, manufacturer, history, etc. It does have some of this printed with each cereal, but it doesn't seem to be sorted that way.

Going back to deserts, there was a Periodic Table of Cupcakes posted in Women's Day earlier this year. There's a part of me that is impressed, because I never would have expected any mainstream periodical (other than the venerable Cook's Illustrated) to expect their readers to enjoy a scientific nod like this. But then another part of me looks at the actual cupcakes printed, and would be entirely confused by the majority of them if it didn't know that really it's probably just a marketing gimmick for their own recipes. Still, it's cool.

The Periodic Table of Candy looks to be entirely parody, listing commercial candy varieties, numbered, and in alphabetical order. I haven't decided yet whether the little girl at the top is cute or frightening.

Our journey is almost over. Back at AllPosters, I also came across a poster of the Periodic Table of Sandwichry. There is no way I can make out anything sensible on this, but as one of the cheaper posted presented, I might be willing to order it along with something else.

To finish up, I present to you the Periodic Tables of Beer Styles and of Mixology. Add these to the category of "Text to small to read, so no clue as to any useful information, including accuracy." Still, it's a nice thought.

T-Shirts For Sale!

2009-12-01T10:13:00.002-07:00

Some of you may recall that when I did my Object Oriented Cooking presentation at the 2009 Utah Open Source Conference, I was wearing a t-shirt with an 8-bit stand mixer on it, instead of one of the bowling shirts that I usually wear. I actually designed and had that shirt made about a year ago. The idea was always to put up a few designs for sale, but I never got around to it; mostly because I only ever came up with one other design that I was even remotely happy with.

Well, a couple of nights ago, I got an idea for another design. It was based on the source code that I used in my presentation for a PB&J sandwich, written in Perl. I drew up the 8-bit graphics, added the source code to the back of the shirt, and after a couple of revisions, posted it for sale.

I now have three shirt designs for sale, in my Spreadshirt store. All feature 8-bit graphics depicting various food-related items. We have the stand mixer that I wore at the conference, a big-ol' jug of moonshine, and of source the PB&J in Perl. And just in time for Christmas too!

So if you want to show your geek side and your food side all at once, head over to my Spreadshirt store and grab a t-shirt. Or direct your friends and/or family members in that direction! I'll post more shirts as ideas come to me, but at least now we have an appetizer to get everyone started.

Mitigating the Evil Maid Attack

2009-10-23T09:27:00.002-06:00

The security world is buzzing with news of the so-called "evil maid" attack. The basic warning is this: no matter how secure you think you are, you aren't. Full-disk encryption is now provably breakable, and without actually having to break the encryption itself. All you need to do is get ahold of a computer that has been shut down, boot to your own boot device, screw with the boot loader, and shut down again... and then come back after the computer has been turned on and logged into by the real user.

This attack has been called the evil maid attack because a hotel maid is in a perfect position to accomplish this. They have access to the room when you're not around, and leaving a "do not disturb" sign on your door is hardly a deterrent. If you stay at a hotel multiple nights in a row, and you leave your computer in your room while you're not around, you are leaving yourself at risk. You may be the subject of a targetted attack, where the attacker poses as a maid or otherwise gains access to your hotel room while you're out. Or an attacker may get an actual job as a maid at a high-profile hotel, where they know that plenty of secrets will always be passing their way, waiting to be stolen.

There are measures you can take to try and prevent this attack from occuring in the first place. Think of them in terms of both intrusion prevention, and intrusion detection. For instance, a BIOS password can be difficult at best to compromise on a laptop. This is an aspect of intrusion prevention. Of course, it is still possible to reset the BIOS password on a laptop, which will then give the attacker access to install the attack MBR. But if you turn on your computer and there is suddenly no password, when you previously had one, then you know that something is amiss. This is a form of intrusion detection.

You could always use a thumb drive to boot your machine, which would make an evil maid attack against your actual hard drive completely worthless. This introduces another aspect of computer security, which security experts will always disparage, called "security through obscurity". If the attacker doesn't know that you use this method, then they won't be able to attack against it. The problem with this method is, now you have to protect both your boot key, and your laptop. If you only use one thumb drive to boot your machine, and that drive somehow gets damaged (water, static electricity, EMF), then you lose the ability to use your computer.

You could change your password on every boot. Using traditional methods, this is is tedius, and inconvenient, and ignores the fact that the modified boot record might be prepared for it anyway. You could use a password dongle, but that suffers from the same limitations as the thumb drive.

You could leave your computer on, of course. If somebody shuts it down and messes with your boot record, then you'll at least be able to detect that. But then your computer is also susceptable to a cold boot attack, which doesn't require the attacker to return later, so that's out. Using a fingerprint scanner for authentication? Your fingerprints are probably everywhere on your computer already, and even the Mythbusters were able to fool these using little more than a photocopy.

So it would seem that leaving your computer off while you're not in the room is safer than leaving it on. Using alternative boot methods helps, but cannot completely prevent. Using a combination of methods is best, and while it may not provide 100% protection, it can slow down the attacker, and that may be enough.

Anybody else have any other methods that I missed?

Whiskerino 2009

2009-10-21T19:45:00.000-06:00

"He that hath a beard is more than a youth, and he that hath no beard is less than a man." - William Shakespeare

The great Whiskerino 2009 is nigh at hand. In short, it is a beard-growing contest. The basic idea is, on November 1st, you shave. Everything comes off. And then you don't shave again until the end of February.

I have a lot of friends who have never seen me without a beard. When I met my wife, I had a goatee, and by the time our first child was born, I was sporting a full beard. Neither has ever seen me without facial hair. I'm a little worried my daughter won't recognize me.

I'm not that worried though. We had a dress code at cooking school that stated that you could have a beard or no beard, but you could not be in the stages of growing a beard. I knew that if we had a three-day weekend, I would have enough time to grow a long enough beard to be within the requirements of the dress code.

Of course I'm planning to enter the Whiskerino. I have a few friends that plan to as well. One rule is, you must post photos of your beard at least every 7 days, and more often towards the end. Never before has my beard been so well documented. I plan to post photos on my blog as well. For those who only read it via RSS, you're out of luck. I will be posting the photos on the side bar on my site.

If anybody out there has been contemplating growing a beard but waiting for the right time, the right time is now. Well, in a week and a half. I issue a challenge to all of my geek friends out there with the ability to grow facial hair. Even Shakespeare knew the importance of a Unix beard. Now is your beard's time to shine!