blog.josephhall.com

Replacing AWStats

2009-05-30T22:06:00.001-06:00

Don't get me wrong; I've been a big fan of AWStats for years. It's great for giving me a high-level overview of how my sites are doing, traffic-wise. The day-by-day breakdown is nice, and if I don't mind looking at the big picture for my site a month at a time (which is usually good), then it's perfect. When I started working on a more complex setup than just a few low-traffic blogs, then it started to show its limitations. Let me tell you what I mean.

In our current setup at work (or at least the one that we've been moving toward), we have three web servers. Each server is behind a load balancer, and serves exactly the same sites. When I started monitoring, we had 200+ sites active. We wanted to have stats for each domain on each server, plus the stats for each domain across all servers, plus the stats for all domains on each individual server.

AWStats requires one separate config file per domain, per server, plus another separate config file per domain for all servers, plus a separate config for the server-wide stats. That means we're looking at 800+ separate config files. I wrote a script to automatically generate all of these for me, but they're still a pain to keep track of.

Before I parse out the log files, I need to combine multiple log files into massive composite log files (domain-wide and server-wide), in chronological order so that AWStats doesn't choke. As you can imagine, this requires a considerable amount of resources.

AWStats also maintains a flat data file per config file per month, all in the same directory. Assuming we keep just a year's worth of data, we're looking at 9600+ separate files in that directory. At this point, one wonders why we can't just store everything in a database.

Now that you've seen the kind of management nightmare that I have to deal with, with just the features that AWStats does have, I think you can see where my frustrations begin. But I can't be content just being unhappy with existing features; I want new features too. And anybody that's ever looked at the main awstats.pl file knows how much of a beast it is to figure out just what's going on, much less change anything.

I ended up adding some glue of my own to make things a little easier. Remember, not only do I have hundreds of config files and thousands of data files, I also have to pull them up in the first place. As it turns out, the script that I wrote to automate building config files also builds an HTML file with links to all of the AWStats pages in it. I even added a quick ping script to periodically hit each domain, and place a colored dot next to the domain name to indicate its active status (green is good, red is bad, blue and yellow are proprietary indicators that a site responded, but not in a normal manner). I even have other indicators set up to tell me things like whether a site has an SSL cert, today's hits so far, yesterday's hits, green up arrows to tell me if today's hits are higher then yesterdays, red down arrows if they are lower, yellow right arrows if the traffic is the same, you get the idea.

Maybe I'm just selfish, but I want a stats program that can handle all of this gracefully, and for not a lot of money. Free (as in freedom, and beer) is ideal. But since I ultimately decided to write my own, it certainly wasn't free as in time. But it was fun, and I learned a lot about AWStats while I was at it.

Incidentally, it turns out AWStats does support clusters kind of like our load balancer setup, but I didn't find that out until I spent a lot of time in the various parts of it. And I haven't taken the time to figure out how they do it; I already had a solution in mind anyway.

I hope this post gives you an idea of why I started thinking about replacing the great AWStats, I program which I still love and respect. If you're interested in looking at my code so far, I have packed it up for public consumption. Information about its operation and shortcomings are in the README.

Clicky!

Not So Inspirational Thoughts

2009-05-22T08:53:00.002-06:00

I believe I've mentioned before that my primary responsibilities at work are as the systems administrator. It's something that Ive dabbled in with several previous companies, and taught a lot of classes on at my last job. Now I do it full time.

I was talking to my brother about it some months ago, explaining a few of the concepts of server security that I've either learned from others before me, or have picked up on my own. We might even have talked a little bit about social engineering and corporate security. After a while, my brother said to me, "so basically, good security is a fascist regime."

I thought about it for a moment, and said, "yeah, I guess it is."

Yesterday I mentioned this conversation to my boss. He laughed and said, "yeah, that makes perfect sense. In fact, any mechanical process that is a democracy is a failure."

Hope you all enjoy those thoughts as we head into the Memorial Day weekend. Me, I've got a few company servers to enforce martial law on.

Soundex

2009-05-08T12:19:00.003-06:00

Today at work we were talking about spelling issues that have come up with phone reps, while trying to look up customer names in our database. For instance, it might sound like the customer is saying her name is Cassy, when in fact she spells it Cassie or Cassi. I used to know a guy who spelled his name Eron (instead of the more traditional Aaron).

How does one deal with this problem? Some of the solutions that were coming up were pretty scary-sounding, and started to drift into areas of serious security concerns. I finally spoke up and said something to the effect of, "if I could offer a suggestion, how do you all feel about looking up records by soundex?" I suddenly had three very interested faces looking in my direction.

Soundex is something that any serious genealogist is well familiar with, but for some reason it doesn't seem to be that common in the programming world. That's a real shame, because it's so useful. And fortunately for Perl and MySQL developers, tools already exist for you.

First, the basics:

Take a word, and write down the first letter of that word.

Drop all of the vowels remaining in the word.

Remove all duplicate letters (i.e. LL becomes L).

If "H" or "W" separate two letters with the same soundex code, the consonant to the right is ignored.

Convert the remaining consonants to numbers:

b, f, p, v => 1

c, g, j, k, q, s, x, z => 2

d, t => 3

l => 4

m, n => 5

r => 6

Save only the first three numbers. If you run out before you reach three digits, pad with zeros.

Using this code, the name "Joseph" would be encoded as "J100". "McAllister" would be "M242". See how easy that is?

MySQL has a couple of built-in functions that utilize soundex. The first, amazingly enough, is the "SOUNDEX()" function. This function doesn't give a standard soundex value; it actually encodes the whole word, giving you a minimum of 4 characters, but an arbitrary maximum. For instance:


mysql> select soundex('mcallister');
+-----------------------+
| soundex('mcallister') |
+-----------------------+
| M24236                | 
+-----------------------+
1 row in set (0.00 sec)

If you want it to return a standard soundex with a maximum of four characters, you can use the "SUBSTRING()" function:


mysql> select substring(soundex('mcallister'),1,4);
+--------------------------------------+
| substring(soundex('mcallister'),1,4) |
+--------------------------------------+
| M242                                 | 
+--------------------------------------+
1 row in set (0.00 sec)

MySQL also has a built-in function called "SOUNDS LIKE" that actually performs a SOUNDEX() function in the background:


mysql> select distinct first_name from customer where first_name sounds like 'corey' limit 10;
+------------+
| first_name |
+------------+
| Corey      | 
| cherie     | 
| Cory       | 
| Carrie     | 
| corri      | 
| cheri      | 
| cesar      | 
| CHERRY     | 
| Cierra     | 
| cora       | 
+------------+
10 rows in set (0.00 sec)

On one hand, we got a few names that most definitely weren't "corey". But on the other hand, the query was inherently case-insenstive, and offered a lot of wiggle room.

Perl also has a couple of Soundex modules available in CPAN, such as Text::Soundex. This module is also pretty easy to use:


use Text::Soundex;
print soundex("Ashcraft"), "\n";       # prints: A226
print soundex_nara("Ashcraft"), "\n";  # prints: A261

That second function, "soundex_nara()", is pretty important for doing US Census work, since the National Archives and Records Administration (NARA) uses a slightly different encoding scheme. Theirs is probably the version that most genealogists are going to be familiar with.

This is something that's definitely handy for looking up a lot of data, when the actual spelling is a little fuzzy. Maybe it will find its way into your database queries at some point now.

The Other Joseph Halls

2009-04-22T07:43:00.000-06:00

I thought I'd take a moment to tell you about some of the other Joseph Halls. As it turns out, it's kind of a common name. When I was a kid, there were four Joseph Halls in the Salt Lake phone book. I don't know how many there are now, since I haven't even opened a phone book for years, but I can tell you that none of them are me (unlisted).

When I was a newish Perl hacker, I came across a book called Effective Perl Programming, by Joseph N Hall. It felt odd to have another one of me in my community, especially since the other one was published, and his co-author was the great Randal Schwartz. I ended up buying his book, mostly because I liked the reviews. The fact that I could keep a Perl book with my name on the cover at my desk was a bonus. As it turns out, it's a pretty good book. If you're into Perl and want to get better, I highly recommend it.

The next Joseph Hall that I would run into was Joseph Lorenzo Hall. You see, around the same time I found Joseph N Hall's book, I ended up buying josephhall.com, josephhall.net and josephhall.org. I did nothing with them for years, and eventually let the .net and .org expire. Not long after, I decided to check up on them to see if they were still available. The .org was not. But I couldn't help find the new owner of josephhall.org fascinating. He's yet another smarter me, with a PhD. I've been following his blog in my feed reader for a while, and yesterday's post was truly interesting to me. He's a Neil Gaiman fan too! I also know from his site and his blog that he's a computer geek and even a bit of a food geek. You should check out his blog just on principle.

That of course leaves josephhall.net. A few months ago I got a call from a man wanting to buy josephhall.com from me. I told him that it was not for sale. He told me that it was for his son, another Joseph Hall, and that it didn't look like I was even using www.josephhall.com. I told him that it was because I had that redirecting to blog.josephhall.com. He looked, and found that I was indeed actively using my site. He eventually realized that it was futile to try and buy my domain, and I wished him luck in finding another solution. As I spoke with him, I ran a whois on josephhall.net and found it avaialable. Figuring he might buy that, I checked an hour later and found that his name was indeed listed. Some weeks later, I visited the site and found out why this new Joseph Hall needed a site: he's apparently an up and coming Elvis impersonator. There's a part of me that thinks that's just awesome. So go check him out as well.

Mirepoix

2009-04-07T21:42:00.002-06:00

One of the most important components of classical French cooking is mirepoix. A classic French mirepoix is composed of a 2:1:1 ratio of onions to carrots to celery, meaning that four cups of mirepoix would contain two cups of onions, one cup of carrots and one cup of celery. Some of you may also have heard of the cajun version of mirepoix, called trinity. The ratio is the same, but the carrots have been replaced with green bell peppers.

Mirepoix is used as the basis of a number of dishes, not only in French cooking, but in other cuisines. The ratios may vary a little, and the name may change (or not even exist at all), but the combination of onions, carrots and celery is classic. Recently some stores have even started selling both mirepoix and "cajun-style mirepoix" in the frozen foods aisles, which is convenient. But if you're willing to take the time to cut your own mirepoix and maybe even freeze it yourself, it will likely be cheaper, plus you can have a little better control over the quality.

I'm going to show you a few knife cuts here suitable for home use. You've probably seen all sorts of ways to cut these vegetables, and if you've been to cooking school, these methods are probably not what you were taught. They certainly weren't what I was taught, and if I were working for somebody like Thomas Keller, I wouldn't be using them either. But since I'm just cooking for me and my family, they will suffice.

Let's start with carrots. Go ahead and wash them, but don't bother to peel them. There's no reason to waste good carrot like that. With most carrots I see in grocery stores, you should be able to chop them in half, then cut the smaller half lengthwise into four pieces, and the larger half lengthwise into six pieces.

Go ahead and cut up a few carrots like this. When you have enough, there's a tip you can use to save yourself some time. Spread them out in a line across your cutting board, and start working your way across with your knife, dicing them up into pieces with as uniform a size as you can manage. Remember, accuracy is more important than speed, especially considering the sharp knife that I expect you to be working with.

Let's turn our attention to the celery. This is actually a little more difficult to cut, because it has kind of an awkward shape to it. The key is to bring it down to the same level as the carrots. First, grab a bunch of celery and cut the end off. Don't go crazy with it, you just want to be able to separate the stalks from each other. Don't be getting all wasteful on me.

When it's separated, go ahead and wash it in cold water. There were plenty of places for dirt to hide when it was together, and you need to get rid of it. With that out of the way, go ahead and lay a stalk down and slice off a piece of it length-wise.

With that piece cut away, go ahead and make the next cut. Be careful with these! It's easy to get carried away and go too fast. I've lost track of the number of times that I started to think I was a big-shot chef with ninja knife skills, only to end up slicing myself. Remember, accuracy before speed.

Now you have a bunch of long sticks of celery, ready to be diced just like the carrots. Lay 'em out the same way, and chop 'em up.

Last up, onions. These are actually very easy to cut, but they have their own dangers. You know what I'm talking about: the tears. I can give you a few tips, but when it comes down to it, you really just need to learn to suck it up. Take a step back if you start crying too hard, give it a break, then go back to it. First things first, you need to cut the stem end off.

Be careful not to cut too much of it off, because you actually need some of it to hold the onion together for a moment. Cut the flower end off too. Then go ahead and slice it in half, from end to end, so that you have two halves that are both held together at the stem.

At this point you can go ahead and peel away the papery part and throw it away. You might even want to put the cut sides face-down on the cutting board for a few minutes. Believe it or not, this seems to help dissipate some of the sting. When you're ready, slice the onion on the side, but not all the way to the stem. You still need that holding it all together.

Keep making slices like this at an angle, working your way around the onion. What you're going for are little fingers of onions held together by the stem. Another little tip here: try not to stand directly over the onions when you're cutting them. I think everybody does it, and it just puts you in the line of fire for onion fumes. Stand back a little and you will have fewer tears.

When you've worked your way around, it's time to actually dice the onion. Being careful not to cut your fingers, you want to slice across the cuts that you've already made. This is much easier, because that stem is still holding the onion together.

Keep going until you get almost to the stem. Feel free to cut as close to the stem as you want, but don't cut up the stem itself, I don't think you'd really like it.

Now that you have your vegetables cut up, you have a few options. If you bought way too many vegetables like I did when I was shooting these pictures, you'll probably want to freeze some of them for later use. You'll want some cookie sheets, or if you have a restaurant supply store nearby you can pick up "half-sheet pans", which are basically just commercial-grade cookie sheets.

I like to lay down a sheet of parchment on each sheet pan. You don't have to, but keep this in mind: the frozen veggies will stick to the pan itself a lot easier than to a sheet of parchment. Lay out the veggies on the pan, in as thin a layer as you can get it. If you have a lot of veggies like I did, you can put a cooling rack on top of the veggies, add another pan and repeat, but I wouldn't go more than three sheet pans high. The more you stack, the longer it will take to freeze, and you really do want these veggies to freeze as quickly as possible.

Move the sheet pans into the freezer and leave them there for at least two or three hours. Try not to open the door for the first couple of hours; it just slows down the freezing process. When they're nice and cold, you can move the veggies into a resealable plastic bin or bag, and then directly back into the freezer (once you've written the day's date on them, of course).

It's up to you whether you want to store them separately, or mix together the ratios yourself for freezer storage. Personally, I like to keep them separate. It offers me a little more freedom if I end up needing, say, a cup of carrot puree. I can't think of any reason offhand why I would need such a thing, but stranger things have happened.

I have another post written that makes excellent use of this mirepoix, but I have no photos to accompany it. Stay tuned, and as soon as I have photos I will post them.

Geek/Maker Calendar

2009-04-01T21:43:00.003-06:00

I don't know if anybody else has noticed, but Make seems to be posting a lot of "Maker Birthdays" lately. I thought it might be fun to put together a calendar with as many days as possible filled with not only birthdays of makers, but also birthdays of other notable geeks, and various other noteworthy days for geeks. I know I'm stretching it on a few of these, but hopefully not too far.

My list so far is not comprehensive, but I'm running low on ideas. I thought I'd open it to my geekier readers, to see if we can fill the comments on this post with other geek days. I have Wikipedia links where available, and links elsewhere otherwise. I'm trying to remain as unbiased as possible, which is why you see certain names on the list that I'm not necessarily a fan of. Please try to be unbiased too. Here's what I have so far:


Jan     1864: George Washington Carver - http://en.wikipedia.org/wiki/George_Washington_Carver
Jan     1942: Brian Kernighan - http://en.wikipedia.org/wiki/Brian_Kernighan
Jan  1, 1894: Satyendra Nath Bose - http://en.wikipedia.org/wiki/Bose-Einstein_condensation
Jan  2, 1920: Isaac Asimov - http://en.wikipedia.org/wiki/Isaac_Asimov
Jan  3, 1892: J.R.R. Tolkien - http://en.wikipedia.org/wiki/J._R._R._Tolkien
Jan  4, 1643: Isaac Newton - http://en.wikipedia.org/wiki/Isaac_Newton
Jan  4, 1809: Louis Braille - http://en.wikipedia.org/wiki/Louis_Braille
Jan  5, 1940: FM Radio first demonstrated to the FCC - http://en.wikipedia.org/wiki/Jan_5
Jan  8, 1942: Stephen Hawking - http://en.wikipedia.org/wiki/Stephen_hawking
Jan 21, 1953: Paul Allen - http://en.wikipedia.org/wiki/Paul_Allen
Feb  4, 1943: Ken Thompson - http://en.wikipedia.org/wiki/Ken_Thompson
Feb  5, 1943: Nolan Bushnell - http://en.wikipedia.org/wiki/Nolan_Bushnell
Feb  8, 1828: Jules Verne - http://en.wikipedia.org/wiki/Jules_Verne
Feb 11, 1847: Thomas Edison - http://en.wikipedia.org/wiki/Thomas_edison
Feb 18, 1745: Alessandro Volta - http://en.wikipedia.org/wiki/Alessandro_Volta
Feb 24, 1955: Steve Jobs - http://en.wikipedia.org/wiki/Steve_jobs
Mar  3, 1847: Alexander Graham Bell - http://en.wikipedia.org/wiki/Alexander_Graham_Bell
Mar 11, 1952: Douglas Adams - http://en.wikipedia.org/wiki/Douglas_Adams
Mar 13, 1733: Joseph Priestley - http://en.wikipedia.org/wiki/Joseph_Priestley
Mar 14, 1879: Albert Einstein - http://en.wikipedia.org/wiki/Albert_Einstein
Mar 14      : Pi Day
Mar 16, 1953: Richard Stallman - http://en.wikipedia.org/wiki/Richard_Stallman
Mar 17, 1948: William Gibson - http://en.wikipedia.org/wiki/William_Gibson
Mar 24      : Ada Lovelace Day - http://blog.makezine.com/archive/2009/03/today_is_ada_lovelace_day_celebrati.html
Mar 31, 1811: Robert Bunsen - http://en.wikipedia.org/wiki/Robert_Bunsen
Apr 16, 1867: Wilbur Wright - http://en.wikipedia.org/wiki/Wright_brothers
May 11, 1918: Richard Feynman - http://en.wikipedia.org/wiki/Richard_Feynman
Jun  6, 1954: Tim O'Reilly - http://en.wikipedia.org/wiki/Tim_O%27Reilly
Jun  8, 1955: Tim Berners-Lee - http://en.wikipedia.org/wiki/Tim_Berners-Lee
Jun 19, 1623: Blaise Pascal - http://en.wikipedia.org/wiki/Blaise_Pascal
Jun 23, 1912: Alan Turing - http://en.wikipedia.org/wiki/Alan_turing
Jun 26, 1824: William Thomson, 1st Baron Kelvin - http://en.wikipedia.org/wiki/William_Thomson,_1st_Baron_Kelvin
Jul 10, 1856: Nikola Tesla - http://en.wikipedia.org/wiki/Nikola_Tesla
Jul 17, 1971: Cory Doctorow - http://en.wikipedia.org/wiki/Cory_Doctorow
Jul 20, 1969: Apollo 11 lands on the moon
Jul 30, 1863: Henry Ford - http://en.wikipedia.org/wiki/Henry_ford
Jul 30, 1962: Alton Brown - http://en.wikipedia.org/wiki/Alton_brown
Aug  8, 1901: Ernest Lawrence - http://en.wikipedia.org/wiki/E._O._Lawrence
Aug 11, 1950: Steve Wozniak - http://en.wikipedia.org/wiki/Steve_Wozniak
Aug 12, 1887: Erwin Schrodinger - http://en.wikipedia.org/wiki/Erwin_Schr%C3%B6dinger
Aug 19, 1871: Orville Wright - http://en.wikipedia.org/wiki/Wright_brothers
Aug 19, 1906: Philo Farnsworth - http://en.wikipedia.org/wiki/Philo_Farnsworth
Aug 20, 1970: John Carmack - http://en.wikipedia.org/wiki/John_Carmack
Sep  9, 1941: Dennis Ritchie - http://en.wikipedia.org/wiki/Dennis_Ritchie
Sep 24, 1954: Larry Wall - http://en.wikipedia.org/wiki/Larry_Wall
Oct  7, 1885: Niels Bohr - http://en.wikipedia.org/wiki/Niels_Bohr
Oct 17, 1984: Randall Munroe - http://en.wikipedia.org/wiki/Randall_Munroe
Oct 28, 1955: Bill Gates - http://en.wikipedia.org/wiki/Bill_gates
Oct 31, 1959: Neal Stephenson - http://en.wikipedia.org/wiki/Neal_Stephenson
Nov  7, 1867: Marie Curie - http://en.wikipedia.org/wiki/Marie_Curie
Nov  8, 1954: Bill Joy - http://en.wikipedia.org/wiki/Bill_Joy
Dec  9, 1906: Grace Hooper - http://en.wikipedia.org/wiki/Grace_Hopper
Dec 10, 1815: Ada Lovelace - http://en.wikipedia.org/wiki/Ada_lovelace
Dec 24, 1818: James Prescott Joule - http://en.wikipedia.org/wiki/James_Prescott_Joule
Dec 26, 1791: Charles Babbage - http://en.wikipedia.org/wiki/Charles_Babbage
Dec 26, 1937: John Conway - http://en.wikipedia.org/wiki/John_H._Conway
Dec 27, 1822: Louis Pasteur - http://en.wikipedia.org/wiki/Louis_Pasteur
Dec 28, 1969: Linus Torvalds - http://en.wikipedia.org/wiki/Linus_Torvalds
Dec 29, 1800: Charles Goodyear - http://en.wikipedia.org/wiki/Charles_Goodyear

I have a few other names that I've come up with that either I don't have birthdays for, or am not sure if they warrant being on the list.


E.G. Kingsford (Kingsford Charcoal)
Jul 22, 1968: Alan Cox - http://en.wikipedia.org/wiki/Alan_Cox
Damian Conway - http://en.wikipedia.org/wiki/Damian_Conway
Dave Bradley (CTRL-ALT-DEL) - http://en.wikipedia.org/wiki/David_Bradley_(engineer)
JD Frazer (User Friendly) - http://en.wikipedia.org/wiki/J.D._Frazer
Guido van Rossum - http://en.wikipedia.org/wiki/Guido_van_Rossum
Robert Morris - http://en.wikipedia.org/wiki/Robert_H._Morris_(cryptographer)

Anyone else want to submit some names?

More Harm Than Good

2009-03-30T20:56:00.003-06:00

I ran across an interesting item in my feed reader the other day. In brief, there is a site that claims to be an energy-efficient version of Google, just because it uses a black background instead of white. The problem is, black screens are actually less energy-efficient on LCD monitors than on CRTs. And since the world is trying to leave CRTs behind, Blackle is actually doing more harm than good with its black backgrounds.

This reminded me of another site that I happened upon recently. The basic gist is, peta, an organization that claims to be saving animals, is actually killing them by the boatload. In my opinion, the problem isn't so much that they're euthanizing animals, as it is that they're trying to cover it up. But the focus of this site seems to be the other part of it: an organization that claims to be saving animals is actually killing animals. More harm than good.

While I was mentioning these things to my brother, he brought up another one. Sorry, no link on this one, but I'm sure there's something out there. Ordinary indandescent bulbs are being replaced by compact flourescent bulbs everywhere. The problem is, flourescent bulbs contain mercury, a known toxin. At our house we've been systematically replacing indandescent bulbs with flourescent bulbs, as the indandescent bulbs burn out. I haven't had a flourescent bulb burn out yet (we moved in less than a year ago, and are supposed to expect many years to come), but when one does burn out we're supposed to take it to a special center or something to dispose of it, because of the mercury.

I don't know about you, but I don't know where this magical, mystical place of compact flourescent bulb recycling is. And I'd be willing to bet that most of America has no clue either. In fact, I'd be surprised if the majority of Americans knew that they were screwing in little mercury capsules into all of those electrical sockets in their house.

I guess there's a few lessons to take away here. I think I may have said some of them before, like don't take medical advice from me because I'm not a doctor and in fact can't even manage to open a computer without cutting myself. Also, logic that seems infallable to you might be absolutely idiotic to somebody who actually knows what they're talking about. And let's not forget that just because some big, faceless organization claims to be helping you help the environment, doesn't mean they're telling the whole story.

Maybe it's time to start questioning things instead of taking them for granted.

UPDATE: Okay, it was weird enough that so many people posted about the CFLs. But I didn't mind because some intelligent things were being said. Then somebody posted without reading the other comments, and it seemed a little less intelligent.

Look, if you're that worried about the CFLs, just get an LED-based light bulb. They have no mercury, they use even less power than the CFLs, and they'll last much, much longer. They're also terribly expensive right now, but if you're that worried about both mercury and incandescent bulbs, they're really your best shot.

And that's it. I'm shutting of comments on this post.

Three Cheesecakes

2009-03-29T20:44:00.004-06:00

I did a little experimenting with cheesecakes recently. Now, when I say experimenting, I mean these probably aren't what you think of when you think of cheesecake. And you know, that's a real shame, because these really are delicious. But if you're feeling a little iffy about them, don't worry. If you remember my mini gnome cheesecakes, you'll know that you can make little tiny versions of the cheesecakes to sample, and if you like it then just quadruple the recipe and make a big cheesecake. That's right, these cheesecakes can be made with empty tuna cans. Let's take them one at a time, and I apologize in advance for the monotonous photos.

Cheddar Apple Cheesecake

This may seem a little strange to anyone that lives outside of New England, but cheddar and apple really do go well together. And while this is a sweet cheesecake, I didn't want it to be too sweet, so I went with a pretzel crust. The sugar is actually an attempt to make the crust hold together a little better. After a few tests both ways, I think it worked. As you can see in the photo, I added a little cinnamon on top of the cheesecakes, which I think really makes it.

Crust:

5 Tbsp pretzel crumbs
1 Tbsp sugar
1 1/2 Tbsp butter

Filling:

8 oz cream cheese, softened
1/4 cup sugar
1 large egg
3 Tbsp unsweetened applesauce
1/2 cup shredded cheddar

Garnish:

Ground cinnamon, as needed

* Preheat oven to 350F.
* Prepare 4 empty tuna cans, as per the mini Gnome cheesecake recipe.
* Melt the butter, and toss with pretzel crumbs and sugar. Divide evenly among the tuna cans, about 2 Tbsp per can.
* Press crusts into the bottom of the cans.
* Blind-bake the crusts at 350F for 10 minutes.
* Allow crusts to cool completely on counter, and drop the oven temperature to 275F.
* Cream together the cream cheese and sugar.
* Mix in the egg.
* Mix in the apple sauce and cheddar.
* Divide the cream cheese mixture among the tuna cans.
* Sprinkle cinnamon on top of the cheesecakes.
* Bake at 275F for 15 minutes.
* Turn off the oven and leave the door open for one minute.
* Close the door and allow the cheesecakes to cool for 45 minutes with the oven.
* Chill overnight before serving.

Bacon and Onion Cheesecake

I'm not going to lie here. Savory cheesecakes freak people out, just like you already started freaking out when you read the name of this cheesecake. I think that this kind of cheesecake is actually what the tuna can form factor is perfect for. Yes, they are kind of intense to eat on their own. No, you wouldn't ever serve it for dessert. Tell you what, instead of thinking of it as cheesecake, how about if you think about it as a dip? The only difference between this and a cheeseball is that this is flat, and tastes way better.

Crust:

5 Tbsp pretzel crumbs
1 Tbsp sugar
1 1/2 Tbsp butter

Filling:

4 strips bacon
1 cup diced onion
1/2 cup diced red bell pepper
1/2 tsp chopped garlic
dash Worcestershire sauce
8 oz cream cheese, softened
1/4 cup sour cream
1 large egg
2 Tbsp whole milk

* Preheat oven to 350F.
* Prepare 4 empty tuna cans, as per the mini Gnome cheesecake recipe.
* Melt the butter, and toss with pretzel crumbs and sugar. Divide evenly among the tuna cans, about 2 Tbsp per can.
* Press crusts into the bottom of the cans.
* Blind-bake the crusts at 350F for 10 minutes.
* Allow crusts to cool completely on counter, and drop the oven temperature to 275F.
* Fry the bacon till nice and crispy. Remove from the pan to cool, but save the drippings.
* Saute the onion and bell pepper in the bacon drippings until nice and browned. When the onion gets transluscent, add the garlic and Worcestershire sauce. Allow to cool completely.
* Cream together the cream cheese and sour cream.
* Mix in the egg.
* Mix in the milk.
* Crumble the bacon, and fold into the cream cheese mixture with the veggies.
* Divide the cream cheese mixture among the tuna cans.
* Bake at 275F for 15 minutes.
* Turn off the oven and leave the door open for one minute.
* Close the door and allow the cheesecakes to cool for 45 minutes with the oven.
* Chill overnight before serving.

Chinese Five-Spice Cheesecake

Okay, so maybe the savory cheesecake was a little weird. And this one probably seems weird too. Don't worry, this is probably the most normal of these cheesecakes. Depending on who makes it, Chinese Five-Spice tends to have ingredients like cinnamon, cloves, nutmeg, ginger, allspice, things like that. Kind of like a pumpkin pie. I even threw in a gingersnap crust, which is excellent. You'll like this one.

Crust:

5 Tbsp gingersnap crumbs
1 Tbsp sugar
1 1/2 Tbsp butter

Filling:

8 oz cream cheese, softened
1/4 cup sugar
1 large egg
1 tsp Chinese Five-Spice powder

* Preheat oven to 350F.
* Prepare 4 empty tuna cans, as per the mini Gnome cheesecake recipe.
* Melt the butter, and toss with gingersnap crumbs and sugar. Divide evenly among the tuna cans, about 2 Tbsp per can.
* Press crusts into the bottom of the cans.
* Blind-bake the crusts at 350F for 10 minutes.
* Allow crusts to cool completely on counter, and drop the oven temperature to 275F.
* Cream together the cream cheese and sugar.
* Mix in the egg.
* Mix in the spices.
* Divide the cream cheese mixture among the tuna cans.
* Bake at 275F for 15 minutes.
* Turn off the oven and leave the door open for one minute.
* Close the door and allow the cheesecakes to cool for 45 minutes with the oven.
* Chill overnight before serving.

Checking for open files

2009-02-27T19:11:00.001-07:00

A coworker presented me with an interesting problem today. We have an SFTP server set up, where a client is uploading a series of files on a pretty regular basis. My coworker needed to process the files as soon as they were finished uploading, but he didn't know how to tell when it was finished. Since the SFTP server would be keeping the file open until it was finished writing it, all we needed to do was find out whether the file was still open.

In Bash, there's an easy way to check for this. When called without any arguments, the lsof program shows a list of all open files on the system. You can also call it with the full path of the filename that you are checking, and it will show only the processes that have that file open. For example, we can look at /dev/null, which seems to pretty much always be open on my system:


$ lsof /dev/null
COMMAND     PID  USER   FD   TYPE DEVICE SIZE NODE NAME
gconfd-2   6668 jhall    0u   CHR    1,3      6663 /dev/null
gconfd-2   6668 jhall    1u   CHR    1,3      6663 /dev/null
gconfd-2   6668 jhall    2u   CHR    1,3      6663 /dev/null
...snip...

The lsof program also has an exit status of 0 if it returns any information, and an exit status of 1 if it did not. Since we're only interested in that error status and not in any of the information returned, we can throw it away by redirecting STDOUT and STDERR to /dev/null with &>:


$ lsof /dev/null &> /dev/null && echo "The file is open" || echo "The file is not open"
The file is open

Breaking this down a little further, if "lsof /dev/null &> /dev/null" was successful (returned an error status of 0), then it will run the first echo command. If not, that will make both the lsof command and the first echo command unsuccessful (error status of 1 for both) and the echo command after the pipes will run.

If it's a little confusing having "/dev/null" in that command twice, try running it on a file that's likely to not be open, like /proc/cpuinfo:


$ lsof /proc/cpuinfo &> /dev/null && echo "The file is open" || echo "The file is not open"
The file is not open

Just a handy little bit of Bash Fu to brighten your day.

Ginger Beer Test #1

2009-02-25T20:41:00.005-07:00

Those who know me well know that one of my favorite things in life is a good ginger beer. Not sugary-sweet ginger ale, but real, spicy, knock-you-off-your-feet ginger beer. I've been wanting to make it for a couple of years now, I just hadn't gotten around to it. Well, this week I finally got off my butt and gave it a try. This batch was a test run, to help me get my bearings. There will be more.

But first, we need to define what exactly makes a good ginger beer. It depends on who you are really, and I'm going to guess that my definition is probably not the same as yours. My three favorite brands are, in order:

Bundaberg

Cock N Bull

Vernors

Okay, so Vernors is really a ginger ale, but it's still up there. There are a few properties to look for in ginger ale and ginger beer. First of all, ginger ales are generally divided into two categories: golden and dry. I've found golden to be a little darker, sweeter, and more strongly-flavored. Unfortunately, it's near-impossible to find. Dry ginger ale is lighter, and extremely common. Being a Utah boy, I grew up on Canada Dry and Schweppes, but when I lived in New Hampshire I got used to Seagrams too.

But these really lack something in my opinion, and that's real ginger. I like mine spicy, but not harsh. Smoothness is also important to me, but not so much as spiciness. And that's where ginger beer comes in. It's stronger, tastes more like ginger, is often brewed, and the good stuff almost always has little fibers of real ginger in it.

I've tried a lot of different kinds of ginger beer, and I've divided (most of) my favorites into two categories: Jamaican-style and Australian-style. Jamaican ginger beer is spicy, strong, and has been known to make me cough, just getting a whiff of a newly-opened bottle. Unfortunately, it generally seems to be harsh, overly-spicy, and flavored with other spices in addition to ginger. In the small, relatively unknown world of ginger beers, this seems to be the most common, and I've probably tried a good 20 or 30 different brands, but few more than once.

Australian ginger beer is different. I've had a good dozen or so different types of this, and few of them miss their mark. They're strongly-flavored, spicy, but also smooth. They seem to be true to the ginger flavor, and when I get ahold of a few bottles, they don't remain full for long. It was this variety that I was looking to reproduce.

The first task of course was to find some recipes. I wasn't sure yet whether I was going to make a composite recipe, I was just looking to see what I could find. Most of the recipes I came across were long and ridiculously complex. One of them had a longer list of equipment than actual ingredients. Finally, I found a recipe on Recipezaar that was simple, but kind of long and very poorly-written. It seemed to be more of a Jamaican recipe, but I figured you've gotta start somewhere.

I didn't use the recipe as-is of course. I made a few changes, and then took notes while I more or less followed their directions. My ingredient list was as follows:

1/2 tsp yeast
1/2 cup warm water (110F)
1 3/4 liter water
3/4 oz/wt freshly-grated ginger
1/4 tsp anise
5 whole cloves
1/2 cup sugar

I sprinkled the year in the 1/2 cup of warm water and set it aside. I then mixed together all of the other ingredients in a sauce pan, brought it to a boil, and then dropped it to a simmer for 5 minutes. I then killed then heat and let it cool on the counter to 110F (actually, 112F to be exact, but anything below 120F is fine), and then poured in the yeast/water mixture.

I poured all of this (whole spices and all) into a clean 2-liter bottle, pressed in the sides a little, screwed on the cap and let it sit overnight on the counter. In the morning, the bottle was completely inflated, and you could tell by feeling it that there was a lot of pressure inside. I moved this to the fridge and let it sit until the next evening.

I tasted the brew before I added the yeast, and I knew I was already in trouble. It was extremely spicy, and the anise overpowered everything. But I was determined to let it age for a couple of days so that I could get a good idea of what it would be like.

When we uncorked it, there was a lot of sediment. This is to be expected with home-brewed drinks. But it's not something I would want to drink, so I poured it through a tea strainer into some cups and tasted it with my wife and her friend visiting from Wisconsin. Unfortunately, this means I also strained out the ginger fibers, but since we were just about to drink it, I figured it wouldn't be adding any more flavor anyway.

First of all, the color. It was more cloudy than I expected. I'm okay with that. Next, the carbonation. It was lightly carbonated, not as much as commercial ginger beer, but still decent. Finally, the flavor. It was very spicy, but not as bad as I had feared. The anise still overpowered everything, but it still had a lot of ginger in the finish, to come back and bite you. I suspect the clove was about right (if you like that sort of thing; I'm currently undecided), but that anise just screwed it all up. It was just a little bit sweet for me, but not bad.

Just a quick note on drinks carbonated with yeast. This is exactly how beer is made. When the yeast gets going, it produces lots of carbon dioxide and a little bit of alcohol. That's what makes bread rise, and makes beer fizzy. Since my brew had only a tiny amount of yeast, and was only allowed to ferment for a couple of days, the alcohol content was negligable. If that is of concern to you, this is the wrong recipe for you.

I will be starting another bottle tomorrow night, when this one is used up. Now that I have my bearings, I think I have an idea of where I want to take it. I'm going to try multiple changes at first, and then fine-tune it as I get closer to what I want. The next batch will have no anise. That was just a bad idea. I will leave the cloves in there, and probably use the same amount of ginger. This time I'm thinking I may add some lemon juice, and drop the sugar down to 1/3 of a cup. I hope to post results this weekend.

Plex Cake

2009-02-18T20:04:00.004-07:00

It's been a while since I posted, and I'm going to blame that at least partly on being busy. But I did want to share a cake with you that I made this past weekend. It was my daughter's second birthday, and she is a huge Yo Gabba Gabba fan. In fact, she has been watching it almost since her first birthday, so it should come as no surprise that I've been looking forward to making this cake for several months.

Plex is a robot member of the cast, and is probably one of the most popular cakes for parents to try to make for their kids. But up until this point, I have't seen one with fondant like this, so I'm going to declare myself in the lead. First, the cake:

The yellow part is a white chocolate rolled fondant recipe that I've been working on for a while. It's not quite to where I want it, but it's getting there. The cake was a chocolate cake based on a recipe I found online. It it filled with a chocolate mousse based on a recipe by Alton Brown and fresh raspberries. Anything you see that is covered with white fondant (except for the eyes) is molded from crispy rice treats. And the antenna is a cigar cookie wrapped in semi-sweet chocolate.

The black square where his eyes are was actually about half an inch higher, but it seemed to have slipped down during transportation. I eventually gave up on trying to make the ball on the top of his antenna red, or trying to make his ears at all. And no, I never planned to attach even parts of his arms to his shoulders. Deal with it.

It was kind of a fun cake to build, but like all cakes that I make like this, I was reminded repeatedly why I don't often make cakes like this. The fondant didn't want to cooperate (I think I may have added just a touch too much white chocolate to it), keeping the chocolate in temper for that antenna was a miracle (the four backup antennas that I made all lost their temper) and speaking of the antenna, when I opened the tin of cigar cookies, they were all broken. Actually, that's pretty much why I decided to wrap it in chocolate; it was actually a couple of pieces of broken cigar cookie, reinforced with chocolate.

I'm going to go ahead and post my fondant recipe here, because I seem to keep losing it. It is not the easiest thing to make, and I only recommend it for those who are truly serious. I use a stand mixer for part of making this, but heed well this warning: if you try to make it from start to finish with any kind of electric mixer, you will burn out the motor! And yes, this recipe is measured entirely by weight. If you're not willing to weigh the ingredients, you're not serious enough to make it.

White Chocolate Rolled Fondant (still considered a test recipe)

0.5 oz/wt powdered gelatin
2.1 oz/wt cold water
0.8 oz/wt glycerin
9.6 oz/wt corn syrup
10.5 oz/wt white chocolate
2 lbs confectioners sugar + extra just in case

* Bloom the gelatin in cold water for 3 minutes.
* Start melting the white chocolate gently over a double-burner.
* Add the glycerin and corn syrup. Heat gently to dissolve, while whisking occassionally.
* Remove everything from the heat, and whisk in the white chocolate.
* Add the mixture and all of the sugar to a stand mixer use the paddle to mix on low.
* When the mixer starts to sound as if it is having trouble, turn it off and finish kneading the mixture by hand. I have found that using a bowl scraper to cut and mix the fondant helps keep your hands clean.
* If the mixture seems to be too wet and sticky, sprinkle in some more sugar.
* When you have formed a homogeneous ball of dough, move into an air-tight container for a couple of hours to let it rest.

Like I said, I still consider this recipe in a testing stage. It clearly requires more than 2 lbs of sugar, but I'm not sure how much more. If I had to guess, I'd say it was just a couple of ounces, maybe as much as half a cup. Also, I originally went with only 10 oz of white chocolate, and the fondant seemed a little too pliable. 10.5 oz made it a little too hard. Next time around, I'm going to go with 10.25 oz of white chocolate and see how it goes. I think that the rest of the ingredients are pretty solid, it's just a matter of getting the best measurements for the sugar and the chocolate.

A Quick SNMP Primer

2009-01-22T19:50:00.002-07:00

Just a quick disclaimer to go with my quick primer: I'm relatively new to SNMP myself. This article doesn't aim or claim to be comprehensive. It's just meant to help you get started.

SNMP is the Simple Network Management Protocol. It has been said that not only is it not simple, but it's usually used for monitoring more than managing devices. I don't fully agree with this sentiment. It does have some weirdness, but once you get over that it's not so bad.

Structure of Management Information

Before we get into practical usage, we need to talk theory. SNMP partly describes a network protocol, but it also describes an organizational structure. This structure is called the Structure of Management Information, or SMI for short. This is a tree that branches out into various nodes called Object Identidiers, or OIDs. Each level of this tree has its own set of numbers for each node. At the moment, we're only concerned with a specific path in this tree, sometimes referred to as MIB-II, or the Management Information Base. The OID for this node is 1.3.6.1.2.1.

This node is actually the root of where our queries are performed. In fact, a lot of times this OID is just printed as mib-2. The nodes underneath this are what we will be querying.

Some of the nodes that we're interested in here are system (1), interfaces (2), at (3), ip (4) and so on. For example, the OID that describes a system is mib-2.system.sysDescr.0. The long name for this is .iso.org.dod.internet.mgmt.mib-2.system.sysDescr.0, and the numerical notation for this is .1.3.6.1.2.1.1.1.0. On my Linux box, this is often the output of the "uname -a" command.

If you need some time to digest this part, go for it. I think this is the biggest thing to overcome with SNMP. When you're ready, come back for the next part.

SNMP Versions

There are three main versions of SNMP that you will run into: 1, 2c and 3. Version 2c is the community version of version 2, and while it does provide a little extra information, it isn't much of an improvement over version 1.

These two versions use passwords called community strings to provide access. There is a read-only community used for monitoring (reading values from the device), and a read/write community used for management (setting values on the device). The most common default community string for read-only access is "public". The most common default community string for read/write access is "private".

Most SNMP-enabled devices have these as their defaults, and as we all know, most sysadmins are still too lazy to change the defaults. Fortunately, the Net-SNMP package in Linux has a little bit better default configuration. It is set to only listen to requests from the local machine. Even better, the Net-SNMP server isn't installed by default on most Linux distributions, so don't worry about your system being vulnerable like that out of the box.

SNMP version 3 introduced the concept of usernames and passwords, rather than just communities. But versions 1, 2c and 3 all suffer from one major shortcoming: they send their authentication and their data in the clear. Fortunately, version 3 also supports encryption for either authentication or data transfer or both.

Setting Up SNMP

Installation is easy, but there are a couple of different packages that you want. On RHEL5 and Fedora 10:


yum install net-snmp net-snmp-utils

On Ubuntu:


apt-get install snmpd snmp

For those of you who are interested, net-snmp == snmpd and net-snmp-utils == snmp.

Once you have it installed, go ahead and blow away the configuration file:


mv /etc/snmp/snmpd.conf /etc/snmp/snmpd.conf.orig

The default configuration is a lot to deal with. We're going to keep it simple. Create a new /etc/snmp/snmpd.conf file with the following line:


rocommunity yourcommunitystring

...where yourcommunitystring is, of course, whatever community string you want to use. This is really all the configuration you need for versions 1 and 2c, though you can add more if you like.

If you want version 3 instead, the configuration is a little different. There are actually two files to edit here, and the second one probably doesn't exist by default. First, edit the /etc/snmp/snmpd.conf file and add the following line:


rouser username

...where username is of course the username that you want to use. You'll also need to set up a password. Of course, we would never want to use a plaintext password if we didn't want to. Depending on your version of Net-SNMP, we have a few options available to us. We can use either MD5 or SHA for authentication, and DES (or AES in more recent versions) for data transfer. I'm going to assume that you want to use SHA and DES. Create if necessary and edit the /var/net-snmp/snmpd.conf file and add the following line:


createUser username SHA password1 DES password2

...where username is of course the username that you specified in the /etc/snmp/snmpd.conf file, password1 is your SHA password and password2 is your DES password. I know it seems kind of scary to leave passwords out in the open in plain text files like this, but don't worry; as soon as SNMP starts up, it will rewrite this file and encrypt the passwords. I should also note that this is the only time you will ever edit this file by hand, and the createUser line is the only line that you will ever add to it.

When you're ready to start the SNMP service, type in:


chkconfig snmpd on (RHEL/Fedora systems only)
/etc/init.d/snmpd start

Viewing SNMP Data

There are several commands available for viewing SNMP data, but I'll start you off with just the snmpwalk and snmpget commands for now. Once you get the hang of them, you can start exploring other commands. Usage does differ slightly between versions 1 and 2c, and version 3, but only in authentication.

If you added the rocommunity line to your /etc/snmp/snmpd.conf file, use the following command to take a look at your MIB-II tree:


snmpwalk -v 2c -c yourcommunitystring localhost | less

We're piping the output of the command to less because there can be a lot of information here. You've probably guessed that the -v option specifies the version (1 or 2c) and the -c option specifies the community string. The first line probably looks something like this:


SNMPv2-MIB::sysDescr.0 = Linux bourdain 2.6.24-22-generic #1 SMP Mon Nov 24 18:32:42 UTC 2008 i686 GNU/Linux

Let's make the names little more verbose. Hit 'q' to exit out of this, and try this command:


snmpwalk -v 2c -c yourcommunitystring -Of localhost | less

For your reference, that is a capital letter O, not the number zero. When you look at the output of this command, it probably looks more like this:


.iso.org.dod.internet.mgmt.mib-2.system.sysDescr.0 = Linux bourdain 2.6.24-22-generic #1 SMP Mon Nov 24 18:32:42 UTC 2008 i686 GNU/Linux

Hey, you want to really have some fun? Try out this command instead:


snmpwalk -v 2c -c yourcommunitystring -On localhost | less

Now you get the name in numerical format:


.1.3.6.1.2.1.1.1.0 = STRING: Linux bourdain 2.6.24-22-generic #1 SMP Mon Nov 24 18:32:42 UTC 2008 i686 GNU/Linux

The amusement never ends. Going back to the full naming format, you'll notice that the first set of lines is in the system node:


.iso.org.dod.internet.mgmt.mib-2.system...

If you scroll down a little, you'll find the interfaces node:


.iso.org.dod.internet.mgmt.mib-2.interfaces...

We can specify to walk only one of these nodes if we like. Since we're only looking at information under the mib-2 node, we can actually just refer to it by that one group name:


snmpwalk -v 2c -c yourcommunitystring -Of localhost system | less

Now you will only see information in the system group. If you know the exact location of a value that you want to look at, you can use the snmpget command to look at it. The syntax is almost identical to snmpwalk:


snmpget -v 2c -c yourcommunitystring -Of localhost system.sysDescr.0

Now that you know how to get information using community strings, let's switch back to SNMPv3. When we specify "-v 3", we must specify the -u option with a username, instead of the -c option. We also need to let it know what passwords we're using, and how to encrypt them. Your basic snmpwalk command will look something like this:


snmpwalk -v 3 -u username -a SHA -A password1 -x DES -X password2 -l authNoPriv localhost | less

If you were able to follow along okay when we set these passwords, then most of this line already makes sense to you. The big things to remember now are what options go with what values, and the security level.

The "-l" option can specify one of three levels: noAuthNoPriv, authNoPriv or authPriv. If for some unknown reason you decide not to use passwords, you would set this to noAuthNoPriv. Otherwise you would use authNoPriv for read-only access or authPriv for read/write access.

I'm not going to bore you with the output of this command, because you've already seen it. You can use -Of or -On to modify the output, you can specify a group, you can do pretty much whatever you were doing with v1 or v2. The only real difference is authentication. And yes, the snmpget command looks pretty much the same.

In fact, once you have a handle on these couple of commands, this would probably be a good time to check the man page for snmpcmd. There is not actually a command called snmpcmd, this page just shows you the options that are common to the whole suite of Net-SNMP commands.

Hopefully this gives you a good foundation for playing with SNMP on your Linux box. Of course, there are volumes written on the subject, and my little artcle doesn't even attempt to cover what they will. But hopefully they will get you over any initial fears about it and get you up and running.

That Wasn't Nice

2009-01-22T07:42:00.000-07:00

Okay, so in retrospect, that wasn't very nice of me. I don't post for a couple of weeks, and then when I do it's a walkthrough in Cacti, a program that I've barely mentioned here before. Even worse, Cacti fits into the category of "arcane, even within the already esoteric Linux comminuty". While my walkthrough doesn't technically cover SNMP, it's almost pointless to expect somebody to understand Cacti if they don't know SNMP.

What I've decided to do to make up for this (to the geek crowd, at least) is write up a brief SNMP primer. I'm still pretty new to it myself, but what I've gotten into so far has proven to be relatively simple, once I got over a few basics that are kind of rough. Expect that primer to be posted shortly.

For those of you who come here for food, I'll try to post something for you soon too. I don't really know what yet. I'll figure it out.

Monitoring Apache Processes with Cacti

2009-01-21T19:06:00.002-07:00

I've been setting up Cacti at work to monitor our servers. Last night it occured to me that it might be useful to monitor Apache processes on our web servers. Cacti already has a graph built in for monitoring processes in general, but I just wanted those that matched "httpd". It was a simple thing that I wanted, but setting it up proved to be non-trivial, so I thought I'd share how I set it up. Keep in mind that this is on a RHEL5.2 box. Your distribution may vary.

First of all, you need a script to gather the information. You already have one mostly built, so use it as a template:


cd /var/lib/cacti/scripts/
cp unix_processes.pl unix_apache_processes.pl

Edit that file so that it looks like this:


#!/usr/bin/perl

open(PROCESS, "ps ax | grep httpd | grep -c : |");
$output = ;
close(PROCESS);
chomp($output);
print $output;

Now you need to set up Cacti to use that script. Go into the Console tab in Cacti and click on "Data Input Methods". Click "Add", and use the following values:


Name: Unix - Get Apache Processes
Input Type: Script/Command
Input String: perl /scripts/unix_apache_processes.pl

Save it, then go back in and click on "Add" for "Output Fields". Use the following values:


Field [Output]: proc
Friendly Name: Number of Apache Processes
Update RRD File: (checked)

Go ahead and save it. Now go into "Graph Templates" and find the "Unix - Processes" template. Put a check next to it, go find the drop-down box next to "Choose an action", set it to "Duplicate" and click "go". Change the title to "Unix - Apache Processes" and save it.

Go back into "Graph Templates" and click on the "Unix - Apache Processes" template that you just created. Find everything that says "Processes" and change it to say "Apache Processes", then save it.

Go into "Data Templates" and find "Unix Processes". Duplicate it like you did with the Graph Template, calling the new template "Apache Processes". Then go back into it and change the "Data Input Method" drop-down to "Unix - Get Apache Processes".

Go back into "Graph Templates", then "Unix - Apache Processes" again and click on "Item # 1". Change the "Data Source" drop-down to "Apache Processes - (proc)" and hit "save". Do the same for "Item # 2", "Item # 3" and "Item # 4".

Now when you go into the "Devices" menu and click on one of your web servers, you should see "Unix - Apache Processes" show up on the drop-down box for "Associated Graph Templates". Go ahead and add it, then go up to the top of the device page and click "Create Graphs for this Host". Put a check next to "Unix - Apache Processes" and click "Create".

Setting up this graph may be pretty straight-forward for the average Cacti user, but there are a lot of steps, and it helps to do them in the right order. This walkthrough should give you a good start for adding more complex graphs to Cacti when you need to.

Pork Enchiladas

2009-01-03T20:24:00.002-07:00

I love enchiladas. I remember my dad making them when I grew up. Of course, his were a little different than mine. For instance, he used corn tortillas. And he seemed to like cheese-only enchiladas. I kind of liked 'em like that. But I've discovered that my wife doesn't seem to like cheese-only enchiladas. And I got fed up with corn tortillas long ago. They crack when you try to roll them, they... well, they crack. Isn't that bad enough?

Lately I've been playing with enchilada sauce. Really, isn't it just a thinned out tomato and chile sauce? I decided to try a little experiment: canned tomato sauce plus taco seasoning. Hey, it's actually not bad. It's a little thick, but it's very close in flavor to some of the store-bought enchilada sauces.

So bearing all this in mind, let me share with you an enchilada recipe that my family has been eating on occassion lately. It's not exactly traditional, but we like it. But if you want to go back to corn tortillas or something, go for it. You won't hurt my feelings.

First off, you'll need your enchilada sauce. Whether you use the "real" stuff or you do it my way, it helps to heat it up. I like to use a wide frying pan, to have enough room to dip the tortillas in later. With my version, you need a can of tomato sauce and a couple of Tablespoons of taco seasoning.

Stir it all together and heat it up to about a simmer. You don't need to boil it or anything, it just needs to be heated a little. If you want, you can thin it out with a little bit of liquid. Water would be fine, but broth would be better. Once it's ready, put it together with the rest of your ingredients and equipment.

We have cheddar cheese, chopped pork, sliced olives, red bell peppers, an 8x8 baking pan, four 8-inch flour tortillas, and a pan full of enchilada sauce. Just a note: about one enchilada in, we put the baking pan next to the enchilada sauce. Step one, coat a tortilla with sauce:

Move it to the baking pan and add a hanful of chopped pork:

Olives go in next:

Chopped red bell peppers:

And of course, some cheese:

Roll it up and repeat with the rest of the tortillas. You'll want to use all of the pork inside, but be sure to save some of the olives, peppers and cheese for the top. At some point, it's going to look kind of like this:

Go ahead and cover it with a bit of cheese:

Add the remaining olives:

And the remaining peppers:

Slide it into the oven at 350F, until the cheese starts to bubble. It'll probably take about 15 minutes or so.

Oh, so tasty.

Fun at Work

2009-01-02T19:25:00.003-07:00

I'm afraid of being found out.

Seriously, I don't know that I've ever had so much fun at work. Don't get me wrong, I loved working at Guru Labs, but the situation is different here. Let me explain.

At Guru Labs, my bosses seemed to have very realistic expectations of me. They hired me, fully knowing that I had gaps in the technical knowledge that they needed me to have. The promise was that they would help fill those gaps, and I would be ready to teach by the time I needed to. If I was not, then they would give me the opportunity to work elsewhere.

I met the challenge, and before long I was knee-deep in Linux knowledge that I never expected to obtain. There were always new things to learn, and I was always surrounded by people willing to teach me. At the same time, I got to be that person for about a couple dozen students a month. I was having a blast, and I wasn't sure I ever wanted it to end.

There was just one problem. As much as there was to learn, it sometimes seemed like there was never time to learn it. There were always classes to teach, and there was course ware to write, work to be done. I was constantly surrounded by cool new technologies that often felt barely touchable. And suddenly, it was gone, and I was left to look for another job which could never hope to be as much fun.

Fortunately, I had prospects in the queue. By the end of the first week I had applied at close to two dozen companies, interviewed at three of them, and had an offer from one of them. By the end of the second week I had already started work at my new company, and was still politely turning down interviews.

I had actually turned down an offer from this company at the same time I originally accepted an offer from Guru Labs. I had a friend that worked there, but I still had to get hired on my own. When I started, I think the idea was for me to watch my new coworkers and get a feel for what needed to be done. They were programmers who had been doing system administration when it was called for, but it wasn't their thing. Thanks to Guru Labs, I was well-trained in system administration and I just kind of took over.

This is a company who was small (maybe 20 employees) when I first interviewed, and who is now in the process of growing into a larger company (60+ employees at this location, plus the warehouse, plus the call center in Chicago, etc), but who still has the kind of server setup that one might expect at a small company. I have taken it upon myself to make the servers enterprise-grade.

This means I get to play with all of the technologies that I didn't really get to play with before. Well, not all of them, but increasingly more. One of my current projects involves four RHEL 5.2 virtual machines installed on my notebook using KVM, which I am using to test out different MySQL Cluster configurations. This week I also started playing with Cacti, and Nagios is on the horizon.

One day I looked at our current backup server and decided that it needed more hard drive space. Really we needed a brand new backup server, but I was willing to settle for a new hard drive. My boss came in to talk about it, and by the time he left I was pricing out components for a brand new backup server. This is not the first time I've had a boss that knew less than me about what I did, but it's definitely the first time a boss has not only admitted it, but taken my suggestions on what needs to be done.

To be honest, I'm not used to this. I was hired based on technical merit and experience, and then given what feels like free reign to do whatever I want. I don't really have free reign, but it feels like it. My boss realizes that I know what needs to be done, and he's willing to let me do it. He doesn't have some kind of personal agenda, he's not caught up in a power struggle, there's none of that. He's trusting to me do what he hired me to do, and his only concern is that I stay focused on the best interests of the company.

It's so weird. Nearly every day when I drive home from work, I am elated about what happened that day and what's going to happen the next. Actually having my opinion listened to on such a regular basis makes me feel like a rock star. And yes, I'm a little worried that I'm going to be found out. Every so often I wonder if my boss is going to realize how much fun I'm having actually doing my job, and start charging me admission.

I still miss Guru Labs. I certainly wouldn't be in the position I am now without the expert training and unparalleled opportunities that I received there. But I guess fate (if you believe in that sort of thing) decided that it was time for me to cut my teeth elsewhere. I guess it truly is bitter-sweet.

Lasagna

2008-12-29T20:08:00.002-07:00

One of my all-time favorite dishes is lasagna. Unfortunately, I don't get to make it very often. It's an easy dish, but to make it right can be time-consuming. It's the type of dish that really benefits from letting the flavors spend some time together. Sure, you can rush this along. Skip over a few steps, cut some corners. But once you've had the taste of a real lasagna done right, you won't go back.

Before you start, make sure you have all of your ingredients together:

1 lb ground beef (80% lean)
Worcestershire sauce, to taste
Italian seasoning, to taste
14 oz mirepoix
1 red bell pepper, diced
30 oz tomato sauce
15 oz canned diced tomatoes
2 teaspoons garlic powder
1 teaspoon smoked paprika
1 lb mozzarella
15 oz ricotta
1 large chicken egg
16 oz dry lasagna noodles
Kosher salt, to taste
freshly ground black pepper, to taste

Some of you are already scoffing at me using things like (presumably store-bought) Italian seasoning and garlic powder. There is no shame in having these things handy. The Italian seasoning I used is pretty dang tasty, and already formulated. I also liked that it, and the garlic powder, were dried. That means you can add them early and have the flavors permeate the dish longer and more easily.

Before you start on any of this, you'll probably want to preheat the oven to 350F, and grease up a 13x9 baking dish with cooking spray. Go ahead and set it aside for later. Our next step is to brown up about a pound of ground beef.

Season to taste with Kosher salt, pepper, Italian seasoning and Worcestershire sauce. I probably used about a Tablespoon of the Italian seasoning and maybe twice that of Worcestershire sauce. Don't cook it too long, just long enough to brown it. And make sure the seasoning is where you want it right now. Trying to compensate later for poor seasoning now isn't going to be easy.

When it's all nice and browned up, spoon it onto a paper towel-lined plate and set it aside. Try to leave as much of the fat in the pan as possible, you'll need it in a moment.

Okay, who here got confused when they saw mirepoix on the ingredient list? Wow, look at all those hands. Don't worry, it's not so bad as you think. Mirepoix is a 2:1:1 ratio of onions to carrots to celery. It's a French thing that works pretty well with Italian dishes. And as it turns out, Kroger has started selling it in the freezer section in little 14 oz bags. Add that, along with the red bell pepper, to the fat and crank the heat to medium-high. We want to get some nice color on these veggies.

Did I mention to make sure these are well-seasoned? Kosher salt, pepper, Italian seasoning, Worcestershire sauce. Make sure you season each layer of this dish before moving onto the next one. Speaking of the next layer, once you have some color on your veggies, go ahead and cool down the pan a little by adding the tomato sauce and the diced tomatoes.

Time for another layer of flavor. Notice a pattern here? Same drill as before, but go ahead and add in the garlic powder and smoked paprika with the other seasonings (Kosher salt, pepper, Italian seasoning, Worcestershire sauce). Let it simmer for a bit; how long is really up to you. I went for maybe five or ten minutes. I also stole some of it at this point to spread out on the bottom of the baking dish.

This is important. There are going to be a few layers of pasta in this dish, and you want to make sure you have a tomato sauce layer directly above and below every single layer of pasta. We're not going to use the pasta yet, just keep it in mind.

While the sauce is simmering, you'd better get the cheese ready. You're going to put together a container of ricotta, about 3/4 lbs of shredded mozz, a single chicken egg, and maybe a Tablespoon of Italian seasoning. I didn't say this in the ingredient list because I forgot it (oops!), but you can add 1/4 cup of grated Parmesan too. Go ahead and mix it all together in a bowl and set it aside. You'll need it in a moment.

When you think you're ready, and you have a layer of sauce in the baking dish, go ahead and add the meat into the sauce. You want to do this after you've sauced the baking dish, because you really don't want the meat stuck to the bottom. You will want a layer of dried noodles, however.

I love this part. Didja notice something about the pasta? Like the fact that we didn't blanch it first? So awesome. See, this is the beauty of lasagna: the moisture from the rest of the dish cooks the noodles for you. Okay, yeah, it'll add another 20 minutes of cooking time or so. But that's okay; the longer it cooks, the better the flavors marry. Your chances of soggy noodles will also decrease. Next up, add a layer of sauce.

This part is actually trickier than it seems. You see, you're going to have four layers of noodles, with sauce layers on both sides. That means eight layers of sauce, and this is number two. The sauce is important: it's responsible for cooking the noodles. But as tricky as that step was, it doesn't compare to this one: add about a third of the cheese mixture. It's easiest if you dollop it in small increments and then spread it with a spatula.

Oh man. You're going to hate me right about now, because it's time to add another tomato sauce layer. And then pasta, tomato sauce, cheese, tomato sauce, etc, until you have your top layer of pasta, and it's covered with sauce. Cover that layer with the last 1/4 lb of shredded mozzarella. Don't worry if it looks a little sparse. It'll be plenty.

Go ahead and sprinkle, really lightly, some Italian seasoning on top. Remember how I forgot the Parm in the cheese mixture? It may not be a bad idea to sprinkle a couple of Tablespoons on top as well. I'll let you add that to your own ingredient list. Now, go ahead and cover it with some aluminum foil. Try not to let the foil touch any of the cheese.

Slide that guy into your 350F oven and give it about 40 minutes. If you think about it, we're really only doing a couple of things here. We're not actually cooking the dish (except for the noodles) because the ingredients are already cooked. That's important to remember with casseroles. You're not cooking them, you're reheating them. Also, we're melting the cheese. But that part really happens at the end. At about 40 minutes, remove the foil and give the lasagna another 10 minutes to melt and maybe even brown the cheese a little.

Oh man, doesn't that look good? Well, bad news. You can't touch it yet. I know it's going to be torture, but you need to recover it with foil and let it sit for another half hour. It's gotta have time to cool, and to finish cooking those noodles. If you were to serve it at this point, it would burn your tongue, and the noodles would be a little too al dente.

A few more points before I let you loose on this dish on your own. First of all, I cheated more on the cheese than I let on. For some reason, Monterrey Jack sounded really good that day, so I used that instead of mozz. In the past, I've also mixed Jack and Cheddar. And one of my all-time favorite lasagnas was dubbed "pain lasagna". Instead of tomato sauce, I used hot enchilada sauce. Instead of meat, I used black beans, pinto beans and corn. I didn't mix them into the sauce, I added them as their own layer underneath the cheese layer. Speaking of cheese, I went with hot pepper Jack and habanero Cheddar that time around. It was so good, it brought tears to my eyes. Actually, the tears were mostly from the pain, but the flavor was good too.

Lasagna has a lot of room for interpretation. At its most basic, it's little more than alternating layers of sauce, cheese and noodles. Beyond that, you're on your own.

Screen for Beginners

2008-12-21T07:47:00.003-07:00

Perhaps I got spoiled at the last couple of companies I was at, but I was shocked when I discovered that my new coworkers had never heard of screen, much less used it. In the past couple of years, I have found screen to be such a valuable program that I can't imagine life without it. I use it on a regular basis both at work and at home, partly because it's so handy and partly because I'm a nerd.

There are plenty of ways to use screen, but there are three things that I like to use it for:

* Maintaining a perisistent shell on one computer that I can access from multiple computers.
* Maintaining a perisistent development environment.
* Logging into somebody else's shell to monitor them, or demonstrate techniques to them.

Screen doesn't ship installed on every distribution, but it's easy to get installed:

RHEL/Fedora: yum install screen
Ubuntu: apt-get install screen

Once it's installed, it's easy to get running. Just type "screen" from the command prompt. The display will blank, and you'll be left at a prompt that looks just like the one you were just at. If you're doing thing in gnome-terminal, you might see something like [screen 0: bash] in your title bar.

At this point, you can use the command line exactly as you did before; you're really just running another shell. Go ahead and run top, to see what I mean. But now you have a series of commands available, each of which starts with CTRL-a. The first one you'll want to be familiar with is CTRL-a d (that's a CTRL-a followed by a d, not CTRL-a CTRL-d). This will detach from your screen session. When you do this, you'll find yourself at the command prompt that you were at before, with "[detached]" displayed above the prompt.

At this point, screen is running, but it's in the background. If you didn't exit out of top, then it is still actively running, you just can't see it. Go ahead and open a second terminal window and type in screen -r. This will reattach to the screen session, and you will see top still running.

That's enough to get you going for a basic persistent shell. I know a lot of people that do this on a server with a public IP address, to keep programs like Irssi open so that they can log into it from work or from home, without having to actually log out of their chat accounts. But let's take this a little further.

You should still have top running. Keep it open and type in CTRL-a c. This will create a second screen, with its own command prompt. Go ahead and type in free.

Now that you have two screens going, type CTRL-a p. This will switch to the previous screen, where top is running. Type CTRL-a n to go to the next screen, where free is running. You could also use CTRL-a 0 and CTRL-a 1 to go to a specific screen number. If you hit CTRL-a c again, you'll get yet another screen.

I've found this to be useful in maintaining a persistent development environment, so that I don't have to leave my notebook at work or close several instances of vim and bash at the end of the day. But let's look at one more feature of screen.

If your terminal is maximized, go ahead and resize it so that it only takes up half the screen. Hit CTRL-a 0 to make sure we're at the screen with top. No go ahead and open up another instance of gnome-terminal and maximize it. Type in screen -x at the prompt. You will see the top screen running, but it won't be taking up the whole window. That's because you've attached to a screen session that was already running somewhere else, and screen is using the display variables from that other screen.

Those of you who have ever tried to walk another user through something on the phone know what a pain it can be to not know exactly what the user is looking at. This can be especially troublesome when a command prompt in Linux is involved. Isn't it much easier to log into their machine, connect to their terminal, and then watch them work?

Now for some cleanup. Hit CTRL-a d to detach from this screen. Go back to the other terminal window where top is still running and hit q to quit. Type exit at the prompt to close that shell. Continue exiting out of screens, using the exit command. When you've exited out of the last command shell, you will be dropped back at your original shell, with the message "[screen is terminating]" above the prompt.

That should be plenty to get you running for now. When you're ready to get into some of the more advanced features of screen, go ahead and run man screen to take a look at the other options available.

Pursuing Other Opportunities

2008-12-10T19:47:00.003-07:00

A few of my readers already know this, but most of you don't. A little over a week ago, I was laid off from Guru Labs. I know that it was not an easy decision for the owners to make, and I don't fault them for it. It was easily the best company that I ever worked for, and I already miss working for them.

This time around, my luck was good. When I first accepted the offer to work for Guru Labs, I had another offer on the table from another company as well. When I politely declined the other company's offer, they told me that if I was ever interested in coming back to let them know. I did so the same day I was laid off, as I was sending my resume to several other companies, and they contacted me almost immediately. They were the first of a series of interviews, and the first to give me an offer.

I think that had I held out, some of the other companies might have given me offers for more money. But this company paid well enough, and the drive into work was pretty close to what it had been. I already knew one of the programmers there, and that sweetened the deal. Today was my first day, and I had fun. In fact, I've already started working on a small project that I will probably lay out in a future post here.

I think I will keep the company anonymous this time around. When I was a guru, I had to be careful how I presented some companies on my blog, and I would often forego posting certain pieces of information entirely, to be on the safe side. Keeping my present employer seperated from my blog will make things a little easier on me.

Four other gurus were laid off at the same time as me. As as this writing, they are all still unemployed, though at least one seems to be on the verge of getting hired. If you are looking for one of the best system administrators you could ever hope to hire, I can hook you up with them. One of them was my instructor when I took the RHCE exam, and parts of his lectures still resonate in my mind. The others have also taught me a world of things (both Linux-related and otherwise) that I will never forget. Each one of them also knows Python to varying degrees, which I know has become increasingly important at many companies.

If you are looking for an admin, click the "contact" link on the right side of my blog and send me a note. I will happily forward your information to these gentlemen. And if you're looking for Linux training, I would still recommend Guru Labs in a heartbeat. You will not find better Linux training than them. They are still around, and while they did lose some excellent instructors, they also kept some excellent instructors. You will not go wrong with them.

Turducken

2008-12-07T07:06:00.000-07:00

Well, I pulled it off: turducken. This is the sort of dish that you learn a lot from while making it. Of course, I learned a few things the hard way. The first lesson that I learned the hard way was to read the instructions from all of my turducken recipes a little more carefully. It's not like baking where you can say, "this is a muffin recipe, that means I can use the muffin method." There's more to the technique.

The idea with a turducken is to remove all of the bones from a chicken and a duck, and most of the bones from a turkey, and stuff the chicken inside the duck, and the duck inside the turkey. Traditionally, there would be stuffing between each bird as well. As it turns out, removing most of the bones from the turkey isn't difficult, so long as you leave the skin on. Doing the same for a chicken and a duck was far more difficult, especially since we decided to remove the skin.

What we ultimately ended up with were a boneless, skinless chicken and duck, quartered. Because we tried to keep it from being quartered, it took much longer than it would have if we had just quartered it properly in the first place. Now, I'm not saying that this is the correct way to make turducken. It's just the way that we ended up with.

Bearing that in mind, let's go through the steps after we deboned all of the birds. With the turkey, you don't have to remove the bones from the wings or the drumsticks; you just have to remove the bones from the body. This isn't bad, and it leaves you with what is essentially a butterflied turkey.

Before you even started the process of deboning these birds, I hope you made some stuffing to stick inside of them. The stuffing should be cooling to room temperature while you debone the birds. We ended up using about 6 cups total. Try to make it a little drier than usual; part of its job is to soak up juices. Go ahead and put down a layer of fully coolled stuffing.

Next up comes the duck. Lay down the duck breast in the middle, and the rest of the meat on the side.

Give it another layer of stuffing.

And then follow up with the chicken, breast in the middle, rest of the meat on the outside.

This is where you use up the last of the stuffing. Don't feel like you have to use it all. The birds are going to be plenty hard to close up anyway, and more stuffing will just make it more difficult.

This part will be interesting. You need to close up the bird, and then keep it closed. At first, this doesn't seem so difficult.

But you'll find that keeping it all together is going to be rough. It'll help to have a second person helping out here, because they'll need to hold the turkey closed while you thread a couple of skewers through the skin.

Next up, roll the turkey over on top of the seam, and move it to a baking pan. You probably can't see it with my baking pan, but I have a rack in the bottom. Also, I would liked to have tied the legs back together, but I couldn't find my butchers twine.

Believe it or not, I think this next part was actually more difficult than building the bird. We had to cook it. We didn't cook it right away, actually. We covered it and put it in the fridge, to cook the next day. We pulled it from the fridge at the same time we started pre-heating the oven, with the hopes that it would come somewhat closer to room temp by the time we stuck it in the oven.

I prepared for this event by watching Alton Brown cook several turkeys, both on Good Eats and on Food Network "All-Star" holiday specials. Stuffing or no, there was always one constant: he did the majority of cooking at 350F. He stated at least once that lower, slower cooking times would contribute to drying out the bird. He also would tend to start the bird at a higher temp, to get some browning going, and then liked to put a foil shield over the breast to keep it from overcooking.

I started by giving the bird a spritz of cooking spray. I knew the oil would aid in proper browning, and that's important. I also fashioned a triangle-shaped shield of foil to cover the breast, before I put it in the oven. The shield would be added later.

I started the cooking process at 400F, to get some decent browning on the bird. Once it looked presentable, I covered the breast with the foil shield and put it back in the oven, at 350F. Since I have also misplaced by probe thermometer, I had to check the internal temp with an instant-read thermometer. My goal was the high 150s, since I knew that carry-over cooking would take it to 165F on the way to my dad's house.

Total cooking time was about 3 hours. Somewhere around the 2 1/2 hour mark, I started to chicken out (no pun intended) and drop the oven temp. It eventually came down to maybe 310F or so. I'm still not convinced this was the right action. When the bird was just shy of 160F, I pulled out and covered it completely with foil. Before I covered it, it looked like this:

When we got it to my dad's house, we let everyone see it before carving it. Then we set about carving it, which should have been a simple task. Since all of the bones are removed from the inside, we should have just been able to cut cross-sections. But since some of the outer bones remained, they initially got in our way. Eventually, I was able to come up with a good cross-section.

Yes, the meat looks a little dry, doesn't it? This probably has to do with my lowering the oven temp, and/or having to open the oven repeatedly to check the temp, since I didn't have a probe thermometer. Fortunately, it wasn't too dry. Just a tad.

I carved up the whole thing, and moved it all to a platter. I kept the drumsticks intact, but laid out the rest in easy-to-serve pieces, nicely garnished with all of the stuffing.

I noticed that the duck was not a gamey as it usually is. I don't know whether this is because it had chicken and turkey juices mixed in with it, or because it was fully-cooked, instead of medium-rare as duck is normally served. I imaging it's a combination of both, but mostly the second. The chicken was decently cooked. Nobody got sick from undercooked stuffing. All in all, I think it turned out better than expected, but it certainly wasn't a professional job. I think next time we do a turducken, I might just order one premade from The South (tm) and see how they did theirs. Still, not a bad first try.

Tutorial: Baklava

2008-12-01T08:23:00.007-07:00

I made some baklava a couple of weeks ago, as a bribe to get my boss to send me to San Francisco. The deal was, if they send me to SF, then the business day immediately following my return, I would bring homemade baklava to work with me.

A few days after presenting my offer, I was presented with my itinerary. Upon my return I picked up some ingredients, got the camera ready, and put together the first baklava that I had made since starting this blog.

I'll admit, baklava looks like a scary thing to make. And unfortunately, few people that make it do it right. Almost every baklava I've ever had has been either too dry or too soggy, and they've also generally been sickeningly sweet. My baklava, as I promised to my boss, is different. It's not too sweet, dry or soggy. It's also not incredibly traditional. Well, screw tradition, I say! This is a dang good baklava!

You may recall a couple of years ago when I make some phyllo cups. If you can handle those, this shouldn't be a problem. You may also want to brush up on making clarified butter. First, the cast of ingredients (a la The Pioneer Woman):

Ingredients

Sugar/Spice Mixture:
1/2 cup sugar
1 tsp cinnamon
1 tsp nutmeg
1/2 tsp cloves
1/2 tsp allspice

Simple Syrup:
3/4 cup sugar
1/4 cup honey
1/4 cup water
juice of half a lemon
juice of half an orange
zest of a lemon
zest of an orange
2 cinnamon sticks, broken in half
6 whole cloves

The Rest of the Baklava:
1 cup pistachios
1 cup pecans
3/4 cup sliced almonds
1/4 to 1/2 lb clarified butter
20 sheets (1/2 lb or 1/2 a box) phyllo dough

To start off, you'll need a few spices, mixed in with some sugar. This is kind of a high-end cinnamon sugar that you can sprinkle later on your toast. In fact, you'll have some left over for that, so make good use of it.

This is also not a bad time to get together the ingredients for a simple syrup. The hardest part is getting some lemon and orange zest ready. Don't bother with a microplane grater on this one; just use a vegetable peeler.

Go ahead and toss the zest in a sauce pan, along with some sugar, honey, water, orange and lemon juice, and whole spices. You can set those aside for now.

Next up, you'll need some nuts. I went with pistachios, pecans and almonds. You can either pulse them in a food processor, or do what I always do and toss them into a zip-top bag and roll them with a rolling pin. The texture is inconsistent, which I actually kind of like.

Now that you have those things out of the way, it's time to move onto the fun stuff: the phyllo dough. You'll want 20 sheets (half a box) of phyllo dough, a stick or two of clarified butter, a 13 x 9 baking dish brushed with clarified butter, and a kitchen towel that is just barely damp.

The towel is to keep the phyllo from drying out while you're working with it. Just between you and me, I don't really use it. I've been working with phyllo for so long that I'm fast enough to go through half a package (that's one whole baklava) without needing it. You'll get there, don't worry. Now go ahead and lay out a sheet of phyllo and brush it with clarified butter.

Repeat until you have 6 sheets laid down. Then go ahead and divide up the nuts into thirds, and sprinkle one third onto the phyllo.

Follow up with a sprinkling of the spice/sugar mixture. I moved mine into a leftover spice jar, to make it easier to sprinkle evenly, and to keep it later for toast. Don't feel like you have to use it all. You just want a thin layer, barely capable of even being called a layer.

Go ahead and lay down another sheet of phyllo and brush with clarified butter. Go easy on this one, since there's a layer of nuts underneath waiting to tear holes in your phyllo. Just between you and me, if you think this part is going to be too hard, feel free to lay out another cutting board and do your layers of phyllo on that, and then move the layers of phyllo on top of the nuts. You'll want four sheets of phyllo on this layer.

Keep going until you have three layers of nuts and four layers of phyllo. The top and bottom layers should be 6 sheets each, and the inner layers should be 4 sheets each. When it's all finished, go ahead and cut it into squares, and then cut the squares into triangles.

Toss it in a 350F oven for about half an hour. While it's baking, you need to cook up the simple syrup. Bring it to a boil, then drop it to a simmer until the baklava comes out of the oven.

Pull the baklava when it reaches a golden-brown color. Mine actually was a little darker than it shows in the photos. Just be careful not to burn it.

Go ahead and pull the large bits (zest and cinnamon) out of the simple syrup with a pair of tongs. Don't worry about the smaller bits, we're going to strain them out. In fact, you might as well just strain them out onto the baklava. The syrup is hot and the pan is hot, so there will be some sizzling. The sizzling will continue for at least a couple of minutes.

Make sure you wait until the baklava cools down before you try serving it. In fact, if you can manage to wait a couple of days, it will actually get even better. At the very least, you should wait overnight. Me, I waited until the sugar was cooled enough not to burn me.

The reviews on this were excellent. I had a couple of coworkers that had previously told me that they hated baklava, tell me that they were now convinced that I was the only person in the world that could actually make good baklava. Several people told me that it definitely wasn't traditional, and everyone liked that it wasn't too sweet. It was a hit with everyone who tried it.

Condensed Pain

2008-11-30T20:08:00.002-07:00

My intention was actually to make another spice blend that I call the Deathcrier, but I was playing around with a few things, and it kind of went in a different direction.

Usually, I mix together whole spices with pre-ground spices, because that's just what I usually have laying around. This time I had whole versions of everything, so I went for it. I also decided to try grinding everything up in my little 2-cup food processor instead of the coffee grinder that I usually use, so that I could make a little more.

Things didn't go as well as I'd hoped. I wanted more heat, so I upped the amount of habaneros. I also discovered that my food processor was good for getting the chiles and spices small enough to fit into a pepper grinder, but not small enough to be called a powder. After 10 to 15 minutes of my fruitless efforts, I eventually gave up and pulled out the coffee grinder. It took 3 batches, but each batch took less than 20 seconds.

At some point, I got some chile in my eye. I got it rinsed out okay, painful though it was for a few minutes. Unfortunately, the water mixing with the chile powder that had gotten on the rest of the left side of my face made half of my face feel like it was on fire. When I finally tasted the chile powder, my mouth was in immediate pain. The first set of words that hit my brain were, "condensed pain". I like it.

Recommended hardware: painters mask, protective goggles for the eyes, rubber gloves, coffee grinder.

Condensed Pain

2 whole New Mexico chiles
3 whole guajillo chiles
6 whole chiles de Arbol
6 whole Japones chiles
3 whole chipotles
12 whole habaneros
1 Tbsp whole all-spice
2 tsp minced dried garlic
1 Tbsp chopped dried onion
1 Tbsp whole achiote
2 tsp whole coriander seed
1 tsp whole cumin seed
1 Tbsp whole black pepper

Utah Chocolate Show 2008 Report

2008-11-23T19:19:00.003-07:00

Well, another chocolate show has come and gone. Even though I've known about this one for months, I still felt like it caught be off-guard. I heard nothing about it from anyone, I just had to keep watching the site, when I remembered. Fortunately, I remembered a few days before the show, and did not miss it. I also only managed to show up for one afternoon, unlike other years where I spent a couple of days there.

I've gotta say, my experience was marred before I even got in the building. I don't know who thought it would be a good idea to do the Gun Show at the same time, but the Sandy Expo Center's parking lot isn't even big enough for one show at a time, much less two. The scary part is, it looked like there was an entire other exhibition hall available, but I could be wrong. The Gun Show might have extended it into it, and I would never have known, because I was only interested in the chocolate.

The show was a little smaller this year than in years past. I attribute this to two things: the wonky economy, and the University of Utah vs BYU game. Sometime around the late afternoon, many of the spectators took off to watch BYU take a thrashing from the Utes. I took off not too long after that, knowing that traffic would be light while everyone was at The Game.

There were a few stars at this year's show that you should know about. As always, Knight Family Honey was around with their amazing honey that comes from (so they told me) a pumpkin field surrounded by an alfalfa field. Okay, so it's not chocolate, but I don't mind. it's fresh, it's raw, and it's darned tasty. Keep in mind, this is coming from somebody that usually doesn't care for honey or its bitter aftertaste. This year they also had honey taffy which was decent, but tasted nothing like honey to me. Still, I liked it and have been snacking on it all day. I have a jar of honey from them too, and I may break that out for a baklava or something.

Tony Caputo's had a booth again this year, and I was glad for it. I'd wondered if Matt Caputo would recognize me, since I haven't really seen him since last year's show, but the moment he saw me he greeted me by name, and asked how things were. He was a bit busy helping other customers, and other employees would often stop by and ask me if I needed help when Matt would have to step away. It was a lot like being in their store, except that they only had their chocolate inventory with them. I was happy to see that they still carry chocolates by Chris Blue over at Chocolatier Blue. I didn't think you could still get that in Utah, since Chris left Utah earlier this year to move to Berkeley and start supplying the French Laundry, in addition to Charlie Trotter's and Chez Panisse. Even if Caputo's didn't have the largest chocolate selection in Utah, I would still stop by there just for Chocolatier Blue.

One of the new stars at the show was a company called Choffy. A few months ago I heard about a company taking coffee beans and treating them like chocolate, going so far as to sell a bar of coffee beans, sugar and cocoa butter. Choffy goes in the other direction. They take the cocoa beans and roast them, grind them and brew them as if they were coffee beans. Personally, I've never liked coffee. I've always thought it tasted and smelled like burned chocolate. That being the case, and considering that people actually like that flavor, I thought it might be a perfect product. At Matt Caputo's urging, I headed over to try some out.

They had three different varieties there. From darkest to lightest, they were Nicaraguan, Ivory Coast, and Ecuadorian. I tried all three, in that order. The Nicaraguan was way too much for me, which didn't surprise the sales rep because I'd told him I wasn't a coffee drinker. Even with cream, I didn't like it. Some sugar might have helped, but by that point I wasn't really interested. Ivory Coast was in the middle, and the show favorite. In fact, by the time I got to the booth, they had some brewed, but none left to sell. This one was drinkable, even without the cream. Still, it wasn't my thing. I decided to try lighter still.

I was surprised at how much I liked the Ecuadorian, until I found out that's what it was. I have always liked beans from there, so it's no surprise that I would like them brewed like this too. It was excellent on its own, but when I added a vanilla-flavored creamer, it got even better. I almost shelled out for a bag right then and there, but I decided to wait a couple of paychecks. Besides, I have no coffee equipment, so that would have been an issue too. Unfortunately, you can't buy their product in stores. They've apparently decided to go with some crappy MLM-like marketing scheme, which would normally mean that I wouldn't buy their product on principle. But you can order it online, so I might break down at some point at pick it up. But if they allowed Caputo's to sell it, I would be even more likely to buy it.

Last of all, Ruth Kendrick, author of the famous Candymaking book by HP Books, made her yearly appearance at the show. As expected, she was teaching a class, this time on tempering chocolate. But this year yielded an unexpected surprise for many fans of hers. Early this year, Ruth took off to the Great White North to take some classes at the Callebaut Chocolate Academy in Montreal. She has taken her years of experience and combined it with her newfound knowledge and formed her own chocolate company, Chocolot, which debuted at this year's show.

While Ruth does sell toffee and candied popcorn, her real jems are her truffles, which represent her entry into the truly high-end world of chocolates. They're as pretty as they are tasty, and are sure to impress. Remember how I said I don't normally like honey? Ruth's chocolates are another exception to the rule. My favorite flavor may very well be the beehive honey. I tried one of the chai teas, and it was also excellent: flavorful, but not overpowering. Also on my favorites list are strawverry sabe, australian ginger and rootbeer float.

Be aware that Ruth is very small-batch, and has limits as to where she can ship (anything out of the state is pretty much out of the question). If you happen to live in Utah or know somebody that does and doesn't mind shipping to you, you need to give her a shout. Her truffles are a little pricey, but the quality is excellent.

I'm a little sad that I was only able to stick around for a few hours this year, but I'm glad that I got at least that. As always, the show was a delight, and I can't wait for next year's. Here's hoping it gets a little more advertising the next time around.

Review: Amano's Jembrana and Cuyagua Bars

2008-11-23T18:46:00.004-07:00

It took some convincing, but I finally talked Art Pollard over at Amano Chocolate into a bar of his new Jembrana chocolate. This is an interesting bar, to be sure. I've been tasting it half a square at a time all weekend. See, that's the thing about Amano: you don't eat a whole bar at once. It would just be a waste of money, and the enjoyment factor would be off.

The beans for this chocolate came out of the island of Jambrana, out in Bali. I understand there aren't a whole lot of beans that come out of Bali, which makes these extra special. Like Art's other chocolates, this doesn't taste like your run-of-the-mill chocolate. It is best consumed only a square at a time.

When you pop a square in your mouth, it starts off with a pretty familiar chocolate flavor. But it takes off pretty quickly, developing into a dark, deep flavor. There are light sour and bitter notes, but no harshness. It's definitely fruity, but not a fresh fruitiness like you get from Amano's bars. It's more developed; the difference between a fresh concord grape, and an aged pinot noir. This bar isn't for kiddies, it's for the big girls and boys.

While I'm talking about Amano, I'd better mention Art's Cuyagua bar. This one came out a while ago, and quite frankly I'm surprised he has any left. They didn't get very many beans for it in the first place, and it's been labelled "Limited Edition" because they don't know if they're ever going to get any more. That's your cue to snap up the last remaining bars before some other guy in New York beats you to it.

This is a single-origin chocolate from Venezuela, from the valley of Cuyagua. It's a lot more accessible than his first two offerings, as the actual chocolate flavor is more pronouced. There is a distinct fruitiness reminiscent of the Madagascar bar. My wife found it to have a very crisp, clean taste, while I picked up a slight bitterness that actually complimented the fruit quite nicely. It has a slightly darker flavor, but not from the overroasting that one might experience in other brands. There is a very light smokiness, that you might not pick up unless you were actually looking for it. It is definitely a bar to be savored slowly, and carefully.

If you are planning a chocolate tasting, this may be a more appropriate bar to include than other Amano varieties. Warning: it will provide a sharp contrast to other brands, and may cause your tasters to be unable to eat conventional chocolate afterwards. If you were to use all four current Amano varieties in your tasting, I think I might recommend Ocumare first, then Cuyagua followed by Jambrana and finally, Madagascar.

Then again, if you're planning on doing such a tasting, I'd suggest you hurry. Like I said, when Cuyagua's gone, it's probably gone for good.

Linkdump 2008-11-22

2008-11-22T07:00:00.003-07:00

Utah Preppers
Safely Gathered In
Chocolot - My friend Ruth just got her chocolate business going, and it looks like she's doing well.