Randomness from the past few weeks ...  

I’ve been pretty busy lately (travel, taxes, work, more travel), so here’s some things I skimmed from my Instapaper and Evernote from the past few weeks that you may find handy or interesting.

Check out an old version of a file from your git repo as a new file

I needed to grab a couple of snapshots of files (here’s how the file looked on date A, date B, and date C). It took me a bit of digging, but the syntax for that is:

git show <sha>:<filename> > <newfilename>

Pretty handy, once you figure it out.

Nate Silver’s new FiveThirtyEight.com launched

Some early highlights:

• Why People Bet on the Favorite
• Toilet Seat Covers: To Use or Not to Use
• Building a Bracket is Hard This Year
• How Statisticians Could Help Find That Missing Plane

If you’re a fan of Nate Silver’s writing, I’ve found the site to be pretty solid so far. It’s still finding it’s legs, I think, but it’s a great extension of what his fivethirtyeight blog used to be.

Speaking of FiveThirtyEight …

Both FiveThirtyEight and Grantland are now hosted on WordPress.com, which means they’re both exposing full text rss feeds. I don’t remember those existing before, and it makes it much easier for me to keep up with articles from both sites.

Finally … OmniFocus 2

I first mentioned signing up for the OmniFocus 2 beta about a year ago. It was in very very rough shape to start with—more of a tech demo than a real beta.

Well, that year as served the app well, as the beta is back. It’s now usable on a daily basis, and the forecast view on the Mac is even more useful than the feature on iOS (because I’m often planning my week at my desk, not on my phone).

There’s still some rough edges (clipping out of Mail didn’t work until I rebooted my Mac, it’s not quite as fast as it probably should be, there’s some UI bits that probably need to be rethought), but it’s really solid and there’s enough that’s goodness that it’s easily worth displacing the venerable OmniFocus 1.

Unintentionally Eating Some Delicious Cookies  

I use a cool little web app called ThinkUp to keep track of stuff I post to Twitter and Facebook. I use the self-hosted version, running on my own server, and I’ve had it running for a year or so and never had a problem.

This weekend, I went to login to see if ThinkUp would show me anything interesting. Except I couldn’t log in. Every time I tried to login, it would just kick me back to the login screen. Clearly something had gone wrong. I watched the login requests via Developer Tools in Safari and Chrome and noticed that I was not getting a PHP session cookie. That’s certainly odd—setting a session cookie is pretty straight forward and I’ve never seen it fail.

As is typical in this sort of issue, I debugged it ass backwards. I spent an hour or so writing test scripts, changing permissions on session directories, and changing session settings before realizing I was debugging things entirely wrong.

My stack looks something like this:

nginx -> varnish -> apache2

I realized that I should start by looking to see if the request to Apache2 was getting the cookie headers back. I ran a quick curl command, and sure enough, the cookie headers were there when talking directly to Apache2. Logically, I then ran the same curl command, changing it to talk to Varnish. Sure enough, the cookie headers were gone.

Finally, I’d figured out where my cookies were getting eaten (haw haw).

Diving into the Varnish config, it was pretty quickly obvious what had happened. When adding Varnish caching to support this here blog, I added this line,

unset beresp.http.set-cookie;

which basically says “get rid of the cookie header we’re sending back to the user”, which allows us to cache more stuff. Of course, that was getting set far too liberally, dropping the PHP session cookie, and making it so I couldn’t login. A couple of tweaks and restart later, and all was well.

This sort of thing happens to me somewhat frequently. I muck around with some settings on my server, and everything works great for my blog or my static site, but I forget about other things I have running, and a few weeks later, I notice they’re broken, but now I have no idea why. It’s a pretty good case for using a tool like doing, to log things I do (that aren’t necessary driven by my OmniFocus to-do list) so that I don’t spend hours debugging my self-inflicted problems.

I couldn't think of a clever title about a hydra. This is a dorky programming post.  

This weekend, I was tooling around with a couple of scripts to do some web benchmarking. Really simply, I just wanted to throw a bunch of requests at a site, see its performance, make a few changes, and do it again. I wrote it out in perl the first time, just shelling out a curl request (not even bothering to use LWP). It was the quick and dirty solution:

for ( 1..10 ) {
    # New host
    my $time = `/usr/bin/curl -o /dev/null -s -w %{time_total} $site/`;
    push( @curr_time, $time );

    # Old host
    $time = `/usr/bin/curl -o /dev/null -s -w %{time_total}  -H'host: $host' $ip{$dc}/$url/`;
    push( @old_time, $time)
}

The whole script (parsing the incoming data, running the curl requests, creating some aggregate data) took about 65 lines with comments.

It was definitely the quick solution to write. But it wasn’t the quick solution to run, as it took the better part of a day to finish running.

On a lark, I remembered that Ruby had a library called Typhoeus that was pretty much designed to do the exact task I was trying to do. And it does it with some concurrency using a queue called Hydra, to help me solve the “quick” side of things. I could have done this in perl with something like AnyEvent, but I figured why not give it a whirl in Ruby.

Took me a bit to get my head back around Ruby, but I was able to crank out the script in about 90 lines (with comments). My first pass through running it (with a 10 threads) it took a little over an hour to rip through the list of sites. In looking at the data, there were a bunch of requests that got back no result, which leads me to believe that maybe I was pushing the concurrency too high. So I dropped that down, and added a retry.

Remember, this is super quick and dirty. If this script proves fruitful, I’ll likely turn it into something reusable, but for now, the guts look like this:

hydra = Typhoeus::Hydra.new(max_concurrency: 5)
50.times do
    request = Typhoeus::Request::new(url)

    request.on_complete do |response|
        if response.code == 0
            puts "Got a 0 response for #{url}"
            unless retried[url] == 1
                puts "Retrying request"
                request = Typhoeus::Request::new(url)
                hydra.queue request
                retried[url] = 1
            end
        elsif response.code == 200
            results_file.write(data[site]["user"] + "," + site + 
                       response.total_time.to_s + "," + response.code.to_s + "\n")      
        end
    end

    puts "queueing request"
    hydra.queue request
end
hydra.run

Basically, we create 50 requests to the same url, queue them up, execute them (up to 5 at a time), and then I write the results out to a file. If I get back a response.code of 0 from the request, I retry it (but only once). If I get back a 200 response, I log it. Otherwise, I just move on.

I’m expecting with retries and the concurrency turned down to 5, that it’ll take a few hours to finish. Typhoeus, with its Hydra queue, is a pretty nifty little framework.

2014 Goals Update  

A few days after the the new year started, I posted about my goals for 2014. I thought that it’d be worth giving an update after a couple of months, as a mechanism to keep myself honest.

Dressing Slightly Better

I think I’ve been achieving this, given that it’s a pretty low bar to hurdle. When I’m out running errands or just hanging around locally, I still tend to dress for comfort (hoodies, tees, jeans). At work, I’ve at least mostly started wearing nicer shirts most days.

It’ll be interesting come spring and summer, when the temperatures give me the desire to wear shorts and tees.

But, for now, I’m doing ok on this one.

Getting Back Into Shape

I’ve been walking more (thanks slightly warmer weather and Celtics home games!), which is definitely helping. More than that, aside from a couple of weeks when the flu was kicking my butt, I’ve been hitting the treadmill a couple of times a week (or more). I’m trying to work myself back into averaging 7 minute miles for a 5k. Right now, I’m not there. I think I can get there over the course of the next couple of months.

When I do this update again, we’ll see where I’m at.

Managing My Time Better

This is where I think I’ve probably done the best. I’ve really been pushing hard to keep my inbox as clear as possible (2 emails right now), get data into OmniFocus and into Evernote so that it’s not cluttering my inbox, and to not spend time aimlessly answering and reading email. I’ve put together something like 20 rules to get useless email out of my inbox, and I add more each week. I’ve been trying to not spend too much time on the computer in the morning, instead using that time to go to the gym, then to start my day fresh, cleaning out my inbox and laying out what’s important for the day.

Next, for me, is to probably try to start doing the paperless thing. I’m thinking very, very hard about getting a ScanSnap, even at the crazy price. The ability to get all of the crazy paper that piles up on my desk (and in drawers, boxes, and closets) into digital format is really, really appealing. And, it should help me continue to steal back a bit time each day.

WordPress Login Security Plugins  

Just some WordPress minutia:

I’d been using Simple Login Lockdown to handle locking IPs out when attempting to many logins. Over the past few months, every now and then I’d notice that I’d get locked out of my own site.

A quick bit of reading turned up that Simple Login Lockdown doesn’t handle being behind a reverse proxy (i.e. Varnish or nginx) particularly well.

A few searches later, and I switched to Limit Login Attempts, which supports running WordPress behind the aforementioned reverse proxies. So far, so good.

I should probably also use something like fail2ban with failed logins, and actually block failed logins at the firewall. Looks like there’s a plugin that does that, WP fail2ban, though I’ve not looked too closely at it. In a real quick overview, it looks pretty nifty.

Always Use Braces (and BSD KNF Style)  

There’s been plenty of coverage of the serious SSL issue that was identified in Apple’s iOS/MacOS stack.

From ImperialViolet, here’s the bug:

if ((err = SSLHashSHA1.update(&hashCtx, &serverRandom)) != 0)
    goto fail;
if ((err = SSLHashSHA1.update(&hashCtx, &signedParams)) != 0)
    goto fail;
    goto fail;
if ((err = SSLHashSHA1.final(&hashCtx, &hashOut)) != 0)
    goto fail;

That second goto fail; is the problem. Since those if clauses aren’t wrapped in braces, if you ever get pat the first two if statements, you’ll always hit the second goto fail;, leading to the error.

On our development team, and in my personal programming, I’m a pretty big believer in wrapping your control statements in braces, and in having the braces on the same line as the control statement. I’m a fan of something close to the BSD KNF Style, using the cuddled else, which looks something like this:

if (foo) {
    stuff;
} else {
    other stuff;
}

The reason I’m a fan of this is two-fold. First, you have braces around your control statements, so you’ll avoid that Apple bug above, and it’s clear what block your code belongs to. Second, and this is why I’m keen on this particular style: you won’t accidentally break up joined control statements.

For example, if you write your code without the cuddled else …

if (foo) {
    stuff;
}
else {
    other stuff;
}

that’s generally no big deal. Until someone isn’t paying attention and breaks up your if/else block, or drops a line of code in there. Now, at best, you’ve got a compilation error. In the worst case, you end up with random code executing (because maybe someone adds a different if statement in between).

Coding styles and guidelines aren’t a replacement for good code review or QA, but they can help to identify or flag issues like this before they slip through.

How do you spell Hadouken anyway?   ⚐

Ever wanted to have a Grover-Dhalsim hybrid fight an Ernie-Ken hybrid via spelling words on the keyboard? All while a 16-bit, hyped up version of the Sesame Street theme plays in the background?

Now you can.

Sesame Street Fighter

Seriously.

(Via Wired Game|Life)

Don't Look a Gift Gazelle in the Mouth  

The iPhone 5S came out at the perfect time, as I had recently dropped my 4S and cracked the screen. I was well past my upgrade date, so the upgrade would be reasonably inexpensive (as iPhones go). And, to top it off, Gazelle had given me an estimate of $80 for my iPhone 4S with a cracked screen.

I went and bought my 5S, got my little Gazelle box in the mail, and, as instructed (really, it’s right in the instructions …), I dropped the box off at the nearest post box. It’s right at the end of my street—pretty convenient.

And that was the last that was ever seen or heard from my iPhone.

After a couple of weeks had gone by, I contacted Gazelle support to ask if they had any news. They pointed me to my local post office. My local post office had no record of ever receiving the package, so they told me to wait a while longer, then file an insurance claim (since Gazelle’s packages are insured!).

So I did that. I’ve talked to Gazelle, and the national USPS, and my local USPS, and round and round.

Yesterday, I found out my insurance claim was rejected by the government.
Today, I found out Gazelle can’t help me because there’s no record of the tracking.

Well, shit.

At this point, I don’t care. The insurance claim was $50, and honestly, I’ve spent more time than its worth trying to get my $50 to prove a point.

I give up, you win, forces of corporate and government inertia. And you win, especially you, dishonest postal service employee who stole my broken phone. [1]

What are the lessons here?

• Gazelle shouldn’t make their boxes conspicuous. Had the box not been so clearly a Gazelle shipping container, it probably gets ignored.
• Gazelle absolutely shouldn’t tell you to drop your package in a post box, when the insurance won’t really apply until it’s tracked by the USPS. Go hand it to the postal service agent by hand.
• I’ll defend the USPS in most cases, but jesus, it’s incredibly obvious what happened and they just don’t seem to give a shit.

In the end, I’m not sure if I’ll use Gazelle again. Maybe I will, but I feel like I got a bit of the run around in this process. They must have dealt with a lost or stolen package before, in this scenario. And the USPS, well, I like my local USPS, but I’ll be handing them things and double-checking tracking from now on, because there’s a scoundrel in their midst.

I do like my new iPhone 5S, though …

(Un)Mistaken Identity  

As the computer dork in the house, I was given the case of “I just bought this book on Amazon, but it’s not showing up in my Kindle app” by Katie. I grabbed her computer and poked around in her account. Even though she had purchased previous books, those weren’t showing up in her order history. Her new book was in her account, but would not show up on her Kindle.

So, I did what any self-respecting dork would do. I logged out and logged back in.

Lo! and behold, the new book happily downloaded to the Kindle.

But the old books were now gone.

If you’re smarter than me, you might have figured out what’s going on by now [1]. But I was still stumped. So I chatted Amazon support. After a couple of minutes with two helpful chat agents, we figured out what it was.

Somehow, Katie had two Amazon accounts with the same email address, that differed only by password.

Well, hell. With some help from the Amazon folks, we got the books all onto one account and got that setup and everything was normal.

But, dear god, in what world would you let the same email address have multiple accounts. How much bad stuff can happen because of that? For one, the one we just talked about, getting content split across multiple accounts.

What happens if I want to change my password, which account does it change? Accounts on Amazon seem to be unique by email plus password, which is such a weird thing. Turns out, this is a pretty common problem.

From a technical/product perspective, this becomes one of those interesting questions: “at what point is the small number of users taking advantage of this feature offset by the customer confusion it causes”. My guess is that there cannot be enough people who intentionally rely on this feature to be worth the negative impact it causes.

Same email, different accounts, poor use of email as identity.

The Wonder of Netflix Streaming (and How Television Watching Will Be Forever Changed)  

This isn’t a groundbreaking opinion, but we’re in the waning days of the television “season”. It may not be for fie or ten years, but the end is nearing. There’s a bunch of reasons why, and many of them are obvious. As technology has advanced, folks don’t want to be tethered to their couches to watch TV—they want to watch a couple of episodes while waiting at the airport, or half an episode waiting in line, or maybe an episode on the laptop at lunch to take a break from the day. Nor, does your average television watcher want to go a year (or often longer) waiting to catch up on the seasons for a show that they missed.

These are all things that are slowly changing with Netflix (and Hulu and each network having its own apps and a million other media that will eventually shake out to a few). Smart networks make sure back episodes and seasons are available for folks who want to catch up. Netflix and Hulu (and cable providers) stream TV and movies to your phone. Much like the early days of MP3s and iPods, the technology is there and, shortly, the networks and content producers will adapt or die.

But, for me, and I think for the television industry, the big change is the ability to make sure that enough episodes, enough content is there for a viewer to get a true feel for the show. Rather than an episode (or two, if you’re lucky), you can put out 6 or 7 (or 13…) episodes and viewers can binge on a few to really decide if they like a show.

The reason that this comes to mind is the show “Chuck”, a show about a nerd who gets a computer stuck in his head and becomes a spy. It was a great show, pretty well reviewed, and ended up lasting five seasons (though mostly because it was on NBC, and it nearly got cancelled every year). Why does this show make me think these changes are imminent and are good for the television industry?

My fiancee.

Had I described “Chuck” to her, I don’t think she would have agreed to watch it. Or, at a minimum, it would have been a struggle. And every week, waiting for another episode, while the show figures itself out, would have increased the chance of her giving up on it.

However, with Netflix, she’s been mainlining “Chuck”. Being able to watch a few episodes in a row, to get the characters and humor, to more easily get the in-jokes and references, it’s a massive difference from when I was watching it live. She adores the show. There are many other shows that have died early, low-rated deaths that I think would have excelled in a Netflix world[1].

At first, networks will struggle. They’ll have to adjust from the shotgun approach to pilots. Instead, picking a few episodes, committing to a half-season, and airing it all at once, or uninterrupted over a few weeks (with episodes online immediately) will allow networks/studios to generate more interest. Of course, this is already happening, at least in part.

When all is said and done, Netflix probably won’t be there. Networks and studios seem to always be behind the times, and they’ll start locking their content down to their own portals (they already are) rather than entirely embracing the new distribution models. But, in the end, television seasons will be much more like the recent “House of Cards” seasons on Netflix than the way “Chuck” was presented on NBC.