Tag Archives: testing

The Paradox of Black Boxes

When I was a young programmer, I was developing an application for a client — it may have been for Windows 3.1 or Mac System 6, I don’t remember — and my code just wouldn’t work. I banged my head against it for quite a while, and just couldn’t see any errors.

So I made an intellectual leap. One of youth and hubris, in part. I concluded that there was a bug in the system’s API. Somehow, in the millions of lines of code that comprise the (then-new) operating system, a bug had been placed that was causing my code to fail.

Incensed, I dashed off a bug report and posted indignantly to a programming community (probably on CompuServe or AppleLink). My unhappiness at the bug was trumped only by my self-righteousness at finding someone else’s.

But of course, you know the end of this story. My indignation and feeling of superiority evaporated quickly when the programming community noted no such error in THEIR code, which took me back to my own — where I found my problem. My happiness at solving the problem was eclipsed by my embarrassment.

This may be an anecdote borne of, as I mentioned, youth and hubris, but it also derives from another phenomenon: the black box. My code was relying on someone else’s code, so when mine didn’t work, it was intuitive to blame the mystery “other” code.

This happens all the time today — likely more than at any time in recent memory — because of the rise of modularized code. It is trivial today to take code libraries from multiple places, call a remote API or two, and package it up as a new application. In fact, in some cases, it increases one’s efficiency by several orders of magnitude.

This is the virtue of the black box. We can drop one in whenever we need specific functionality and we don’t have to do much to make it work.

The difficulty, however, is in provenance. Who knows how that code was written, or how well it works? You can get a sense of the latter often by doing some web searching, but if you are using a narrow-niche kind of code where few use it, or you are using a “black box” in a corner-case/niche-y way, you may not be able to rely on the commons.

So what’s the answer? Surely it’s not to build everything yourself from scratch. If that were true, we’d all be releasing our own versions of iPhone operating systems to go with our applications, and it would be worse than the Android fragmentation! (Kidding, kidding. Mostly.) Rather, it’s to be judicious when using external libraries and APIs, to do so where you can degrade gracefully if something breaks, and to code defensively, handling any errors or anomalies that could involve the foreign code.

Because even though it’s unlikely, it’s possible that next time, it really WILL be the black box code that’s broken, and not my application. And that will make my younger self smile.

Advertising Saturation

I was at a restaurant the other night, and needed to use the restroom. As I was doing so, I could not help but notice that there was an ad in the restroom (for neither the restaurant nor its restroom services). That was a little offputting by itself, but moreso was it’s location: the bottom of the urinal.

Upon further checking (advertising research, not checking bathrooms), it appears this is quite the growth location for advertising — some entrepreneur has even come up with “interactive” urinal advertising. My, oh my.

It got me thinking (no pun intended) about advertising saturation. There has, for many years, been an evolving science of determining when one’s ads have reached as many people as they’re going to. This is the point, it’s theorized, that you can stop spending money on that campaign — to do more reaps diminishing returns.

But what about the saturation to the viewer? It is increasingly difficult to go through a day (let alone a few hours) without seeing ads. Surely this overload of messaging dilutes its effectiveness.

This comes up with our clients regularly. When working on commercial sites that wish to advertise their own wares, or publishing platforms that feature the advertising of sponsors, there are always conversations about placement, frequency, and the balance of content vs. ads. The initial tendency is often to create as many ad placement opportunities as possible — to really “get the message out.”

In our usability and interface testing, though, we find that there is a balance point, after which additional branding and marketing messages simply aren’t seen, and by that time, the effectiveness of the ones that are seen is also diminished. While we haven’t studied the results of our testing over the years, my guess is that the balance point has shifted over the years, as viewers get both more sophisticated and more tired of seeing wall-to-wall selling.

It’s a reinforcement for doing user testing, to be sure. But it’s also an incentive to look at more integrative ways to communicate advertising — and more restrained ways of papering it across the Internet!