PeteSearch

Which iOS versions are Jetpac users running?

Photo by Visual Media

I just hit a nasty bug in the Jetpac iPad app that only seems to affect users on iOS 6.0.1. Unfortunately it seems to be deep in the OS's Facebook integration code, so I wasn't able to put in a very satisfactory fix. Since the problem didn't occur in earlier versions of the OS, and was fixed in later ones, I needed to figure out how much more time I should spend on this bug, versus all the other issues we're wrestling with.

To get a good idea of the user impact, I needed to know how many of our users are on that exact iOS version. Happily we've got quite a robust analytics setup now, so it just took me a couple of minutes to pull that out. It was pretty interesting to see how fast iOS 6 has been adopted, so I thought I'd share it. It's based on a few thousand users from the last couple of weeks, and the sample is for iPad users who like travel apps, but I can't imagine it's too skewed from the general app-installing population:

For the major versions, 86% were on iOS6, 14% were on iOS5, and we've dropped support for iOS4 so we had practically no users there. Here's the details on the minor ones:

6.1.2 0%

6.1.0 39%

6.0.2 5%

6.0.1 38%

6.0.0 5%

5.1.1 12%

5.1.0 0%

5.0.1 1%

5.0.0 0%

4.3.5 0%

4.3.3 0%

4.3.2 0%

For my purposes, this means that 6.0.1 is still fairly strong, and I have more work to do! It's good news that in just five months, 86% of our users have moved over to a version of 6 though, that will definitely make on-going support a lot easier! It's surprising that the 6.1.2 user numbers are so low, but the cut-off time was earlier yesterday, so it may be that few of our users had upgraded by that point.

Five short links

Photo by Martin Fisch

Facial profiling for the detection of mal-intent using thermal imaging - I've been out of the loop on how far image processing has come in detecting emotions. If you think a computer that recognises your face is uncanny, how about one that can tell how you're feeling better than a human?

FakeBelieve - Juicy details on how the amazing photographic paintings of Ransom Mitchell are created. I love how they're hacking reality, with hardly a CPU in sight.

Beauty in the Breakdown - Beautiful writing about a climber's addiction to 'benzo' pills, and the interplay between anxiety, obsession, and all kinds of addiction.

Quandl - It's not a new idea, but this is a good implementation of a search engine for time-series data sets.

Bulk loading data into Redshift - I've been very impressed by what I've seen of Amazon's new database aimed at scalable analytics processing, especially since bulk loading seems to be a first class citizen, rather than something you have to hack on after the fact like the old SimpleDB.

A pub that's also a theater?!?

I love drinking beer, and I love watching plays. I usually have to elbow my way into a crowded bar in intermission to combine the two, which is far from ideal. Imagine my delight when I ran across the concept of the San Francisco Theater Pub! The beer's right there, and so is the play. It's genius!

Tonight was my first chance to sample the format, and I had a great time. The bar had a good selection, including some delicious 101 North stout, with a nice clean Radeberger lager to clean it off. The play itself was an odd duck, just ten lines, translated from the original German of Heiner Müller. They repeated it eight times with different troupes, so I think I have it down from memory:

#1 - Can I lay my heart at your feet?

#2 - Only if you don't dirty my floor.

#1 - My heart is clean.

#2 - That remains to be seen

#1 - I can't get it out.

#2 - Would you like me to help?

#1 - Only if it's not too much trouble.

#2 - It will be my pleasure... I can't get it out either. I'll operate! What else is my pocket knife for? Persevere, don't despair. We have it. It's a brick. Your heart is a brick.

#1 - But it only throbs for you.

Everyone had a good time interpreting it with a five or ten minute performance each. Here's what I remember from each of them:

A cleaning lady and a client, with #1 silent and unwrapping the words from her body, and scrawling the final line on the window of the pub as she left. There was some strong physical comedy, it had obviously been well thought out.

A tango-dancing pair of mute ghosts, who grabbed two audience members and gave them cue card with lines to read as they snapped their fingers. The mimed direction aimed at the hapless audience members was a highlight, especially when they forced one of them to re-read his line with 'more balls'.

One pair of actors who set out to direct two volunteers from the audience. They went through a whole dress-rehearsal, notes, and preview performance with the couple, and both the volunteers threw themselves into the roles.

A country singer versus an opera singer, up against a lady with some swish moves, all doing their lines in rounds, gradually moving to a climax on the final line. The sheer power of the voices together was remarkable.

A boy band version, played by two girls in skater pants doing synchronized dance moves and lip-syncing to a cheesy track in-between their lines. It was surprisingly moving, with some great acting.

A strange accordion player up against a burlesque lady with a power drill and a latin lover. The instrument came in handy, it became part of the conversation.

Three girls who had a well-rounded dance routine, and some film-score music to run the lines against. The coordination and thoughtfulness of the numbers they performed was impressive.

A pre-recorded silent movie version with some impressive acrobatics, recorded in the bar during the day when it was empty. They did a good job with a screen that they set up to look like a thought bubble from one of the actors who was slumped on the bar.

I was never bored! Despite a mime density that would normally have me running screaming from a venue, the evening was very entertaining. Everyone in the audience was very engaged, though I forgot my first rule and ended up in the splash zone in my position at the bar. The players did a great job working in a very limited space, and did some very imaginative work with the strange shapes they had to play with. Nobody was taking themselves too seriously but the actors had obviously put some serious effort in, and it paid off in some touching performances of a very difficult work. I'm looking forward to making it there again for the Taming of the Shrew next month. It will be challenging since misogyny's the backbone of the plot, but I'm confident they can pull it off intelligently!

Things that happen to startup founders

Photo by USACE Europe District

You get into a lot of debt living off an anaemic salary and go bankrupt.

Your spouse breaks up with you.

You are fired from your own company by an outside CEO.

Your company is acquired, but at a level where only the investors get any money back.

Talking with an old startup colleague, I realized how many of our circle had been through rough times. All of these have happened to our close friends. In an abstract sense we always knew the odds, but it's different when you see it playing out in real time with people you care about. Startup failure is a lot easier to deal with as an abstract possibility than a looming likelihood. Until you've been through it, you can't imagine how messy and prolonged the death of a startup can be. The final end of my Mailana was mercifully quick, but the months leading up to it were one of the toughest times of my life.

The Valley doesn't like to talk about failure and pain, but you can't understand how precious the successes are without knowing how hard it can be. I feel extremely lucky and glad to be here at Jetpac now, but some of my friends genuinely regret the time they sank into their startups. Not all entrepreneurial careers have happy endings, and there's a difference between knowing that, and living it.

Five short links

Photo by Spodzone

Why the Open Data movement is a joke - An impassioned rant, but it misses the point when it accuses the movement of cloaking itself in the mantle of progressive politics. Tom seems to expect it to attack 'corporate interests', but it's a single-interest lobby, not a political party with a broad agenda.

Excel horror stories - Read through these to understand the real problems that will emerge from our current love affair with data.

Daft Punks Derezzed/TRON - A fan-based video created using the Motion app I helped build at Apple. Pure nostalgia for me, especially spotting the Kaleidotile filter.

Email transparency at Stripe - I know this makes me odd, but I've loved the idea of opening my email to everyone I work with ever since I ran across JP Rangaswami's use of it at BT.

How to succeed in journalism when you can't afford an internship - The depressing answer in this case is 'inherit some money'. Over the past couple of decades we've succeeded in creating an innovative new barrier to poor people becoming professionals in all sorts of industries.

What's the SF apartment market really like?

Photo by Post Bear

For the last six weeks I've been looking for an apartment in San Francisco. I thought I'd have to elbow through screaming mobs at every showing, but it really hasn't been too bad.

I have to admit that parts of San Francisco have become eye-wateringly expensive. The apartment above me is up for $2,700, when it was $1,700 just two years ago. My girlfriend has a good job, I'm finally earning money again, and we're moving in together, so by pooling what we currently pay for separate apartments we can just about manage $3,500 a month. I sometimes wonder if we're idiots for being willing to pay that much, but we love San Francisco and easy access to the city's delights is worth a lot to us.

I also worry that I'm part of the barbarian horde of technologists driving out true San Franciscans. In my defence I'm here to enjoy the city's crazy culture, not destroy it! Our weekends revolve around the theaters, galleries, concerts, festivals, and readings. Every artist needs an audience, and we're it. We're even regulars at one man shows! I can imagine the frustration of locals who are finding themselves facing rising rents, but the underlying problem is a growing city that's building less than 300 new apartments a year.

The good news (for us) is that even in our preferred neighborhoods of Noe Valley or the Castro there are lots of two bedrooms available. Every weekend we visit two or three open houses, and at many of we're the only ones there. Even when there are multiple people, I've never seen the lines of check-waving Googlers that popular lore had us expecting. Several of the places have been on Craigslist for over a month now, with price drops and one that even began offering two weeks free rent.

It doesn't mean everything is rosy though, we've had some memorable experiences:

- The owner who spent the whole tour with floss in his teeth, and started absent-mindedly flossing a few minutes in. He also explained that the metal front door was very secure, but it didn't really matter because the muggers just wait outside until you open it.

- The landlord who explained he'd installed a coin-operated washer and dryer in the unit to prevent 'laundry parties'. His landlord friends had warned him that tenants would invite their friends over to drink wine and do all their laundry too.

- The posting that announced a compulsory $4,000 'concierge fee' that appeared to result in you being given a map of the neighborhood and some coupons when you moved in. Thankfully nobody took up that offer, and the place is still available two months on, despite the fee being dropped.

- The realtor who gave me the wrong address, arrived 25 minutes late, and then realized she'd lost the keys and couldn't let me in.

- The flat with a garden full of carefully stacked jars of mysterious yellow liquid.

It might just be because I grew up with the awful London rental market, but San Francisco only feels normally terrible right now. Rents have got to the point where they're so out of reach, they've peaked. If you're lucky enough to be able to pay the prices, or can widen your search to less desirable neighborhoods like the Sunset, don't be scared off from looking around. You might be surprised what you can find!

How to track iOS memory crashes

Photo by Fingle

I love being able to use HTML5 content within Jetpac, but hosting it in Apple's UIWebView component can use a lot of memory. That matters because iOS apps crash when they run out of memory, and to make things worse they crash so hard that you don't even get a report! The process is killed, even low-level exit handlers don't get run, our code is shut down with no chance to do anything.

We do sometimes see low memory warnings, but these aren't as useful as you might think. They can occur in fairly benign circumstances and be cured by Javascript running a round of garbage collection, which means they aren't great predictors of crashes. They also don't always fire before there's a memory exhaustion crash, so we can't rely on taking preventative measures in those handlers.

To help understand what's going on, I've added low-level OS memory tracking instrumentation to help us track the free memory situation over time. I've combined it with our home-brewed Javascript function tracing to get quite a fine-grained view of which operations are using the most space, and we found some fascinating issues, like simple Canvas drawing operations appearing to leak a full image's worth of memory every time! 

It's important for us to understand how widespread the crashes are in the wild though, and without crash reports we can't keep track of how well we're doing with our fixes. I've been talking to the folks at Crittercism, who we use and love for our general crash reporting, and they don't yet have a solution, but I did have a brain wave that I'm trying.

We have no chance to run code if the app crashes hard, but we do if the user deliberately quits by pressing the home button. We have an in-house activity log server, so if we fire off an event when a user starts up the app, and one when they deliberately close it, we can estimate how many times it crashed. We get great reports from Crittercism for normal crashes, so by subtracting those we can figure out roughly how many users are affected by this. The numbers will be a bit biased by lost connections (since we need to communicate with our log server to record an app close), but it will be far better than nothing.

I've submitted a new version of the app with the logging included, so I should have a better idea of how this works in practice within the next few weeks. Here's hoping it helps!

Five short links

Mural by Monte Thrasher

Heads by Monte Thrasher - Normally my short link images are side-notes, but the pentagonal helmet image led me to discover what I think is my favorite mural ever. Check out Twiggy, the world's ugliest dog, and the inflatable skull, and much more.

Stately - US state outlines as a font, in their correct positions. I love the explosion of font hacks we've seen recently!

Carmen Geocoder - The MapBox folks have done some great detective work to find open data sources for this project. I'm looking forward to seeing how this might work together with the Data Science Toolkit.

R Language interface to the DSTK - On that topic, it was great to see Ryan Elmore release an interface to the toolkit in R.

Big Data - Beyond the Hype - An opinionated and thought-provoking exploration of where the data world is headed.

The dignity of customer service

Photo by Ricky Brigante

When I started my first job as a supermarket clerk, I dreaded going in. I needed the money to feed my weed and D&D habit though, so I gritted my teeth and dragged myself to work every Saturday. After a few weeks something strange happened - I found myself enjoying my time behind the checkout!

I'd grown up with the idea of any kind of service job as shameful and embarassing. In Britain, there were still a lot of holdover attitudes from the Downton Abbey days of servants and lords. Somebody working in a job where you had to do things for people off the street risked losing face and descended down the class hierarchy. The usual defense was surliness; "You're looking down on me? I'll show you I'm not at your beck and call!".

I was lucky enough to be at Tescos, which in the early 90's was the scrappy upstart of the supermarket world, and their killer advantage was a different approach to customer service. As my supervisor patiently explained to me:

"The women you'll see have just spent an hour dragging screaming toddlers around the store after a full day of work. They'll likely be in a foul mood by the time they reach your checkout, but don't take it personally. They're so focused on their own worries, they don't even see you! If they lash out, just listen to them patiently, and let them know they're being heard. They take their cues off you, so if you react calmly instead of being upset, most of them calm down too. And if you take the high ground and smile sweetly, you don't give the few nasty ones the satisfaction of getting to you. This is the hardest part of your job, so take pride in doing it well."

It was never an easy job but I found there was a real dignity in it, once I treated good customer service as something I could take internal pride in, rather than being shameful and servile.

I've been thinking about this a lot after reading Andrew Sullivan's posts on the subject. He's made the same journey from Britain to the US, and shares my joy in the American dedication to good customer service. A lot of his readers don't agree, complaining that it's soul-violating to be forced to "treat callers like royalty". I'm not trying to romanticize any entry-level job, but a professional attitude to customer interactions was the best armor I had against the emotional assaults that the general public inflicts. It gave me good boundaries so I could approach awkward customers as a problem to be handled, not a reflection on my own self-worth.

This became especially important once I moved to Manchester, and to pay for college spent a year working at an infamous chain called Kwik Save. It was known for its "No Frills" brand, and everything about the store lived up to the slogan. Even box cutters were precious items given to honored employees, while the rest of us improvised using keys or pens to poke through the sticky tape. Management also embraced a distinctly old-fashioned approach to customer service - "The people who shop here are scum, we have to treat them like scum" as the manager Mr Albinson put it!

I kept to my Tesco training as much as I could, but I found being in an environment of bad customer service was far more soul-destroying than my old job. Employees would argue and even yell at shoppers, and they'd get sucked into all sorts of petty disputes. Everyone left work a lot more upset than they ever did at Tescos. Losing the shield of professionalism made the inevitable friction with customers far more soul-destroying than it had to be.

That all means that articles like Timothy Noah's leave me hopping mad. A close member of my family is a long-time Pret employee, with multiple awards for great customer service (which are good chunks of cash, and shared with the whole team), and he's not been brainwashed by a cultish corporation. He's a good guy working a tough job, and part of it is treating customers extremely well. Sure, Pret employees may not be allowed to have an off day, but only in the same way that they aren't allowed to drop the sandwiches on the floor. Doctors, lawyers, and anyone who has to deal with the public has a work persona they need to adopt to effectively do their job. It's patronizing to assume that clerks and servers aren't making the same kind of tradeoffs as people in more prestigious professions.

Expecting everyone who wants to give you money to deal with your emotional baggage is a luxury few of us can afford. There are a lot of genuine problems out there, like the awful working conditions that many service employees have to suffer, but most companies that care about employees appearing happy have figured out that treating them decently is a big help. Don't take away the dignity of great customer service givers by assuming they're silently suffering as they smile, and need your protection. They're secure on their side of the professional barrier, and the most helpful thing you can do is give them the respect they deserve.

How do analytics really work at a small startup?

I was lucky enough to spend a few hours today with my friend Kevin Gates, one of the creators of Google's internal business intelligence systems, and it turned out to be a very thought-provoking chat. His mind was somewhat boggled that we were so data-obsessed at such an early stage in our life. Most people running analytics work at a large company and have a big stream of users to run experiments on. Our sample sizes are much smaller, which makes even conceptually simple approaches like A/B tests problematic. Just waiting long enough to get a statistically-significant results becomes a big bottleneck.

We've found ways around a lot of the technical issues, for example focusing on pre/post testing rather than A/B to speed up the process, but there's a bigger philosophical question. Is it even worth focusing on data when you only have tens of thousands of users?

The key for us is that we're using the information we get primarily for decision-making (should we build out feature X?) rather than optimization (how can we improve feature X?). Our quest is to understand what users are doing and what they want. Everything we're looking at should be actionable, should answer a product question we're wrestling with. To help answer that, I sketched out a diagram of how the information flows through our tools to the team:

The silhouettes show where people are looking at the results of our data crunching. The primary things that everyone on our team religiously watches are the daily report emails, and the UserTesting.com videos that show ordinary people using new features of our app. The daily reports are built on top of our analytics database, which is a Postgres machine with a homebrewed web UI to create, store, and regularly run reports on the event logs it holds. We built this when our requirements expanded beyond KissMetrics more funnel-focused UI, but we still use their web interface for some of our needs. Qualaroo is an awesome offshoot of KissMetrics that we use for in-app surveys, and we also refer to MailChimp's Mandrill dashboard and Urban Airship's statistics to understand how well our emails and push notifications are working. We have to use AppAnnie to keep track of our iOS download numbers and reviews over time.

We also have about twenty key statistics that we automatically add to a 'State of the App' Google Docs spreadsheet every day. This isn't something we constantly refer to, but it is crucial when we want to understand trends over weeks or months.

Over the last 18 months we've experimented with a lot of different approaches and sources of data, but these are the ones that have proved their worth in practice. It doesn't look the same as a large company approach to analytics, but this flow has been incredibly useful in our startup environment. It has helped us to make better and faster decisions, and most importantly spot opportunities we'd never have seen otherwise. If you're a small company and are feel like you're too early to start on analytics, you may be surprised by how easy it is to get started and how much you get out of it. Give simple services like KissMetrics a try, and I bet you'll end up hooked!