The Bay Area doesn’t deserve Caltrain

It deserves something much, much better.

I live in San Francisco and work in downtown Mountain View. I don’t enjoy driving too much and so, I’m grateful that I’m able to take public transportation to work. A smaller commute would be nice, but it is not bad at all.

That is, until Caltrain fails. Again. And again. Note that I’m not blaming Caltrain for the fatalities, but I am complaining about how they respond to such events. It took me almost 3 hours to get home last night. I went though something similar a few months ago. Unfortunately, this is only one of the problems with Caltrain.

We pay our taxes; I pay more than enough for the monthly pass. We deserve something better. Here are just a few things that are “broken” (notwithstanding Caltrain’s well publicized financial troubles)

  • Ancient hardware: this is the Silicon Valley; the so-called center of the tech universe; the birth place of many a great technology companies. Our public transportation should be leading the rest of the country and indeed the world. Instead, we are stuck with decades old diesel-powered engines and several outdated coaches. On more than one occasion, my ride was interrupted due to “mechanical failures” and one evening, the train simply shut down at Menlo Park and we had to be transferred to the next train. BART has its share of problems, but it beats Caltrain any day — it goes under the damn Bay!! Plans for high-speed rail and electrification remain just that — plans.
  • Ancient protocol: post any fatality, Caltrain seems like a headless chicken. As it is, there’s no official way to track the trains or get status updates. Riders have graciously setup a Twitter feed for posting updates. When an accident happens, there’s no authoritative communication channel — no number to call, no real-time updates. Hell, even most officials present on site have no clue what is going on! There’s a PA system on some of the stations but it is largely useless. What I expect? The ability to track the location of each train on a map, in real-time; a social media strategy that is able to communicate in a timely and effective manner; a customer education strategy so we know what to expect and what to do when a fatality happen (and they do seem to happen with eery regularity).
  • Ancient software: Caltrain remains a “proof of payment” system — that is, unlike most well-functioning public transit systems around the world, Caltrain requires travelers to purchase a ticket and carry a proof of purchase for the duration of the journey. While this is a feasible approach (the Indian railways does something similar), it starts breaking down quickly for a metropolitan area where most people won’t be riding for more than 30 minutes. It is harder to keep track of payments; it is error prone in that people may forget to buy tickets (so Caltrain loses money); it introduces more humans in the equation (conductors etc). To make things worse, Caltrain doesn’t sell tickets onboard. Until recently, this proof-of-payment happened to be an actual piece of paper. After several millions of dollars and some failed pilots, Caltrain finally transitioned to the Clipper Card a few months ago.

You’d imagine that with the Clipper Card in place, things would be smooth. Well they are smoother, but the system remains extremely un-user friendly. It is as-if they intend to confuse riders. For instance, you are supposed to tag-on before getting on the train and tag-off, except if you have a monthly pass, in which case you are supposed to tag-on and tag-off exactly once, that too, on your first ride of the month. That’s not all — even though I have a monthly pass (so I’ve already paid $170+ on the card), I’m still required to maintain a $1.25 cash balance on the card at all times.

Imagine you are a visiter to San Francisco. How confusing would all this seem to you? Sure there’s rationale for everything, but I’m sure if you asked a couple of smart people to think about this for a few days, they could come up with a better solution.

Arghhhh!

Some thoughts on iCloud

Sorry, all the sensationalist headlines were taken, so I had to pick something boring.

As we all know by now (read: probably 1% of the world’s population), at WWDC earlier this week, Apple spilled the beans on the upcoming iCloud, among other things. In this post, I wanted to share some of my thoughts on the much hyped iCloud (not that there is any dearth of opinions and articles on the subject, thanks to the echo-chamber that is Twitterverse and Blogosphere)

iCloud

First off, some quick bullets summarizing what it is:

  • iCloud aims to make cloud storage painless, the idea being that your data should be available to you from all your devices, all the time.
  • It’s automatic and transparent. Apple is baking iCloud support deep into 9 different applications: iTunes, Photo Stream, Apps, Books, Documents, Backup, Contacts, Calendar and Mail. And that’s just the beginning.
  • It’s free. Upto 5GB — excluding purchased music, books, apps and photo stream.
  • Sync over the air: iCloud can sync across devices over wireless. As a concrete example, you’ll no longer need a cable to sync and backup your iPhone with your laptop.

Here are some cool things about iCloud:

  • Scan and skip upload (iTunes only): when dealing with large data sets (such as your movies and music collection), one of the main impediments to using cloud storage is the overhead of doing the initial import. With a 1Mbps uplink, a 10GB music collection will take a full day to upload. Of course, if the file you are trying to upload already exists somewhere in the cloud, you don’t need to upload it and this is exactly what iCloud does. Because of the iTunes store, Apple already has a library of 18 million songs (and counting) and detecting if two files are for the same song is a lot easier than for many other media types (say images or movies).
  • Storage APIs for developers: APIs are all the rage these days. By exposing the right set of APIs, Apple could attract developers to build iCloud functionality on other platforms (Android, for example). Unfortunately, the API is fairly limited at this point (key-value store or documents).
  • HP, Teradata, maybe EMC are rumored to have supplied bulk of the hardware in the spanking new datacenter that will be the backbone for iCloud.
  • Despite all the hoopla around “cloud” recently, it was still grounded firmly within the tech circles. Apple has the ability, experience and motivation to take cloud computing truly mainstream with iCloud.

What is NOT so cool:

  • Apple has a habit of exaggerating the novelty and efficacy of their features (remember Spaces?) Scan and skip upload is nothing new: it is just deduplication under the wraps — a well known technique in storage systems. Videos and photos will still have to be uploaded though — there’s no real shortcut for those. Of course, there are techniques to dedup arbitrary data and I hope Apple is leveraging them.
  • In the same vein, syncing of Mail, Calendar and Contacts is just catch up. Ever used Google? Likewise for Docs and Books. The delivery model is different — Apple apps work with the local data and sync when there’s connectivity. They haven’t touched upon conflict resolution, disconnected clients etc.
  • Implications for Dropbox: transparent, automatic sync across multiple devices is a phenomenally hard problem. Apple makes it sound like they’ve nailed it. It took Dropbox several years to address all the performance and security concerns. I’d wager Apple will run into its share of snags along the way.
  • Apples all the way: despite their claims, iCloud is designed to lock you in. Sure you may be able to leverage some of the features by installing additional software on a PC. But unless you are using an Apple device, you won’t get the full experience or service. Want your “reading list” available on Android (or Chome, for that matter)? Tough luck. Want your music available to other music players (open source players like Banshee and Amarok, god forbid)? How about your photo stream in Picasa?

Finally, there’s no doubt that iCloud will drastically alter the cloud landscape. However, Apple is focused mainly on the personal cloud — which is a good thing, they are playing to their strengths. It is also a great opportunity because the enterprise cloud market is still wide open. The requirements, challenges and “killer apps” in that market are very very different than the personal/consumer cloud market. Should be fun!

Indian Sweets in the Kitchen

JohnC wrote a cute post over on his blog that filled me with nostalgia. I figured I’d respond with a counter-point and shamelessly stole the title from the original post :) So go read his post first.

Welcome back. I’ve been meaning to write a series of posts about my startup experience(s) and John’s post is good inspiration. If nothing else, I’ll try to post something to supplement or respond to his posts.

Back to Kaju Barfi (also known as Kaju Katli by many). First, a picture:

Kaju Barfi
Image Courtesy: slimwithyoga.com

John makes a good point about being in a diverse environment surrounded by people who come from different cultures and backgrounds. I have seen myself grow personally and intellectually in similar situations. While I agree that US immigration law needs serious reform, let there be no doubt that there are very few countries that are as immigrant-friendly as the United States. Countless people from all over the world have come to the US, made it their home and contributed to all walks of society. I can not imagine Americans having the same kind of success in India as Indians have had in the US.

Another experience probably most Indians in the US would share is this: only when you are outside India do you realize how little you know about your country. Over the years I’ve been asked all sorts of questions about India — from naive ones about elephants and snake-charmers to hard-hitting ones about religion, freedom and corruption.

While we are on the subject of Kaju Barfi, do you see the silver material coating the surface of the barfi’s in the picture above? Here’s an advice — do NOT microwave that thing! Or any other Indian sweet that has the silver foil. It is a question that comes up often: what is it? what is it’s purpose? The silver foil is commonly called “vark” in India and yes, traditionally it is meant to be a super-thin silver foil. It servers no particular purpose other than to give a grandiose look to the sweets — it is edible and does not modify the taste of the sweets.

You may worry about consuming metal with your sweets and if you think that many people likely don’t use silver anymore, you’d probably be right. But relative to the challenges our world faces, I’d say its a minor concern. Hundreds of millions who are eating the silver foiled sweets daily are doing just fine.

John, I’ll bring you a box next time I’m in India!

The silent victories of open source

Tux, the Linux penguin
Image via Wikipedia

For years, free/libre/open source software (henceforth referred to as FLOSS) have proclaimed, year after year, how that year is the year of Linux, or the year that open source will become mainstream, or the year that open source will finally take off etc. But it never has, at least traditionally speaking. Linux based desktops haven’t penetrated either the enterprise or consumer markets; with a few notable exceptions (Apache httpd, for instance), most FLOSS products — be it office software like OpenOffice, multimedia software such as Gimp or Inkscape — remain popular with economically insignificant niches. And yet, this year, more than ever before, open source forges ahead with its silent victories.

Consider the following shifts:

  • all the top brands of the day — Apple, Google, Facebook, Twitter, Amazon — they ALLstand tall on the shoulders of FLOSS giants.
  • Contributing software back to the open source community is becoming increasingly common, even expected. Take a look at the GitHub repositories of Twitter and Facebook, or the various Google projects. In fact, when screening engineering candidates, I often look for and encourage people to talk about their open source contributions.
  • Most of the activity around “big data” and “cloud computing” is being driven in large part by FLOSS, whether it is the Hadoop-powered ecosystem or the Xen/Linux powered Amazon Web Services.
  • Given the current smartphone landscape, it is highly likely that Android will become ubiquitous on tablet devices and a variety of consumer smart phones. Already, Android has more search mindshare than Linux, despite the fact that Linux is part of the Android stack.
  • If you start a software company today, I would bet that you will find yourself bootstrapping almost entirely using open source software. The entire development process — from the GCC compiler toolchain, to the build systems, to the scripting languages, to the version control systems, to the code review systems, to the continuous integration systems — everything is dominated by FLOSS products. Good bug trackers and enterprise Wikis are the last bastions but it is just a matter of time.

I’ve had a chance to see the enterprise software market up close and increasingly find more and more open source everywhere I look. FLOSS has not arrived, it has taken over.

Disposable Film Festival

What is the Disposable Film Festival? From the horse’s mouth:

Selected by MovieMaker Magazine as one America’s “coolest film festivals,” the Disposable Film Festival was created in 2007 by Eric Slatkin and Carlton Evans to celebrate the creative potential of disposable video: short films made on everyday equipment like cell phones, pocket cameras, and other inexpensive video capture devices.

DFF

Surabhi is one of the finalists in the Competitive Shorts Program. The opening night event is going to take place at the historic Castro Theater in San Francisco on March 24th. The theater can seat close to 1,500 people, but the groupon is already sold out (thats 700 seats gone) and the show is likely to sell out very soon. You can still get your tickets over at Brown Paper Tickets. The exciting line-up consists of 25 super-short films from all over the world.

Finally, here’s the promo video to pique your interest: