Category: Featured

The Bay Area doesn’t deserve Caltrain

It deserves something much, much better.

I live in San Francisco and work in downtown Mountain View. I don’t enjoy driving too much and so, I’m grateful that I’m able to take public transportation to work. A smaller commute would be nice, but it is not bad at all.

That is, until Caltrain fails. Again. And again. Note that I’m not blaming Caltrain for the fatalities, but I am complaining about how they respond to such events. It took me almost 3 hours to get home last night. I went though something similar a few months ago. Unfortunately, this is only one of the problems with Caltrain.

We pay our taxes; I pay more than enough for the monthly pass. We deserve something better. Here are just a few things that are “broken” (notwithstanding Caltrain’s well publicized financial troubles)

  • Ancient hardware: this is the Silicon Valley; the so-called center of the tech universe; the birth place of many a great technology companies. Our public transportation should be leading the rest of the country and indeed the world. Instead, we are stuck with decades old diesel-powered engines and several outdated coaches. On more than one occasion, my ride was interrupted due to “mechanical failures” and one evening, the train simply shut down at Menlo Park and we had to be transferred to the next train. BART has its share of problems, but it beats Caltrain any day — it goes under the damn Bay!! Plans for high-speed rail and electrification remain just that — plans.
  • Ancient protocol: post any fatality, Caltrain seems like a headless chicken. As it is, there’s no official way to track the trains or get status updates. Riders have graciously setup a Twitter feed for posting updates. When an accident happens, there’s no authoritative communication channel — no number to call, no real-time updates. Hell, even most officials present on site have no clue what is going on! There’s a PA system on some of the stations but it is largely useless. What I expect? The ability to track the location of each train on a map, in real-time; a social media strategy that is able to communicate in a timely and effective manner; a customer education strategy so we know what to expect and what to do when a fatality happen (and they do seem to happen with eery regularity).
  • Ancient software: Caltrain remains a “proof of payment” system — that is, unlike most well-functioning public transit systems around the world, Caltrain requires travelers to purchase a ticket and carry a proof of purchase for the duration of the journey. While this is a feasible approach (the Indian railways does something similar), it starts breaking down quickly for a metropolitan area where most people won’t be riding for more than 30 minutes. It is harder to keep track of payments; it is error prone in that people may forget to buy tickets (so Caltrain loses money); it introduces more humans in the equation (conductors etc). To make things worse, Caltrain doesn’t sell tickets onboard. Until recently, this proof-of-payment happened to be an actual piece of paper. After several millions of dollars and some failed pilots, Caltrain finally transitioned to the Clipper Card a few months ago.

You’d imagine that with the Clipper Card in place, things would be smooth. Well they are smoother, but the system remains extremely un-user friendly. It is as-if they intend to confuse riders. For instance, you are supposed to tag-on before getting on the train and tag-off, except if you have a monthly pass, in which case you are supposed to tag-on and tag-off exactly once, that too, on your first ride of the month. That’s not all — even though I have a monthly pass (so I’ve already paid $170+ on the card), I’m still required to maintain a $1.25 cash balance on the card at all times.

Imagine you are a visiter to San Francisco. How confusing would all this seem to you? Sure there’s rationale for everything, but I’m sure if you asked a couple of smart people to think about this for a few days, they could come up with a better solution.


The silent victories of open source

Tux, the Linux penguin
Image via Wikipedia

For years, free/libre/open source software (henceforth referred to as FLOSS) have proclaimed, year after year, how that year is the year of Linux, or the year that open source will become mainstream, or the year that open source will finally take off etc. But it never has, at least traditionally speaking. Linux based desktops haven’t penetrated either the enterprise or consumer markets; with a few notable exceptions (Apache httpd, for instance), most FLOSS products — be it office software like OpenOffice, multimedia software such as Gimp or Inkscape — remain popular with economically insignificant niches. And yet, this year, more than ever before, open source forges ahead with its silent victories.

Consider the following shifts:

  • all the top brands of the day — Apple, Google, Facebook, Twitter, Amazon — they ALLstand tall on the shoulders of FLOSS giants.
  • Contributing software back to the open source community is becoming increasingly common, even expected. Take a look at the GitHub repositories of Twitter and Facebook, or the various Google projects. In fact, when screening engineering candidates, I often look for and encourage people to talk about their open source contributions.
  • Most of the activity around “big data” and “cloud computing” is being driven in large part by FLOSS, whether it is the Hadoop-powered ecosystem or the Xen/Linux powered Amazon Web Services.
  • Given the current smartphone landscape, it is highly likely that Android will become ubiquitous on tablet devices and a variety of consumer smart phones. Already, Android has more search mindshare than Linux, despite the fact that Linux is part of the Android stack.
  • If you start a software company today, I would bet that you will find yourself bootstrapping almost entirely using open source software. The entire development process — from the GCC compiler toolchain, to the build systems, to the scripting languages, to the version control systems, to the code review systems, to the continuous integration systems — everything is dominated by FLOSS products. Good bug trackers and enterprise Wikis are the last bastions but it is just a matter of time.

I’ve had a chance to see the enterprise software market up close and increasingly find more and more open source everywhere I look. FLOSS has not arrived, it has taken over.

How do you use Twitter/Buzz/Facebook?

No no, I’m not late to the party and I’m not asking literally how does one use the above mentioned services. Rather, I’m asking how does one put these various services to use. When do you post something on Twitter but not on Buzz, Facebook but not on Twitter; or do you post everything everywhere ( style)? I’m not a heavy hitter by any means and my usage of social networks is mediocre at best. Yet I myself confounded with all of the various services and their accompanying warts and virtues. Don’t you?

To help sort out my thoughts, I drew a picture (don’t you dare judge me for my lack of creativity!):


Below I elaborate more on how I currently use each of the services.


  • I tend to use it for technical and/or non-personal content. Things that I would want to publicize.
  • Unlike Buzz/Facebook, I don’t pay too much attention to who is following me. Most tweets are public anyways.
  • The 140 character limit is sometimes amusing, but often irritating. Are people still using regular SMS with Twitter?
  • Multiple startups devoted to managing Twitter “noise” is not encouraging.
  • @ replies are bandaid. Twitter is a broadcast-and-forget medium — I can’t have (or follow) a conversation on it.


  • Use it for sharing random, personal updates (or things I find interesting :p)
  • Mostly on because of network effect (read: don’t want to be left off the social bandwagon).
  • Like that I can “Like” most things and actually follow the conversation via comments.
  • Always worried if my privacy settings are working and if there’s a new “default” I need to worry about.
  • Pay more attention to who I friend. The noise level is still quite high despite that.


  • Usage domain similar to that of Facebook. Unlike Facebook, can choose to make posts Public.
  • Love the email integration. Conversely, API/clients still have to catch up to Twitter.
  • Supports likes, comments and “resharing”.
  • Privacy is modeled around my contacts (chat or otherwise), which seems natural.

I’m fine with using Twitter for all of my public posts. The main confusion lies between Buzz and Facebook. Facebook obviously has more social traction. That said, Buzz is just more convenient to use (because of the email integration mostly). Of course, all of the various connectors available (Twitter <-> Buzz, Twitter <-> Facebook, multicast via or Chromedeck etc) make the whole thing even more confusing. At the end of the day, I might just go back to not using anything on a regular basis.

How are you using Twitter, Buzz and Facebook?

Startup Infrastructure: Where Linux Fails

Category:WikiProject Cryptography participants
Image via Wikipedia

It is no secret that I’m an open source evangelist and so when it was time to set up internal infrastructure at work, naturally the first order of business was to evaluate the various OSS projects out there — everything from wikis, bug trackers, source control, code review and project management. Running Ubuntu LTS (10.04) on all of our servers was a no-brainer and there were plenty of excellent options for most everything else as well (a follow-up post on our final choices later). The Linux ecosystem is fabulous for most of the infrastructure needs of a startup, but I learnt the hard way that there are still some areas where Linux needs a lot of work before it can become competitive with proprietary, non-Linux solutions.


Centralized account management (users and groups) and authentication is critical component in any IT deployment, no matter the size. Even for a small startup, creating users/groups repeatedly for each new server, separate authentication mechanisms for each new service is simply not scalable. That is precisely why Active Directory is so ubiquitous at enterprises.

LDAP was the obvious solution in Linux-land and I figured it would be trivial to setup an OpenLDAP server that can manage user/group information for us. It would also be the single authentication source for all servers and services. I was so wrong.

After struggling with OpenLDAP for several painful hours, I gave up — the documentation is fragmented, Google doesn’t help much and personally I think the LDAP creators had never heard of “usability” when designing it. The seemingly simple task of creating some new users and groups involved several black-magic incantations of the LDAP command line tools. Getting servers to authenticate against the resulting directory was even harder.

Just as I was about to throw in the towel and setup an AD instance in-house, I stumbled upon the 389 Directory Server (now known as the Fedora Directory Server). With a new found hope, I set about installing it on Ubuntu and hit another roadblock — there are no up-to-date packages of FDS for Ubuntu. Reluctantly, I setup a Fedora instance (the only one so far) and installed FDS. Thankfully, Red Hat has put together really comprehensive documentation and guides for the Directory Server, which was invaluable.

From there on, it was mostly downhill (only a few minor hiccups). Finally we have a nice GUI to manage users and groups, and all servers/services authenticate against a single Directory Server. But the journey was unnecessarily painful. Here’s what I’d like to see:

  • Up-to-date packages of FDS for Ubuntu. Sane defaults and functionality out-of-the-box
  • Ready to consume documentation on how to integrate LDAP with various web applications, Linux distros etc (I’ll put together some of this soon)
  • More awareness — I should have found FDS a lot sooner than I did, but it is certainly not very well marketed
  • Single sign on: This is a whole different beast

Remote Access

At my previous company, we had a Cisco VPN solution. There were plenty of Cisco compatible VPN clients on Windows and Mac. In fairness, it was relatively easy to get vpnc working on Ubuntu as well. In fact, with Network Manager, you can manage your VPN connections using a simple and intuitive UI. But the setup was not very reliable and my connections would get dropped relatively frequently. It was impossible to have a long-running VPN session without disruption. I’m not sure if the problem was with the Cisco hardware or the Ubuntu vpnc client; I did see similar issues with the built-in VPN client on Mac OS X.

But at least VPN on Linux works. I can’t say the same about other remote access mechanisms, in particular IPSec and L2TP over IPSec. It took me some time to figure out which package to use (Strongswan, Openswan, iked etc etc); another couple of hours to get the Openswan configuration just right; several hours of struggling to automatically setup DNS lookups when using the IPSec connection (gave up and ended up using entries in /etc/hosts!). There is no UI in Network Manager to manage IPSec connections either. Strongswan does have a NM plugin, but that only works for IKEv2 (certificate based authentication), while I had to use IKEv1 (shared key based authentication).

At the end of the day, I do have a working IPSec tunnel and it is definitely more reliable than the Cisco VPN (been up for more than 2 days without disruption). But all this can and should become a lot more seamless.

These are a few areas where Linux failed me in setting up the infrastructure for a startup; it shines most everywhere else. Hopefully these last few kinks will get ironed out soon.

The San Francisco Taiko Dojo

My wife and I had been thinking about learning Taiko, so after some quick Googling, one fine Tuesday we dropped in at the San Francisco Taiko Dojo to “observe” the adult beginners class. We only stayed the first hour or so, and it was interesting to say the least. First, there was the intimidating workout: everyone was counting in Japanese; the workout included sets of 60 pushup, situps, scissor kicks and tricep dips! And then there was the class itself — there seemed to be no “orientation” for beginners or a structured way to learn the ropes; everyone there just seemed to know what they were doing; there seemed to be a lot of understood etiquettes — there was an expected way of doing pretty much everything. Suffice to say that we decided to start classes the following week.

BTW, if don’t know what Taiko is or have never heard Taiko, I refer you to the mighty Wikipedia and the mightier YouTube:

I’m on a temporary hiatus from Taiko right now, but I had an amazing experience the few months I spent with SF Taiko Dojo.

Yes, there are rules and etiquettes. But in a society where anything goes and freedom rules and any kind of “discipline” is often frowned upon, SFTD was almost refreshing. In many ways, it was reminiscent of the Gurukul system in ancient India.

Taiko itself is a wonderful art form. There is something powerful about a Taiko performance. A single drum is an excellent percussion device, but in a group, Taiko takes a life of its own. Like most art forms, you can pick up the basics real quick. But to go deep into Taiko, you need time, patience and a lot of hard work. The veterans at SFTD have been playing for 10-15 years and still learning.

Needless to add, Taiko is also a fantastic full body workout. It is a combination of dance, drumming, music and more. The classes are fun, but you do need serious commitment if you want to become an advanced Taiko player. The folks in the adult beginners class are a merry bunch. Before our first class, I was extremely anxious, trying to memory numerals in Japanese from Wikipedia and worried whether I’ll be able to keep up with everyone. There was help every step of the way. The class won’t stop for you, but it will not leave you behind either :)

But the best part of SFTD is the opportunity to learn from Sensei Tanaka. His accomplishments in the world of Taiko are well known, so I won’t enlist them here. What surprised me was the humility and generosity and the energy he brings with him, even after doing this for more than four decades. He could easily delegate the adult beginners class to one of his many advanced students; yet he still routinely teaches the class himself, ever so patient and understanding. Better yet, his expertise in Taiko is matched only by his wistful humor.

So if you are in the Bay Area and are looking for some inspiration, do checkout San Francisco Taiko Dojo.