Disaster averted


I messed up big time today. Ok, so it all started off with my FTP server. I usually allow anonymous connections, and on occasion also allow uploading files into incoming. Once in a while I notice some bot trying to dump gigs of crap onto my machine, so I turn the upload off. Today was one of those days, and today I turned off the FTP server completely for good measure.

After that I thought I’d clean the mess a bit up, so I was just looking around to see what files had been created. Now there was this one file named “/” (thats right, the slash character). Amusingly, just before looking at this file I had deleted a directory, so my shell history had ‘rm -rf’ all over it. Now, being the over-confident idiot that I sometimes am, I went ahead and pressed ‘up’ and removed the earlier directory and replaced it with an escaped slash ‘\/’. But just before pressing enter, I pressed tab (BIG mistake) and my bash auto completion “unescaped” the slash again, and bang! There it was…. happily deleting away my root partition!! :-O

By the time I pressed Ctrl-C it was too late. Most of the files in /bin, /usr/bin and /usr/sbin had disappearead. I couldn’t even ls or cp, let alone SSH or apt-get anything. One foolish thing let to another — I could have simply popped in Knoppix and restored the lost files. Since my home partition and all critical data are on a separate partition, I wasn’t worried. Though, I might have lost all of my blog data and my photo gallery… but I was ready to live with that. After all, this IS my lab machine, meant for research and not for hosting software for my amusement.

Anyways, in my rush, I popped in a CD of the debian installer instead, and was soon on my way to a fresh install. During the disk partitioning, I carefully left the home partition untouched and formatted the original root partition. It was only much later when grub was setting up that I realized that I had formatted the home partition instead!!! :-O I was so lost! Fortunately, most of my critical stuff was in a subversion repository, and I had freshly updated local copies on my laptop. The current project stuff was all on another remote machine. But I’d still lose a LOT of data, and configuration. On the flip side, I was able to properly restore my blog, gallery and groupware setup. All in all it wasn’t very pleasant.

And I learned a lot of lessons: a) BACK UP. b) keep sensitive data safe from the rest of the crap. c) DO NOT fool around with the lab machine. d) have a knoppix/ubuntu live CD always at hand. Hope this doesn’t happen again

Leave a Reply