A minor web disaster

If you tried to view the site over the past day or so, you probably ran into errors or a page telling you the site was “down for maintenance”.

Yesterday morning, I accidentally deleted the entire content of the web site.  All of it.  It was a simple mistake; much simpler than it should have been, in fact.  I guess I shouldn’t be making updates before I’ve finished my second cup of coffee.  I didn’t panic too badly, knowing that the irreplaceable data — the databases — were intact and backed up at least daily.  I also thought I had all of the web content backed up.  That turned out not to be the case…I quickly found that the web content, the actual HTML and PHP files, were not as current as I thought they were.  My backup routine was (as I now know) not backing up everything that I thought it was.  So, I called my web hosting provider, knowing that they back up nightly and could quickly restore the content.

Sure, they can restore it — for a hefty fee.  Since I have been a loyal, long time customer, my stuff is on an older server that they have not bothered to update with the automated restore option that would let me restore the backup files myself, for free, immediately.  It would cost me, I was told, in the 3-figure range.  After struggling for a few hours I finally agreed to pay their fee, and only after the fee was collected was I told it “could take up to 24 hours”.  Great.  Never mind that, had I been a new customer, I could have simply clicked a button to automatically restore the files from the previous night’s backup, at no cost.

Well, a day later with no files restored, I called the support line again.  Now they tell me their normal time to restore files from a backup is 7 to 10 days.  Seriously, I’m not making this up.  Now, I used to be responsible for the team that handled  server backups and restores at a fairly well known technology company.  I have also been the guy responsible for restoring files and entire filesystems to servers when the application team needed it done.  Restoring a file from backup should take no more than an hour or two, and that’s if your server admins are really busy.  I ended up restoring what I had, fixing a number of minor problems and as of this evening am working to re-create or locate a few missing image files.

All in all, this has been the worst customer service experience I have had in years.  This company has managed to turn me from a loyal, happy, long-term customer into an unsatisfied soon-to-be-ex customer who will now actively discourage anyone who asks from doing business with them.  And for what?  What would it have cost them to have someone restore the files I needed?  Zero.  Not a penny.

Having a good product is important.  Having good customer service is even more important.  I try to remind myself of that often.

2 Comments on A minor web disaster

  1. Howard Collier
    August, 8th 2010 at 8:57 PM

    A fee based, windoze/Linux, over the Internet backup solution for me has been to use “Carbonite.com” and it runs in the background 24/7. And an older method I’ve used was to have a secondary HD installed, D:, which I ran a batch file to copy data file from drive C:. If it’s a Linux box, use Crontab to sked a backup session.

  2. admin
    August, 8th 2010 at 10:31 PM

    Thanks for the comment. The web site is hosted on a commercial hosting service; I had been using used automated batch jobs to copy the content to my local server. Unfortunately, after rebuilding my local server I didn’t rebuild my cron job to image the web site so my local image was over a month old. That’s fixed now. :) In the process I found some cool features of NcFTP that I didn’t know about before that make it even easier.

    By the way… when you police up that brass, I’ll be happy to take all the excess .357, .40 and .44 Mag you have. :)

Leave a Reply

CAPTCHA Image CAPTCHA Audio
Refresh Image

WP Login