RapidSSL 1 – GoDaddy 0

My new company’s website uses SSL. I ordered an “extended validation” certificate from GoDaddy, instead of my usual CA, RapidSSL/GeoTrust, because GoDaddy’s EV certificates were cheap. EV certificates are security theater more than anything else, I probably should not have bothered.

Immediately after switching from my earlier “snake oil” self-signed test certificate to the production certificate, I saw SSL errors on Google Chrome for Mac and Safari for Mac, i.e. the two browsers that use OS X’s built-in crypto and certificate store. I suppose I should have tested the certificate on another server before going live, but I trusted GoDaddy (they are my DNS registrars, and competent, if garish).

Big mistake.

I called their tech support hotline, which is incredibly grating because of the verbose phone tree that keeps trying to push add-ons (I guess it is consistent with the monstrosity that is their home page).

After a while, I got a first-level tech. He asked whether I saw the certificate error on Google Chrome for Windows. At that point, I was irate enough to use a four-letter word. Our customers are Android mobile app developers. A significant chunk of them use Macs, and almost none (less than 5%) use IE, so know-nothing “All the world is IE” demographics are not exactly applicable.

After about half an hour of getting the run-around and escalating to level 2, with my business partner Michael getting progressively more anxious in the background, the level 1 CSR tells me the level 2 one can’t reproduce the problem (I reproduced it on three different Macs in two different locations). I gave them an ultimatum: fix it within 10 minutes or I would switch. At this point, the L1 CSR told me he had exhausted all his options, but I could call their “RA” department, and offered to switch me. Inevitably, the call transfer failed.

I dialed their SSL number, and in parallel started the certificate application process on RapidSSL. They offered a free competitive upgrade, I tried it, and within 3 minutes I had my fresh new, and functional certificate, valid for 3 years, all for free and in less time than it takes to listen to GoDaddy’s obnoxious phone tree (all about “we pride ourselves in customer service” and other Orwellian corporate babble).

I then called GoDaddy’s billing department to get a refund. Surprisingly, the process was very fast and smooth. I guess it is well-trod.

The moral of the story: GoDaddy—bad. RapidSSL—good.

Update (2012-08-26)

I switched my DNS business from GoDaddy to in December 2011 after Bob Parsons’ despicable elephant-hunting stunt.

Clueless SaaS providers can leave you with egg on your face

While cleaning out my spam folders, I noticed a disturbing trend: a number of the spam were sent to vendor-specific email addresses I had set up to communicate with Parallels, Joyent and Shoeboxed. As a security measure, I do not give my personal email address to vendors, only aliases. The email address I used in the past for Dell was, for instance (I now use a different domain). A few years back, I started receiving pornographic spam at that address, which led me to think either Dell had secretly adopted a radically new diversification plan, or that their customer database had been compromised. Needless to say, this did not reflect well on Dell. I canceled that alias and stopped dealing with Dell.

I contacted the support for the three vendors. Joyent got back to me, and said:

We have traced this back to a third-party provider that was used to distribute service notifications. We have been in contact with this service provider, and they have determined that subscriber email addresses of their clients were compromised. They have launched their own investigation, which is ongoing, and have also reached out to their local FBI office.

After some digging, I found some interesting posts. Some email marketing company called iContact, that I had never heard about before, was the source of the compromise. They claim to be SAS-70 compliant, but of course like most bureaucratic certifications, SAS-70 is mostly security theater that makes sysadmins’ life miserable for no meaningful security benefit (SAS-70 auditors, on the other hand, profit handsomely).

Just another example of how outsourcing critical functions to outside vendors can backfire spectacularly and take down your own reputation in the process.

Broken SPF records

I have SPF verification enabled on my mail server. While SPF is no panacea for the problem of spam, it is quite effective at ensuring spammers do not forge the sending address to impersonate someone else, and cause some poor innocent soul to receive in a boomerang effect the torrent of complaints hurled at them.

Unfortunately far too many lame organizations (cough, Google) qualify their SPF record using a too permissive ?all or ~all clause, which means they have servers other than those listed, and thus their SPF record is useless for filtering purposes.

In the last month, I noticed the opposite problem: I did not receive emails from Eurostar and BookMooch because their SPF records did not list the mail servers they actually use. If they are not clueful enough to manage a simple list of IP addresses, or have basic change management discipline, they should do us all a favor and ditch the SPF record they clearly are incapable of maintaining.

Fie on parasitic US cellcos

The Economist has an excellent article on how Indian mobile phone companies cut costs. They have an ARPU of $6.50 a month yet operate with a 40% gross margin. If US cellcos were run as efficiently, they would have a 1200% gross margin on the $51 monthly ARPU!

The time has long come to stop coddling grossly inefficient and anti-competitive cellular carriers in the West. They are no longer fledgling businesses in the shadow of landlines, quite the opposite, in fact. One good place to start would be to require them to offer consumers the choice of carrier for international calls and for roaming, as is the case with landlines. Their rates are simply extortionate.

At long last real broadband in San Francisco

I upgraded my broadband connection yesterday from a puny 3-6Mbps/384-768K DSL connection to 20Mbps symmetrical Metro Ethernet service from an outfit called WebPass. My current ISP, Raw Bandwidth, has excellent service with no restrictions on hosting servers or traffic shaping shenanigans unlike the likes of Comcast, but they are still hobbled by the AT&T last-mile connection.

WebPass finesses around the incumbent monopoly by using newer buildings’ data-grade wiring plant to bring 100MBps Ethernet connections right into your home (all they had to do was change a wall plate and patch some cables in the closet) and use microwave links to backhaul traffic to their data center. They claim to use a mesh network for backhaul, but I think this just means a standard network of microwave links where some sites have to hop multiple microwave links to get to the transit connection, rather than a purely centralized hub and spoke model. In my case their offices are a mere two blocks away. This would allow me the pleasure of ditching the scumbags at AT&T altogether (were it not for the fact my building requires an entirely unnecessary landline for its security system).

AT&T is probably the worst telco in the US now, and is notorious for starving its infrastructure of investment to maximize short-term profits, unlike Verizon, who is investing heavily in its FiOS optical network. Unfortunately San Francisco is in AT&T territory and will not get true optical networks anytime soon. Municipalities can usually reassign the cable franchise every so many years, but there is no such provision for involuntary transfer of telcos that I know of.

The new service is $45 a month with no installation fee, vs. $70 a month for Raw Bandwidth, but it does not include a static IP address (they do offer it as part of their prohibitively expensive metered business service). Configuring my home router (a Cisco 877) to use both connections was incredibly painful, but I will run the two ISPs side by side for the next few months. If WebPass proves as reliable as Raw bandiwdth, I may just have to find a work-around for the static IP issue, or just rely on DHCP lease pinning.

If you live in San Francisco, or are moving there, definitely have a look at the buildings they have covered. The service is a glimpse of what people not in broadband backwater USA get.