Fazal Majid's low-intensity blog

Sporadic pontification

Migrating to Hugo

I have been meaning to move away from Wordpress to a static site generator for a very long time, due to:

  • The slowness of WP, since every page request makes multiple database calls due to the spaghetti code nature of WP and its plugin architecture. Caching can help somewhat, but it has brittle edge cases.
  • Its record of security holes. I mitigated this somewhat by isolating PHP as much as possible.
  • It is almost impossible to follow front-end optimization best-practices like minimizing the number of CSS and JS files because each WP plugin has its own

My original plan was to go with Acrylamid, but about a year ago I started experimenting with Hugo. Hugo is blazing fast because it is implemented in Go rather than a slow language like Python or Ruby, and this is game-changing. Nonetheless, it took me over a year to migrate. This post is about the issues I encountered and the workflow I adopted to make it work.

Wordpress content migration

There is a migration tool, but it is far from perfect despite the author’s best efforts, mostly because of the baroque nature of Wordpress itself when combined with plugins and an old site that used several generations of image gallery technology.

Unfortunately, that required rewriting many posts, specially those with photos or embedded code.

Photo galleries

Hugo does not (yet) support image galleries natively. I started looking at the HugoPhotoSwipe project, but got frustrated by bugs in its home-grown YAML parser that broke round-trip editing, and made it very difficult to get galleries with text before and after the gallery proper. The Python-based smartcrop for thumbnails is also excruciatingly slow.

I wrote hugopix to address this. It uses a simpler one-way index file generation method, and the much faster Go smartcrop implementation by Artyom Pervukhin.

Broken asset references

Posts with photo galleries were particularly broken, due to WP’s insistence on replacing photos with links to image pages. I wrote a tool to help me find broken images and other assets, and organize them in a more rational way (e.g. not have PDFs or source code samples be put in static/images).

It also has a mode to identify unused assets, e.g. 1.5GB of images that no longer belong in the hugo tree as their galleries are moving elsewhere.

Password-protected galleries

I used to have galleries of family events on my site, until an incident where some Dutch forum started linking to one of my cousin’s wedding photos and making fun of her. At that point I put a pointed error message for that referrer and controlled access using WP’s protected feature. That said, private family photos do not belong on a public blog and I have other dedicated password-protected galleries with Lightroom integration that make more sense for that use case, so I just removed them from the blog, shaving off 1.5GB of disk in the bargain.

There are systems that can provide search without any server component, e.g. the JavaScript-based search in Sphinx, and I looked at some of the options referenced by the Hugo documentation like the Bleve-based hugoidx but the poor documentation gave me pause, and I’d rather not run Node.js on my server as needed by hugo-lunr.

Having recently implemented full-text search in Temboz using SQLite’s FTS5 extension, I felt more comfortable building my own search server in Go. Because Hugo and fts5index share the same Go template language, this makes a seamless integration in the site’s navigation and page structure easy.

Theme

There is no avoiding this, moving to a new blogging system requires a rewrite of a new theme if you do not want to go with a canned theme. Fortunately, Hugo’s theme system is sane, unlike Wordpress’, because it does not have to rely on callbacks and hooks as much as with WP plugins.

One pet peeve of mine is when sites change platform with new GUIDs or permalinks in the RSS feeds, causing a flood of old-new articles to appear in my feed reader. Since I believe in showing respect to my readers, I had to avoid this at all costs, and also put in place redirects as needed to avoid 404s for the few pages that did change permalinks (mostly image galleries).

Doing so required copying the embedded RSS template and changing:

<guid>{{ .Permalink }}</guid>

to:

<guid isPermaLink="false">{{ .Params.rss_guid | default .Permalink }}</guid>

The next step was to add rss_guid to the front matter of the last 10 articles in my legacy RSS feed.

How big can a panorama get?

I use the Kolor AutoPano Giga panorama-stitching software, recently acquired by GoPro, but I have yet to produce a gigapixel panorama like those they pioneered. This brings up an interesting question: given a camera and lens, what would the pixel size of the largest 360° stitched panorama be?

Wikipedia to the rescue: using the formula for the solid angle of a pyramid, the full panorama size of a camera with m megapixels on a sensor of a x b using a focal length of f would be:

m * π / arctan(ab / 2f / sqrt(4f2 + a2 + b2))

(this applies only to rectilinear lenses, not fisheyes or other exotics).

Here is a little JavaScript calculator to apply the formula (defaults are for the Sony RX1RII, the highest resolution camera I own):

MP
mm actual 35mm equivalent

MP

The only way I can break through the gigapixel barrier with a prime lens is using my 24MP APS-C Fuji X-T2 with a 90mm lens.

Scanner group test

TL:DR—avoid scanners with Contact Image Sensors if you care at all about color fidelity.

Vermeer it is not

After my abortive trial of the Colortrac SmartLF Scan, I did a comparative test of scanning one of my daughter’s A3-sized drawings on a number of scanners I had handy.

Scanner Sensor Scan
Colortrac SmartLF Scan CIS ScanLF.jpg
Epson Perfection Photo V500 Photo (manually stitched) CCD Epson_V500.jpeg
Epson Perfection V19 (manually stitched) CIS Epson_V19.jpg
Fujitsu ScanSnap S1500M (using a carrier sheet and the built-in stitching) CCD S1500M_carriersheet.jpg
Fujitsu ScanSnap SV600 CCD SV600.jpg
Fuji X-Pro2 with XF 35mm f/1.4 lens, mounted on a Kaiser RS2 XA copy stand with IKEA KVART 3-spot floor lamp (CCT 2800K, a mediocre 82 CRI as measured with my UPRtek CV600) CMOS X-Pro2.jpg

I was shocked by the wide variance in the results, as was my wife. This is most obvious in the orange flower on the right.

Comparison

I scanned a swatch of the orange using a Nix Pro Color Sensor (it’s the orange square in the upper right corner of each scan in the comparison above). When viewed on my freshly calibrated NEC PA302W SpectraView II monitor, the Epson V500 scan is closest, followed by the ScanSnap SV600.

The two scanners using Contact Image Sensor (CIS) technology yielded dismal results. CIS are used in low-end scanners, and they have the benefit of low power usage, which is why the only USB bus-powered scanners available are all CIS models. CIS sensors begat the CMOS sensors used by the vast majority of digital cameras today, superseding CCDs in that application, I would not have expected such a gap in quality.

The digital camera scan was also quite disappointing. I blame the poor quality of the LEDs in the IKEA KVART three-headed lamp I used (pro tip: avoid IKEA LEDs like the plague, they are uniformly horrendous).

I was pleasantly surprised by the excellent performance of the S1500M document scanner. It is meant to be used for scanning sheaves of documents, not artwork, but Fujitsu did not skimp and used a CCD sensor element, and it shows.

Pro tip: a piece of anti-reflective Museum Glass or equivalent can help with curled originals on the ScanSnap SV600. I got mine from scraps at a framing shop. I can’t see a trace of reflections on the scan, unlike on the copy stand.

Colortrac SmartLF Scan review

TL:DR summary

Pros:

  • Scans very large documents
  • Easy to use
  • Packs away in a convenient carrying case

Cons:

  • So-so color fidelity
  • Hard to feed artwork straight
  • Dust and debris can easily get on the platen, ruining scans
  • Relatively expensive for home use

Review

One thing you do not lack for when your child enters preschool is artwork. They generate prodigious amounts of it, with gusto, and they are often large format pieces on 16×24″ paper (roughly ISO A2). The question is, what do you do with the torrent?

I decided I would scan them, then file them in Ito-Ya Art Profolios, and possibly make annual photobooks for the grandparents. This brings up the logistical challenge of digitizing such large pieces. Most flatbed scanners are limited to 8.5×14″ (US Legal) format. Some like the Epson Expression 11000XL and 12000XL can scan 11×17″ (A3), as can the Fujitsu ScanSnap SV600 book scanner, but that is not fully adequate either. One option would be to fold the artwork up, scan portions then stitch them together in AutoPanoGiga or Photoshop, but that would be extremely cumbersome, specially when you have to do a couple per day. I do not have access to a color copier at my office, and most of these are only A3 anyway.

I purchased a Kaiser RS2 XA copy stand (cheaper to get it direct from Europe on eBay than from the usual suspects like B&H) and got a local framing shop to cut me a scrap of anti-reflective Museum Glass. This goes up to 16×20″ for the price of a midrange flatbed scanner, but it is tricky to set up lights so they don’t induce reflections (no AR coating is perfect), perfectly aligning the camera with the baseboard plane is difficult (I had to shim it using a cut-up credit card), and this still doesn’t solve the problem of the truly large 16×24″ artwork (stands able to handle larger formats are extremely expensive and very bulky).

I then started looking at large-format scanners like those made by Contex or Océ. They are used by architecture firms to scan blueprints and the like, but they are also extremely large, and cost $3000-5000 for entry-level models, along with onerous DRM-encumbered software that requires license dongles and more often than not will not run on a Mac. They are also quite bulky, specially if you get the optional stands.

That is why I was pleasantly surprised to learn British company Colortrac makes a model called the SmartLF Scan! (I will henceforth omit the over-the-top exclamation mark). It is self-contained (can scan to internal memory or a USB stick, although it will also work with a computer over USB or Ethernet, Windows-only, unfortunately), available in 24″ or 36″ wide versions, is very compact compared to its peers, and is even supplied with a nifty custom-fitted wheeled hard case. The price of $2,000 ($2,500 for the 36″ version), while steep for home use, is well within the range of enthusiast photo equipment. I sold a few unused cameras to release funds for one.

Once unpacked, the scanner is surprisingly light. It is quite wide, obviously, to be able to ingest a 24″ wide document (see the CD jewel case in the photo above for scale). There is a LCD control panel and a serviceable keypad-based (not touch) UI. The power supply is of the obnoxious wall-wart type. I wish they used text rather than inscrutable icons in the UI—it is much more informative and usable to see a menu entry for 400dpi resolution rather than checkerboard icons with various pitches.

After selecting your settings (or saving them as defaults), you load paper by feeding it from the front, face up. It is quite hard to feed large-format paper straight, and this is compounded by the lack of guides. On the other hand it is hard to see how Colortrac could have fitted photocopier-style guide rails in such a compact design, and they would be likely to break.

The scanner is simplex, not duplex, unsurprisingly at that price point. The sensor is on top of the feed, which helps control dust and debris sticking to it, but when scanning painted artwork, there will inevitably be crumbs of paint that will detach and stick to the sensor platen. This manifests itself as long dark vertical lines spoiling subsequent scans, something I occasionally also see on my Fujitsu ScanSnap document scanner. Cleaning the Colortrac is way easier than on the ScanSnap, as unfolding rear legs and releasing front catches opens it wide, and a few passes with optical cleaning wipes (I use Zeiss’ single-use ones) will do the trick.

By the manufacturer’s own admission, the scanner is designed to scan technical drawings, not art. It uses a linear contact image sensor (CIS) like lower-end flatbed scanners and document scanners, unlike the higher-fidelity charge-coupled device (CCD) sensors used in higher-end graphics arts and photo scanners. The light source is a row of point light LEDs that casts relatively harsh shadows on the paper. They do make CCD scanners for graphics arts, but they start at $10,000… Contex makes an A2 flatbed CCD scanner, the HD iFlex, but it costs $6,700 (at Jet.com of all places), their iQ Quattro 2490 at $4,500 is the most viable step-up (it uses a CIS, but offers 16-bit color, AdobeRGB and beyond gamut, calibration and magnetic paper guides).

The scanner’s resolution is 600dpi. Scanning 16×24″ originals at that resolution yields a 138MP file that is nearly a gigabyte in size. The 400dpi setting yields a much more reasonable 200MB or so, and compressing them further using tiffcp with zip compression (not an option on the scanner) yields 130-140MB files.

Unfortunately, I ended up returning it. There was a 1cm scratch in the glass platen, which manifested itself as streaks. It takes quite a bit to scratch glass (I don’t think it was Lexan or similar), and I wasn’t scanning sandpaper, so it must have been a factory defect or a customer return. When I looked at the color fidelity of the scans, I was not inclined to order a replacement, and got a Fujitsu ScanSnap SV600 from Japan instead from an Amazon third-party reseller (25% savings over the US price, even if you usually forgo a US warranty on grey-market imports).

Avery 22807 template for InDesign

The Avery 22807 2-inch circular stickers are a good alternative to Moo, PSPrint et al when you need a small quantity of stickers in a hurry. Unfortunately Avery has not seen it fit to provide usable InDesign templates as they do with some of their other sticker SKUs, only Microsoft Word, which is needless to say inadequate. A search for “Avery 22807 Indesign template” yielded some, but they have issues with missing linked PDF files.

I reverse-engineered the Microsoft template to build one of my own, with dimensions (including the tricky almost-but-not-quite square grid spaced at 5/8″ horizontally but 7/12″ vertically) to simplify “Step and Repeat…”.

I have only tested this with my InDesign CS6, not sure if it will work with older versions.

Avery 22807 2-inch circular labels.indt

On the bugginess of El Capitan

I never updated my home Mac Pro to El Capitan. To paraphrase Borges, each successive Apple OS release since Snow Leopard makes you long for the previous one. Unfortunately I have no choice but to run the latest OS X release on my work Macs as that is usually required to run the latest Xcode, itself required for the App Store.

I did not realize how bad El Capitan was until I upgraded my work iMac (27-inch 5K model) to Sierra last week. Previously, I would experience a mean time between crashes of around 3 days. I thought it was flaky hardware (the problems started from when I unboxed the computer), but couldn’t find the time to take it to the Genius Bar. I had also experienced the same problem with my old home 2009 Nehalem Mac Pro, which I had taken to the office, in fact that’s why I bought the iMac in the first place (and the first one I ordered had to go back because of defective pins in the RAM expansion slots). The Mac Pro had previously been rock-steady at home.

Since I upgraded to Sierra, I haven’t had a single crash. The only possible conclusion is that El Capitan bugs were to blame. The only thing unusual about this iMac is I upgraded the RAM from OWC, but the memory passes testing using Micromat’s TechTool.

I am not one to look at the Steve Jobs era with rosy-tinted glasses, OS X has never had the same level of stability as Solaris or even Linux, but Apple’s hardware and software quality has really gone to the dogs of late, something Lloyd Chambers dubs Apple Core Rot.

I am now starting to hedge my bets and am testing Ubuntu for my laptop computing needs, first by repurposing my 2008-vintage first-generation MacBook Air that is no longer supported by OS X anyway (works, but painfully slow) and soon with a shiny new HP Spectre on order.

Avoiding counterfeit goods on Amazon: mission impossible?

I mentioned previously that I seldom shop for electronics on Amazon.com any more, preferring B&H Photo whenever possible. I now have another reason: avoiding counterfeit goods.

My company boardroom is in an electromagnetic war zone—dozens of competing WiFi access points combined with electronic interference from the US-101 highway just outside make WiFi reception tenuous at best, and unusable more often than not. To work around this, we set up a wired Ethernet switch, and since most of our staff use MacBook Airs, Apple USB Ethernet adapters purchased from Amazon. When I side-graded from my 15″ Retina MacBook Pro to a much more portable 12″ Retina MacBook, I wasn’t able to connect using the dongle, and the name of the device was interspersed with Chinese characters. At first I thought it was an issue with my Satechi USB-C hub, but I experienced the same problems via a genuine Apple USB-C multiport adapter as well.

Eventually I figured out the Ethernet dongles were counterfeit. The packaging, while very similar to Apple’s, was just a tiny bit off, like amateurish margins between the Apple logo and the edges of the card. On the dongles themselves, the side regulatory disclosures sticker was inset, not flush with the body of the adapter.

Counterfeiting is a major problem. By some accounts, one third of all Sandisk memory cards worldwide are counterfeits. In some cases like chargers or batteries, your equipment could be at risk, or even your very life. The counterfeit adapters we purchased from Amazon did not come from Amazon themselves but from a third-party merchant participating in the Amazon marketplace. To Amazon’s credit, we returned them for a prompt, no questions asked refund even though we bought them over six months ago, but it is hard to believe Amazon is unaware of the problem rather than willfully turning a blind eye to it.

My first reaction was to tell our Office Manager to make sure to buy only from Amazon rather than third-party merchants (pro tip: including “amazon” in your Amazon search terms will do that in most cases). Unfortunately, that may not be enough. Amazon has a “fulfilled by Amazon” program for merchants where you ship your goods to them, and they handle warehousing and fulfillment. These “fulfilled by Amazon” items are also more attractive to Prime members. One option Amazon offers is Stickerless, commingled inventory where the items you send are put into a common bin. Amazon still has the ability to trace the provenance of the item through its inventory management, but for purposes of order fulfillment they will be handled just like Amazon’s own stock. Some categories like groceries and beauty products are excluded, but electronics are not.

The implications are huge: even if the vendor is Amazon itself, you cannot be sure that the item is not counterfeit. All the more reason to buy only from trustworthy, single-vendor sites like B&H, even if shipping is a bit slower.

Chrome and AES-256 security: it’s not me, it’s you

This blog now supports the HTTP/2 protocol, courtesy of nginx 1.9.5 (PDF).

In the process, I was stymied by an “ERR_SPDY_INADEQUATE_TRANSPORT_SECURITY” error from Google Chrome. HTTP/2 mandates TLS de facto, if not in the strict letter of the specification, and it also forbids a number of obsolete or weaker SSL/TLS ciphers to only permit ones that are truly secure. After some considerable digging, I found out the issue is Google Chrome on Mac and Android (presumably Windows as well) does not support 256-bit AES in HTTP/2, and my server was set up to only accept 256-bit encryption (only the best will do for my readers!). The error message was misleading: it’s not the server but Chrome’s crypto which is lacking.

It seems the cryptographers at Google feel 128-bit AES in Galois Counter Mode is good enough, and they did not want to be too far apart from Firefox (which does not support it either, and just fails without even the courtesy of an error message). In contrast, Safari on Yosemite supports AES-256-CBC (not ideal, I know, but that’s also what Chrome supports if HTTP/2 is turned off) and AES-256-GCM on El Capitan and iOS 9. Here are the settings your browser uses:

This is disappointing. AES-256-GCM is supported in hardware on most Intel hardware nowadays (all but lowest-end chips have the AES-NI instructions) and in the ARMv8-A architecture supported by most smartphones and mobile devices today, where the extra CPU load would matter most. I wonder how much of this is driven by Google’s fondness for Dan Bernstein’s ChaCha20+Poly1305 algorithms. Excellent as they may be, they are not implemented in hardware on the most common platforms, nor implemented at all in OpenSSL. It is quite disconcerting that my phone has better crypto than my desktop browser.

I ended up resolving the issue by loosening my cipher list from AES256+EECDH to EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH, but Chrome really should catch up and not let itself be hobbled by the increasingly irrelevant Firefox and its hoary NSS crypto.

I probably sound harsher than I intended towards the Google crypto team. The backward compatibility issues they have to deal with, from poorly designed TLS standards to broken web server software, intrusive anti-virus or corporate proxy servers mean a lot of their energy goes into exception cases, rather than implementing the latest and greatest in crypto algorithms.

Update (2017-01-18):

It looks like Chrome silently added AES256-GCM support last year, as it now negotiates the ECDHE-RSA-AES256-GCM-SHA384 cipher on aes256gcm.majid.org.

Heat sealers: organizer’s secret weapon

Professional Organizers will wax lyrical about label-makers, file folders, and the like, but one unheralded gizmo I have found surprisingly effective is a heat sealer, in my case the AIE-200C. It’s made right here in California, and very robust, although if I were to buy one again, I would probably spring for the 12″ version. You put stuff in a polyethylene bag (up to 6 mil or 0.15mm thick, but 4 mil seems like the optimum for robustness while remaining flexible and see-through), put the open end under the sealer, set the thermostat, press and cut the excess bag with the built-in cutter. It makes a 1mm wide heat weld in the bag, which is now airtight and water-proof. You can also buy rolls for massive capacity, but that seemed like overkill.

The great advantage of a heat sealer over ziploc bags is that you cut the bag to size, instead of having items floating around in an oversized bag, which means it’s much tidier, and also takes up less space. Cables are much more manageable when individually bagged so they cannot tangle together, for instance. They are also perfect for infrequently used supplies, random parts for the house or appliances, or infrequently used tools.

Organized Cables

My current approach to organizing random stuff is to bag it, optionally include a description written on an index card if it is not immediately obvious what it is, seal it then dump it in a Rubbermaid plastic bin. When the bin is full, I will take an inventory in a spreadsheet (more specifically OmniOutliner and Delicious Library). In a year’s time, I will cull them as needed.

This system is close in spirit to my paperless workflow: do not exhaust yourself attempting to physically organize the long tail of stuff that doesn’t fit in an established category with a well-defined home. Just put them in numbered containers and keep an index on a computer where they are much easier to search. There are also smartphone apps to streamline this inventory task like Home Inventory Photo Remote.

ArtisanState review

TL:DR—avoid them.

I seldom print photos any more. When I do, I prefer to make photobooks, as the format is way more convenient than loose prints, takes little space, and looks more polished than a traditional photo album.

Unfortunately, most photobooks are printed on HP Indigo digital presses, which use a technology somewhat similar to a laser printer, but capable of better quality photo reproduction. Indigo presses were originally designed to produce personalized junk-mail, not high-quality photo reproduction, and the quality, while decent, is not at the same level as that of true RA-4 photo paper exposed with a laser or LED light source as done by most digital minilabs (e.g. Fuji Frontier or Noritsu QSS) or higher-end imaging systems like the Océ/Cymbolic Lightjet or Durst Lambda.

There are higher-quality options. AdoramaPix has a good reputation for its albums, which are printed on RA-4 paper and bound in a lay-flat binding without a gutter, a technique that lends itself specially well to panoramic prints. They also have a “Hudson” line of premium albums where the photos are laminated on thick cardstock for a more luxurious feel. In researching this flush-mount process, I discovered a company called ArtisanState. It is based here in San Francisco (manufacturing is in China), their pricing seemed attractive, so I decided to give them a try.

I used a selection of my panoramic prints and ordered a 6×8 album bound in genuine leather. They offer two grades of paper, Fuji Crystal Archive Lustre and Fuji Crystal Archive Pearl Metallic, and I opted for the latter. Metallic paper, first introduced by Kodak under the Endura brand, has mica particles embedded in the RC paper base under the photographic emulsion. The photo looks as if it is painted on metal, which can be spectacular, specially with specular highlights (although I would not recommend it for portraiture such as a wedding album, the fashion industry seems to be quite fond of it). The Fuji lustre has a grainy finish that resists fingerprints, but I don’t find it attractive, and would much prefer a satin finish without an obvious texture like the one Moo uses in its business cards.

When I received my album after 2 weeks, I was impressed by the reproduction quality and the metallic effect, but there was also a very visible texture on the pages, similar to an orange peel. After some research, I found that mounting metallic papers seems to cause orange peel unless done very carefully using a low temperature on the mounting press, and they are the exception to the general rule of thumb that Fuji products are superior to Kodak’s (although true to form, Kodak’s bean counters degraded the quality of the product by cutting corners to shave costs).

At the price they charge ($104 list, but I got it at 40% promotional discount), you can rightfully expect perfection. I wrote to ArtisanState to complain, got the run-around, and reviews online suggest my experience with unresponsive support is far from unusual. I am going to try again with AdoramaPix: they may be more expensive, but the product won’t be made in China and in the end you get what you pay for.

Divine Dark Chocolate Hazelnut Truffle

Divine Chocolate is owned by a Ghanaian cocoa farmers’ cooperative. All the profits go back to the farmers, unlike the “Fairtrade” scam where the expensive certification primarily benefits self-aggrandizing Western auditors and marketers. For that reason alone it is a brand I would like to love. Unfortunately, my experience with their products to date has been underwhelming—not bad per se, just very ho-hum.

I experienced chocolate cravings today and stopped by the SF SOMA Whole Foods despite its mediocre range (Whole Foods’ selection is mostly abysmal, but they are the only grocery within walking distance of my office). They had a new bar by Divine, and I tried it out. This tuned out to be good call.

The bar itself is really a dark chocolate gianduja, I guessed they dumbed down the name to “truffle” to avoid confusing the mainstream consumer. I personally prefer a lighter, milk chocolate based giandujas, my benchmark being the Venchi Blend bars and the Callebaut blocks meant for bakers, but this bar has a clean taste, and the hazelnut taste comes out well.

It is not as good as the Poco Dolce Bittersweet Hazelnut bar, but is also significantly cheaper at $4 each. I am not sure how long they can keep the price, given the bar is 20% hazelnuts by weight, and that the price of hazelnuts on world markets has jumped by 60% due to poor Turkish harvests (Turkey produces 70% of the world’s supply of hazelnuts, and 25% of the world’s hazelnuts are snapped up by Ferrero, makers of Nutella).

The slow decline of Amazon Prime

I have been an Amazon Prime customer since it was introduced, almost a decade ago in 2005. They recently raised the price to $99, which is not unreasonable given inflation and the rise of fuel and shipping costs. Unfortunately, the service has also degraded, and I am considering dropping it for that reason.

It really hit me this week. I ordered a bunch of loupes from eBay last weekend, as Schneider stopped manufacturing them 2 years ago and they are now officially listed as discontinued, and old-new stock of other reputable makers like Leica, Cabin/Mamiya or Rodenstock are starting to dwindle. At the same time, I ordered a few items using Amazon Prime (5 orders in all, 3 from Amazon themselves, 2 from third-party vendors but fulfilled by Amazon). All my eBay items have already arrived, including some shipped all the way from Canada that arrived yesterday, but only one of the Amazon items has arrived. Something is seriously wrong when Amazon’s vaunted logistics cannot match individual sellers on the fleabay.

The problem cannot be laid at the door of the shipping companies, the problem is that Amazon is taking longer to ship the items in the first place. It is an open question whether that delay is intentional (as seems to be the case for Amazon free super shipping orders, or when Netflix delays heavy customers’ DVDs to rate-limit them and thus reduce its shipping costs).

One other factor that has decreased the value of the service is the increasing proportion of items that are part of Amazon’s obnoxious add-on item program. Contrary to Amazon’s statements, many of the items downgraded to add-on status are not ones that were unavailable previously, but rather items that were previously eligible for Prime but no longer are. If I have to accumulate $25 in orders, I might as well go back to the free super saver shipping.

To sweeten the sticker shock, Amazon is bundling streaming video and music, and the Kindle lending library. Those services have essentially zero value for me, as the movie selection is as dismal as Netflix’s (mostly C-list or really old movies, hardly anything you might want to watch), streaming does not have good classical music coverage, and I refuse to use Kindle due to their predatory practices.

I find I am buying considerably less from Amazon these days:

  • Since they introduced sales tax, they are often not competitive with bricks-and-mortar retailers like Target (which will give you an extra 5% discount for using their REDcard) or B&H.
  • I refuse to buy books from Amazon (eBooks from iBooks or straight from the publisher like O’Reilly).
  • I buy my classical music from ArkivMusic (for CDs and SACDs, and they have their own $20/year equivalent of Prime) or FLAC sites like B&W Society of Sound, Linn, eClassical and Chandos.
  • I get my photo gear, computers and other electronics from B&H whenever possible, and that probably accounts for the bulk of my former Amazon dollars.

What’s left?

  • Oddball items hard to source otherwise
  • Tools
  • Household supplies (although I get most of these from Costco or Soap.com, admittedly an Amazon company now).
  • Very occasionally some specialty grocery items and clothes.

I used Amazon’s handy order history export (temperamental, it fails if you have Amazon Honor System transactions in the selected date range) to calculate how much I spend with them (removing Adorama as they are a big outlier), and the trend is clearly unfavorable to Amazon since the high water mark of 2011. Their changes to Prime (pay more for worse service) are certainly not helping.

amazon

A Passel of Miniature Tripods

Miniature tripods are a handy thing to carry in a camera bag. While they cannot replace a full-size tripod, they can allow you to take a shot where otherwise impossible. Here are a few worth your consideration.

Tested but not shown:

  • Joby Gorillapod: total garbage, unfit for purpose.
  • Pedco Ultrapod: the tripod itself is reasonably decent, but its ballhead is poor

My scissors collection

Mac Pro first impressions

I received my late 2013 black cylinder Mac Pro last Monday. I ordered the 6-core model with D700 GPUs, since the higher-core models can’t Turbo boost to the full 3.9GHz the 4-core and 6-core ones can, and thus for most poorly-parallelized apps will underperform. I had to get a Promise Pegasus J4 with a pair of Samsung 840 Evo SSDs to hold my files, as I need nearly 2TB to do so and that is not available on the internal SSD, and it would be a shame to hobble this machine with spinning rust.

Some notes I have not seen in the many reviews sloshing around the Internet this far:

  • It is not actually black, rather a dark metallic gray, the color of hematite.
  • The TOSlink digital optical out is now capable of 192kHZ/24 bit audio, whereas the old Mac Pro was limited to 96 kHz. Unfortunately, it is very hard to find POF cables and DACs that can reliably sustain that data rate.
  • It is dead quiet compared to the old Mac Pro, and even my work iMac.
All in all, a remarkable engineering feat. A HP Z820 may have more memory capacity, expandability and total horsepower in its BMW-designed case, but Apple is the one pushing the envelope in terms of design.

Externalities again

I just wasted half an hour of my life on the phone with my credit card company’s fraud department, as someone attempted to buy expensive tickets from an airline in Panama. Most likely my card number was compromised by Target, although it could also be due to Adobe.

It is actually surprising such breaches do not occur on a daily basis—the persons paying for the costs of a compromise (the card holder, defrauded merchants and their credit card companies via the cost of operating their fraud departments) are not the same as those paying for the security measures that would prevent the said breach, a textbook example of what economists call an externality. There are reputational costs to a business that has a major security breach, but they are occurring so often consumers are getting numbed to them.

Many states have mandatory breach disclosure laws, following California’s example. It is time for legislatures to take the next step and impose statutory damages for data breaches, e.g. $100 per compromised credit card number, $1000 per compromised social security number, and so on. In Target’s case, 40 million compromised credit cards multiplied by $100 would mean $4 billion in damages. That would make management take notice and stop paying mere lip service to security. It might also jump-start the long overdue migration to EMV chip-and-PIN cards in the United States.

Wotancraft Etan review

The real electromagnetic emissions danger

I live 1.2km away from Sutro Tower in San Francisco. At my wife’s request I was trying to calculate the safe radius at which emissions from the transmitters at Sutro Tower are of the same power as a cell phone held a meter away, with back-of-the-envelope calculations using the inverse square law and Wikipedia’s table of radio powers.

I was shocked to find out the total power from the transmitters is about 8 megawatts, not in the kilowatt range I was expecting, and once reached 29MW. For comparison, the power of France’s first-generation PWR nuclear reactors is 900MW, and a typical cellular tower is 100W to 500W. If I use 2W as the reference, this yields a “safe” radius of 2km, which excludes many desirable San Francisco neighborhoods like Twin Peaks, Forest Hill or Noe Valley.

I looked up the most recent Environmental Impact Report following the DTV transition, and it mentions a FCC maximum allowed flux level of 0.2mW/cm2, and the measured levels in the Midtown Terrace neighborhood immediately adjacent to Sutro Tower reach 4% of this max level.

On further investigation, this is not one of those situations where US standards are significantly more lax than those in Europe, as France or the UK have the same level, derived from an international NGO called the ICNIRP. Interestingly, according to the WHO the maximum allowed emissions in such environmental paragons as Russia and China are one hundredth as high as those in the US or Europe and are just as science-based as those from ICNIRP (remember, for all its faults, the Soviet Union ranked very highly in maths and physics education & research, and in health care).

The ICNIRP/FCC standard is equivalent to a 25W isotropic emitter within a 1 meter radius, or 12x 2G GSM cell phones. Anyone who has experienced the squeal of unshielded and unpowered speakers next to an actively transmitting GSM phone will be skeptical about their claims that this is a safe level. Their methodology is based solely on the thermal effects of non-ionizing radiation, as if this were a mere microwave oven shielding exercise, and assumes that cells are otherwise unaffected by electromagnetism or cumulative exposure. This seems unwarrantedly optimistic.

People worry about cancer risks associated with radio frequency emissions from cell phone towers and cell phones themselves, but the real risk comes from overlooked obsolete technologies like TV and FM radio.

What to do? Getting a site survey from a Professional Engineer using calibrated equipment costs $1,500, which is something you would only do as part of a final inspection while buying a house. Most RF power meters sold on places like Amazon, usually in the $300 range, are pieces of junk with suggested applications like detecting paranormal activity and ghosts. Most likely solid engineering and metrology are optional given their application domain. Professional T&M gear like an Agilent V3500A or a Wandel & Goltermann/Narda EMR-300 cost $2,000 and $6,000 respectively, so the DIY route is also expensive.

Update (2014-03-08):

My father worked on some projects in the Soviet Union in the Seventies. He told me their workplace safety standards were much more stringent than the ones in the West. Workers were not allowed to lift weights above 25kg, for instance.

Fixing Mac software update NSURLErrorDomain error -1012

Software Update for system components on my home Mac Pro has not worked in a while, and I have had to resort to manually downloading and applying updates. The updates just wouldn’t appear in the Mac App Store app where they normally should.

After upgrading to Mavericks, I finally figured out why. Instead of silently ignoring the updates, Mavericks displays a not-so-helpful error message “NSURLErrorDomain Error -1012”. On inspecting network traffic from the App Store app, I noticed it connects using TLS 1.2 to swdist.apple.com, then aborts. It then hit me – in 2011, after Comodo was hacked, apparently by elements affiliated with the Iranian government, I revoked the trust setting on their root certificates. The certificate for swdist.apple.com is signed by Comodo, and thus Software Update could no longer establish a secure connection to Apple and that’s why it was failing.

This is not the only time a Certificate Authority was hacked. Dutch CA Diginotar, which included the Dutch government among its clients, suffered a breach, apparently also involving Iran. Microsoft, Mozilla, Google and Apple promptly revoked Diginotar’s root CA certificates, which quickly led to the company going out of business. I guess Comodo is larger (the EFF calls them “too big to fail”) and better politically connected (it helps when you have people like Phillip Hallam-Baker on the payroll), and managed to elude the same punishment it richly deserved.

Apple should really step up its game and ditch a security provider which demonstrated incompetence at its alleged core competency, and I filed Radar bug report 15328323 to urge them to do so. In the meantime, the way to fix the error message is to temporarily reinstate trust in the Comodo root CA.

Update (2015-10-29)

At some point in the last 2 years they switched from Comodo to Symantex (probably 2014-04-13 when the current certificate was issued). Unfortunately, Symantec has its own problems.