Fazal Majid's low-intensity blog

Sporadic pontification

Fazal

Here, take my money. Please. Pretty please?

Eighty percent of success is showing up. — Woody Allen

My company, Kefta, helps its clients, usually Fortune 500 companies with e-commerce operations, improve their online conversion rates. We typically increase sales by 10–20%. This is not rocket science, more akin to Retail 101, simple things like modifying pages to stop showing offers for products we know the user has already purchased, or making offers more relevant when we know the prospect is interested in a specific product (e.g. because they come from Google after searching for that keyword).

Sometimes I wonder if what we are doing is not too sophisticated by far, when I see particularly boneheaded practices at places that really should know better. Dell is often touted as a model of logistical and operational excellence, and for being a web-centric company. My experience is that many products they carry are not listed on the web site and can only be ordered by phone. You also have to phone to get a discount.

Despite being a telecoms engineer by training, I loathe phones. Phones are great for keeping an emotional connection with friends and family, but are a staggeringly inefficient form of communication for business purposes. They do not leave an audit trail, and even when they do (my voice mail system automatically forwards them to me by email as a MIME-encoded WAV attachment), they hog disk space and are not searchable. You can scan an email in a few seconds, but are forced to listen to voice mail at whatever pace it was dictated. Well, at least with WAV attachments, I can skip back to write down a phone number without having to replay the whole message.

Coming back to Dell, I recently needed to buy a Gigabit Ethernet switch from them. I sent an email to my rep, which he promptly ignored. I tried calling, at least 4 or 5 times, but my only option was voice-mail jail. In the end, I passed the buck to a junior colleague, who tried to leave voice mail and discovered he couldn’t because it was full. With persistence, he managed to get Dell to condescend to taking our order. No customer should have to go through so many hoops just so the vendor can take their money.

I am ragging on Dell, but most IT vendors do as poorly. I can understand expensive support calls receiving lower priority and resources than sales calls — after all, the company already has your money. Not having their act together for the simple matter of order-taking simply boggles the mind. Workflow systems, automatic call distributors and other technologies designed to prevent this have been available for many years. It looks like nobody has bothered to go through the user experience, even though these bugs (and many other glaring deficiencies like session timeouts) could be caught by the most cursory of inspections.

Dell sends an automated satisfaction survey after a sale. Unlike the order-taking process, the survey follows up if you do not respond… That said, it is the usual worthless multiple-choice question format asking me to answer irrelevant questions on a scale of 1 to 10. I don’t recall if the form had a box for free-form comments, but even if it did, the survey design is not-so-subtly signaling that no human is ever going to read what you type there, and thus it is not worth the effort to fill it. The numeric answers are probably going to be collated by an automated report nobody pays any attention to anymore, because garbage-in, garbage-out.

If you are serious about customer feedback, make it open and free-form, and make sure each and every feedback is read by a human (they come quite cheap in the Midwest and the developing world). They should be acknowledged personally (not with an automated reply) and followed through until the issue is either resolved or a decision is taken not to implement the changes suggested (because they are too expensive, impractical or whatever other reason). In both cases, inform the user who bothered to give feedback — most large companies pay a fortune in market research while at the same time ignoring the free (and usually very valuable) insights submitted by their customers. Granted, you cannot always resolve every complaint by unreasonable customers, but feedback on process issues should always be taken into consideration.

Sometimes dropped orders are due to active incompetence rather than careless neglect. While implementing a campaign for one of our clients, we realized there was a bug in one of their ordering forms that would cause them to drop an order. Our software sits on top of the client’s website and monitors it precisely for exception cases like these, and we told them we could, at no extra charge for them, send the dropped order details to an email address of their choosing so the order could be re-entered manually. They declined our offer for various reasons related to internal politics and trade union issues, essentially they were refusing to bend down and pick up money lying on the floor (our estimate was they were losing tens to hundreds of thousands of dollars of customer lifetime value every month due to inaction).

You don’t have to endure a multi-million dollar ERP or CRM implementation to improve follow-through. Where there is a will, there is a way, and a little creative thinking will usually find a work-around that can get the job done until a more robust solution can be deployed. One of our clients, a major bank, was in the early stages of developing their e-commerce, and simultaneously in the throes of a Siebel implementation. Their online forms would simply send an email to a branch office for manual processing. We were implementing a satisfaction survey for them, and offered to send an email automatically to a supervisor if the customer’s order had not been processed, at least until Siebel came on-line. Poor man’s workflow, but email workflows are often quite effective, specially for remedial situations like these.

As I mentioned, sometimes I think I am in the wrong business, and should instead start a consultancy to teach some clue to large companies that have grown complacent. But then again, that is assuming somebody cares, beyond paying lip service to Customer Relationship Management. There is no point in setting up complex systems to build a lifelong relationship with repeat customers if you can’t even take their orders in the first place.

Temboz 0.7 released

I have released version 0.7 of Temboz. The main improvements in the new version are a better user interface, ad filtering, and garbage collection of articles older than 6 months. Several facilities have also been added to make it easier to write and test filtering rules – you can now add comments to a rule, or purge and reload a feed from the feed details page to see if changes rules are kicking in or not.

Temboz now also has a publicly accessible CVStrac with a documentation Wiki and a bug-tracking database (where change requests can also be submitted). The Wiki is publicly read-only for now, but if you would like to contribute to it, drop me an email and I will create an account with edit privileges for you.

The megapixel myth revisited

Introduction

As my family’s resident photo geek, I often get asked what camera to buy, specially now that most people are upgrading to digital. Almost invariably, the first question is “how many megapixels should I get?”. Unfortunately, it is not as simple as that, megapixels have become the photo industry’s equivalent of the personal computer industry’s megahertz myth, and in some cases this leads to counterproductive design decisions.

A digital photo is the output of a complex chain involving the lens, various filters and microlenses in front of the sensor, and the electronics and software that post-process the signals from the sensor to produce the image. The image quality is only as good as the weakest link in the chain. High quality lenses are expensive to manufacture, for instance, and often manufacturers skimp on them.

The problem with megapixels as a measure of camera performance is that not all pixels are born equal. No amount of pixels will compensate for a fuzzy lens, but even with a perfect lens, there are two factors that make the difference: noise and interpolation.

Noise

All electronic sensors introduce some measure of electronic noise, among others due to the random thermal motion of electrons. This shows itself as little colored flecks that give a grainy appearance to images (although the effect is quite different from film grain). The less noise, the better, obviously, and there are only so many ways to improve the signal to noise ratio:

  • Reduce noise by improving the process technology. Improvements in this area occur slowly, typically each process generation takes 12 to 18 months to appear.
  • Increase the signal by increasing the amount of light that strikes each sensor photosite. This can be done by using faster lenses or larger sensors with larger photosites. Or by only shooting photos in broad daylight where there are plenty of photons to go around.

Fast lenses are expensive to manufacture, specially fast zoom lenses (a Canon or Nikon 28-70mm f/2.8 zoom lens costs well over $1000). Large sensors are more expensive to manufacture than small ones because you can fit fewer on a wafer of silicon, and as the likelihood of one being ruined by an errant grain of dust is higher, large sensors have lower yields. A sensor twice the die area might cost four times as much. A “full-frame” 36mm x 24mm sensor (the same size as 35mm film) stresses the limits of current technology (it has 4 times the die size of the latest-generation “Sandy Bridge” quad-core Intel Core i7), which is why the cheapest full-frame bodies like the Canon EOS 5DmkII or Nikon D700 cost $2,500, whereas a DSLR with an APS-C sized sensor (that has 40% the surface area of a full-frame sensor) can be had for under $500. Larger professional medium-format digital backs can easily reach $25,000 and higher.

This page illustrates the difference in size of the sensors on various consumer digital cameras compared to those on some high-end digital SLRs. Most compact digital cameras have tiny 1/1.8″ or 2/3″ sensors at best (these numbers are a legacy of TV camera tube ratings and do not have a relationship with sensor dimensions, see DPReview’s glossary entry on sensor sizes for an explanation).

For any given generation of cameras, the conclusion is clear – bigger pixels are better, they yield sharper, smoother images with more latitude for creative manipulation of depth of field. This is not true across generations, however, Canon’s EOS-10D had twice as many pixels as the two generations older EOS-D30 for a sensor of the same size, but it still manages to have lower noise thanks to improvements in Canon’s CMOS process. Current bodies like the 7D have 6 times the pixels of the D30 while still having better noise levels, the benefits of 10 years of progress from sensor engineers.

The problem is, as most consumers fixate on megapixels, many camera manufacturers are deliberately cramming too many pixels in too little silicon real estate just to have megapixel ratings that look good on paper. The current batch of point-and-shoot cameras cram 14 million pixels in tiny 1/2.3″ sensors. Only slightly less egregious, the premium-priced Canon G12 puts 10.1M pixels in a 1/1.7″ sensor, the resulting photosites are 1/10 the size of those on the similarly priced 10 megapixel Nikon D3000. Canon Digital Rebel T3i (EOS-D600) and 1/16 of those on the significantly more expensive 21MP Canon 5DmkII.

Predictably, the noise levels of the G12 are poor in anything but bright sunlight, just as a “150 Watts” ghetto blaster is incapable of reproducing the fine nuances of real music. The camera masks this with digital signal processing that conceals noise by smoothing pictures, thus smudging noise but also removing the details those extra megapixels were supposed to deliver. The DSLR will yield far superior images in most circumstances, but naive purchasers could easily be swayed by the 2 extra megapixels into buying the inferior yet overpriced Sony product. Unfortunately, there is a Gresham’s law at work and manufacturers are still racing to the bottom, although: Nikon and Canon have also introduced 8 megapixel cameras with tiny sensors pushed too far. You will notice that for some reason camera makers seldom show sample images taken in low available light…

Interpolation

Interpolation (along with its cousin, “digital zoom”) is the other way unscrupulous marketers lie about their cameras’ real performance. Fuji is the most egregious example with its “SuperCCD” sensor, that is arranged in diagonal lines of octagons rather than horizontal rows of rectangles. Fuji apparently feel this somehow gives them the right to double the pixel rating (i.e. a sensor with 6 million individual photosites is marketed as yielding 12 megapixel images). You can’t get something for nothing, this is done by guessing the values for the missing pixels using a mathematical technique named interpolation. This makes the the image look larger, but does not add any real detail. You are just wasting disk space storing redundant information. My first digital camera was from Fuji, but I refuse to have anything to do with their current line due to shenanigans like these.

Most cameras use so-called Bayer interpolation, where each sensor pixel has a red, green or blue filter in front of it (the exact proportions are actually 25%, 50% and 25% as the human eye is more sensitive to green). An interpolation algorithm reconstructs the three color values from adjoining pixels, thus invariably leading to a loss of sharpness and sometimes to color artifacts like moiré patterns. Thus, a “6 megapixel sensor” has in reality only 1.5-2 million true color pixels.

A company called Foveon makes a distinctive sensor that has three photosites stacked vertically in the same location, yielding more accurate colors and sharper images. Foveon originally took the high road and called their sensor with 3×3 million photosites a 3MP sensor, but unfortunately they were forced to align themselves with the misleading megapixel ratings used by Bayer sensors.

Zooms

A final factor to consider is the zoom range on the camera. Many midrange cameras come with a 10x zoom, which seems mighty attractive in terms of versatility, until you pause to consider the compromises inherent in a superzoom design. The wider the zoom range, the more aberrations and distortion there will be that degrade image quality, such as chromatic aberration (a.k.a. purple fringing), barrel or pincushion distortion, and generally lower resolution and sharpness, specially in the corners of the frame.

In addition, most superzooms have smaller apertures (two exceptions being the remarkable constant f/2.8 aperture 12x Leica zoom on the Panasonic DMC-FZ10 and the 28-200mm equivalent f/2.0-f/2.8 Carl Zeiss zoom on the Sony DSC-F828), which means less light hitting the sensor, and a lower signal to noise ratio.

A reader was asking me about the Canon G2 and the Minolta A1. The G2 is 2 years older than the A1, and has 4 million 9 square micron pixels, as opposed to 5 million 11 square micron sensors, and should thus yield lower image quality, but the G2’s 3x zoom lens is fully one stop faster than the A1’s 7x zoom (i.e. it lets twice as much light in), and that more than compensates for the smaller pixels and older sensor generation.

Recommendations

If there is a lesson in all this, it’s that unscrupulous marketers will always find a way to twist any simple metric of performance in misleading and sometimes even counterproductive ways.

My recommendation? As of this writing, get either:

  • An inexpensive (under $400, everything is relative) small sensor camera rated at 2 or 3 megapixels (any more will just increase noise levels to yield extra resolution that cannot in any case be exploited by the cheap lenses usually found on such cameras). Preferably, get one with a 2/3″ sensor (although it is becoming harder to find 3 megapixel cameras nowadays, most will be leftover stock using an older, noisier sensor manufacturing process).
  • Or save up for the $1000 or so that entry-level large-sensor DSLRs like the Canon EOS-300D or Nikon D70 will cost. The DSLRs will yield much better pictures including low-light situations at ISO 800.
  • Film is your only option today for decent low-light performance in a compact camera. Fuji Neopan 1600 in an Olympus Stylus Epic or a Contax T3 will allow you to take shots in available light without a flash, and spare you the “red-eyed deer caught in headlights” look most on-camera flashes yield.

Conclusion

Hopefully, as the technology matures, large sensors will migrate into the midrange and make it worthwhile. I for one would love to see a digital Contax T3 with a fast prime lens and a low-noise APS-size sensor. Until then, there is no point in getting anything in between – midrange digicams do not offer better image quality than the cheaper models, while at the same time being significantly costlier, bulkier and more complex to use. In fact, the megapixel rat race and the wide-ranging but slow zoom lenses that find their way on these cameras actually degrade their image quality over their cheaper brethren. Sometimes, more is less.

Updates

Update (2005-09-08):

It seems Sony has finally seen the light and is including a large sensor in the DSC-R1, the successor to the DSC-F828. Hopefully, this is the beginning of a trend.

Update (2006-07-25):

Large-sensor pocket digicams haven’t arrived yet, but if you want a compact camera that can take acceptable photos in relatively low-light situations, there is currently only one game in town, the Fuji F30, which actually has decent performance up to ISO 800. That is in large part because Fuji uses a 1/1.7″ sensor, instead of the nasty 1/2.5″ sensors that are now the rule.

Update (2007-03-22):

The Fuji F30 has been superseded since by the mostly identical F31fd and now theF40fd. I doubt the F40fd will match the F30/F31fd in high-ISO performance because it has two million unnecessary pixels crammed in the sensor, and indeed the maximum ISO rating was lowered, so the F31fd is probably the way to go, even though the F40 uses standard SD cards instead of the incredibly annoying proprietary Olympus-Fuji xD format.

Sigma has announced the DP-1, a compact camera with an APS-C size sensor and a fixed 28mm (equivalent) f/4 lens (wider and slower than I would like, but since it is a fixed focal lens, it should be sharper and have less distortion than a zoom). This is the first (relatively) compact digital camera with a decent sensor, which is also a true three-color Foveon sensor as cherry on the icing. I lost my Fuji F30 in a taxi, and this will be its replacement.

Update (2010-01-12):

We are now facing an embarrassment of riches.

  • Sigma built on the DP1 with the excellent DP2, a camera with superlative optics and sensor (albeit limited in high-ISO situations, but not worse than film) but hamstrung by excruciatingly slow autofocus and generally not very responsive. In other words, best used for static subjects.
  • Panasonic and Olympus were unable to make a significant dent in the Canon-Nikon duopoly in digital SLRs with their Four-Thirds system (with one third less surface than an APS-C sensor, they really should be called “Two-Thirds”). After that false start, they redesigned the system to eliminate the clearance required for a SLR mirror, leading to the Micro Four Thirds system. Olympus launched the retro-styled E-P1, followed by the E-P2, and Panasonic struck gold with its GF1, accompanied by a stellar 20mm f/1.7lens (equivalent to 40mm f/1.7 in 35mm terms).
  • A resurgent Leica introduced the X1, the first pocket digicam with an APS-C sized sensor, essentially the same Sony sensor used in the Nikon D300. Extremely pricey, as usual with Leica. The relatively slow f/2.8 aperture means the advantage from its superior sensor compared to the Panasonic GF1 is negated by the GF1’s faster lens. The GF1 also has faster AF.
  • Ricoh introduced its curious interchangeable-camera camera, the GXR, one option being the A12 APS-C module with a 50mm f/2.5 equivalent lens. Unfortunately, it is not pocketable

According to Thom Hogan, Micro Four Thirds grabbed in a few months 11.5% of the market for interchangeable-lens cameras in Japan, something Pentax, Samsung and Sony have not managed despite years of trying. It’s probably just a matter of time before Canon and Nikon join the fray, after too long turning a deaf ear to the chorus of photographers like myself demanding a high-quality compact camera. As for myself, I have already voted with my feet, successively getting a Sigma DP1, Sigma DP2 and now a Panasonic GF1 with the 20mm f/1.7 pancake lens.

Update (2010-08-21):

I managed to score a Leica X1 last week from Camera West in Walnut Creek. Supplies are scarce and they usually cannot be found for love or money—many unscrupulous merchants are selling their limited stock on Amazon or eBay, at ridiculous (25%) markups over MSRP.

So far, I like it. It may not appear much smaller than the GF1 on paper, but in practice those few millimeters make a world of difference. The GF1 is a briefcase camera, not really a pocketable one, and I was subconsciously leaving at home most of the time. The X1 fits easily in any jacket pocket. It is also significantly lighter.

High ISO performance is significantly better than the GF1 – 1 to 1.5 stops. The lens is better than reported in technical reviews like DPReview’s—it exhibits curvature of field, which penalizes it in MTF tests.

The weak point in the X1 is its relatively mediocre AF performance. The GF1 uses a special sensor that reads out at 60fps, vs. 30fps for most conventional sensors (and probably even less for the Sony APS-C sensor used in the X1, possibly the same as in the Nikon D300). This doubles the AF speed of its contrast-detection algorithm over its competitors. Fuji recently introduced a special sensor that features on-chip phase-detection AF (the same kind used in DSLRs), let’s hope the technology spreads to other manufacturers.

 

Free association

When I first heard Microsoft chose “Vista” as the official name for the much-delayed Longhorn release of Windows, I immediately thought of this (Quicktime, 657KB). Does this mean I am a bad person?

Explosion in San Francisco

I work at 153 Kearny in the San Francisco Financial district. At around 9:55AM, I heard a loud thud (not a sharp crack) and a pressure wave that rattled the windows. My first thought was naturally that a terrorist attack had occurred, and we decided to evacuate the office (as did the other tenants). Within 2 minutes, we were all out, and there were already first responders on the scene ushering us away from the scene. The awning on the Ralph Lauren store at the corner of Post and Kearny was in flames; I only took this one photo.

Ralph Lauren store in flames

I did not see any wounded people, and most passersby left the scene in an orderly fashion. Many people a mere two blocks away were unaware that anything happened. Reports are contradictory, so after making sure all my staff were accounted for, I sent them all home — if it were a gas explosion or an electrical fire, it could be dangerous to return until we have an all-clear from the authorities.