Mylos longer articles

The megapixel myth – a pixel too far?

Revised introduction

This article remains popular thanks to Google and the like, but it was written 7 years ago and the models described are ancient history. The general principles remain, you are often better off with a camera that has fewer but better quality pixels, though the sweet spot shifts with each successive generation. The more reputable camera makers have started to step back from the counterproductive megapixel race, and the buying public is starting to wise up, but this article remains largely valid.

My current recommendations are:

  • Dispense with entry-level point and shoot cameras. They are barely better than your cameraphone
  • If you must have a pocketable camera with a zoom lens, get the Canon S95, Panasonic LX5, Samsung TL500 or Olympus XZ-1. Be prepared to pay about $400 for the privilege.
  • Upping the budget to about $650 and accepting non-zoom lenses gives you significantly better optical and image quality, in cameras that are still pocketable like the Panasonic GF2, Olympus E-PL2, Samsung NX100, Ricoh GXR and Sony NEX-5.
  • The Sigma DP1x and DP2x offer stunning optics and image quality in a compact package, but are excruciatingly slow to autofocus. If you can deal with that, they are very rewarding.
  • The fixed-lens Fuji X100 (pretty much impossible to get for love or money these days, no thanks to the Sendai earthquake) and Leica X1 offer superlative optics, image and build quality in a still pocketable format. The X1 is my everyday-carry camera, and I have a X100 on order.
  • If size and weight is not an issue, DSLRs are the way to go in terms of flexibility and image quality, and are available for every budget. Models I recommend by increasing price range are the Pentax K-x, Canon Rebel T3i, Nikon D7000, Canon 5DmkII, Nikon D700 and Nikon D3S.
  • A special mention for the Leica M9. It is priced out of most people’s reach, and has poor low-light performance, but delivers outstanding results thanks to Leica lenses and its sensor devoid of anti-alias filters.

Introduction

As my family’s resident photo geek, I often get asked what camera to buy, specially now that most people are upgrading to digital. Almost invariably, the first question is “how many megapixels should I get?”. Unfortunately, it is not as simple as that, megapixels have become the photo industry’s equivalent of the personal computer industry’s megahertz myth, and in some cases this leads to counterproductive design decisions.

A digital photo is the output of a complex chain involving the lens, various filters and microlenses in front of the sensor, and the electronics and software that post-process the signals from the sensor to produce the image. The image quality is only as good as the weakest link in the chain. High quality lenses are expensive to manufacture, for instance, and often manufacturers skimp on them.

The problem with megapixels as a measure of camera performance is that not all pixels are born equal. No amount of pixels will compensate for a fuzzy lens, but even with a perfect lens, there are two factors that make the difference: noise and interpolation.

Noise

All electronic sensors introduce some measure of electronic noise, among others due to the random thermal motion of electrons. This shows itself as little colored flecks that give a grainy appearance to images (although the effect is quite different from film grain). The less noise, the better, obviously, and there are only so many ways to improve the signal to noise ratio:

  • Reduce noise by improving the process technology. Improvements in this area occur slowly, typically each process generation takes 12 to 18 months to appear.
  • Increase the signal by increasing the amount of light that strikes each sensor photosite. This can be done by using faster lenses or larger sensors with larger photosites. Or by only shooting photos in broad daylight where there are plenty of photons to go around.

Fast lenses are expensive to manufacture, specially fast zoom lenses (a Canon or Nikon 28-70mm f/2.8 zoom lens costs over $1000). Large sensors are more expensive to manufacture than small ones because you can fit fewer on a wafer of silicon, and as the likelihood of one being ruined by an errant grain of dust is higher, large sensors have lower yields. A sensor twice the die area might cost four times as much. A “full-frame” 36mm x 24mm sensor (the same size as 35mm film) stresses the limits of current technology (it has nearly 8 times the die size of the latest-generation “Prescott” Intel Pentium IV), which is why the full-frame Canon EOS 1Ds costs $8,000, and professional medium-format digital backs can easily reach $25,000 and higher.

This page illustrates the difference in size of the sensors on various consumer digital cameras compared to those on some high-end digital SLRs. Most compact digital cameras have tiny 1/1.8″ or 2/3″ sensors at best (these numbers are a legacy of TV camera tube ratings and do not have a relationship with sensor dimensions, see DPReview’s glossary entry on sensor sizes for an explanation).

For any given generation of cameras, the conclusion is clear – bigger pixels are better, they yield sharper, smoother images with more latitude for creative manipulation of depth of field. This is not true across generations, however, Canon’s EOS-10D has twice as many pixels as the two generations older EOS-D30 for a sensor of the same size, but it still manages to have lower noise thanks to improvements in Canon’s CMOS process.

The problem is, as most consumers fixate on megapixels, many camera manufacturers are deliberately cramming too many pixels in too little silicon real estate just to have megapixel ratings that look good on paper. Sony has introduced a 8 megapixel camera, the DSC-F828, that has a tiny 2/3″ sensor. The resulting photosites are 1/8 the size of those on the similarly priced 6 megapixel Canon Digital Rebel (EOS-D300), and 1/10 the size of those on the more expensive 8 megapixel DSLR Canon EOS-1D Mark II.

Predictably, the noise levels of the 828 are abysmal in anything but bright sunlight, just as a “150 Watts” ghetto blaster is incapable of reproducing the fine nuances of classical music. The lens also has its issues, for more details see the review. The Digital Rebel will yield far superior images in most circumstances, but naive purchasers could easily be swayed by the 2 extra megapixels into buying the inferior yet overpriced Sony product. Unfortunately, there is a Gresham’s law at work and manufacturers are racing to the bottom: Nikon and Canon have also introduced 8 megapixel cameras with tiny sensors pushed too far. You will notice that for some reason camera makers seldom show sample images taken in low available light…

Interpolation

Interpolation (along with its cousin, “digital zoom”) is the other way unscrupulous marketers lie about their cameras’ real performance. Fuji is the most egregious example with its “SuperCCD” sensor, that is arranged in diagonal lines of octagons rather than horizontal rows of rectangles. Fuji apparently feel this somehow gives them the right to double the pixel rating (i.e. a sensor with 6 million individual photosites is marketed as yielding 12 megapixel images). You can’t get something for nothing, this is done by guessing the values for the missing pixels using a mathematical technique named interpolation. This makes the the image look larger, but does not add any real detail. You are just wasting disk space storing redundant information. My first digital camera was from Fuji, but I refuse to have anything to do with their current line due to shenanigans like these.

Most cameras use so-called Bayer interpolation, where each sensor pixel has a red, green or blue filter in front of it (the exact proportions are actually 25%, 50% and 25% as the human eye is more sensitive to green). An interpolation algorithm reconstructs the three color values from adjoining pixels, thus invariably leading to a loss of sharpness and sometimes to color artifacts like moiré patterns. Thus, a “6 megapixel sensor” has in reality only 1.5-2 million true color pixels.

A company called Foveon makes a distinctive sensor that has three photosites stacked vertically in the same location, yielding more accurate colors and sharper images. Foveon originally took the high road and called their sensor with 3×3 million photosites a 3MP sensor, but unfortunately they were forced to align themselves with the misleading megapixel ratings used by Bayer sensors.

Zooms

A final factor to consider is the zoom range on the camera. Many midrange cameras come with a 10x zoom, which seems mighty attractive in terms of versatility, until you pause to consider the compromises inherent in a superzoom design. The wider the zoom range, the more aberrations and distortion there will be that degrade image quality, such as chromatic aberration (a.k.a. purple fringing), barrel or pincushion distortion, and generally lower resolution and sharpness, specially in the corners of the frame.

In addition, most superzooms have smaller apertures (two exceptions being the remarkable constant f/2.8 aperture 12x Leica zoom on the Panasonic DMC-FZ10 and the 28-200mm equivalent f/2.0-f/2.8 Carl Zeiss zoom on the Sony DSC-F828), which means less light hitting the sensor, and a lower signal to noise ratio.

A reader was asking me about the Canon G2 and the Minolta A1. The G2 is 2 years older than the A1, and has 4 million 9 square micron pixels, as opposed to 5 million 11 square micron sensors, and should thus yield lower image quality, but the G2’s 3x zoom lens is fully one stop faster than the A1’s 7x zoom (i.e. it lets twice as much light in), and that more than compensates for the smaller pixels and older sensor generation.

Recommendations

If there is a lesson in all this, it’s that unscrupulous marketers will always find a way to twist any simple metric of performance in misleading and sometimes even counterproductive ways.

My recommendation? As of this writing, get either:

  • An inexpensive (under $400, everything is relative) small sensor camera rated at 2 or 3 megapixels (any more will just increase noise levels to yield extra resolution that cannot in any case be exploited by the cheap lenses usually found on such cameras). Preferably, get one with a 2/3″ sensor (although it is becoming harder to find 3 megapixel cameras nowadays, most will be leftover stock using an older, noisier sensor manufacturing process).
  • Or save up for the $1000 or so that entry-level large-sensor DSLRs like the Canon EOS-300D or Nikon D70 will cost. The DSLRs will yield much better pictures including low-light situations at ISO 800.
  • Film is your only option today for decent low-light performance in a compact camera. Fuji Neopan 1600 in an Olympus Stylus Epic or a Contax T3 will allow you to take shots in available light without a flash, and spare you the “red-eyed deer caught in headlights” look most on-camera flashes yield.

Conclusion

Hopefully, as the technology matures, large sensors will migrate into the midrange and make it worthwhile. I for one would love to see a digital Contax T3 with a fast prime lens and a low-noise APS-size sensor. Until then, there is no point in getting anything in between – midrange digicams do not offer better image quality than the cheaper models, while at the same time being significantly costlier, bulkier and more complex to use. In fact, the megapixel rat race and the wide-ranging but slow zoom lenses that find their way on these cameras actually degrade their image quality over their cheaper brethren. Sometimes, more is less.

Updates

Update (2005-09-08):

It seems Sony has finally seen the light and is including a large sensor in the DSC-R1, the successor to the DSC-F828. Hopefully, this is the beginning of a trend.

Update (2006-07-25):

Large-sensor pocket digicams haven’t arrived yet, but if you want a compact camera that can take acceptable photos in relatively low-light situations, there is currently only one game in town, the Fuji F30, which actually has decent performance up to ISO 800. That is in large part because Fuji uses a 1/1.7″ sensor, instead of the nasty 1/2.5″ sensors that are now the rule.

Update (2007-03-22):

The Fuji F30 has been superseded since by the mostly identical F31fd and now the F40fd. I doubt the F40fd will match the F30/F31fd in high-ISO performance because it has two million unnecessary pixels crammed in the sensor, and indeed the maximum ISO rating was lowered, so the F31fd is probably the way to go, even though the F40 uses standard SD cards instead of the incredibly annoying proprietary Olympus-Fuji xD format.

Sigma has announced the DP-1, a compact camera with an APS-C size sensor and a fixed 28mm (equivalent) f/4 lens (wider and slower than I would like, but since it is a fixed focal lens, it should be sharper and have less distortion than a zoom). This is the first (relatively) compact digital camera with a decent sensor, which is also a true three-color Foveon sensor as cherry on the icing. I lost my Fuji F30 in a taxi, and this will be its replacement.

Update (2010-01-12):

We are now facing an embarrassment of riches.

  • Sigma built on the DP1 with the excellent DP2, a camera with superlative optics and sensor (albeit limited in high-ISO situations, but not worse than film) but hamstrung by excruciatingly slow autofocus and generally not very responsive. In other words, best used for static subjects.
  • Panasonic and Olympus were unable to make a significant dent in the Canon-Nikon duopoly in digital SLRs with their Four-Thirds system (with one third less surface than an APS-C sensor, they really should be called “Two-Thirds”). After that false start, they redesigned the system to eliminate the clearance required for a SLR mirror, leading to the Micro Four Thirds system. Olympus launched the retro-styled E-P1, followed by the E-P2, and Panasonic struck gold with its GF1, accompanied by a stellar 20mm f/1.7 lens (equivalent to 40mm f/1.7 in 35mm terms).
  • A resurgent Leica introduced the X1, the first pocket digicam with an APS-C sized sensor, essentially the same Sony sensor used in the Nikon D300. Extremely pricey, as usual with Leica. The relatively slow f/2.8 aperture means the advantage from its superior sensor compared to the Panasonic GF1 is negated by the GF1’s faster lens. The GF1 also has faster AF.
  • Ricoh introduced its curious interchangeable-camera camera, the GXR, one option being the A12 APS-C module with a 50mm f/2.5 equivalent lens. Unfortunately, it is not pocketable

According to Thom Hogan, Micro Four Thirds grabbed in a few months 11.5% of the market for interchangeable-lens cameras in Japan, something Pentax, Samsung and Sony have not managed despite years of trying. It’s probably just a matter of time before Canon and Nikon join the fray, after too long turning a deaf ear to the chorus of photographers like myself demanding a high-quality compact camera. As for myself, I have already voted with my feet, successively getting a Sigma DP1, Sigma DP2 and now a Panasonic GF1 with the 20mm f/1.7 pancake lens.

Update (2010-08-21):

I managed to score a Leica X1 last week from Camera West in Walnut Creek. Supplies are scarce and they usually cannot be found for love or money—many unscrupulous merchants are selling their limited stock on Amazon or eBay, at ridiculous (25%) markups over MSRP.

So far, I like it. It may not appear much smaller than the GF1 on paper, but in practice those few millimeters make a world of difference. The GF1 is a briefcase camera, not really a pocketable one, and I was subconsciously leaving at home most of the time. The X1 fits easily in any jacket pocket. It is also significantly lighter.

High ISO performance is significantly better than the GF1 – 1 to 1.5 stops. The lens is better than reported in technical reviews like DPReview’s—it exhibits curvature of field, which penalizes it in MTF tests.

The weak point in the X1 is its relatively mediocre AF performance. The GF1 uses a special sensor that reads out at 60fps, vs. 30fps for most conventional sensors (and probably even less for the Sony APS-C sensor used in the X1, possibly the same as in the Nikon D300). This doubles the AF speed of its contrast-detection algorithm over its competitors. Fuji recently introduced a special sensor that features on-chip phase-detection AF (the same kind used in DSLRs), let’s hope the technology spreads to other manufacturers.

The Ghola asset management program

I am now using Kavasoft Shoebox and thus this whole entry is obsolete and kept only for historical purposes. It is interesting to see one of my requirements anticipated Aperture’s stacks.

Introduction

I am in the process of migrating from Windows to Mac OS X as my primary home computing environment. IMatch is one of the key applications I need to migrate, but it is not available on the Mac. Ghola is an attempt to reproduce the key functionality of IMatch, and possibly go beyond. It is also a good way to learn Cocoa programming (my last programming experience on the Mac goes back to Think C 4.0 accessing raw QuickDraw calls).

Use cases

  1. Assign categories to images or a folder of images.
  2. Search for images matchin an expression of categories.
  3. Restructure a category (e.g. splitting a category grown too large into multiple, more manageable subcategories).

Requirements (incomplete)

The following are the key features from IMatch that need to be carried over:

  • The flexible set-oriented category system.

  • Support for RAW images (CRW and NEF).

  • Fast retrieval performance.

  • Extensible metadata.

  • Offline media support.

The following are the requirements for Ghola beyond IMatch:

  • Ability to group very similar variants of an image together. This would allow to group an image original, retouched versions, cropped and resized versions, or multiple very similar images taken in succession, yet manage them as a single logical unit. The role of each image in the group would be identified as well as part of its group membership, and each group would have a leader used by default.

  • Manage assets beyond images, such as PDF files.

  • Highly efficient categorization user interface.

  • Scriptable in Python.

  • HTML gallery generation, integrated with Mylos.

Implementation directions

The system core will be implemented in Python, and C if necessary. It should be as portable as possible. Possibly even multi-user and remotable.

SQLite will be used as the core database. Fast, simple, easy to manage.

GUI front-ends will use the native toolkits (PyObjC) whenever possible for optimal user experience and Aqua compliance.

A command-line UI could be more efficient for category assignment, if it is augmented with features like completion.

We may have to use bitmap indexes for efficient category indexing. Boolean operations on Python long integers are surprisingly fast, and C might not be needed at this point.

Colophon

The System name is a reference to Frank Herbert’s Dune (the original series, of course, not the opportunistic add-ons by Brian Herbert. It has the nice side benefit of not being already used by another open-source program.

Mylos

I switched to WordPress at the end of 2009 for the reasons expressed elsewhere, then to Hugo in 2017, which is going back in the opposite direction, and this entry is here for historical purposes only.

Mylos is my home-grown weblog management software. I wrote my first web pages by hand in Emacs and RCS in 1993, but stopped maintaining them in 1996 or so. I only restarted one with Radio last year. After a year of weblogging, however, I find I am frustrated by the limitations of Radio as well as its web-based user interface (I am one of those rare people who prefer command-line user interfaces and non-WYSIWYG HTML editors). I guess I could have extended Radio using UserLand’s Frontier language it is implemented in, but I have no interest in learning yet another oddball scripting language.

I decided in April 2003 to roll my own system, implemented in Python. In my career at various ISPs, I had to kill home-grown content-management system (CMS) projects gone awry, and I was certainly aware that these projects have a tendency to go overboard. Still, it has taken me three months of (very) part-time work to get the system to a point where it generates usable pages and imports my legacy pages from Radio without a hitch.

The implemented requirements for Mylos are:

  • Migration of my existing Radio weblog entries and stories (done, but not in an entirely generic fashion, is theme-dependent)
  • All pages are static HTML, no requirements for CGI scripts, PHP, databases or the like
  • Implemented and extensible in Python
  • Separation of content and presentation using themes (based on Webware Python Server Pages and CSS)
  • Support for navigational hierarchy
  • Articles are stored as regular files on the filesystem where they can be edited using conventional tools if necessary, no need for proprietary databases
  • Extensible article metadata
  • Atom 1.0 syndication, with separate feeds for subcategories
  • Use only relative URLs in hyperlinks to allow easy relocation
  • Automatic entry HTML cleanup for XHTML compliance
  • A CSS-based layout where the blogroll doesn’t wrap around short bodies (e.g. on permalink pages for short articles).
  • reasonable defaults, e.g. don’t try to create a weblog entry for an image that is colocated with an article, just copy it
  • Built-in multithreaded external link validation.
  • Automatic URL remapping (/mylos/ becomes relative to the Mylos root, relative URLs in an entry are automatically prefixed in containers like home pages).
  • Ability to review an article before publishing
  • Lynx compliance
  • Automatically cache external images in weblog entries in case they disappear (but do not use them as such due to potential copyright issues)
  • Set robots meta tag so only permalinks are indexed and cached by search engines, for better relevance to search engine users (albeit at the cost of lower rankings for the home page).
  • Sophisticated image galleries fully integrated with the navigation
  • Automatic code fragment colorization using Pygments

These features are planned but not yet implemented:

  • Keyword index.
  • Enhanced support for books via Allconsuming and Amazon.
  • Automated dependency tracking to re-render only the pages affected by a change (via SCons)
  • Multi-threaded rendering (via SCons)
  • Automatically add height, width and alt tags to img tags
  • Auto abbreviation glossary as tooltip help using tags
  • Typographically clean results, as done by SmartyPants
  • Feedback loop via on-page comments
  • Notification of new comments by email
  • Ability to promote a weblog entry to a story if it reaches critical mass

These features are “blue-sky”, don’t hold your breath for them:

  • Updates by email
  • User-submitted ratings for articles
  • Support for multilingual weblogs

Features thet are not planned at all (anti-requirements) include:

  • Synchronization or upload to server – rsync does this far better
  • Text editor – use $VISUAL or $EDITOR, whether Emacs, vi, or whatever
  • Web user interface – Radio’s web interface has very poor usability in my personal opinion, and this is due to the fact it is web-based, not any fault of Userland’s
  • RSS 1.0 – RDF seems like an exercise in intellectual masturbation
  • Blogger API or similar – although someone else could certainly write a bridge in Python if needed

The software is currently not in a state where it can be used by anyone else. I am not sure if there is any demand for such a tool in any case, if so, I would certainly consider documenting it better and making a SourceForge project out of it.

By the way, the system is named “Mylos” after a city in the magnificent illustrated series “Les Cités Obscures” by Belgian architects and writers Schuiten and Peeters, more specifically L’Enfant Penchéee

Cover for L'Enfant Penchée

Louvre Panorama

For your enjoyment, a 360° immersive panorama I took of the Louvre courtyard in 1998. Java-enabled browser required (I tested this with Mozilla 1.1 and IE 6 with the Sun JRE 1.4.0 plug-in).

Windows configuration management

The key to running a reasonably reliable Windows system is configuration management. A typical Windows will have tens of thousand of files and hundreds of software components installed. It’s a numbers game: the more components interacting on the system, the greater the probability that two of them will conflict.

Windows gets a lot of heat from Unix zealots (I am one myself) for being unreliable, but any operating system that attempts to comprehensively support all the wide variety of oddball peripherals and software out there is going to experience the same integration problems; certainly, Linux is converging towards Windows in terms of the number of security advisories released. Of course, using an obsolete version like 98 or ME without modern protected memory is a prescription for disaster, but the NT-based versions, i.e. 2000 and XP can have reasonable reliability, at least for desktop usage.

The rest of this article describes my strategy minimizing entropy in my Windows systems.

Separation

The way I approach my Windows configuration is to establish a clear separation between Operating System/Applications and Data.

The OS and Applications do not mean anything special to me other than the amount of work required to reinstall. Data represent actual productive work on my part and must be protected. I separate OS/Applications from data clearly, and make regular checkpoints of OS/Apps after installation and every now and then before I make major changes like installing an application or OS service packs. If my system becomes unstable at some point in time, I can easily revert to a known stable configuration.

The specific tools used to provide this backup of system configuration are a question of personal choice. A number of commercial software like Roxio GoBack or Powerquest SecondChance (since discontinued) purport to do this, as does Windows XP.

I personally don’t trust these programs all that much, and prefer to make a total backup of my system using Norton Ghost. To ensure my data is not erased when I restore from a Ghost image, I have at least two partitions on each of my systems:

  1. C: for Windows and applications (NTFS)
  2. D: for my personal data (NTFS on desktops, FAT32 on laptops)
  3. I: for my Ghost images on desktops (FAT32), on a different drive than C: so I can survive a drive failure

That way I can destroy C: at my leisure, in the worst case I will have to reinstall a couple of applications and reapply some settings that were lost since the last release. My data sits safely on the D: partition (and backups).

Backup strategy

I don’t trust CD-R media or removable drive cartridges for backup purposes, and tape is either too slow or too expensive in the case of DLT. I keep full duplicates of my data partition and some Ghost images on a pair of 100GB external FireWire drives, one I keep at home and one at work. I rotate them weekly so even if my house burns down I will have lost in the worst case only a week of work or photos.

Limitations of this method

This technique doesn’t work very well if the underlying hardware configuration changes too often, and assumes a linear install history. If I install software A, then B, then C, I can go back from A+B+C to A+B or A but not to A+C.

How-to

This section shows how to extricate the data from OS/Apps which Windows and most apps usually try and commingle. The TweakUI utility from Microsoft is an absolute must-have. It is a control panel that allows you to change the behavior of the OS in vital ways that are not accessible otherwise short of editing the Registry directly.

Outlook

Outlook files are the single largest data files on my system (Ghost images do not count). By default, Outlook will create its PST file in the Documents and Settings directory. You can either relocate this directory to the D: partition, or create a new PST file in a location of your choice and use Advanced properties in the properties dialog for the PST in Outlook to make it the default location for POP delivery, after which you can close the old one and delete it.

My Documents

And derivatives like “My Pictures”, can be relocated to D: using TweakUI.

Favorites

For IE users, TweakUI allows you to relocate the Favorites directory to another place than the default. This way, you will not lose your favorites when you have to restore your system.

For Netscape/Mozilla users, the Profile manager utility allows you to set up a new profile with files that are stored where you want, e.g. on the D: partition.