Mylos

Migrating from Cyrus to Dovecot

I ran the Cyrus IMAP server for almost a year on my home server, but I recently switched to Dovecot. I originally used Cyrus because of its demonstrated scalability and in part because it is a product of my father’s alma mater, but it is quite hard to set up, and quite brittle to changes in its dependencies.

The last straw was when I tried unsuccessfully to set up another instance of Cyrus on a server, with the exact same configuration files and permissions, but different versions of the Berkeley DB and Cyrus SASL libraries, and it simply wouldn’t cooperate. In disgust, I downloaded Dovecot, compiled it and installed it in less time it took me just to figure out that Cyrus wouldn’t allow me to authenticate because the ever-crufty SASL library failed in a new inscrutable way. I had also never managed to get Cyrus’ SSL to work reliably, it is nearly effortless with Dovecot.

Dovecot is much easier to build and manage, does not have dependencies on unreliable cruft like the Cyrus SASL library, and is much easier to integrate with procmail, SpamAssassin and other goodies thanks to its use of the Maildir format rather than a proprietary database cum filesystem structure like Cyrus. From what I have seen of the internals of the Cyrus 2.2 “skiplist” database back-end (which replaced the BerkeleyDB back-end used in previous releases), I have a hard time believing it is significantly more efficient than Dovecot, if at all.

One problem was migrating my email – I have pretty much all my email since 1995 in my IMAP mailbox, migrated from various Emacs Babyl mailbox files or Exchange PSTs over time. The Dovecot Wiki points to this migration script, but for recent versions of Cyrus like the 2.2.12 I ran, it has two major shortcomings:

  1. It will not preserve the flag that indicates whether an email was read or not.
  2. It does not preserve the delivery timestamp for the emails so they all look as if they were delivered at the time you did the conversion.

I wrote my own migration script in Python, cyrus2maildir.py, to avoid these shortcomings. It does not preserve the “replied to” flag, but the Read flag is carried over, as is the delivery timestamp (in some edge cases like emails you sent, it has to guess, though). This is not a very polished program because I spent far more time on it than I had anticipated, and basically stopped once I got it working, but it should be usable, or at least a starting point for anyone with some Python skills. Of course, it can also be used by users of other Maildir++ based servers like Courier.

The script should guess most parameters, and will dump the emails to a directory named Maildir.cyrus/ in your home directory. By default, your user will not have read access to the Cyrus mail spool, you may have to alter permissions (I do not recommend running the script as root). For the syntax, just type: cyrus2maildir.py -h

On an unrelated note, Solaris 10 users may find the SMF manifest and method useful to automate service management and fault recovery. To install them, copy the manifest to /var/svc/manifest/site and the method to /lib/svc/method and install them into SMF with the command: svccfg import /var/svc/manifest/site/imap-dovecot.xml

Temboz 0.8 released

I am pleased to announce the release of Temboz 0.8.

The main change in this release is its ability to work with either SQLite 2.x or SQLite 3.x. SQLite 3.x is now the recommended version, see the Temboz Wiki for upgrade instructions. SQlite 3.x improves performance, database file sizes and concurrency, but it also introduced a condition where Temboz could deadlock, hence the long incubation time for this release.

Another enhancement is the ability to sort feeds by Signal to Noise Ratio (SNR). The default view for the all feeds page will list high-quality feeds with unread articles first. If you are catching up with many articles, it pays to concentrate on the richest lodes of information first, and possibly prune those that no longer provide an adequate level of interesting information..

I have a number of feature requests I received from users or thought up myself. You are welcome to suggest others on the ticket page for Temboz CVStrac.

Shun till done

One of the benefits of having written my own feed aggregator, Temboz, is that I get to implement the functionality I want to make my life more productive. The most essential one is filtering out articles in subjects I am not interested. One of the first companies to make the cut of those I shun entirely was SCO. I consider them so despicable I don’t even want to hear about them, unlike the hyperventilating Slashdot crowd. Since then, a number of other companies have joined them, most recently Sony. And in the spirit of “a pox on both houses”, I now tune out anything related to either HD-DVD or Blu-Ray.

That said, this form of shunning could fairly be described as passive-aggressive, not constructive. The fact I will no longer entertain the idea of buying a Playstation 3, and probably many others like me, will probably have some effect on Sony’s sales, but their marketing people analyzing sales figures will almost certainly have no clue their contemptible attitude to DRM is costing them. Much of modern economics is grounded in information theory, specially how markets break down in the presence of asymmetric information. If you want your product choices to have real impact, you need to go further.

Corporations are not monolithic entities, all carefully tailored appearances to the contrary notwithstanding. That’s why the New York Attorney-General, Eliot Spitzer is so effective against crooked Wall Street firms (like his predecessor Rudy Giuliani before him), corrupt record labels and the like. Even if a fine is small compared to a firm’s profits, it has to come out of somebody’s budget, and few careers survive that kind of blow. One fired employee, even a high-ranking one, is not much either compared to the typical large corporation’s staffing, but it will have a disproportionate effect on the remaining employees’ behavior. This has been very visible at IBM and now Microsoft after their anti-trust cases, even if in the latter case it seems to be more of a subconscious hesitancy to get anything done.

The average consumer does not have the punitive powers of a Spitzer at his disposal, but there is another way. Unless a corporation is terminally dysfunctional, it will have clear lines of accountability, all the way to the CEO reporting to the Board. The power we have is to dispel the cloud of obfuscation that some may use to to keep their upper management in the dark about the consequences of their actions.

Writing a letter to the CEO can be surprisingly effective. You have to keep in mind the average public company CEO works upwards of 70 hours per week. That lack of free time, combined with their affluence means they are usually out of touch with reality, and need to be reminded of it. This has to be a letter, preferably on good stationery, typewritten but hand-signed. Emails simply do not carry much weight in the worlds of politics or business, because they are so easily written and thus not evidence of commitment. Usually you want to skip flunkies and go straight to the top, but if the CEO is on record supporting the policies you object to, you will have to copy your letter to the president or the chairman of the Board.

Many companies have started monitoring blogs and forums for possible PR headaches (or subcontract this reputation monitoring work to specialized firms). Blogging about your experiences is another good way to get their attention.

There are limits to what an individual can accomplish. If a corporation is dead-set in its ways, collective action is required. In many ways, the American consumer movement has atrophied since the days of Ralph Nader’s crusade against Detroit, whereas the opposite trend holds true in Europe. There are many fine organizations like the EFF that are fighting for your rights, and sometimes even public officials, local or national.

The San Francisco chocolate lover’s shortlist

Here are my picks for the best chocolate places in the city (note: updated 2013-04-20):

  • Chocolate merchants: Noe Valley’s Chocolate Covered has made leaps and strides in the last 5 years, and beat previous favorite Fog City News
  • Honorable mentionFog City News, an impressive lineup tended by the knowledgeable owner, Adam Smith. Also the chocolate section at Rainbow Co-op.
  • Chocolate bouchées: Cocoa Bella. This shop is a chocolate integrator: it collects chocolates from small chocolatiers across the world and brings them under a single roof. They also make hot chocolate.
  • Honorable mention: Michael Recchiuti makes scrumptious confections, and his burnt caramel chocolate covered hazelnuts are to die for, as are many of his bars. Try also his Chocolate Lab in the Dogpatch for a cafe experience.
  • Chocolate maker: Guittard. This fourth-generation family of chocolatiers, originally from France, have been supplying professionals like Recchiuti for a century and half. The best dessert I ever had in America was a Guittard chocolate and cherry cake at Eno in Atlanta, of all places. They now have a retail line of very high quality.
  • Dishonorable mentions:
    • Scharffen-Berger: part of the evil Hershey empire, who are lobbying to have FDA standards watered down (so mockolate made with margarine can be passed off as real chocolate)
    • Tcho: overrated, and very simplistic, although their “Tchunky Tchotella” bar is amusing
    • Dandelion: sleazy hipster outfit that turns good raw ingredients into crude dreck
    • L’Amourette: another overrated local brand. The packaging for their “70% Dark Chocolate Gold” screams “Venezuela” and “Sur Del Lago”, but only mentions in small type they use the inferior Trinitario cacao instead of the noble Criollo the provenance (and price) would imply.
  • Hot chocolate: Christopher Elbow on Gough & Hayes has an intense hazelnut-flavored hot chocolate.
  • Honorable mention: Charles Chocolates (disclaimer: I am an investor)
  • Chocolate pastries: Cafe Madeleine, a.k.a. Jil’s Patisserie, formerly of Burlingame, now made in their New Montgomery Street shop (with two additional locations on California and O’Farrell).
  • Honorable mentions: Miette in the Ferry Building. Tartine’s chocolate hazelnut tart. B Patisserie’s chocolate Kouign Amann.

See also my Google map of the best sweet treats in San Francisco