Seth Woolley's Man Viewer

Manual for maildir - man 5 maildir

([section] manual, -k keyword, -K [section] search, -f whatis)
man plain no title

MAILDIR(5)                                                          MAILDIR(5)



NAME
       maildir(1,5) - E-mail directory

SYNOPSIS
       $HOME/Maildir

DESCRIPTION
       A  ``Maildir''  is  a  structured directory that holds E-mail messages.
       Maildirs were first implemented by  the  Qmail  mail(1,8)  server.   Qmail's
       maildirs  were a simple data structure, nothing more than a single col-
       lection of E-mail messages.  Courier builds upon  Qmail's  maildirs  to
       provide extended functionality, such as folders and quotas.  This docu-
       ment describes Courier's extended maildirs, without explicitly  identi-
       fying  Courier-specific extensions.  See maildir(1,5)(5) in(1,8) Qmail's documen-
       tation for the original definition of maildirs.

       Traditionally, E-mail folders were saved as plain  text  files,  called
       ``mboxes''.   Mboxes  have known limitations.  Only one application can
       use an mbox at the same time.  Locking is required in(1,8)  order  to  allow
       simultaneous  concurrent  access(2,5) by different applications.  Locking is
       often problematic, and not very reliable  in(1,8)  network-based  filesystem
       requirements.   Some network-based filesystems don't offer any reliable
       locking mechanism at all.  Furthermore, even bulletproof locking  won't
       prevent  occasional mbox corruption.  A process can be killed or termi-
       nated in(1,8) the middle of updating an mbox.  This will  likely  result  in(1,8)
       corruption, and a loss of most messages in(1,8) the mbox.

       Maildirs  allow  multiple  concurrent access(2,5) by different applications.
       Maildirs do not require locking.  Multiple applications  can  update(7,n)  a
       maildir(1,5) at the same time(1,2,n), without stepping on each other's feet.

   MAILDIR CONTENTS
       A  ``maildir(1,5)''  is a directory that's created by maildirmake(1).  Natu-
       rally, maildirs should not have any group or world permissions,  unless
       you want other people to read(2,n,1 builtins) your mail.  A maildir(1,5) contains three sub-
       directories: tmp, new, and cur.  These  three  subdirectories  comprise
       the primary folder, where new mail(1,8) is delivered by the system.

       Folders  are additional subdirectories in(1,8) the maildir(1,5) whose names begin
       with a period: such as .Drafts or .Sent.  Each folder  itself  contains
       the  same  three  subdirectories,  tmp, new, and cur, and an additional
       zero-length file(1,n) named(5,8) maildirfolder, whose purpose is  to  inform  any
       mail(1,8)  delivery  agent that it's really delivering to a folder, and that
       the mail(1,8) delivery agent should look(1,8,3 Search::Dict) in(1,8) the  parent  directory  for  any
       maildir-related information.

       Folders  are  not  physically  nested.   A folder subdirectory, such as
       .Sent does not itself contain any subfolders.  The  main  maildir(1,5)  con-
       tains  a  single, flat list of subfolders.  These folders are logically
       nested, and periods serve to separate folder hierarchies.  For example,
       .Sent.2002  is  considered to be a subfolder called ``2002'' which is a
       subfolder of ``Sent''.

   FOLDER NAME ENCODING
       Folder names can contain any  Unicode  character,  except  for  control
       characters.   US-ASCII  characters, U+0x0020 - U+0x007F, except for the
       period, forward-slash, and ampersand  characters  (U+0x002E,  U+0x002F,
       and  U+0x0026) represent themselves.  The ampersand is represent by the
       two character sequence ``&-''.  The period, forward slash, and non  US-
       ASCII Unicode characters are represented using the UTF-7 character set(7,n,1 builtins),
       and encoded with a modified form of base64-encoding.

       The ``&'' character starts the modified  base64-encoded  sequence;  the
       sequence  is terminated by the ``-'' character.  The sequence of 16-bit
       Unicode characters is written in(1,8) big-endian order,  and  encoded  using
       the  base64-encoding  method described in(1,8) section 5.2 of RFC 1521, with
       the following modifications:

        The ``='' padding character is omitted.  When decoding, an incomplete
         16-bit character is discarded.

        The comma character, ``,'' is used in(1,8) place of the ``/'' character in(1,8)
         the base64 alphabet.

       For example, the word ``Resume'' with both "e"s being the e-acute char-
       acter,  U+0x00e9, is encoded as ``R&AOk-sum&AOk-'' (so a folder of that
       name would be a maildir(1,5) subdirectory called ``.R&AOk-sum&AOk-'').

   OTHER MAILDIR CONTENTS
       Software that uses maildirs may also create  additional  files  besides
       the  tmp,  new, and cur subdirectories -- in(1,8) the main maildir(1,5) or a sub-
       folder -- for its own purposes.

   MESSAGES
       E-mail messages are stored in(1,8) separate, individual  files,  one  E-mail
       message  per file.  The tmp subdirectory temporarily stores E-mail mes-
       sages that are in(1,8) the process of being delivered to this maildir.   tmp
       may also store other kinds of temporary files, as long as they are cre-
       ated in(1,8) the same way that message files are created in(1,8)  tmp.   The  new
       subdirectory  stores messages that have been delivered to this maildir(1,5),
       but have not yet been seen by any mail(1,8) application.  The cur  subdirec-
       tory  stores messages that have already been seen by mail(1,8) applications.

   ADDING NEW MAIL TO MAILDIRS
       The following process delivers a new message to the maildir:

       A new unique filename is created  using  one  of  two  possible  forms:
       ``time.MusecPpid.host'',  or  ``time.MusecPpid_unique.host''.  ``time(1,2,n)''
       and ``usec'' is the current system time(1,2,n), obtained from gettimeofday(2).
       ``pid''  is  the  process number of the process that is delivering this
       message to the maildir.  ``host(1,5)'' is the name of the machine where  the
       mail(1,8)  is  being  delivered.  In the event that the same process creates
       multiple messages, a suffix unique to each message is appended  to  the
       process  id;  preferrably  an  underscore,  followed  by  an increasing
       counter. This applies whether messages created by  a  process  are  all
       added to the same, or different, maildirs.  This protocol allows multi-
       ple processes running on multiple  machines  on  the  same  network  to
       simultaneously create new messages without stomping on each other.

       The  filename  created in(1,8) the previous step is checked for existence by
       executing the stat(1,2)(2) system call.   If  stat(1,2)(2)  results  in(1,8)  ANYTHING
       OTHER than the system error(8,n) ENOENT, the process must sleep(1,3) for two sec-
       onds, then go back and create another  unique  filename.   This  is  an
       extra  step  to  insure  that  each new message has a completely unique
       filename.

       Other applications that wish to use tmp for  temporary  storage  should
       observe  the  same  protocol (but see READING MAIL FROM MAILDIRS below,
       because old files in(1,8) tmp will be eventually deleted).

       If the stat(1,2)(2) system call returned ENOENT, the process may proceed  to
       create the file(1,n) in(1,8) the tmp subdirectory, and save the entire message in(1,8)
       the new file.  The message saved MUST NOT  have  the  ``From_''  header
       that  is  used to mboxes.  The message also MUST NOT have any ``From_''
       lines in(1,8) the contents of the message prefixed by the ``>'' character.

       When saving the message, the number of bytes returned by  the  write(1,2)(2)
       system  call  must  be checked, in(1,8) order to make sure that the complete
       message has been written out.

       After the message is saved, the file(1,n) descriptor  is  fstat(2)-ed.   The
       file(1,n)'s  device  number, inode number, and the its byte size, are saved.
       The file(1,n) is closed and is then immediately moved/renamed into  the  new
       subdirectory.   The name of the file(1,n) in(1,8) new should be ``time.MusecPpid-
       VdevIino.host,S=cnt'', or ``time.MusecPpidVdevIino_unique.host,S=cnt''.
       ``dev''  is the message's device number, ``ino'' is the message's inode
       number (from the previous fstat(2) call); and ``cnt'' is the  message's
       size, in(1,8) bytes.

       The  ``,S=cnt''  part optimizes Courier's maildir(1,5) quota(1,8) enhancement; it
       allows the size of all the mail(1,8) stored in(1,8) the maildir(1,5) to  be  added  up
       without  issuing  the  stat(1,2)(2)  system call for each individual message
       (this can be quite a performance drain with  certain  network  filesys-
       tems).

   READING MAIL FROM MAILDIRS
       Applications that read(2,n,1 builtins) mail(1,8) from maildirs should do it in(1,8) the following
       order:

       When opening a maildir(1,5) or a maildir(1,5) folder, read(2,n,1 builtins) the  tmp  subdirectory
       and delete any files in(1,8) there that are at least 36 hours old.

       Look for new messages in(1,8) the new subdirectory.  Rename new/filename, as
       cur/filename:2,info(1,5,n).  Here, info(1,5,n) represents the state of  the  message,
       and  it  consists of zero or more boolean flags chosen from the follow-
       ing: ``D'' - this is a 'draft' message, ``R'' - this message  has  been
       replied  to,  ``S'' - this message has been viewed (seen), ``T'' - this
       message has been marked to be deleted (trashed), but is not yet removed
       (messages  are  removed  from  maildirs simply by deleting their file(1,n)),
       ``F'' - this message has been marked by the  user,  for  some  purpose.
       These flags must be stored in(1,8) alphabetical order.  New messages contain
       only the :2, suffix, with no flags, indicating that the  messages  were
       not seen, replied, marked, or deleted.

       Maildirs  may  have  maximum  size quotas defined, but these quotas are
       purely voluntary.  If you  need  to  implement  mandatory  quotas,  you
       should  use  any quota(1,8) facilities provided by the underlying filesystem
       that is used to store the maildirs.  The maildir(1,5) quota(1,8)  enhancement  is
       designed to be used in(1,8) certain situations where filesystem-based quotas
       cannot be used for some reason.   The  implementation  is  designed  to
       avoid the use of any locking.  As such, at certain times the calculated
       quota(1,8) may be imprecise, and certain anomalous situations may result  in(1,8)
       the  maildir(1,5)  actually  going over the stated quota. One such situation
       would be when applications create messages without updating  the  quota(1,8)
       estimate  for  the  maildir.   Eventually it will be precisely recalcu-
       lated, but wherever possible new messages should be created in(1,8)  compli-
       ance with the voluntary quota(1,8) protocol.

       The  voluntary  quota(1,8) protocol involves some additional procedures that
       must be followed when creating or  deleting  messages  within  a  given
       maildir(1,5)  or  its  subfolders.   The  deliverquota(8)  command is a tiny
       application that delivers a single message to a maildir(1,5) using the  vol-
       untary  quota(1,8)  protocol,  and  hopefully it can be used as a measure of
       last resort.  Alternatively,  applications  can  use  the  libmaildir.a
       library  to handle all the low-level dirty details for them. The volun-
       tary quota(1,8) enhancement is described in(1,8) the maildirquota(7) man(1,5,7) page.

   MAILDIR QUOTAS
       This is a voluntary mechanism for enforcing "loose" quotas on the maxi-
       mum sizes of maildirs.  This mechanism is enforced in(1,8) software, and not
       by the operating system.  Therefore it is only effective as long as the
       maildirs  themselves  are not directly accessible by their users(1,5), since
       this mechanism is trivially disabled.

       If possible, operating system-enforced quotas are  preferrable.   Where
       operating  system  quota(1,8) enforcement is not available, or not possible,
       this voluntary quota(1,8) enforcement mechanism might be an acceptable  com-
       promise.   Since  it's enforced in(1,8) software, all software that modifies
       or accesses the maildirs is required to voluntary obey  and  enforce  a
       quota.   The voluntary quota(1,8) implementation is flexible enough to allow
       non quota-aware applications to also access(2,5) the maildirs,  without  any
       drastic  consequences.   There  will  be some non-drastic consequences,
       though.  Of course, non quota-aware applications will not  enforce  any
       defined  quotas.   Furthermore,  this voluntary maildir(1,5) quota(1,8) mechanism
       works by estimating the current size  of  the  maildir(1,5),  with  periodic
       exact  recalculation.   Obviously  non quota-aware maildir(1,5) applications
       will not update(7,n) the maildir(1,5) size estimation, so the  estimate  will  be
       thrown off for some period of time(1,2,n), until the next recalculation.

       This  voluntary  quota(1,8) mechanism is designed to be a reasonable compro-
       mise between effectiveness, and performance.   The  entire  purpose  of
       using  maildir-based  mail(1,8) storage is to avoid any kind of locking, and
       to permit parallel access(2,5) to mail(1,8) by multiple applications.   In  order
       to  compute  the  exact  size  of a maildir(1,5), the maildir(1,5) must be locked
       somehow to prevent any modifications while its contents are  added  up.
       Obviously  something  like  that  defeats the original purpose of using
       maildirs, therefore the voluntary quota(1,8) mechanism does not use locking,
       and  that's  why the current recorded maildir(1,5) size is always considered
       to be an estimate.  Regular size recalculations will compensate for any
       occasional  race  conditions  that  result in(1,8) the estimate to be thrown
       off.

       A quota(1,8) for an existing maildir(1,5) is  installed  by  running  maildirmake
       with  the  -q  option,  and  naming an existing maildir.  The -q option
       takes a parameter, quota(1,8), which is  a  comma-separated  list  of  quota(1,8)
       specifications.  A quota(1,8) specification consists of a number followed by
       either 'S', indicating the maximum message size in(1,8) bytes, or 'C', maxi-
       mum number of messages. For example:




              maildirmake -q 5000000S,1000C ./Maildir
       This  sets  the  quota(1,8)  to  5,000,000 bytes or 1000 messages, whichever
       comes first.




              maildirmake -q 1000000S ./Maildir
       This sets the quota(1,8) to 1,000,000 bytes, without limiting the number  of
       messages.

       A  quota(1,8)  of  an  existing  maildir(1,5)  can  be  changed  by rerunning the
       maildirmake command with a new -q option.  To delete a quota(1,8)  entirely,
       delete the Maildir/maildirsize file.

SEE ALSO
       maildirmake(1).



Double Precision, Inc.           12 July 2004                       MAILDIR(5)

References for this manual (incoming links)