
Mailbox Creation Distribution
*****************************

By default, when creating a mailbox in Cyrus IMAP:

* the backend with the most free disk space is selected on the
  Murder frontend

* the partition with the most free disk space is selected on the
  backend

This may not be the most appropriate backend or partition to create
the new mailbox on, and Cyrus IMAP therefor allows for a variety of
additional modes of calculating and selecting the most appropriate
backend and partition. The exact mode for the selection is controlled
with the "imapd.conf" setting "serverlist_select_mode" on the frontend
and "partition_select_mode" on the backend.

Alternatively, a default backend can be configured with the
"defaultserver" setting on a frontend, and a default partition can be
configured with the "defaultpartition" on a backend.

Prior to Cyrus IMAP version 2.5, when creating a mailbox, should no
target partition have been specified, the mailbox is either created
on:

* the configured "defaultserver", or

* the server that is found to have the most free disk space, and

* the configured "defaultpartition", or

* the most fitting partition if "defaultpartition" is not
  configured.

New configuration options are available since Cyrus IMAP 2.5, that
allow more weighted and better balanced backend and partition
selection.


Selection Mode
==============

Among the partitions, how the most fitting one is selected depends on
the configured selection mode: "partition_mode" setting in
"/etc/imapd.conf"


Available Selection Modes
-------------------------

* random

     Choice is (pseudo-)random. Each partition has the same
     probability of being selected.

* freespace-most

     The partition which has the most free space (counted in KiB
     units) is selected.

     Note: Example of "freespace-most" selection on backendSuppose
       that the configured partitions are:

       * "part1" which has a 1000GiB capacity, 400GiB being free
         (that is, 40% free space)

       * "part2" which has a 1000GiB capacity, 600GiB being free
         (that is, 60% free space)

       * "part3" which has a 100GiB capacity, 30GiB being free (that
         is, 30% free space)

       * "part4" which has a 100GiB capacity, 70GiB being free (that
         is, 70% free space)

       In that case "part2" will be selected as most fitting, since
       600GiB of free space is the biggest of all partitions.

* freespace-percent-most

     The partition which has the most percentage of free space is
     selected.

     Note: Example of "freespace-percent-most" selection on
       backendIn the same conditions, "part4" would be selected as
       most fitting, since 70% of free space is the biggest of all
       partitions.

* freespace-percent-weighted

     For each partition, the percentage of free space is its weight.
     Then a weighted choice is performed to select one of those
     partitions.

     As such, the more free space the partition has, the more chances
     it has to be selected.

     Note: Example of "freespace-percent-weighted" selection on
       backendIn the same conditions, the weight of each partition
       would be:

       * 40 for "part1"

       * 60 for "part2"

       * 30 for "part3"

       * 70 for "part4"

       The sum of each weight being 200, the probability for each
       partition to be selected as most fitting would be:

       * 20% for "part1"

       * 30% for "part2"

       * 15% for "part3"

       * 35% for "part4"


Usage convergence
-----------------

In "freespace-percent-weighted" mode, partitions percentage usages
converge towards 100%. So if they have different usages, those
differences will stay and only really diminish upon reaching 100% of
usage.

You may also observe growing differences between partitions usage when
those partitions do not have the same total disk space.

* freespace-percent-weighted-delta

As for "freespace-percent-weighted", a weight is associated to each
partition. It is computed as follows: (*percentage of freespace of
partition*) - (*lowest percentage of freespace of all partitions*) +
0.5

Then a weighted choice is performed to select one of those partitions.

As such, considering the percentages of usage, the more the partition
is lagging behind the most used partition (which is the one with the
lowest percentage of free space), the more chances it has to be
selected.


Computed weight
---------------

The added 0.5 in partitions weight is so that selection gets smoother
when all partitions get close to each other.

Note: **Example of ``freespace-percent-weighted-delta`` Selection on
  Backend**In the same conditions, the weight of each partition would
  be:

  * 40 - 30 + 0.5 = 10.5 for "part1"

  * 60 - 30 + 0.5 = 30.5 for "part2"

  * 30 - 30 + 0.5 = 0.5 for "part3"

  * 70 - 30 + 0.5 = 40.5 for "part4"

  Then the probability for each partition to be selected as most
  fitting would be:

  * 12.8% for "part1"

  * 37.2% for "part2"

  * 0.6% for "part3"

  * 49.4% for "part4"

  **Usage convergence**In "freespace-percent-weighted-delta" mode,
  partitions percentage usages converge towards the most used one. And
  then partitions usages grow equally.


Special cases
=============


What happens when two partitions are equal as most fitting?
-----------------------------------------------------------

Suppose you are using the "freespace-most" selection mode, that two
(or more) partitions have the same free size, and that this freespace
happens to be the biggest one of all configured partitions.

In that case, only one of those partitions will be selected. You may
not know in advance which one will be: it depends of the order in
which configured partitions are stored in memory ("hashtable"). In
particular, it may not be the first one that appears listed in your
"/etc/imapd.conf" configuration file.

Also note that since the selected partition will now have less free
space, it shall not be seen as most fitting next time.


What happens when two partitions point to the same device?
----------------------------------------------------------

Suppose you are using the "freespace-most" or "freespace-percent-most"
selection mode, and that two (or more) partitions actually point to
the same device (that is the device id is the same).

In that case, only one of those partitions will be checked, as if the
others were not configured. You may not know in advance which one will
be: it depends of the order in which configured partitions are stored
in memory ("hashtable"). In particular, it may not be the first one
that appears listed in your "/etc/imapd.conf" configuration file.


Partitions Exclusion
====================

Partitions listed in the "partition_mode_exclude" setting are
permanently excluded from being selected.

* partition_mode_exclude

     Listed partitions names are separated by space or comma. Only
     configured partition names (and not paths) are expected.

     Note: Example of permanent partition exclusionWith the
       following configuration in */etc/imapd.conf*, "parta" and
       "partb" would be permanently excluded from being selected
       :partition-part1: /path/to/part1 partition-part2:
       /path/to/part2 partition-part3: /path/to/part3 partition-part4:
       /path/to/part4 partition-parta: /path/to/parta partition-partb:
       /path/to/partbpartition_mode_exclude: parta partbWhen using a
       selection mode other than "random", partitions are
       automatically excluded if their usage percentage is beyond the
       "partition_mode_soft_usage_limit" integer setting.

* partition_mode_soft_usage_limit

     If all partitions are beyond the configured value, this feature
     is automatically deactivated. A partition is thus selected as if
     the setting was not set.

     Note: Example of partition exclusion using
       "partition_mode_soft_usage_limit"In the same conditions than
       <xref linkend="exam-Deployment_Guide-
       Available_Selection_Modes_on_Backend-
       Example_of_freespace_most_Selection_on_Backend" />, setting
       "partition_mode_soft_usage_limit" to "50" would exclude
       partitions "part1" and "part3" since their disk usage is
       respectively 60% and 70%.But setting the option to "20" would
       have no effect, since the usage of all partitions is beyond
       20%.

     Warning: **Exclusion is not absolute**Partitions are only
       excluded when creating a new user mailbox according to the
       configured selection mode.If you explicitly specify an excluded
       partition, the mailbox will be created on that
       partition.Mailboxes previously created on such partitions are
       of course still accessible, and subfolders are by default still
       created on the same partition as the parent folder.


Partitions Usage Data Reset
===========================

By default partitions usage data are retrieved only once upon service
initialization. This only concerns selection modes other than
"random".

If you tend to use the same service instance for a long lapse of time
and performs a large amount of mailboxes creation, it may be useful to
configure the "partition_mode_usage_reinit" so that partitions usage
data are refreshed after the configured number of creation requests.


Mailbox Creation Distribution Through "murder frontend"
=======================================================

Upon creating a user mailbox, if the target server is not given as
extra parameter, the mailbox is either created on

* the configured "defaultserver" backend

* the most fitting backend among the servers listed in the
  "serverlist" setting, if "defaultserver" is not configured


Selection Mode
--------------

Among the backends, how the most fitting one is selected depends on
the configured selection mode: "serverlist_mode" setting in
"/etc/imapd.conf".

The principle is similar to the mailbox creation distribution on
backend (see <xref linkend="sect-Deployment_Guide-
Mailbox_Creation_Distribution-On_backend" />).


Available Selection Modes on Frontend
-------------------------------------

* random

     Choice is (pseudo-)random. Each backend has the same probability
     of being selected.

* freespace-most

     The backend which has the most free space (counted in KiB units)
     is selected. The considered free space is the sum of all
     available partitions free space on the backend.

     Note: Example of "freespace-most" Selection on FrontendSuppose
       that the configured backends are:

       * "backend1" which has a 2000GiB capacity, 1000GiB being free
         (that is, 50% free space), composed of 2 partitions

       * "part1" which has a 1000GiB capacity, 500GiB being free
         (that is, 50% free space)

       * "part2" which has a 1000GiB capacity, 500GiB being free
         (that is, 50% free space)

            * "backend2" which has a 2000GiB capacity, 900GiB being
              free (that is, 45% free space), composed of 2 partitions

            * "part1" which has a 1000GiB capacity, 200GiB being
              free (that is, 20% free space)

            * "part2" which has a 1000GiB capacity, 700GiB being
              free (that is, 70% free space)

                 * "backend3" which has a 200GiB capacity, 110GiB
                   being free (that is, 55% free space), composed of 2
                   partitions

                 * "part1" which has a 100GiB capacity, 30GiB being
                   free (that is, 30% free space)

                 * "part2" which has a 100GiB capacity, 80GiB being
                   free (that is, 80% free space)

       * In that case "backend1" will be selected as most fitting,
         since 1000GiB of free space is the biggest of all backends.

* freespace-percent-most

     On each backend, the partition with the most percentage of free
     space is considered. The selected backend is the one whose
     partition has the most percentage of free space.

     Note: Example of "freespace-percent-most" Selection on
       FrontendIn the same conditions than <xref linkend="exam-
       Deployment_Guide- Available_Selection_Modes_on_Frontend-
       Example_of_freespace_most_Selection_on_Frontend" /> "backend3"
       would be selected as most fitting, since it has a partition
       with 80% of free space which is the biggest of all backends.

* freespace-percent-weighted

     On each backend, the partition with the most percentage of free
     space is considered: it is the backend weight. Then a weighted
     choice is performed to select one of the backends.

     Note: Example of "freespace-percent-weighted" Selection on
       FrontendIn the same conditions than <xref linkend="exam-
       Deployment_Guide- Available_Selection_Modes_on_Frontend-
       Example_of_freespace_most_Selection_on_Frontend" />, the weight
       of each backend would be:

          * 50 for "backend1"

          * 70 for "backend2"

          * 80 for "backend3"

       The sum of each weight being 200, the probability for each
       backend to be selected as most fitting would be:

          * 25% for "backend1"

          * 35% for "backend2"

          * 40% for "backend3"

* freespace-percent-weighted-delta

On each backend, the partition with the most percentage of free space
is considered. As for "freespace-percent-weighted", a weight is
associated to each backend. It is computed as follows: (*percentage of
freespace on backend*) - (*lowest percentage of freespace of all
backends*) + 0.5

Then a weighted choice is performed to select one of the backends.

   Note: **Example of ``freespace-percent-weighted-delta`` Selection
     on Frontend**In the same conditions than <xref linkend="exam-
     Deployment_Guide-Available_Selection_Modes_on_Frontend-
     Example_of_freespace_most_Selection_on_Frontend" />, the weight
     of each backend would be:

     * 50 - 50 + 0.5 = 0.5 for "backend1"

     * 70 - 50 + 0.5 = 20.5 for "backend2"

     * 80 - 50 + 0.5 = 30.5 for "backend3"

     Then the probability for each backend to be selected as most
     fitting would be:

     * 1.0% for "backend1"

     * 39.8% for "backend2"

     * 59.2% for "backend3"


Backends Exclusion
==================

When using a selection mode other than "random", backends are
automatically excluded if their considered usage percentage is beyond
the "serverlist_mode_soft_usage_limit" integer setting.

* "partition_mode_soft_usage_limit"

If all backends are beyond the configured value, this feature is
automatically deactivated. A backend is thus selected as if the
setting was not set.

   Note: Example of partition exclusion using
     "serverlist_mode_soft_usage_limit"In the same conditions than
     <xref linkend="exam-Deployment_Guide-
     Available_Selection_Modes_on_Frontend-
     Example_of_freespace_most_Selection_on_Frontend" />, using
     "freespace-most" selection mode, setting
     "serverlist_mode_soft_usage_limit" to "49" would exclude
     "backend1" and "backend2" since in that mode they have a disk
     usage of 50% and 55%. In other modes it would however only
     exclude "backend1" whose considered partition has a disk usage of
     50%, while on "backend2" the considered partition has a disk
     usage of 30%.

   Warning: **Exclusion is not absolute**Backends are only excluded
     when creating a new user mailbox according to the configured
     selection mode.If you explicitly specify an excluded backend, the
     mailbox will be created on that backend.Mailboxes previously
     created on such backends are of course still accessible.


Backends Usage Data Reset
=========================

By default backends usage data are retrieved only once upon service
initialization. This only concerns selection modes other than
"random".

If you tend to use the same service instance for a long lapse of time
and performs a large amount of mailboxes creation, it may be useful to
configure the "serverlist_mode_usage_reinit" so that backends usage
data are refreshed after the configured number of creation requests.
