doc: TODO and release notes updates ahead of 1.6

[public-inbox.git] / Documentation / public-inbox-v2-format.pod

diff --git a/Documentation/public-inbox-v2-format.pod b/Documentation/public-inbox-v2-format.pod
index 0a391e1e3fb0a4a183e2029467ae06373f2aa6fe..86a9b8f224905fa3ae89613e880c71c8dc130afd 100644 (file)
--- a/Documentation/public-inbox-v2-format.pod
+++ b/Documentation/public-inbox-v2-format.pod
@@ -2,7 +2,7 @@
 
 =head1 NAME
 
 
 =head1 NAME
 

-public-inbox v2 repository description
+public-inbox v2 format description

 
 =head1 DESCRIPTION
 
 
 =head1 DESCRIPTION
 


@@ -113,9 +113,14 @@ improved with high-quality and high-quantity solid-state storage.
 Issuing TRIM commands with L<fstrim(8)> was necessary to maintain
 consistent performance while developing this feature.
 
 Issuing TRIM commands with L<fstrim(8)> was necessary to maintain
 consistent performance while developing this feature.
 

-Rotational storage devices are NOT recommended for indexing of
-large mail archives; but are fine for backup and usable for
-small instances.
+Rotational storage devices perform significantly worse than
+solid state storage for indexing of large mail archives; but are
+fine for backup and usable for small instances.
+
+As of public-inbox 1.6.0, the C<publicInbox.indexSequentialShard>
+option of L<public-inbox-index(1)> may be used with a high shard
+count to ensure individual shards fit into page cache when the entire
+Xapian DB cannot.

 
 Our use of the L</OVERVIEW DB> requires Xapian document IDs to
 remain stable.  Using L<public-inbox-compact(1)> and
 
 Our use of the L</OVERVIEW DB> requires Xapian document IDs to
 remain stable.  Using L<public-inbox-compact(1)> and


@@ -159,7 +164,7 @@ top-level of the directory.
 
 =head1 OBJECT IDENTIFIERS
 
 
 =head1 OBJECT IDENTIFIERS
 

-There are three distinct type of identifiers.  content_id is the
+There are three distinct type of identifiers.  content_hash is the

 new one for v2 and should make message removal and deduplication
 easier.  object_id and Message-ID are already known.
 
 new one for v2 and should make message removal and deduplication
 easier.  object_id and Message-ID are already known.
 


@@ -179,11 +184,11 @@ The email header; duplicates allowed for archival purposes.
 This remains a searchable field in Xapian.  Note: it's possible
 for emails to have multiple Message-ID headers (and L<git-send-email(1)>
 had that bug for a bit); so we take all of them into account.
 This remains a searchable field in Xapian.  Note: it's possible
 for emails to have multiple Message-ID headers (and L<git-send-email(1)>
 had that bug for a bit); so we take all of them into account.

-In case of conflicts detected by content_id below, we generate a new
-Message-ID based on content_id; if the generated Message-ID still
+In case of conflicts detected by content_hash below, we generate a new
+Message-ID based on content_hash; if the generated Message-ID still

 conflicts, a random one is generated.
 
 conflicts, a random one is generated.
 

-=item content_id
+=item content_hash

 
 A hash of relevant headers and raw body content for
 purging of unwanted content.  This is not stored anywhere,
 
 A hash of relevant headers and raw body content for
 purging of unwanted content.  This is not stored anywhere,


@@ -193,7 +198,7 @@ For now, the relevant headers are:
 
        Subject, From, Date, References, In-Reply-To, To, Cc
 
 
        Subject, From, Date, References, In-Reply-To, To, Cc
 

-Received, List-Id, and similar headers are NOT part of content_id as
+Received, List-Id, and similar headers are NOT part of content_hash as

 they differ across lists and we will want removal to be able to cross
 lists.
 
 they differ across lists and we will want removal to be able to cross
 lists.
 


@@ -203,7 +208,7 @@ raw body risks being broken by list signatures; but we can use
 filters (e.g. PublicInbox::Filter::Vger) to clean the body for
 imports.
 
 filters (e.g. PublicInbox::Filter::Vger) to clean the body for
 imports.
 

-content_id is SHA-256 for now; but can be changed at any time
+content_hash is SHA-256 for now; but can be changed at any time

 without making DB changes.
 
 =back
 without making DB changes.
 
 =back


@@ -226,11 +231,11 @@ no sense in a public archive.
 =head1 THANKS
 
 Thanks to the Linux Foundation for sponsoring the development
 =head1 THANKS
 
 Thanks to the Linux Foundation for sponsoring the development

-and testing of the v2 repository format.
+and testing of the v2 format.

 
 =head1 COPYRIGHT
 
 
 =head1 COPYRIGHT
 

-Copyright 2018-2019 all contributors L<mailto:meta@public-inbox.org>
+Copyright 2018-2020 all contributors L<mailto:meta@public-inbox.org>

 
 License: AGPL-3.0+ L<http://www.gnu.org/licenses/agpl-3.0.txt>
 
 
 License: AGPL-3.0+ L<http://www.gnu.org/licenses/agpl-3.0.txt>