doc/cmds.texi | 12 ++++++++++-- doc/faq.texi | 4 ++-- doc/news.texi | 40 ++++++++++++++++++++++++++++++++++++++++ doc/state.texi | 71 ++++++++++++++++++++++++++++++++++++++++++++++++++--- usage.go | 4 ++++ diff --git a/doc/cmds.texi b/doc/cmds.texi index 79bae4bac9b0fb2031b82f47fac022a23bede4480ee37b2739edf75876fc9f2f..d70e48b03705636dc01dd1ee9392eada1b3d6973a0e7a0f85b06ad54eb9aa2a9 100644 --- a/doc/cmds.texi +++ b/doc/cmds.texi @@ -139,7 +139,7 @@ Dependency @url{https://en.wikipedia.org/wiki/DOT_(graph_description_language), DOT} graph generator. For example to visualize your dependencies with GraphViz: @example -$ redo target [...] # to assure that **/.redo/*.rec are filled up +$ redo target [...] # to assure that **/.redo/*.dep are filled up $ redo-dot target [...] > whatever.dot $ dot -Tpng whatever.dot > whatever.png # possibly add -Gsplines=ortho @end example @@ -152,6 +152,14 @@ differs from updated ctimes, out-of-date algorithm will fallback to rereading the whole files for hash calculation, that is much slower. If you do not want to rebuild your targets from the ground, then @command{redo-depfix} can traverse through all dependency files and - check if they have non-altered ctime values and update them in place. + recalculate dependency information. + + Also if it finds legacy @command{.rec} dependency files, it converts + them to binary format. + +@pindex redo-dep2rec +@item redo-dep2rec + Convert specified @file{.dep} file to recfile on stdout. + Aimed to be used mainly for debugging purposes. @end table diff --git a/doc/faq.texi b/doc/faq.texi index 46f1f335ade8504d0a7e2130850f7d75203dbd669c347eee9d016c5fe296949a..0d02063a38c879f2529c6dba9fb358410b75f4503fc449904cfb476bf5cd5734 100644 --- a/doc/faq.texi +++ b/doc/faq.texi @@ -43,7 +43,7 @@ and transparently compressed, as ZFS with LZ4/Zstandard and Skein/BLAKE[23] algorithms demonstrate us. @command{goredo} includes @command{redo-stamp}, that really records the -stamp in the @file{.rec} file, but it does not play any role later. It +stamp in the @file{.dep} file, but it does not play any role later. It is stayed just for compatibility. @section Can removed .do lead to permanent errors of its non existence? @@ -53,7 +53,7 @@ assume that @file{.do}-less target now is an ordinary source-file? I have no confidence in such behaviour. So it is user's decision how to deal with it, probably it was just his inaccuracy mistake. If you really want to get rid of that dependency knowledge for @file{foo/bar} target, -then just remove @file{foo/.redo/bar.rec}. +then remove @file{foo/.redo/bar.dep}. @section Does redo-always always rebuilds target? diff --git a/doc/news.texi b/doc/news.texi index fdcb09418bce18c5805123796022458bb2672a1be97f7b2f0118532b42bcb8d9..9cada3f58a0f028ee3f7322135563c866c8a6bb359d0dc6b93f7a58dd12305a4 100644 --- a/doc/news.texi +++ b/doc/news.texi @@ -2,6 +2,46 @@ @node News @cindex news @unnumbered News +@anchor{Release 2_0_0} +@section Release 2.0.0 +@itemize +@item +Huge quantity of performance optimisations. + +@item +Fixed possible unexpected lock file closing. + +@item +When resulting target has the same contents, it does not replace already +existing one. That was done previously. But now it also copies the file's +mode flags to the target (for example making it executable). + +@item +If @command{redo-*} command runs under control of another (top-level) +redo, then it does not parse the flags as options and treat each +argument as a target, allowing passing the targets with dashes in the +beginning. + +@item +Prevented possible concurrent stderr writing by multiple running +targets. + +@item +@command{redo-depfix} command now always rewrites dependency files and +calculates checksums of the files. + +@item +Own binary format is used for dependency files (@file{.dep}), instead of +recfile (@file{.rec}) one. It is several times smaller and much faster +to parse. You must run @command{redo-depfix} to convert all existing +@file{.redo/*.rec} files to that binary format. + +@item +@command{redo-dep2rec} command appeared, that can convert @file{.dep} to +recfile on stdout. + +@end itemize + @anchor{Release 1_32_0} @section Release 1.32.0 @itemize diff --git a/doc/state.texi b/doc/state.texi index 38cc997c8a3e7eb1780579bf2a93c2f2de8ee31f0d6cd29f86e4114eb204e7fe..10b088d2cdeddd0ddb91af138e9e310f01ded8c941f562900efc1c8cf2664233 100644 --- a/doc/state.texi +++ b/doc/state.texi @@ -5,9 +5,72 @@ @unnumbered State Dependency and build state is kept inside @file{.redo} subdirectory in each directory related the build. Each corresponding target has its own, -recreated with every rebuild, @file{.rec} file. It is -@url{https://www.gnu.org/software/recutils/, recfile}, that could have -various dependency information. For example: +recreated with every rebuild, binary @file{.dep} file. + +It starts with the header: + +@verbatim +"GOREDO" || 0x01 || BuildUUID +@end verbatim + +@code{0x01} is format's version number. @code{BuildUUID} is 128-bit UUID. + +After the header comes concatenation of length-prefixed chunks. Length +is the length of the whole chunk, including the size of the encoded +length itself. It is 16-bit big-endian integer. After the length comes +single byte type of the chunk: + +@table @asis + +@item ifcreate + +@verbatim +LEN || 0x01 || target +@end verbatim + +@code{target} is UTF-8 encoded target's name. + +@item ifchange + +@verbatim +LEN || 0x02 || + size || inodeNum || + ctimeSec || ctimeNsec || + mtimeSec || mtimeNsec || + hash || target +@end verbatim + +@code{target} is UTF-8 encoded target's name. +@code{hash} is 256-bit BLAKE3 hash. +All other inode-related fields are 64-bit big-endian integers. + +@item always + +@verbatim +LEN || 0x03 +@end verbatim + +@item stamp + +@verbatim +LEN || 0x04 || hash +@end verbatim + +@code{hash} is 256-bit BLAKE3 hash. + +@item ifchange-nonexistent + +@verbatim +LEN || 0x05 || target +@end verbatim + +@code{target} is UTF-8 encoded target's name. + +@end table + +That @file{.dep} file can be converted to human-readable +@url{https://www.gnu.org/software/recutils/, recfile} format. +For example: @example Build: 80143f04-bfff-4673-950c-081d712f573d @@ -31,5 +94,5 @@ Type: stamp Hash: 5bbdf635932cb16b9127e69b6f3872577efed338f0a4ab6f2c7ca3df6ce50cc9 @end example -And the schema definition: +And its schema definition: @verbatiminclude ../dep.rec diff --git a/usage.go b/usage.go index 86f46d03a4256d92f1abe311f41782699f0983cd4f0140a2000d9656ac58b66e..4a0655a62174e95c54bbaef9ebe7f322e2f8b087d88909c95d9ae4376dcb0532 100644 --- a/usage.go +++ b/usage.go @@ -105,6 +105,10 @@ case CmdNameRedoAffects: d = `Usage: redo-affects target [...] List all targets that will be affected by changing the specified ones.` + case CmdNameRedoDep2Rec: + d = `Usage: redo-dep2rec .../.redo/file.dep + +Convert binary .dep file to recfile and write it to stdout.` case CmdNameRedoDepFix: d = `Usage: redo-depfix