]> Sergey Matveev's repositories - bfs.git/log
bfs.git
8 months agoFormatting fixes
Tavian Barnes [Wed, 27 Sep 2023 16:11:15 +0000 (12:11 -0400)]
Formatting fixes

8 months agobit: Fix UINTPTR_WIDTH typo
Tavian Barnes [Tue, 26 Sep 2023 19:42:35 +0000 (15:42 -0400)]
bit: Fix UINTPTR_WIDTH typo

8 months agobit: Use predefined __*_WIDTH__ macros if we can
Tavian Barnes [Tue, 26 Sep 2023 19:05:43 +0000 (15:05 -0400)]
bit: Use predefined __*_WIDTH__ macros if we can

8 months agodstring: New dchar typedef for dynamic strings
Tavian Barnes [Tue, 26 Sep 2023 16:48:21 +0000 (12:48 -0400)]
dstring: New dchar typedef for dynamic strings

8 months agoci/freebsd: Switch to Tailscale OAuth
Tavian Barnes [Mon, 25 Sep 2023 19:26:52 +0000 (15:26 -0400)]
ci/freebsd: Switch to Tailscale OAuth

8 months agoUse the new list macros
Tavian Barnes [Mon, 25 Sep 2023 17:58:19 +0000 (13:58 -0400)]
Use the new list macros

8 months agolist: New for_[s]list() macros
Tavian Barnes [Mon, 25 Sep 2023 17:53:15 +0000 (13:53 -0400)]
list: New for_[s]list() macros

8 months agolist: Unify formatting
Tavian Barnes [Mon, 25 Sep 2023 17:34:17 +0000 (13:34 -0400)]
list: Unify formatting

8 months agolist: New [S]LIST_EMPTY() macros
Tavian Barnes [Mon, 25 Sep 2023 17:29:03 +0000 (13:29 -0400)]
list: New [S]LIST_EMPTY() macros

8 months agolist: New [S]LIST_ITEM_INIT() macros
Tavian Barnes [Mon, 25 Sep 2023 17:16:35 +0000 (13:16 -0400)]
list: New [S]LIST_ITEM_INIT() macros

8 months agolist: Use (void)(...) rather than do { ... } while (0)
Tavian Barnes [Mon, 25 Sep 2023 17:03:51 +0000 (13:03 -0400)]
list: Use (void)(...) rather than do { ... } while (0)

This makes everything usable in expression contexts.

8 months agoREADME: Add Guix and OpenBSD packages
Tavian Barnes [Thu, 14 Sep 2023 16:14:26 +0000 (12:14 -0400)]
README: Add Guix and OpenBSD packages

8 months agobftw: Share the bftw_state between iterations of ids/eds
Tavian Barnes [Wed, 13 Sep 2023 15:39:50 +0000 (11:39 -0400)]
bftw: Share the bftw_state between iterations of ids/eds

8 months agoparse: Give more ephemeral_fds to -no{user,group}
Tavian Barnes [Tue, 12 Sep 2023 19:04:30 +0000 (15:04 -0400)]
parse: Give more ephemeral_fds to -no{user,group}

Fewer than 3 can lead to

    Assertion failed: (retval->write_queue != -1), function __open_cached_connection, file /usr/src/lib/libc/net/nscachedcli.c, line 224.

on a FreeBSD system with LDAP accounts.

8 months agoRelease 3.0.2
Tavian Barnes [Thu, 7 Sep 2023 01:15:50 +0000 (21:15 -0400)]
Release 3.0.2

8 months agobfstd: Work around a FreeBSD-specific msan issue
Tavian Barnes [Wed, 6 Sep 2023 21:13:04 +0000 (17:13 -0400)]
bfstd: Work around a FreeBSD-specific msan issue

Link: https://github.com/llvm/llvm-project/issues/65532
8 months agobfstd: Skip a whole loop in dollar_quote() if possible
Tavian Barnes [Wed, 6 Sep 2023 20:28:12 +0000 (16:28 -0400)]
bfstd: Skip a whole loop in dollar_quote() if possible

8 months agobfstd: Fix printable_len() off-by-one
Tavian Barnes [Wed, 6 Sep 2023 20:23:10 +0000 (16:23 -0400)]
bfstd: Fix printable_len() off-by-one

If xmbrtowc() fails, or if xiswprint() is false, then we shouldn't
include that wide char in the printable length.

Fixes: 19c96abe0a1ee56cf206fd5e87defb1fd3e0daa5
8 months agobfstd: Fix an OOB string index in xmbrtowc()
Tavian Barnes [Wed, 6 Sep 2023 18:59:59 +0000 (14:59 -0400)]
bfstd: Fix an OOB string index in xmbrtowc()

This bug could be reproduced with something like

    $ bfs -samefile $'\xFA\xFA'
    bfs: error: bfs: dstrnescat@src/dstring.c:252: wordesc() result truncated

or worse, with -DNDEBUG,

    $ bfs -samefile $'.....................\xFA\xFA'
    bfs: error: bfs -samefile $'.....................\xFA\xFA\x00\x55\x53\x45\x52\x3D\x74\x61\x76\x69\x61\x6E\x61\x74\x6F\x72
    bfs: error:               ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    bfs: error: No such file or directory.

which prints the memory after the end of the string (in this case, the
environment variable USER=tavianator).

The bug was caused by the line `*i += len`, which was intended to be
`*i = len`.  But actually, the right behaviour seems to be `*i += 1`.

Fixes: 19c96abe0a1ee56cf206fd5e87defb1fd3e0daa5
8 months agobftw: Enforce the dirlimit strictly
Tavian Barnes [Wed, 6 Sep 2023 13:30:47 +0000 (09:30 -0400)]
bftw: Enforce the dirlimit strictly

The previous accounting didn't fully control the number of allocated
bfs_dirs, as the dirlimit was incremented once we popped the directory,
not when we freed it.

9 months agobfs uses the C17 standard version
Alejandro Lazaro [Fri, 1 Sep 2023 15:50:26 +0000 (17:50 +0200)]
bfs uses the C17 standard version

`bfs` uses C17 since 3.0

9 months agopwcache: Don't use _SC_GET{PW,GR}_R_SIZE_MAX
Tavian Barnes [Thu, 31 Aug 2023 15:07:38 +0000 (11:07 -0400)]
pwcache: Don't use _SC_GET{PW,GR}_R_SIZE_MAX

They tend be 1024, which is a lot of memory per user/group.  128 is
usually enough, so start there instead.

9 months agopwcache: Arena-allocate struct passwd/group
Tavian Barnes [Thu, 31 Aug 2023 14:55:39 +0000 (10:55 -0400)]
pwcache: Arena-allocate struct passwd/group

9 months agoalloc: New varena_grow() function
Tavian Barnes [Thu, 31 Aug 2023 14:42:42 +0000 (10:42 -0400)]
alloc: New varena_grow() function

9 months agotrie: New trie_clear() function
Tavian Barnes [Thu, 31 Aug 2023 14:16:35 +0000 (10:16 -0400)]
trie: New trie_clear() function

9 months agoalloc: New [v]arena_clear() functions
Tavian Barnes [Thu, 31 Aug 2023 14:16:15 +0000 (10:16 -0400)]
alloc: New [v]arena_clear() functions

9 months agoWork around https://github.com/llvm/llvm-project/issues/64946
Tavian Barnes [Thu, 24 Aug 2023 03:19:55 +0000 (23:19 -0400)]
Work around https://github.com/llvm/llvm-project/issues/64946

9 months agoioq: New ioq_slot_monitor() helper
Tavian Barnes [Tue, 8 Aug 2023 20:03:11 +0000 (16:03 -0400)]
ioq: New ioq_slot_monitor() helper

9 months agodocs/bfs.1: Document each -regextype
Tavian Barnes [Tue, 15 Aug 2023 19:57:02 +0000 (15:57 -0400)]
docs/bfs.1: Document each -regextype

9 months agodocs/bfs.1: Document that -help/-version exit immediately
Tavian Barnes [Tue, 15 Aug 2023 18:35:55 +0000 (14:35 -0400)]
docs/bfs.1: Document that -help/-version exit immediately

9 months agodocs/bfs.1: Don't unnecessarily quote {}
Tavian Barnes [Tue, 15 Aug 2023 18:30:41 +0000 (14:30 -0400)]
docs/bfs.1: Don't unnecessarily quote {}

9 months agodocs/bfs.1: Clarify the effects of $POSIXLY_CORRECT
Tavian Barnes [Tue, 15 Aug 2023 18:30:17 +0000 (14:30 -0400)]
docs/bfs.1: Clarify the effects of $POSIXLY_CORRECT

9 months agodocs/bfs.1: Document use of $PATH
Tavian Barnes [Tue, 15 Aug 2023 18:28:10 +0000 (14:28 -0400)]
docs/bfs.1: Document use of $PATH

9 months agodocs/bfs.1: Document that -prune does not work together with -depth
Tavian Barnes [Tue, 15 Aug 2023 18:27:01 +0000 (14:27 -0400)]
docs/bfs.1: Document that -prune does not work together with -depth

9 months agodocs/bfs.1: Document the -type d,f syntax
Tavian Barnes [Tue, 15 Aug 2023 18:26:23 +0000 (14:26 -0400)]
docs/bfs.1: Document the -type d,f syntax

9 months agodocs/bfs.1: Use a bulleted list for -size units
Tavian Barnes [Tue, 15 Aug 2023 18:25:42 +0000 (14:25 -0400)]
docs/bfs.1: Use a bulleted list for -size units

9 months agodocs/bfs.1: Give an example with a space separating date from time
Tavian Barnes [Tue, 15 Aug 2023 18:21:47 +0000 (14:21 -0400)]
docs/bfs.1: Give an example with a space separating date from time

9 months agodocs/bfs.1: Make bold/italic formatting more consistent
Tavian Barnes [Tue, 15 Aug 2023 18:19:59 +0000 (14:19 -0400)]
docs/bfs.1: Make bold/italic formatting more consistent

9 months agotests: Add missing .out file
Tavian Barnes [Mon, 14 Aug 2023 20:05:35 +0000 (16:05 -0400)]
tests: Add missing .out file

9 months agoparse: Allow -files0-from an empty set of paths
Tavian Barnes [Mon, 14 Aug 2023 19:59:04 +0000 (15:59 -0400)]
parse: Allow -files0-from an empty set of paths

This follows a behaviour change in GNU findutils 4.9.0.

9 months agobfstd: Don't label a declaration
Tavian Barnes [Tue, 8 Aug 2023 19:11:45 +0000 (15:11 -0400)]
bfstd: Don't label a declaration

9 months agostat: Don't check for statx() mask bits we already checked for
Tavian Barnes [Mon, 7 Aug 2023 23:11:18 +0000 (19:11 -0400)]
stat: Don't check for statx() mask bits we already checked for

9 months agobfstd: Check multiple chars at once for isascii()
Tavian Barnes [Mon, 7 Aug 2023 23:41:12 +0000 (19:41 -0400)]
bfstd: Check multiple chars at once for isascii()

9 months agobfstd: Speed up wordesc() by caching isprint()/isspace()
Tavian Barnes [Mon, 7 Aug 2023 22:29:19 +0000 (18:29 -0400)]
bfstd: Speed up wordesc() by caching isprint()/isspace()

10 months agobit: Add a cast to squelch -Wsign-compare
Tavian Barnes [Mon, 24 Jul 2023 16:13:38 +0000 (12:13 -0400)]
bit: Add a cast to squelch -Wsign-compare

I see this only with musl-gcc for some reason.

10 months agoREADME: Official Homebrew formula
Martin Polden [Sat, 22 Jul 2023 10:51:35 +0000 (12:51 +0200)]
README: Official Homebrew formula

An official `bfs` formula has been added to Homebrew now.

10 months agoREADME.md: fix reversed examples
Wayne Scott [Sat, 22 Jul 2023 09:55:58 +0000 (05:55 -0400)]
README.md: fix reversed examples

10 months agoRename Makefile to GNUmakefile
Raf Czlonka [Thu, 20 Jul 2023 15:58:51 +0000 (16:58 +0100)]
Rename Makefile to GNUmakefile

10 months agoREADME: Add official Arch Linux package
Tavian Barnes [Thu, 20 Jul 2023 12:45:11 +0000 (08:45 -0400)]
README: Add official Arch Linux package

10 months agoCFLAGS adjustment in Makefile
Jason Stewart [Wed, 19 Jul 2023 16:50:14 +0000 (12:50 -0400)]
CFLAGS adjustment in Makefile

`-flto` to `-flto=auto` to eliminate `using serial compilation of 3 LTRANS jobs` gcc warning
see https://stackoverflow.com/questions/72218980/gcc-v12-1-warning-about-serial-compilation

10 months agoRelease 3.0.1
Tavian Barnes [Tue, 18 Jul 2023 16:23:21 +0000 (12:23 -0400)]
Release 3.0.1

10 months agobftw: Use bftw_file->next for multiple lists
Tavian Barnes [Tue, 18 Jul 2023 16:14:07 +0000 (12:14 -0400)]
bftw: Use bftw_file->next for multiple lists

A file can be on the to_open and to_read lists at the same time, but
otherwise only one list, so we can save memory by sharing the pointers.

10 months agobftw: Use a larger ioq depth
Tavian Barnes [Tue, 18 Jul 2023 15:56:41 +0000 (11:56 -0400)]
bftw: Use a larger ioq depth

Now that the dirlimit provides backpressure on the number of open
directories, we can use a uniformly larger queue depth for increased
performance.  The current parameters were tuned with a small grid search
on my workstation.

10 months agobftw: Add a queue of directories to unwrap
Tavian Barnes [Tue, 18 Jul 2023 15:43:21 +0000 (11:43 -0400)]
bftw: Add a queue of directories to unwrap

For !BFS_USE_UNWRAPDIR, if a file is still pinned in bftw_closedir(), it
has to stay open until its pincount drops to zero.  Since this happens
in bftw_ioq_pop(), we can't immediately call bftw_unwrapdir() as that
adds to the ioq.  Instead, add it to a list that gets drained by the
next bftw_gc().

10 months agobftw: Add dirs to the end of the queue in bftw_ioq_pop()
Tavian Barnes [Tue, 18 Jul 2023 00:30:57 +0000 (20:30 -0400)]
bftw: Add dirs to the end of the queue in bftw_ioq_pop()

I tried this before in #105 but it led to performance regressions.  The
key to avoiding those regressions is to put some backpressure on how
many bfs_dir's can be allocated simultaneously.

10 months agobftw: Use separate queues for open and closed directories
Tavian Barnes [Mon, 17 Jul 2023 22:40:32 +0000 (18:40 -0400)]
bftw: Use separate queues for open and closed directories

10 months agobftw: Check that file->fd == bfs_dirfd(file->dir) earlier
Tavian Barnes [Mon, 17 Jul 2023 23:58:39 +0000 (19:58 -0400)]
bftw: Check that file->fd == bfs_dirfd(file->dir) earlier

This has the potential to fail on at least one known platform: macports
with the legacysupport implementation of fdopendir().

Link: https://github.com/macports/macports-ports/pull/19047#issuecomment-1636059809
10 months agobftw: Reserve space in the cache before opening files
Tavian Barnes [Mon, 17 Jul 2023 23:05:23 +0000 (19:05 -0400)]
bftw: Reserve space in the cache before opening files

This fixes a storm of EMFILE retries observed with -j1 on a very large
directory tree.

Fixes: 7888fbababd22190e9f919fc272957426a27969e
10 months agobftw: Pass the whole bftw_state to bftw_openat()
Tavian Barnes [Mon, 17 Jul 2023 23:03:30 +0000 (19:03 -0400)]
bftw: Pass the whole bftw_state to bftw_openat()

This required shuffling a lot of code around.  Hopefully the new order
makes more sense.

10 months agobftw: Add bfs_dir allocation wrappers
Tavian Barnes [Mon, 17 Jul 2023 21:15:47 +0000 (17:15 -0400)]
bftw: Add bfs_dir allocation wrappers

10 months agoRelease 3.0
Tavian Barnes [Fri, 14 Jul 2023 01:19:09 +0000 (21:19 -0400)]
Release 3.0

10 months agobuild: Move some flags around
Tavian Barnes [Fri, 14 Jul 2023 01:00:43 +0000 (21:00 -0400)]
build: Move some flags around

10 months agobfstd: Add an ASCII fast path to wordesc()
Tavian Barnes [Fri, 14 Jul 2023 00:00:11 +0000 (20:00 -0400)]
bfstd: Add an ASCII fast path to wordesc()

10 months agoalloc: Use a different error code for size overflows
Tavian Barnes [Thu, 13 Jul 2023 20:23:31 +0000 (16:23 -0400)]
alloc: Use a different error code for size overflows

This should help debuggability, and also squelches a GCC warning.

10 months agocolor: Only highlight the trailing slash on ENOTDIR
Tavian Barnes [Thu, 13 Jul 2023 20:00:02 +0000 (16:00 -0400)]
color: Only highlight the trailing slash on ENOTDIR

10 months agocolor: Don't break up leading and trailing dirs
Tavian Barnes [Thu, 13 Jul 2023 19:45:08 +0000 (15:45 -0400)]
color: Don't break up leading and trailing dirs

10 months agocolor: TTY-escape filenames
Tavian Barnes [Thu, 13 Jul 2023 19:22:42 +0000 (15:22 -0400)]
color: TTY-escape filenames

10 months agocolor: Get rid of EXT_MAX
Tavian Barnes [Thu, 13 Jul 2023 17:40:46 +0000 (13:40 -0400)]
color: Get rid of EXT_MAX

10 months agobfstd: Support wordesc() without allocating
Tavian Barnes [Thu, 13 Jul 2023 17:30:16 +0000 (13:30 -0400)]
bfstd: Support wordesc() without allocating

10 months agobfstd: Quote the whole string the same way in wordesc()
Tavian Barnes [Thu, 13 Jul 2023 16:36:10 +0000 (12:36 -0400)]
bfstd: Quote the whole string the same way in wordesc()

10 months agobfstd: Use $'\n' etc. over $'\x0A'
Tavian Barnes [Thu, 13 Jul 2023 15:07:05 +0000 (11:07 -0400)]
bfstd: Use $'\n' etc. over $'\x0A'

10 months agobfstd: Escape '!' in wordesc()
Tavian Barnes [Thu, 13 Jul 2023 15:06:51 +0000 (11:06 -0400)]
bfstd: Escape '!' in wordesc()

10 months agoparse: Reject -j0
Tavian Barnes [Wed, 12 Jul 2023 18:12:27 +0000 (14:12 -0400)]
parse: Reject -j0

10 months agotests/bfs: Add tests for -j
Tavian Barnes [Wed, 12 Jul 2023 18:10:29 +0000 (14:10 -0400)]
tests/bfs: Add tests for -j

10 months agoioq: Try harder to avoid setting IOQ_BLOCKED
Tavian Barnes [Wed, 12 Jul 2023 17:12:27 +0000 (13:12 -0400)]
ioq: Try harder to avoid setting IOQ_BLOCKED

10 months agoioq: Separate slot and queue operations
Tavian Barnes [Tue, 11 Jul 2023 14:21:26 +0000 (10:21 -0400)]
ioq: Separate slot and queue operations

10 months agoeval: Don't oversubscribe the CPU by default
Tavian Barnes [Tue, 11 Jul 2023 12:54:30 +0000 (08:54 -0400)]
eval: Don't oversubscribe the CPU by default

10 months agobftw: Try to close files asynchronously
Tavian Barnes [Tue, 4 Jul 2023 19:37:54 +0000 (15:37 -0400)]
bftw: Try to close files asynchronously

10 months agoioq: Implement async close() and closedir()
Tavian Barnes [Tue, 4 Jul 2023 19:17:23 +0000 (15:17 -0400)]
ioq: Implement async close() and closedir()

10 months agobftw: If the ioq is full, try to pop before ioq_opendir()
Tavian Barnes [Tue, 4 Jul 2023 19:03:37 +0000 (15:03 -0400)]
bftw: If the ioq is full, try to pop before ioq_opendir()

10 months agoioq: New ioq_capacity() function
Tavian Barnes [Tue, 4 Jul 2023 19:01:18 +0000 (15:01 -0400)]
ioq: New ioq_capacity() function

10 months agowordesc: Also escape non-printable chars
Tavian Barnes [Thu, 6 Jul 2023 18:16:20 +0000 (14:16 -0400)]
wordesc: Also escape non-printable chars

10 months agoUse strcmp() instead of fnmatch() if possible
Tavian Barnes [Wed, 5 Jul 2023 14:45:56 +0000 (10:45 -0400)]
Use strcmp() instead of fnmatch() if possible

10 months agodiag: Shell-escape expression arguments
Tavian Barnes [Thu, 6 Jul 2023 12:59:37 +0000 (08:59 -0400)]
diag: Shell-escape expression arguments

10 months agoioq: Don't write to an empty slot in ioqq_trypop()
Tavian Barnes [Tue, 4 Jul 2023 17:31:54 +0000 (13:31 -0400)]
ioq: Don't write to an empty slot in ioqq_trypop()

10 months agolist: Fix some parameter docs
Tavian Barnes [Tue, 4 Jul 2023 17:30:10 +0000 (13:30 -0400)]
list: Fix some parameter docs

11 months agocolor: Don't print unnecessary reset sequences
Tavian Barnes [Fri, 30 Jun 2023 15:46:22 +0000 (11:46 -0400)]
color: Don't print unnecessary reset sequences

11 months agodocs: Start preparing for the 3.0 release
Tavian Barnes [Fri, 30 Jun 2023 15:20:43 +0000 (11:20 -0400)]
docs: Start preparing for the 3.0 release

11 months agocolor: Compare values too when deciding to smart-case
Tavian Barnes [Fri, 30 Jun 2023 14:25:42 +0000 (10:25 -0400)]
color: Compare values too when deciding to smart-case

11 months agocolor: Implement smart casing
Tavian Barnes [Thu, 29 Jun 2023 19:17:27 +0000 (15:17 -0400)]
color: Implement smart casing

Since coreutils 9.2, ls does case-sensitive extension matching if the
same extension is capitalized differently in $LS_COLORS.  Implement the
same logic.

Link: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=33123
Link: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=9086
11 months agoparse: Warn about errors parsing $LS_COLORS
Tavian Barnes [Thu, 29 Jun 2023 17:53:48 +0000 (13:53 -0400)]
parse: Warn about errors parsing $LS_COLORS

11 months agodstring: Add some exact-size utility functions
Tavian Barnes [Thu, 29 Jun 2023 17:53:03 +0000 (13:53 -0400)]
dstring: Add some exact-size utility functions

11 months agodstring: Allow dstreserve(NULL, n)
Tavian Barnes [Thu, 29 Jun 2023 17:52:39 +0000 (13:52 -0400)]
dstring: Allow dstreserve(NULL, n)

11 months agotests/bfs/color: Add mixed-case extensions
Tavian Barnes [Tue, 27 Jun 2023 16:07:14 +0000 (12:07 -0400)]
tests/bfs/color: Add mixed-case extensions

11 months agotests: Get more tests passing with --bfs=tests/find-color.sh
Tavian Barnes [Tue, 27 Jun 2023 15:52:24 +0000 (11:52 -0400)]
tests: Get more tests passing with --bfs=tests/find-color.sh

11 months agothread: Wrap more pthread APIs
Tavian Barnes [Mon, 26 Jun 2023 19:04:13 +0000 (15:04 -0400)]
thread: Wrap more pthread APIs

11 months agoioq: New ioq_cancel() function
Tavian Barnes [Mon, 26 Jun 2023 15:47:41 +0000 (11:47 -0400)]
ioq: New ioq_cancel() function

11 months agoioq: Don't check NDEBUG manually in ioqq_trypop()
Tavian Barnes [Mon, 26 Jun 2023 15:35:23 +0000 (11:35 -0400)]
ioq: Don't check NDEBUG manually in ioqq_trypop()

11 months agobuild/tsan: Fix target_clones override
Tavian Barnes [Sat, 24 Jun 2023 17:22:38 +0000 (13:22 -0400)]
build/tsan: Fix target_clones override

11 months agobfstd: Add a getprogname() wrapper
Tavian Barnes [Sat, 24 Jun 2023 16:47:09 +0000 (12:47 -0400)]
bfstd: Add a getprogname() wrapper