]>
Sergey Matveev's repositories - bfs.git/log
Tavian Barnes [Wed, 27 Sep 2023 16:11:15 +0000 (12:11 -0400)]
Formatting fixes
Tavian Barnes [Tue, 26 Sep 2023 19:42:35 +0000 (15:42 -0400)]
bit: Fix UINTPTR_WIDTH typo
Tavian Barnes [Tue, 26 Sep 2023 19:05:43 +0000 (15:05 -0400)]
bit: Use predefined __*_WIDTH__ macros if we can
Tavian Barnes [Tue, 26 Sep 2023 16:48:21 +0000 (12:48 -0400)]
dstring: New dchar typedef for dynamic strings
Tavian Barnes [Mon, 25 Sep 2023 19:26:52 +0000 (15:26 -0400)]
ci/freebsd: Switch to Tailscale OAuth
Tavian Barnes [Mon, 25 Sep 2023 17:58:19 +0000 (13:58 -0400)]
Use the new list macros
Tavian Barnes [Mon, 25 Sep 2023 17:53:15 +0000 (13:53 -0400)]
list: New for_[s]list() macros
Tavian Barnes [Mon, 25 Sep 2023 17:34:17 +0000 (13:34 -0400)]
list: Unify formatting
Tavian Barnes [Mon, 25 Sep 2023 17:29:03 +0000 (13:29 -0400)]
list: New [S]LIST_EMPTY() macros
Tavian Barnes [Mon, 25 Sep 2023 17:16:35 +0000 (13:16 -0400)]
list: New [S]LIST_ITEM_INIT() macros
Tavian Barnes [Mon, 25 Sep 2023 17:03:51 +0000 (13:03 -0400)]
list: Use (void)(...) rather than do { ... } while (0)
This makes everything usable in expression contexts.
Tavian Barnes [Thu, 14 Sep 2023 16:14:26 +0000 (12:14 -0400)]
README: Add Guix and OpenBSD packages
Tavian Barnes [Wed, 13 Sep 2023 15:39:50 +0000 (11:39 -0400)]
bftw: Share the bftw_state between iterations of ids/eds
Tavian Barnes [Tue, 12 Sep 2023 19:04:30 +0000 (15:04 -0400)]
parse: Give more ephemeral_fds to -no{user,group}
Fewer than 3 can lead to
Assertion failed: (retval->write_queue != -1), function __open_cached_connection, file /usr/src/lib/libc/net/nscachedcli.c, line 224.
on a FreeBSD system with LDAP accounts.
Tavian Barnes [Thu, 7 Sep 2023 01:15:50 +0000 (21:15 -0400)]
Release 3.0.2
Tavian Barnes [Wed, 6 Sep 2023 21:13:04 +0000 (17:13 -0400)]
bfstd: Work around a FreeBSD-specific msan issue
Link: https://github.com/llvm/llvm-project/issues/65532
Tavian Barnes [Wed, 6 Sep 2023 20:28:12 +0000 (16:28 -0400)]
bfstd: Skip a whole loop in dollar_quote() if possible
Tavian Barnes [Wed, 6 Sep 2023 20:23:10 +0000 (16:23 -0400)]
bfstd: Fix printable_len() off-by-one
If xmbrtowc() fails, or if xiswprint() is false, then we shouldn't
include that wide char in the printable length.
Fixes: 19c96abe0a1ee56cf206fd5e87defb1fd3e0daa5
Tavian Barnes [Wed, 6 Sep 2023 18:59:59 +0000 (14:59 -0400)]
bfstd: Fix an OOB string index in xmbrtowc()
This bug could be reproduced with something like
$ bfs -samefile $'\xFA\xFA'
bfs: error: bfs: dstrnescat@src/dstring.c:252: wordesc() result truncated
or worse, with -DNDEBUG,
$ bfs -samefile $'.....................\xFA\xFA'
bfs: error: bfs -samefile $'.....................\xFA\xFA\x00\x55\x53\x45\x52\x3D\x74\x61\x76\x69\x61\x6E\x61\x74\x6F\x72
bfs: error: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
bfs: error: No such file or directory.
which prints the memory after the end of the string (in this case, the
environment variable USER=tavianator).
The bug was caused by the line `*i += len`, which was intended to be
`*i = len`. But actually, the right behaviour seems to be `*i += 1`.
Fixes: 19c96abe0a1ee56cf206fd5e87defb1fd3e0daa5
Tavian Barnes [Wed, 6 Sep 2023 13:30:47 +0000 (09:30 -0400)]
bftw: Enforce the dirlimit strictly
The previous accounting didn't fully control the number of allocated
bfs_dirs, as the dirlimit was incremented once we popped the directory,
not when we freed it.
Alejandro Lazaro [Fri, 1 Sep 2023 15:50:26 +0000 (17:50 +0200)]
bfs uses the C17 standard version
`bfs` uses C17 since 3.0
Tavian Barnes [Thu, 31 Aug 2023 15:07:38 +0000 (11:07 -0400)]
pwcache: Don't use _SC_GET{PW,GR}_R_SIZE_MAX
They tend be 1024, which is a lot of memory per user/group. 128 is
usually enough, so start there instead.
Tavian Barnes [Thu, 31 Aug 2023 14:55:39 +0000 (10:55 -0400)]
pwcache: Arena-allocate struct passwd/group
Tavian Barnes [Thu, 31 Aug 2023 14:42:42 +0000 (10:42 -0400)]
alloc: New varena_grow() function
Tavian Barnes [Thu, 31 Aug 2023 14:16:35 +0000 (10:16 -0400)]
trie: New trie_clear() function
Tavian Barnes [Thu, 31 Aug 2023 14:16:15 +0000 (10:16 -0400)]
alloc: New [v]arena_clear() functions
Tavian Barnes [Thu, 24 Aug 2023 03:19:55 +0000 (23:19 -0400)]
Work around https://github.com/llvm/llvm-project/issues/64946
Tavian Barnes [Tue, 8 Aug 2023 20:03:11 +0000 (16:03 -0400)]
ioq: New ioq_slot_monitor() helper
Tavian Barnes [Tue, 15 Aug 2023 19:57:02 +0000 (15:57 -0400)]
docs/bfs.1: Document each -regextype
Tavian Barnes [Tue, 15 Aug 2023 18:35:55 +0000 (14:35 -0400)]
docs/bfs.1: Document that -help/-version exit immediately
Tavian Barnes [Tue, 15 Aug 2023 18:30:41 +0000 (14:30 -0400)]
docs/bfs.1: Don't unnecessarily quote {}
Tavian Barnes [Tue, 15 Aug 2023 18:30:17 +0000 (14:30 -0400)]
docs/bfs.1: Clarify the effects of $POSIXLY_CORRECT
Tavian Barnes [Tue, 15 Aug 2023 18:28:10 +0000 (14:28 -0400)]
docs/bfs.1: Document use of $PATH
Tavian Barnes [Tue, 15 Aug 2023 18:27:01 +0000 (14:27 -0400)]
docs/bfs.1: Document that -prune does not work together with -depth
Tavian Barnes [Tue, 15 Aug 2023 18:26:23 +0000 (14:26 -0400)]
docs/bfs.1: Document the -type d,f syntax
Tavian Barnes [Tue, 15 Aug 2023 18:25:42 +0000 (14:25 -0400)]
docs/bfs.1: Use a bulleted list for -size units
Tavian Barnes [Tue, 15 Aug 2023 18:21:47 +0000 (14:21 -0400)]
docs/bfs.1: Give an example with a space separating date from time
Tavian Barnes [Tue, 15 Aug 2023 18:19:59 +0000 (14:19 -0400)]
docs/bfs.1: Make bold/italic formatting more consistent
Tavian Barnes [Mon, 14 Aug 2023 20:05:35 +0000 (16:05 -0400)]
tests: Add missing .out file
Tavian Barnes [Mon, 14 Aug 2023 19:59:04 +0000 (15:59 -0400)]
parse: Allow -files0-from an empty set of paths
This follows a behaviour change in GNU findutils 4.9.0.
Tavian Barnes [Tue, 8 Aug 2023 19:11:45 +0000 (15:11 -0400)]
bfstd: Don't label a declaration
Tavian Barnes [Mon, 7 Aug 2023 23:11:18 +0000 (19:11 -0400)]
stat: Don't check for statx() mask bits we already checked for
Tavian Barnes [Mon, 7 Aug 2023 23:41:12 +0000 (19:41 -0400)]
bfstd: Check multiple chars at once for isascii()
Tavian Barnes [Mon, 7 Aug 2023 22:29:19 +0000 (18:29 -0400)]
bfstd: Speed up wordesc() by caching isprint()/isspace()
Tavian Barnes [Mon, 24 Jul 2023 16:13:38 +0000 (12:13 -0400)]
bit: Add a cast to squelch -Wsign-compare
I see this only with musl-gcc for some reason.
Martin Polden [Sat, 22 Jul 2023 10:51:35 +0000 (12:51 +0200)]
README: Official Homebrew formula
An official `bfs` formula has been added to Homebrew now.
Wayne Scott [Sat, 22 Jul 2023 09:55:58 +0000 (05:55 -0400)]
README.md: fix reversed examples
Raf Czlonka [Thu, 20 Jul 2023 15:58:51 +0000 (16:58 +0100)]
Rename Makefile to GNUmakefile
Tavian Barnes [Thu, 20 Jul 2023 12:45:11 +0000 (08:45 -0400)]
README: Add official Arch Linux package
Jason Stewart [Wed, 19 Jul 2023 16:50:14 +0000 (12:50 -0400)]
CFLAGS adjustment in Makefile
`-flto` to `-flto=auto` to eliminate `using serial compilation of 3 LTRANS jobs` gcc warning
see https://stackoverflow.com/questions/
72218980 /gcc-v12-1-warning-about-serial-compilation
Tavian Barnes [Tue, 18 Jul 2023 16:23:21 +0000 (12:23 -0400)]
Release 3.0.1
Tavian Barnes [Tue, 18 Jul 2023 16:14:07 +0000 (12:14 -0400)]
bftw: Use bftw_file->next for multiple lists
A file can be on the to_open and to_read lists at the same time, but
otherwise only one list, so we can save memory by sharing the pointers.
Tavian Barnes [Tue, 18 Jul 2023 15:56:41 +0000 (11:56 -0400)]
bftw: Use a larger ioq depth
Now that the dirlimit provides backpressure on the number of open
directories, we can use a uniformly larger queue depth for increased
performance. The current parameters were tuned with a small grid search
on my workstation.
Tavian Barnes [Tue, 18 Jul 2023 15:43:21 +0000 (11:43 -0400)]
bftw: Add a queue of directories to unwrap
For !BFS_USE_UNWRAPDIR, if a file is still pinned in bftw_closedir(), it
has to stay open until its pincount drops to zero. Since this happens
in bftw_ioq_pop(), we can't immediately call bftw_unwrapdir() as that
adds to the ioq. Instead, add it to a list that gets drained by the
next bftw_gc().
Tavian Barnes [Tue, 18 Jul 2023 00:30:57 +0000 (20:30 -0400)]
bftw: Add dirs to the end of the queue in bftw_ioq_pop()
I tried this before in #105 but it led to performance regressions. The
key to avoiding those regressions is to put some backpressure on how
many bfs_dir's can be allocated simultaneously.
Tavian Barnes [Mon, 17 Jul 2023 22:40:32 +0000 (18:40 -0400)]
bftw: Use separate queues for open and closed directories
Tavian Barnes [Mon, 17 Jul 2023 23:58:39 +0000 (19:58 -0400)]
bftw: Check that file->fd == bfs_dirfd(file->dir) earlier
This has the potential to fail on at least one known platform: macports
with the legacysupport implementation of fdopendir().
Link: https://github.com/macports/macports-ports/pull/19047#issuecomment-1636059809
Tavian Barnes [Mon, 17 Jul 2023 23:05:23 +0000 (19:05 -0400)]
bftw: Reserve space in the cache before opening files
This fixes a storm of EMFILE retries observed with -j1 on a very large
directory tree.
Fixes: 7888fbababd22190e9f919fc272957426a27969e
Tavian Barnes [Mon, 17 Jul 2023 23:03:30 +0000 (19:03 -0400)]
bftw: Pass the whole bftw_state to bftw_openat()
This required shuffling a lot of code around. Hopefully the new order
makes more sense.
Tavian Barnes [Mon, 17 Jul 2023 21:15:47 +0000 (17:15 -0400)]
bftw: Add bfs_dir allocation wrappers
Tavian Barnes [Fri, 14 Jul 2023 01:19:09 +0000 (21:19 -0400)]
Release 3.0
Tavian Barnes [Fri, 14 Jul 2023 01:00:43 +0000 (21:00 -0400)]
build: Move some flags around
Tavian Barnes [Fri, 14 Jul 2023 00:00:11 +0000 (20:00 -0400)]
bfstd: Add an ASCII fast path to wordesc()
Tavian Barnes [Thu, 13 Jul 2023 20:23:31 +0000 (16:23 -0400)]
alloc: Use a different error code for size overflows
This should help debuggability, and also squelches a GCC warning.
Tavian Barnes [Thu, 13 Jul 2023 20:00:02 +0000 (16:00 -0400)]
color: Only highlight the trailing slash on ENOTDIR
Tavian Barnes [Thu, 13 Jul 2023 19:45:08 +0000 (15:45 -0400)]
color: Don't break up leading and trailing dirs
Tavian Barnes [Thu, 13 Jul 2023 19:22:42 +0000 (15:22 -0400)]
color: TTY-escape filenames
Tavian Barnes [Thu, 13 Jul 2023 17:40:46 +0000 (13:40 -0400)]
color: Get rid of EXT_MAX
Tavian Barnes [Thu, 13 Jul 2023 17:30:16 +0000 (13:30 -0400)]
bfstd: Support wordesc() without allocating
Tavian Barnes [Thu, 13 Jul 2023 16:36:10 +0000 (12:36 -0400)]
bfstd: Quote the whole string the same way in wordesc()
Tavian Barnes [Thu, 13 Jul 2023 15:07:05 +0000 (11:07 -0400)]
bfstd: Use $'\n' etc. over $'\x0A'
Tavian Barnes [Thu, 13 Jul 2023 15:06:51 +0000 (11:06 -0400)]
bfstd: Escape '!' in wordesc()
Tavian Barnes [Wed, 12 Jul 2023 18:12:27 +0000 (14:12 -0400)]
parse: Reject -j0
Tavian Barnes [Wed, 12 Jul 2023 18:10:29 +0000 (14:10 -0400)]
tests/bfs: Add tests for -j
Tavian Barnes [Wed, 12 Jul 2023 17:12:27 +0000 (13:12 -0400)]
ioq: Try harder to avoid setting IOQ_BLOCKED
Tavian Barnes [Tue, 11 Jul 2023 14:21:26 +0000 (10:21 -0400)]
ioq: Separate slot and queue operations
Tavian Barnes [Tue, 11 Jul 2023 12:54:30 +0000 (08:54 -0400)]
eval: Don't oversubscribe the CPU by default
Tavian Barnes [Tue, 4 Jul 2023 19:37:54 +0000 (15:37 -0400)]
bftw: Try to close files asynchronously
Tavian Barnes [Tue, 4 Jul 2023 19:17:23 +0000 (15:17 -0400)]
ioq: Implement async close() and closedir()
Tavian Barnes [Tue, 4 Jul 2023 19:03:37 +0000 (15:03 -0400)]
bftw: If the ioq is full, try to pop before ioq_opendir()
Tavian Barnes [Tue, 4 Jul 2023 19:01:18 +0000 (15:01 -0400)]
ioq: New ioq_capacity() function
Tavian Barnes [Thu, 6 Jul 2023 18:16:20 +0000 (14:16 -0400)]
wordesc: Also escape non-printable chars
Tavian Barnes [Wed, 5 Jul 2023 14:45:56 +0000 (10:45 -0400)]
Use strcmp() instead of fnmatch() if possible
Tavian Barnes [Thu, 6 Jul 2023 12:59:37 +0000 (08:59 -0400)]
diag: Shell-escape expression arguments
Tavian Barnes [Tue, 4 Jul 2023 17:31:54 +0000 (13:31 -0400)]
ioq: Don't write to an empty slot in ioqq_trypop()
Tavian Barnes [Tue, 4 Jul 2023 17:30:10 +0000 (13:30 -0400)]
list: Fix some parameter docs
Tavian Barnes [Fri, 30 Jun 2023 15:46:22 +0000 (11:46 -0400)]
color: Don't print unnecessary reset sequences
Tavian Barnes [Fri, 30 Jun 2023 15:20:43 +0000 (11:20 -0400)]
docs: Start preparing for the 3.0 release
Tavian Barnes [Fri, 30 Jun 2023 14:25:42 +0000 (10:25 -0400)]
color: Compare values too when deciding to smart-case
Tavian Barnes [Thu, 29 Jun 2023 19:17:27 +0000 (15:17 -0400)]
color: Implement smart casing
Since coreutils 9.2, ls does case-sensitive extension matching if the
same extension is capitalized differently in $LS_COLORS. Implement the
same logic.
Link: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=33123
Link: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=9086
Tavian Barnes [Thu, 29 Jun 2023 17:53:48 +0000 (13:53 -0400)]
parse: Warn about errors parsing $LS_COLORS
Tavian Barnes [Thu, 29 Jun 2023 17:53:03 +0000 (13:53 -0400)]
dstring: Add some exact-size utility functions
Tavian Barnes [Thu, 29 Jun 2023 17:52:39 +0000 (13:52 -0400)]
dstring: Allow dstreserve(NULL, n)
Tavian Barnes [Tue, 27 Jun 2023 16:07:14 +0000 (12:07 -0400)]
tests/bfs/color: Add mixed-case extensions
Tavian Barnes [Tue, 27 Jun 2023 15:52:24 +0000 (11:52 -0400)]
tests: Get more tests passing with --bfs=tests/find-color.sh
Tavian Barnes [Mon, 26 Jun 2023 19:04:13 +0000 (15:04 -0400)]
thread: Wrap more pthread APIs
Tavian Barnes [Mon, 26 Jun 2023 15:47:41 +0000 (11:47 -0400)]
ioq: New ioq_cancel() function
Tavian Barnes [Mon, 26 Jun 2023 15:35:23 +0000 (11:35 -0400)]
ioq: Don't check NDEBUG manually in ioqq_trypop()
Tavian Barnes [Sat, 24 Jun 2023 17:22:38 +0000 (13:22 -0400)]
build/tsan: Fix target_clones override
Tavian Barnes [Sat, 24 Jun 2023 16:47:09 +0000 (12:47 -0400)]
bfstd: Add a getprogname() wrapper