summaryrefslogtreecommitdiff
AgeCommit message (Collapse)AuthorFilesLines
2023-10-22bcachefs: bkey_on_stackKent Overstreet12-104/+205
This implements code for storing small bkeys on the stack and allocating out of a mempool if they're too big. Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com> Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-10-22bcachefs: Make memcpy_to_bio() param constKent Overstreet2-2/+2
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com> Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-10-22bcachefs: Use wbc_to_write_flags()Kent Overstreet1-4/+3
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com> Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-10-22bcachefs: Fix erorr path in bch2_write()Kent Overstreet1-1/+6
The error path in bch2_write wasn't updated when the end_io callback was added to bch_write_op. Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com> Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-10-22bcachefs: Set lost+found mode to 0700Justin Husted2-2/+2
For security and conformance with other filesystems, the lost+found directory should not be world or group accessible. Signed-off-by: Justin Husted <sigstop@gmail.com> Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-10-22bcachefs: Be slightly less tricky with union usageKent Overstreet1-7/+11
This is to fix a valgrind complaint - the code was correct, but too tricky for valgrind to know that. Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com> Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-10-22bcachefs: Remove some BKEY_PADDED usesKent Overstreet2-16/+9
Prep work for extents with inline data Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com> Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-10-22bcachefs: Go back to 16 bit mantissa bkey floatsKent Overstreet1-71/+28
The previous optimizations means using 32 bit mantissas are now a net loss - having bkey_float be only 4 bytes is good for prefetching. Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com> Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-10-22bcachefs: Fall back to slowpath on exact comparisonKent Overstreet3-61/+48
This is basically equivalent to the original strategy of falling back to checking against the original key when the original key and previous key didn't differ in the required bits - except, now we only fall back when the search key doesn't differ in the required bits, which ends up being a bit faster. Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com> Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-10-22bcachefs: kill BFLOAT_FAILED_PREVKent Overstreet3-58/+4
The assumption underlying BFLOAT_FAILED_PREV was wrong; the comparison we're doing in bset_search_tree() doesn't have to tell the pivot apart from the previous key, it just has to tell if search is definitely greater than or equal to the pivot. Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com> Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-10-22bcachefs: bch2_read_extent() microoptimizationsKent Overstreet1-11/+13
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com> Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-10-22bcachefs: Pipeline binary searches and linear searchesKent Overstreet1-45/+69
This makes prefetching for the linear search at the end of the lookup much more effective, and is a couple percent speedup. Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com> Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-10-22bcachefs: Make __bch2_bkey_cmp_packed() smallerKent Overstreet1-16/+10
We can probably get rid of the version that dispatches based on type checking too. Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com> Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-10-22bcachefs: Inline fast path of bch2_increment_clock()Kent Overstreet2-5/+15
Shaving more cycles. Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com> Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-10-22bcachefs: Avoid calling bch2_btree_iter_relock() in bch2_btree_iter_traverse()Kent Overstreet1-4/+10
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-10-22bcachefs: Add an option for fsck error ratelimitingKent Overstreet3-4/+21
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com> Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-10-22bcachefs: Don't BUG_ON() sector count overflowKent Overstreet1-12/+14
Return an error instead (still work in progress...) Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com> Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-10-22bcachefs: Some reflink fixesKent Overstreet2-8/+9
len might fit into a loff_t when aligned_len does not - make sure we use a u64 for aligned_len. Also, we weren't always extending the inode correctly. Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com> Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-10-22bcachefs: Fix setting of attributes mask in getattrKent Overstreet1-0/+5
Discovered by xfstests generic/553 Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com> Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-10-22bcachefs: Eliminate function calls in DIO fastpathsKent Overstreet1-11/+15
We can assume that usually buffered and O_DIRECT IO won't be mixed, and the calls to flush the page cache won't be needed. Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-10-22bcachefs: DIO write path only needs to shoot down pagecache once, not twiceKent Overstreet1-19/+9
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-10-22bcachefs: Add pagecache_add lock to buffered IO path, fault pathKent Overstreet1-2/+2
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-10-22bcachefs: Fix uninitialized field in hash_check_init()Justin Husted1-0/+1
The chain_end field was not initialized before use in hash_set_chain_start. Signed-off-by: Justin Husted <sigstop@gmail.com> Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-10-22bcachefs: Don't hold inode lock longer than necessary in dio write pathKent Overstreet1-35/+54
In theory we should be able to do (non appending/extending) dio writes without taking the inode lock at all - but this gets us most of the way there. Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-10-22bcachefs: Avoid atomics in write fast pathKent Overstreet4-50/+65
This adds some horrible hacks, but the atomic ops for closures were getting to be a pretty expensive part of the write path. We don't want to rip out closures entirely from the write path, because they're used for e.g. waiting on the allocator, or waiting on the journal flush, and that stuff would get really ugly without closures. Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com> Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-10-22bcachefs: Don't use extent_ptr_decoded_append() in write path (fixup patch)Kent Overstreet1-1/+3
bch2_extent_ptr_decoded_append() is more general than we need here; we know we're initializing a new extent so e.g. we're going to need the crc entry. Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com> Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-10-22bcachefs: bch2_btree_iter_fix_key_modified()Kent Overstreet3-8/+31
This is considerably cheaper than bch2_btree_node_iter_fix(), for cases where the key was only modified and key ordering isn't changing. Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com> Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-10-22bcachefs: Inline more of bch2_trans_commit hot pathKent Overstreet6-97/+99
The main optimization here is that if we let bch2_replicas_delta_list_apply() fail, we can completely skip calling bch2_bkey_replicas_marked_locked(). And assorted other small optimizations. Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-10-22bcachefs: Trust btree alloc info at runtimeKent Overstreet3-1/+4
This lets us avoid a cache miss in the write path. Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com> Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-10-22bcachefs: More bset.c microoptimizationKent Overstreet3-8/+16
Improve a few paper cuts that've shown up during profiling. Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com> Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-10-22bcachefs: Initialize btree_node flags field in bch2_btree_root_alloc.Justin Husted1-0/+1
Valgrind data indicated that the flags field was only partially initialized when written to disk. Signed-off-by: Justin Husted <sigstop@gmail.com> Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-10-22bcachefs: Further padding fixes in bch2_journal_super_entries_add_common()Justin Husted1-11/+24
The previous patch 128cb1a to fix uninitialized data was incorrect and did not initialize the padding space correctly. Furthermore, several other cases in this function do not initialize their padding space correctly. Move initialization into some helper functions in a more robust way. Signed-off-by: Justin Husted <sigstop@gmail.com> Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-10-22bcachefs: Initialize padding space after alloc bkeyJustin Husted3-5/+16
Packed bkeys are padded up to 64 bit alignment, but the alloc bkey type was not clearing the pad bytes after the last data byte. This left the key possibly containing some random garbage at the end. This problem was found using valgrind. This patch also changes a path with the inode bkey to clear in the same way. Signed-off-by: Justin Husted <sigstop@gmail.com> Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-10-22bcachefs: Add missing error checking in bch2_find_by_inum_trans()Kent Overstreet1-3/+8
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-10-22bcachefs: Fix an error path raceKent Overstreet1-1/+6
On IO error, bch2_writepages_io_done() will set the page state to indicate nothing's already reserved (since the write didn't happen, we don't know what's already reserved). This can race with the buffered IO path, in between getting a disk reservation and calling bch2_set_page_dirty(). Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-10-22bcachefs: Don't reuse bio in retry pathKent Overstreet1-0/+1
We can't reuse bios without reinitializing them, and in the retry path it's safer to just make sure we don't reuse them at all. Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-10-22bcachefs: Don't use rep movsq for small memcopiesKent Overstreet3-4/+22
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-10-22bcachefs: Avoid calling iter_prev() in extent update pathKent Overstreet1-15/+8
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-10-22bcachefs: kill bch2_extent_merge_inline()Kent Overstreet1-287/+146
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-10-22bcachefs: Don't use FUA unnecessarilyKent Overstreet1-1/+4
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-10-22bcachefs: Refactor bch2_trans_commit() pathKent Overstreet10-228/+259
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-10-22bcachefs: Make btree_node_type_needs_gc() cheaperKent Overstreet3-29/+24
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-10-22bcachefs: Make replicas_delta_list smallerKent Overstreet3-10/+20
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-10-22bcachefs: Don't submit bio in write path under lockKent Overstreet1-3/+6
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-10-22bcachefs: Drop bch_write_op->io_wqKent Overstreet2-3/+0
This is dead code Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-10-22bcachefs: Limit bios in writepages path to 256MKent Overstreet1-0/+1
This works around a bug where bio_full() doesn't check for bio->bi_iter.bi_size overflowing - and, we don't really want to build bios that are that big anyways. Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-10-22bcachefs: Fix an iterator counting bugKent Overstreet1-14/+1
The iterator counting assumed we're doing an obvious optimization when only updating the refcount on indirect extents - but we're not doing it yet. Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-10-22bcachefs: Fix flushing held btree writes when there's a fs errorKent Overstreet1-0/+3
Previously, we'd go into an infinite loop. Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-10-22bcachefs: Fix iterator counting for reflink pointers (again)Kent Overstreet1-1/+2
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
2023-10-22bcachefs: Fix a debug assertionKent Overstreet1-0/+7
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
generated by cgit v1.2.3 (git 2.39.1) at 2024-12-12 18:36:45 +0000