diff options
| author | Hugh Dickins <hughd@google.com> | 2022-02-14 18:31:48 -0800 |
|---|---|---|
| committer | Matthew Wilcox (Oracle) <willy@infradead.org> | 2022-02-17 11:57:02 -0500 |
| commit | 34b6792380ce4f4b41018351cd67c9c26f4a7a0d (patch) | |
| tree | 8fe823dca2e14be9871318630740fd42372d3fa6 /mm/internal.h | |
| parent | 07ca760673088f262da57ff42c15558688565aa2 (diff) | |
mm/munlock: mlock_pte_range() when mlocking or munlocking
Fill in missing pieces: reimplementation of munlock_vma_pages_range(),
required to lower the mlock_counts when munlocking without munmapping;
and its complement, implementation of mlock_vma_pages_range(), required
to raise the mlock_counts on pages already there when a range is mlocked.
Combine them into just the one function mlock_vma_pages_range(), using
walk_page_range() to run mlock_pte_range(). This approach fixes the
"Very slow unlockall()" of unpopulated PROT_NONE areas, reported in
https://lore.kernel.org/linux-mm/70885d37-62b7-748b-29df-9e94f3291736@gmail.com/
Munlock clears VM_LOCKED at the start, under exclusive mmap_lock; but if
a racing truncate or holepunch (depending on i_mmap_rwsem) gets to the
pte first, it will not try to munlock the page: leaving release_pages()
to correct it when the last reference to the page is gone - that's okay,
a page is not evictable anyway while it is held by an extra reference.
Mlock sets VM_LOCKED at the start, under exclusive mmap_lock; but if
a racing remove_migration_pte() or try_to_unmap_one() (depending on
i_mmap_rwsem) gets to the pte first, it will try to mlock the page,
then mlock_pte_range() mlock it a second time. This is harder to
reproduce, but a more serious race because it could leave the page
unevictable indefinitely though the area is munlocked afterwards.
Guard against it by setting the (inappropriate) VM_IO flag,
and modifying mlock_vma_page() to decline such vmas.
Signed-off-by: Hugh Dickins <hughd@google.com>
Acked-by: Vlastimil Babka <vbabka@suse.cz>
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Diffstat (limited to 'mm/internal.h')
| -rw-r--r-- | mm/internal.h | 3 |
1 files changed, 2 insertions, 1 deletions
diff --git a/mm/internal.h b/mm/internal.h index a43d79335c16..b3f0dd3ffba2 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -412,7 +412,8 @@ void mlock_page(struct page *page); static inline void mlock_vma_page(struct page *page, struct vm_area_struct *vma, bool compound) { - if (unlikely(vma->vm_flags & VM_LOCKED) && + /* VM_IO check prevents migration from double-counting during mlock */ + if (unlikely((vma->vm_flags & (VM_LOCKED|VM_IO)) == VM_LOCKED) && (compound || !PageTransCompound(page))) mlock_page(page); } |