fdo#72755: Only use double mmap as fallback

...when write+exec mmap fails (due to SELinux deny_execmem).  This avoids the
tmp file creation in environments that don't need it and which in turn have
problems of their own with that tmp file business.

An alternative would be to first check whether SELinux deny_execmem is enforced
and only then try double mmap first.  An advantage could be that it might avoid
false SELinux alerts in that case.  The disadvantage would be the overhead of
introducing a conditional dependency on libselinux here.  And given that for one
deny_execmem typically appears to be off by default (as at least both
contemporary GNOME desktop and OpenJDK malfunction when it is enabled), and for
another I guess deny_execmem could still change its value between the time of
checking for it and the time of requesting a write+exec mmap, that just does not
seem worth it.

Change-Id: I3560803139b630557b6219d3db52945c7e0cdcd2

1 files changed, 6 insertions, 7 deletions

diff --git a/bridges/source/cpp_uno/shared/vtablefactory.cxx b/bridges/source/cpp_uno/shared/vtablefactory.cxx
index ca77aea72e50..226a994bf664 100644
--- a/bridges/source/cpp_uno/shared/vtablefactory.cxx
+++ b/bridges/source/cpp_uno/shared/vtablefactory.cxx
@@ -229,9 +229,14 @@ bool VtableFactory::createBlock(Block &block, sal_Int32 slotCount) const
     sal_Size size = getBlockSize(slotCount);
     sal_Size pagesize = sysconf(_SC_PAGESIZE);
     block.size = (size + (pagesize - 1)) & ~(pagesize - 1);
-    block.start = block.exec = NULL;
     block.fd = -1;
 
+    // Try non-doublemmaped allocation first:
+    block.start = block.exec = rtl_arena_alloc(m_arena, &block.size);
+    if (block.start != nullptr) {
+        return true;
+    }
+
     osl::Security aSecurity;
     OUString strDirectory;
     OUString strURLDirectory;
@@ -290,12 +295,6 @@ bool VtableFactory::createBlock(Block &block, sal_Int32 slotCount) const
 
         strDirectory.clear();
     }
-    if (!block.start || !block.exec || block.fd == -1)
-    {
-       //Fall back to non-doublemmaped allocation
-       block.fd = -1;
-       block.start = block.exec = rtl_arena_alloc(m_arena, &block.size);
-    }
     return (block.start != 0 && block.exec != 0);
 }