Re-apply r267784, r267824 and r267830.

I have updated the compiler-rt tests.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@267903 91177308-0d34-0410-b5e6-96231b3b80d8

4 files changed, 137 insertions, 24 deletions

diff --git a/docs/ControlFlowIntegrity.rst b/docs/ControlFlowIntegrity.rst
index c403610952..eed5ac5120 100644
--- a/docs/ControlFlowIntegrity.rst
+++ b/docs/ControlFlowIntegrity.rst
@@ -25,13 +25,25 @@ As currently implemented, all schemes rely on link-time optimization (LTO);
 so it is required to specify ``-flto``, and the linker used must support LTO,
 for example via the `gold plugin`_.
 
-To allow the checks to be implemented efficiently, the program must be
-structured such that certain object files are compiled with CFI
+To allow the checks to be implemented efficiently, the program must
+be structured such that certain object files are compiled with CFI
 enabled, and are statically linked into the program. This may preclude
-the use of shared libraries in some cases. Experimental support for
-:ref:`cross-DSO control flow integrity <cfi-cross-dso>` exists that
-does not have these requirements. This cross-DSO support has unstable
-ABI at this time.
+the use of shared libraries in some cases.
+
+The compiler will only produce CFI checks for a class if it can infer hidden
+LTO visibility for that class. LTO visibility is a property of a class that
+is inferred from flags and attributes. For more details, see the documentation
+for :doc:`LTO visibility <LTOVisibility>`.
+
+The ``-fsanitize=cfi-{vcall,nvcall,derived-cast,unrelated-cast}`` flags
+require that a ``-fvisibility=`` flag also be specified. This is because the
+default visibility setting is ``-fvisibility=default``, which would disable
+CFI checks for classes without visibility attributes. Most users will want
+to specify ``-fvisibility=hidden``, which enables CFI checks for such classes.
+
+Experimental support for :ref:`cross-DSO control flow integrity
+<cfi-cross-dso>` exists that does not require classes to have hidden LTO
+visibility. This cross-DSO support has unstable ABI at this time.
 
 .. _gold plugin: http://llvm.org/docs/GoldPlugin.html
 
@@ -233,11 +245,6 @@ A :doc:`SanitizerSpecialCaseList` can be used to relax CFI checks for certain
 source files, functions and types using the ``src``, ``fun`` and ``type``
 entity types.
 
-In addition, if a type has a ``uuid`` attribute and the blacklist contains
-the type entry ``attr:uuid``, CFI checks are suppressed for that type. This
-allows all COM types to be easily blacklisted, which is useful as COM types
-are typically defined outside of the linked program.
-
 .. code-block:: bash
 
     # Suppress checking for code in a file.
@@ -247,8 +254,6 @@ are typically defined outside of the linked program.
     fun:*MyFooBar*
     # Ignore all types in the standard library.
     type:std::*
-    # Ignore all types with a uuid attribute.
-    type:attr:uuid
 
 .. _cfi-cross-dso:
 
@@ -260,6 +265,11 @@ flow integrity mode, which allows all CFI schemes listed above to
 apply across DSO boundaries. As in the regular CFI, each DSO must be
 built with ``-flto``.
 
+Normally, CFI checks will only be performed for classes that have hidden LTO
+visibility. With this flag enabled, the compiler will emit cross-DSO CFI
+checks for all classes, except for those which appear in the CFI blacklist
+or which use a ``no_sanitize`` attribute.
+
 Design
 ======
 
diff --git a/docs/LTOVisibility.rst b/docs/LTOVisibility.rst
new file mode 100644
index 0000000000..21a3157c83
--- /dev/null
+++ b/docs/LTOVisibility.rst
@@ -0,0 +1,111 @@
+==============
+LTO Visibility
+==============
+
+*LTO visibility* is a property of an entity that specifies whether it can be
+referenced from outside the current LTO unit. A *linkage unit* is a set of
+translation units linked together into an executable or DSO, and a linkage
+unit's *LTO unit* is the subset of the linkage unit that is linked together
+using link-time optimization; in the case where LTO is not being used, the
+linkage unit's LTO unit is empty. Each linkage unit has only a single LTO unit.
+
+The LTO visibility of a class is used by the compiler to determine which
+classes the virtual function call optimization and control flow integrity
+features apply to. These features use whole-program information, so they
+require the entire class hierarchy to be visible in order to work correctly.
+
+If any translation unit in the program uses either of the virtual function
+call optimization or control flow integrity features, it is effectively an
+ODR violation to define a class with hidden LTO visibility in multiple linkage
+units. A class with public LTO visibility may be defined in multiple linkage
+units, but the tradeoff is that the virtual function call optimization and
+control flow integrity features can only be applied to classes with hidden LTO
+visibility. A class's LTO visibility is treated as an ODR-relevant property
+of its definition, so it must be consistent between translation units.
+
+In translation units built with LTO, LTO visibility is based on symbol
+visibility or, on the Windows platform, the dllimport and dllexport
+attributes. When targeting non-Windows platforms, classes with a visibility
+other than hidden visibility receive public LTO visibility. When targeting
+Windows, classes with dllimport or dllexport attributes receive public LTO
+visibility. All other classes receive hidden LTO visibility. Classes with
+internal linkage (e.g. classes declared in unnamed namespaces) also receive
+hidden LTO visibility.
+
+A class defined in a translation unit built without LTO receives public
+LTO visibility regardless of its object file visibility, linkage or other
+attributes.
+
+This mechanism will produce the correct result in most cases, but there are
+two cases where it may wrongly infer hidden LTO visibility.
+
+1. As a corollary of the above rules, if a linkage unit is produced from a
+   combination of LTO object files and non-LTO object files, any hidden
+   visibility class defined in both a translation unit built with LTO and
+   a translation unit built without LTO must be defined with public LTO
+   visibility in order to avoid an ODR violation.
+
+2. Some ABIs provide the ability to define an abstract base class without
+   visibility attributes in multiple linkage units and have virtual calls
+   to derived classes in other linkage units work correctly. One example of
+   this is COM on Windows platforms. If the ABI allows this, any base class
+   used in this way must be defined with public LTO visibility.
+
+Classes that fall into either of these categories can be marked up with the
+``[[clang::lto_visibility_public]]`` attribute. To specifically handle the
+COM case, classes with the ``__declspec(uuid())`` attribute receive public
+LTO visibility. On Windows platforms, clang-cl's ``/MT`` and ``/MTd``
+flags statically link the program against a prebuilt standard library;
+these flags imply public LTO visibility for every class declared in the
+``std`` and ``stdext`` namespaces.
+
+Example
+=======
+
+The following example shows how LTO visibility works in practice in several
+cases involving two linkage units, ``main`` and ``dso.so``.
+
+.. code-block:: none
+
+    +-----------------------------------------------------------+  +----------------------------------------------------+
+    | main (clang++ -fvisibility=hidden):                       |  | dso.so (clang++ -fvisibility=hidden):              |
+    |                                                           |  |                                                    |
+    |  +-----------------------------------------------------+  |  |  struct __attribute__((visibility("default"))) C { |
+    |  | LTO unit (clang++ -fvisibility=hidden -flto):       |  |  |    virtual void f();                               |
+    |  |                                                     |  |  |  }                                                 |
+    |  |  struct A { ... };                                  |  |  |  void C::f() {}                                    |
+    |  |  struct [[clang::lto_visibility_public]] B { ... }; |  |  |  struct D {                                        |
+    |  |  struct __attribute__((visibility("default"))) C {  |  |  |    virtual void g() = 0;                           |
+    |  |    virtual void f();                                |  |  |  };                                                |
+    |  |  };                                                 |  |  |  struct E : D {                                    |
+    |  |  struct [[clang::lto_visibility_public]] D {        |  |  |    virtual void g() { ... }                        |
+    |  |    virtual void g() = 0;                            |  |  |  };                                                |
+    |  |  };                                                 |  |  |  __attribute__(visibility("default"))) D *mkE() {  |
+    |  |                                                     |  |  |    return new E;                                   |
+    |  +-----------------------------------------------------+  |  |  }                                                 |
+    |                                                           |  |                                                    |
+    |  struct B { ... };                                        |  +----------------------------------------------------+
+    |                                                           |
+    +-----------------------------------------------------------+
+
+We will now describe the LTO visibility of each of the classes defined in
+these linkage units.
+
+Class ``A`` is not defined outside of ``main``'s LTO unit, so it can have
+hidden LTO visibility. This is inferred from the object file visibility
+specified on the command line.
+
+Class ``B`` is defined in ``main``, both inside and outside its LTO unit. The
+definition outside the LTO unit has public LTO visibility, so the definition
+inside the LTO unit must also have public LTO visibility in order to avoid
+an ODR violation.
+
+Class ``C`` is defined in both ``main`` and ``dso.so`` and therefore must
+have public LTO visibility. This is correctly inferred from the ``visibility``
+attribute.
+
+Class ``D`` is an abstract base class with a derived class ``E`` defined
+in ``dso.so``.  This is an example of the COM scenario; the definition of
+``D`` in ``main``'s LTO unit must have public LTO visibility in order to be
+compatible with the definition of ``D`` in ``dso.so``, which is observable
+by calling the function ``mkE``.
diff --git a/docs/UsersManual.rst b/docs/UsersManual.rst
index 34e4408719..7a9bdb96c5 100644
--- a/docs/UsersManual.rst
+++ b/docs/UsersManual.rst
@@ -1056,17 +1056,8 @@ are listed below.
 .. option:: -fwhole-program-vtables
 
    Enable whole-program vtable optimizations, such as single-implementation
-   devirtualization and virtual constant propagation. Requires ``-flto``.
-
-   By default, the compiler will assume that all type hierarchies are
-   closed except those in the ``std`` namespace, the ``stdext`` namespace
-   and classes with the ``__declspec(uuid())`` attribute.
-
-.. option:: -fwhole-program-vtables-blacklist=path
-
-   Allows the user to specify the path to a list of additional classes to
-   blacklist from whole-program vtable optimizations. This list is in the
-   :ref:`CFI blacklist <cfi-blacklist>` format.
+   devirtualization and virtual constant propagation, for classes with
+   :doc:`hidden LTO visibility <LTOVisibility>`. Requires ``-flto``.
 
 .. option:: -fno-assume-sane-operator-new
 
diff --git a/docs/index.rst b/docs/index.rst
index 6028fb8ec2..f287911d3c 100644
--- a/docs/index.rst
+++ b/docs/index.rst
@@ -31,6 +31,7 @@ Using Clang as a Compiler
    SanitizerStats
    SanitizerSpecialCaseList
    ControlFlowIntegrity
+   LTOVisibility
    SafeStack
    Modules
    MSVCCompatibility