Intel compiler docs and new compatible packages approach (conan-io#1479)

* Intel compiler docs and new compatible packages approach

* Review

Author: lasote

creating_packages/define_abi_compatibility.rst

@@ -169,12 +169,12 @@ in the range ``(v >= "4.5" and v < "5.0")`` will have exactly the same package I
 used to build it. It worked like an information erasure, once the binary is built, it is not possible to know which
 gcc was used to build it.
 
-Using ``CompatiblePackage``, it is possible to define compatible binaries that have different package IDs. With
-this ``CompatiblePackage`` feature, it is possible to have a different binary for each ``gcc`` version, so the 
+But it is possible to define compatible binaries that have different package IDs. For instance, it
+is possible to have a different binary for each ``gcc`` version, so the
 ``gcc 4.8`` package will be a different one with a different package ID than the ``gcc 4.9`` one, and still define
 that you can use the ``gcc 4.8`` package when building with ``gcc 4.9``.
 
-With ``CompatiblePackage`` we can define an ordered list of compatible packages, that will be checked in order if
+We can define an ordered list of compatible packages, that will be checked in order if
 the package ID that our profile defines is not available. Let's see it with an example:
 
 Lets say that we are building with a profile of ``gcc 4.9``. But for a given package we want to
@@ -191,14 +191,15 @@ That can be defined as:
         def package_id(self):
             if self.settings.compiler == "gcc" and self.settings.compiler.version == "4.9":
                 for version in ("4.8", "4.7"):
-                    compatible_pkg = CompatiblePackage(self)
+                    compatible_pkg = self.info.clone()
                     compatible_pkg.settings.compiler.version = version
                     self.compatible_packages.append(compatible_pkg)
 
 Note that if the input configuration is ``gcc 4.8``, it will not try to fallback to binaries of ``gcc 4.7`` as the
 condition is not met.
 
-The ``CompatiblePackage()`` copies the values of ``settings``, ``options`` and ``requires`` from the current instance of the recipe so they can be modified to model the compatibility.
+The ``self.info.clone()`` method copies the values of ``settings``, ``options`` and ``requires`` from the current instance of
+the recipe so they can be modified to model the compatibility.
 
 It is the responsibility of the developer to guarantee that such binaries are indeed compatible. For example in:
 
@@ -211,7 +212,7 @@ It is the responsibility of the developer to guarantee that such binaries are in
         default_options = {"optimized": 1}
         def package_id(self):
             for optimized in range(int(self.options.optimized), 0, -1):
-                compatible_pkg = CompatiblePackage(self)
+                compatible_pkg = self.info.clone()
                 compatible_pkg.options.optimized = optimized
                 self.compatible_packages.append(compatible_pkg)
 
@@ -226,6 +227,88 @@ the ``optimized`` option to conditionally require different dependencies, that w
 step is processed after the whole dependency graph has been built, so it is not possible to define how dependencies are resolved
 based on this compatibility model, it only applies to use-cases where the binaries can be *interchanged*.
 
+Check the :ref:`Compatible Compilers<compatible_compilers>` section to see another example of how to take benefit of compatible packages.
+
+
+.. _compatible_compilers:
+
+Compatible Compilers
+--------------------
+
+Some compilers make use of a base compiler to operate, for example, the ``intel`` compiler uses
+the ``Visual Studio`` compiler in Windows environments and ``gcc`` in Linux environments.
+
+The ``intel`` compiler is declared this way in the :ref:`settings.yml<settings_yml>`:
+
+ .. code-block:: yaml
+
+    intel:
+        version: ["11", "12", "13", "14", "15", "16", "17", "18", "19"]
+        base:
+            gcc:
+                <<: *gcc
+                threads: [None]
+                exception: [None]
+            Visual Studio:
+                <<: *visual_studio
+
+Remember, you can :ref:`extend Conan<extending>` to support other compilers.
+
+
+You can use the ``package_id()`` method to define the compatibility between the packages generated by the ``base`` compiler and the ``parent`` one.
+You can use the following helpers together with the :ref:`compatible packages<compatible_packages>` feature to:
+
+    - Consume native ``Visual Studio`` packages when the input compiler in the profile is ``intel`` (if no ``intel`` package is available).
+    - The opposite, consume an ``intel`` compiler package when a consumer profile specifies ``Visual Studio`` as the input compiler (if no
+      ``Visual Studio`` package is available).
+
+- ``base_compatible()``: This function will transform the settings used to calculate the package ID into the "base" compiler.
+
+  .. code-block:: python
+
+      def package_id(self):
+
+          if self.settings.compiler == "intel":
+              p = self.info.clone()
+              p.base_compatible()
+              self.compatible_packages.append(p)
+
+  Using the above ``package_id()`` method, if a consumer specifies a profile with a intel profile (**-s compiler=="intel"**) and there is no binary available, it will resolve to a
+  Visual Studio package ID corresponding to the base compiler.
+
+
+- ``parent_compatible(compiler="compiler", version="version")``: This function transforms the settings of a compiler into the settings of a
+  parent one using the specified one as the base compiler. As the details of the "parent" compatible cannot be guessed, you have to provide them as **keyword args** to the
+  function. The "compiler" argument is mandatory, the rest of keyword arguments will be used to initialize the ``info.settings.compiler.XXX`` objects
+  to calculate the correct package ID.
+
+  .. code-block:: python
+
+      def package_id(self):
+
+         if self.settings.compiler == "Visual Studio":
+            compatible_pkg = self.info.clone()
+            compatible_pkg.parent_compatible(compiler="intel", version=16)
+            self.compatible_packages.append(compatible_pkg)
+
+  In this case, for a consumer specifying Visual Studio compiler, if no package is found, it will search for an "intel" package for the version 16.
+
+
+Take into account that you can use also these helpers without the "compatible packages" feature:
+
+  .. code-block:: python
+
+      def package_id(self):
+
+         if self.settings.compiler == "Visual Studio":
+            self.info.parent_compatible(compiler="intel", version=16)
+
+In the above example, we will transform the package ID of the ``Visual Studio`` package to be the same as the ``intel 16``, but you won't
+be able to differentiate the packages built with ``intel`` with the ones built by ``Visual Studio`` because both will have the same package ID,
+and that is not always desirable.
+
+
+
 
 .. _problem_of_dependencies:
 
@@ -271,6 +354,7 @@ Then, **a new binary for MyLib/1.0 is required to be built for the new dependenc
 ``addition()`` version. Even in the case that ``MyLib/1.0`` doesn't have any change in its code lines neither in the recipe, the resulting
 binary rebuilding ``MyLib`` requires ``MyOtherLib/2.1`` and the package to be different.
 
+
 .. _package_id_mode:
 
 Using package_id() for Package Dependencies
@@ -564,6 +648,7 @@ The default behavior produces a *conaninfo.txt* that looks like:
 
 Changing the default package-id mode
 ++++++++++++++++++++++++++++++++++++
+
 It is possible to change the default ``semver_direct_mode`` package-id mode, in the
 *conan.conf* file:
 
@@ -591,6 +676,26 @@ that only generates 1 package-id for all possible configurations and versions of
 
 Remember that *conan.conf* can be shared and installed with :ref:`conan_config_install`.
 
+Take into account that you can combine the :ref:`compatible packages<compatible_packages>` with the package-id modes.
+
+For example, if you are generating binary packages with the default ``recipe_revision_mode``,
+but you want these packages to be consumed from a client with a different mode activated,
+you can create a compatible package transforming the mode to ``recipe_revision_mode`` so the package
+generated with the ``recipe_revision_mode`` can be resolved if no package for the default mode is found:
+
+.. code-block:: python
+
+    from conans import ConanFile, CompatiblePackage
+
+    class Pkg(ConanFile):
+        ...
+
+        def package_id(self):
+            p = self.info.clone()
+            p.requires.recipe_revision_mode()
+            self.compatible_packages.append(p)
+
+
 Library Types: Shared, Static, Header-only
 ++++++++++++++++++++++++++++++++++++++++++
 

howtos.rst

@@ -12,6 +12,7 @@ This section shows common solutions and different approaches to typical problems
    howtos/makefiles
    howtos/manage_gcc_abi
    howtos/vs2017_cmake
+   howtos/intel_compiler
    howtos/manage_cpp_standard
    howtos/run_conan_in_docker
    howtos/python_code_reuse

howtos/intel_compiler.rst

@@ -0,0 +1,9 @@
+.. _howto_intel_compiler:
+
+Working with Intel compiler
+===========================
+
+The ``Intel`` compiler is a particular case, as it uses ``Visual Studio`` compiler in Windows environments
+and ``gcc`` in Linux environments. If you are wondering how to manage the compatibility between the packages generated
+with ``intel`` and the generated with the pure base compiler (``gcc`` or ``Visual Studio``) check the
+:ref:`Compatible Packages<compatible_packages>` and :ref:`Compatible Compilers<compatible_compilers>` sections.

integrations.rst

@@ -10,6 +10,7 @@ packages can be consumed, created, and continuously deployed/tested with each, a
 .. toctree::
    :maxdepth: 2
 
+   integrations/compilers
    integrations/build_system
    integrations/ides
    integrations/ci

integrations/compilers.rst

@@ -0,0 +1,27 @@
+.. _integration_compilers:
+
+Compilers
+=========
+
+
+Conan can work with any compiler, the most common ones are already declared in the default :ref:`settings.yml<settings_yml>`:
+
+- `sun-cc`
+- `gcc`
+- `Visual Studio`
+- `clang`
+- `apple-clang`
+- `qcc`
+- `intel`
+
+.. note ::
+
+    Remember that you can :ref:`customize Conan <extending>` to extend the supported compilers, build systems, etc.
+
+
+.. important::
+
+    If you work with a compiler like ``intel`` that uses ``Visual Studio`` in Windows environments
+    and ``gcc`` in Linux environments and you are wondering how to manage the compatibility between the packages generated
+    with ``intel`` and the generated with the pure base compiler (``gcc`` or ``Visual Studio``) check the
+    :ref:`Compatible Packages<compatible_packages>` and :ref:`Compatible Compilers<compatible_compilers>` sections.

reference/config_files/settings.yml.rst

@@ -30,15 +30,15 @@ are possible. These are the **default** values, but it is possible to customize
             version: ["5.0", "6.0", "7.0", "8.0"]
         Linux:
         Macos:
-            version: [None, "10.6", "10.7", "10.8", "10.9", "10.10", "10.11", "10.12", "10.13", "10.14"]
+            version: [None, "10.6", "10.7", "10.8", "10.9", "10.10", "10.11", "10.12", "10.13", "10.14", "10.15"]
         Android:
             api_level: ANY
         iOS:
-            version: ["7.0", "7.1", "8.0", "8.1", "8.2", "8.3", "9.0", "9.1", "9.2", "9.3", "10.0", "10.1", "10.2", "10.3", "11.0", "11.1", "11.2", "11.3", "11.4", "12.0", "12.1"]
+            version: ["7.0", "7.1", "8.0", "8.1", "8.2", "8.3", "9.0", "9.1", "9.2", "9.3", "10.0", "10.1", "10.2", "10.3", "11.0", "11.1", "11.2", "11.3", "11.4", "12.0", "12.1", "12.2", "12.3", "12.4", "13.0", "13.1"]
         watchOS:
-            version: ["4.0", "4.1", "4.2", "4.3", "5.0", "5.1"]
+            version: ["4.0", "4.1", "4.2", "4.3", "5.0", "5.1", "5.2", "5.3", "6.0", "6.1"]
         tvOS:
-            version: ["11.0", "11.1", "11.2", "11.3", "11.4", "12.0", "12.1"]
+            version: ["11.0", "11.1", "11.2", "11.3", "11.4", "12.0", "12.1", "12.2", "12.3", "12.4", "13.0"]
         FreeBSD:
         SunOS:
         AIX:
@@ -53,18 +53,18 @@ are possible. These are the **default** values, but it is possible to customize
             version: ["5.10", "5.11", "5.12", "5.13", "5.14"]
             threads: [None, posix]
             libcxx: [libCstd, libstdcxx, libstlport, libstdc++]
-        gcc:
+        gcc: &gcc
             version: ["4.1", "4.4", "4.5", "4.6", "4.7", "4.8", "4.9",
                       "5", "5.1", "5.2", "5.3", "5.4", "5.5",
                       "6", "6.1", "6.2", "6.3", "6.4",
-                      "7", "7.1", "7.2", "7.3",
+                      "7", "7.1", "7.2", "7.3", "7.4",
                       "8", "8.1", "8.2", "8.3",
                       "9", "9.1", "9.2"]
             libcxx: [libstdc++, libstdc++11]
             threads: [None, posix, win32] #  Windows MinGW
             exception: [None, dwarf2, sjlj, seh] # Windows MinGW
             cppstd: [None, 98, gnu98, 11, gnu11, 14, gnu14, 17, gnu17, 20, gnu20]
-        Visual Studio:
+        Visual Studio: &visual_studio
             runtime: [MD, MT, MTd, MDd]
             version: ["8", "9", "10", "11", "12", "14", "15", "16"]
             toolset: [None, v90, v100, v110, v110_xp, v120, v120_xp,
@@ -74,21 +74,31 @@ are possible. These are the **default** values, but it is possible to customize
             cppstd: [None, 14, 17, 20]
         clang:
             version: ["3.3", "3.4", "3.5", "3.6", "3.7", "3.8", "3.9", "4.0",
-                      "5.0", "6.0", "7.0",
-                      "8", "9"]
+                      "5.0", "6.0", "7.0", "7.1",
+                      "8", "9", "10"]
             libcxx: [libstdc++, libstdc++11, libc++, c++_shared, c++_static]
             cppstd: [None, 98, gnu98, 11, gnu11, 14, gnu14, 17, gnu17, 20, gnu20]
         apple-clang:
             version: ["5.0", "5.1", "6.0", "6.1", "7.0", "7.3", "8.0", "8.1", "9.0", "9.1", "10.0", "11.0"]
             libcxx: [libstdc++, libc++]
             cppstd: [None, 98, gnu98, 11, gnu11, 14, gnu14, 17, gnu17, 20, gnu20]
+        intel:
+            version: ["11", "12", "13", "14", "15", "16", "17", "18", "19"]
+            base:
+                gcc:
+                    <<: *gcc
+                    threads: [None]
+                    exception: [None]
+                Visual Studio:
+                    <<: *visual_studio
         qcc:
             version: ["4.4", "5.4"]
             libcxx: [cxx, gpp, cpp, cpp-ne, accp, acpp-ne, ecpp, ecpp-ne]
 
     build_type: [None, Debug, Release, RelWithDebInfo, MinSizeRel]
     cppstd: [None, 98, gnu98, 11, gnu11, 14, gnu14, 17, gnu17, 20, gnu20]  # Deprecated, use compiler.cppstd
 
+
 As you can see, the possible values ``settings`` can take are restricted in the same file. This is done to ensure matching naming and
 spelling as well as defining a common settings model among users and the OSS community.
 If a setting is allowed to be set to any value, you can use ``ANY``.