deps: add docs to upgrade openssl

This document is intended to describe the procedure to upgrade openssl
from 1.0.1m to 1.0.2a in io.js.

Author: Shigeki Ohtsu

deps/openssl/doc/UPGRADING.md

@@ -0,0 +1,190 @@
+## How to upgrade openssl library in io.js
+
+This document is intended to describe the procedure to upgrade openssl
+from 1.0.1m to 1.0.2a in io.js.
+
+
+### Build System and Upgrading Overview
+The openssl build system is based on the perl script of Configure.
+For example, running `Configure linux_x86-64` in the openssl
+repository generates `Makefile` and `opensslconf.h` for the target
+architectures of linux_x86_64.
+The `Makefile` contains the list of asm files which are generated by
+perl scripts during build so that we can get the most of use of the
+hardware performance according to the type of cpus.
+
+`Configure TABLE` shows various build parameters that depend on each
+os and arch.
+
+In io.js, build target is defined as `--dest-os` and `--dest-cpu` in
+configure options which are different from the one that is defined in
+openssl and it's build system is gyp that is based on python,
+therefore we cannot use the openssl build system directly.
+
+In order to build openssl with gyp in iojs, files of opensslconf.h and
+asm are generated in advance for several supported platforms.
+
+Here is a map table to show conf(opensslconf.h) and asm between
+the openssl target and configuration parameters of os and cpu in iojs.
+The tested platform in CI are also listed.
+
+| --dest-os | --dest-cpu | conf | asm  | openssl target     | CI |
+|---------- |----------- |----- |----- |------------------- |--- |
+| linux     | ia32       | o    | o    |linux-elf           | o  |
+| linux     | x32        | o    | x(*2)|linux-x32           | x  |
+| linux     | x64        | o    | o    |linux-x86_64        | o  |
+| linux     | arm        | o    | o    |linux-arm           | o  |
+| linux     | arm64      | o    | o    |linux-aarch64       | o  |
+| mac       | ia32       | o    | o    |darwin-i386-cc      | o  |
+| mac       | x64        | o    | o    |darwin64-x86_64-cc  | o  |
+| win       | ia32       | o    | o(*3)|VC-WIN32            | x  |
+| win       | x64        | o    | o    |VC-WIN64A           | o  |
+| solaris   | ia32       | o    | o    |solaris-x86-gcc     | o  |
+| solaris   | x64        | o    | o    |solaris64-x86_64-gcc| o  |
+| freebsd   | ia32       | o    | o    |BSD-x86             | o  |
+| freebsd   | x64        | o    | o    |BSD-x86_64          | o  |
+| openbsd   | ia32       | o    | o    |BSD-x86             | x  |
+| openbsd   | x64        | o    | o    |BSD-x86_64          | x  |
+| others    | ia32       | x(*1)| o    | -                  | x  |
+| others    | x64        | x(*1)| o    | -                  | x  |
+| others    | arm        | x(*1)| o    | -                  | x  |
+| others    | arm64      | x(*1)| o    | -                  | x  |
+| others    | others     | x(*1)| x(*2)| -                  | x  |
+
+- (*1) use linux-elf as a fallback configuration
+- (*2) no-asm used
+- (*3) currently masm (Microsoft Macro Assembler) is used but it's no
+longer supported in opnessl. We need to move to use nasm or yasm.
+
+All parameters such as sources, defines, cflags and others generated
+in openssl Makefile are wrote down into `deps/openssl/openssl.gypi`.
+
+The header file of `deps/openssl/openssl/crypto/opensslconf.h` are
+generated by `Configure` and varies on each os and arch so that we
+made a new `deps/openssl/config/opensslconf.h`, where it includes each
+conf file from `deps/openssl/config/archs/*/opensslconf.h` by using
+pre-defined compiler macros. This procedure can be processed
+automatically with `deps/openssl/config/Makefile`
+
+Assembler support is one of the key features in openssl, but asm files
+are dynamically generated with
+`deps/openssl/openssl/crypto/*/asm/*.pl` by perl during
+build. Furthermore, these perl scripts check the version of asm
+compiler and generate asm files according to the supported
+instructions in each compiler.
+
+Since perl is not a build requirement in iojs, they all should be
+generated in advance and statically stored in the repository. We
+provide two sets of asm files, one is asm_latest(avx2 and addx
+supported) in `deps/openssl/asm` and the other asm_obsolete(without
+avx1/2 and addx) in `deps/openssl/asm_obsolute`, which depends on
+supported features in asm compilers. Each directory has a `Makefile`
+to generate asm files with perl scripts in openssl sources.
+
+`configure` and gyp check the version of asm compilers such as gnu
+as(gas), llvm and Visual Studio. `deps/openssl/openssl.gypi`
+determines what asm files should be used, in which the asm_latest
+needs the version of gas >= 2.23, llvm >= 3.3 or MSVS_VERSION>='2012'
+(ml64 >= 12) as defined in
+https://github.com/openssl/openssl/blob/OpenSSL_1_0_2-stable/crypto/sha/asm/sha512-x86_64.pl#L112-L129,
+otherwise asm_obsolete are used.
+
+The following is the detail instruction steps how to upgrade openssl
+version from 1.0.1m to 1.0.2a in iojs.
+
+### 1. Replace openssl source in `deps/openssl/openssl`
+Remove old openssl sources in `deps/openssl/openssl` .
+Get original openssl sources from
+https://www.openssl.org/source/openssl-1.0.2a.tar.gz and extract all
+files into `deps/openssl/openssl` .
+
+### 2. Apply private patches
+There are three kinds of private patches to be applied in openssl-1.0.2a.
+
+- The two fixes of assembly error on ia32 win32. masm is no longer
+  supported in openssl. We should move to use nasm or yasm in future
+  version of iojs.
+
+- The fix of openssl-cli built on win. Key press requirement of
+  openssl-cli in win causes timeout failures of several tests.
+
+- Backport patches for alt cert feature from openssl-1.1.x. Root certs
+  of 1024bit RSA key length were deprecated in io.js. When a tls
+  server has a cross root cert, io.js client leads CERT_UNTRUSTED
+  error because openssl does not find alternate cert chains. This fix
+  supports its feature but was made the current master which is
+  openssl-1.1.x. We backported them privately into openssl-1.0.2 on
+  iojs.
+
+### 3. Replace openssl header files in `deps/openssl/openssl/include/openssl`
+all header files in `deps/openssl/openssl/include/openssl/*.h` are
+symbolic links in the distributed release tar.gz. They cause issues in
+Windows. They are replaced into the files to include a real header
+file such as
+````
+#include "../../crypto/aes/aes.h"
+````
+### 4. Change `opensslconf.h` so as to fit each platform.
+The opensslconf.h in each target was created in advance by typing
+`deps/openssl/openssl/Configure {target}` and copied
+into `deps/openssl/conf/archs/{target}/opensslconf.h`.
+`deps/openssl/conf/openssconf.h` includes each file according to its
+target by checking pre-defined compiler macros. These can be generated
+by using `deps/openssl/conf/Makefile`
+
+We should remove OPENSSL_CPUID_OBJ define in opensslconf.h because it
+causes build error when --openss-no-asm option is specified. Instead,
+the OPENSSL_CPUID_OBJ is defined in `deps/openssl/openssl.gypi`
+according to the configure options.
+
+One fix of opensslconf.h is needed in 64-bit MacOS.
+The current openssl release does not use RC4 asm since it explicitly
+specified as `$asm=~s/rc4\-[^:]+//;` in
+https://github.com/openssl/openssl/blob/OpenSSL_1_0_1-stable/Configure#L584
+But iojs has used RC4 asm on MacOS for long time. Fix type of RC4_INT
+into `unsigned int` in opensslconf.h of darwin64-x86_64-cc to work on
+the RC4 asm.
+
+### 5. Update openssl.gyp and openssl.gypi
+Sources, cflags and define parameters that depends on each target can
+be obtained via `Configure TABLE`. Its list is put in the table of
+[define and cflags changes in openssl-1.0.2a](openssl_define_list.pdf)
+
+There is no way to verify all necessary sources automatically. We can
+only carefully look at the source list and compiled objects in
+Makefile of openssl and compare the compiled objects that stored
+stored under `out/Release/obj.target/openssl/deps/openssl/' in iojs.
+
+### 6. ASM files for openssl
+We provide two sets of asm files. One is for the latest asm compiler
+and the other is the older one.
+
+### 6.1. asm files for the latest compiler
+This was made in `deps/openssl/asm/Makefile`
+- Updated asm files for each platforms which are required in
+  openssl-1.0.2a.
+- Some perl files need CC and ASM envs. Added a check if these envs
+  exist. Followed asm files are to be generated with CC=gcc and
+  ASM=nasm on Linux. See
+  `deps/openssl/openssl/crypto/sha/asm/sha512-x86_64.pl`
+- Added new 32bit targets/rules with a sse2 flag (OPENSSL_IA32_SSE2)
+  to generate asm for use SSE2.
+- Generating sha512 asm files in x86_64 need output filename which
+  has 512. Added new rules so as not to use stdout for outputs.
+- PERLASM_SCHEME of linux-armv4 is `void` as defined in openssl
+  Configure. Changed its target/rule and all directories are moved
+  from arm-elf-gas to arm-void-gas.
+- add a new rule for armv8 asm generation
+
+With export environments of CC=gcc and ASM=nasm, then type make
+command and check if new asm files are generated.
+
+### 6.2.asm files for the older compiler
+For older asm compiler, the version check of CC and ASM should be
+skipped in generating asm file with perl scripts.
+Copy files from `deps/openssl/asm` into
+`deps/openssl/asm/asm_obsolete` and change rules to generate asm files
+into this directories and remove the check of CC and ASM envs.
+
+Without environments of CC and ASM, then type make command and check
+if new asm files for older compilers are generated.