3.0.0 Network Refactoring (espressif#8760)

* Create ESP_NetworkInterface class and have Ethernet extending it

* Update CMakeLists.txt

* Split networking from WiFi (H2 can now use Ethernet)

Now all libs have been checked yet. More to do on WiFi side

* Fix build errors

* Guard WiFi classes and fix RMII ETH examples

* Decouple network related libraries from WiFi

* Fix examples and WiFiUpdate

* Guard WiFiProv lib to compile only on WiFi chips

* Add periman string for network and "fix" mdns on the first ETH

* Revert back location of Client/Server/Udp in order to accept some PRs

* Fix periman

* Some fixes from merging master

* Fix web server missing fs.h

* Move Client, Server and Udp out of WiFi

* More fixes

* more fixes

* Fix CMakekLists and rework lib menu dependencies

* Fix CMake issues

* move back WiFiClient to rebase with master

* Update ETH_TLK110.ino

* Move back WiFiClient

* Update progress

* Update WiFiGeneric.cpp

* More fixes

* Switch AP to the new interface

* Cleanup

* Rename AP methods

* Add extra interface info for Printable

* Rename IPv6 getters to clarify that they are returning LinkLocal address

cc @sgryphon

* Rename network classes

cc @sgryphon

* Update NetworkManager.h

* Rename WiFi Server and UDP

* Rename WiFiClient and WiFiClientSecure

* Update CMakeLists.txt

* Update on-push.sh

* Rename Network library

* Remove unnecessary guard

* Get the correct interface MAC address for mDND Workstation service

* Apply suggestions from code review



---------

Co-authored-by: Me No Dev <[email protected]>
Co-authored-by: Lucas Saavedra Vaz <[email protected]>

Author: 3 people

libraries/NetworkClientSecure/README.md

@@ -0,0 +1,133 @@
+NetworkClientSecure
+================
+
+The NetworkClientSecure class implements support for secure connections using TLS (SSL).
+It inherits from NetworkClient and thus implements a superset of that class' interface.
+There are three ways to establish a secure connection using the NetworkClientSecure class:
+using a root certificate authority (CA) cert, using a root CA cert plus a client cert and key,
+and using a pre-shared key (PSK).
+
+Using a root certificate authority cert
+---------------------------------------
+This method authenticates the server and negotiates an encrypted connection.
+It is the same functionality as implemented in your web browser when you connect to HTTPS sites.
+
+If you are accessing your own server:
+- Generate a root certificate for your own certificate authority
+- Generate a cert & private key using your root certificate ("self-signed cert") for your server
+
+If you are accessing a public server:
+- Obtain the cert of the public CA that signed that server's cert
+Then:
+- In NetworkClientSecure use setCACert (or the appropriate connect method) to set the root cert of your
+  CA or of the public CA
+- When NetworkClientSecure connects to the target server it uses the CA cert to verify the certificate
+  presented by the server, and then negotiates encryption for the connection
+
+Please see the NetworkClientSecure example.
+
+Using a bundle of root certificate authority certificates 
+---------------------------------------------------------
+This method is similar to the single root certificate verfication above, but it uses a standard set of 
+root certificates from Mozilla to authenticate against, while the previous method only accepts a single 
+certificate for a given server. This allows the client to connect to all public SSL servers.
+
+To use this feature in PlatformIO:
+1. create a certificate bundle as described in the document below, or obtain a pre-built one you trust:
+https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-reference/protocols/esp_crt_bundle.html 
+(gen_crt_bundle.py can be found in the /tools folder)
+   a. note: the full bundle will take up around 64k of flash space, but has minimal RAM usage, as only
+      the index of the certificates is kept in RAM
+2. Place the bundle under the file name "data/cert/x509_crt_bundle.bin" in your platformio project
+3. add "board_build.embed_files = data/cert/x509_crt_bundle.bin" in your platformio.ini
+4. add the following global declaration in your project:
+   extern const uint8_t rootca_crt_bundle_start[] asm("_binary_data_cert_x509_crt_bundle_bin_start");
+5. before initiating the first SSL connection, call
+   my_client.setCACertBundle(rootca_crt_bundle_start);
+
+To use this feature in Arduino IDE:
+If the Arduino IDE added support for embedding files in the meantime, then follow the instructions above.
+If not, you have three choices:
+1. convert your project to PlatformIO
+2. create a makefile where you can add the idf_component_register() declaration to include the certificate bundle
+3. Store the bundle as a SPIFFS file, but then you have to load it into RAM in runtime and waste 64k of precious memory
+
+Using a root CA cert and client cert/keys
+-----------------------------------------
+This method authenticates the server and additionally also authenticates
+the client to the server, then negotiates an encrypted connection.
+
+- Follow steps above
+- Using your root CA generate cert/key for your client
+- Register the keys with the server you will be accessing so the server can authenticate your client
+- In NetworkClientSecure use setCACert (or the appropriate connect method) to set the root cert of your
+  CA or of the public CA, this is used to authenticate the server
+- In NetworkClientSecure use setCertificate, and setPrivateKey (or the appropriate connect method) to
+  set your client's cert & key, this will be used to authenticate your client to the server
+- When NetworkClientSecure connects to the target server it uses the CA cert to verify the certificate
+  presented by the server, it will use the cert/key to authenticate your client to the server, and
+  it will then negotiate encryption for the connection
+
+Using Pre-Shared Keys (PSK)
+---------------------------
+
+TLS supports authentication and encryption using a pre-shared key (i.e. a key that both client and
+server know) as an alternative to the public key cryptography commonly used on the web for HTTPS.
+PSK is starting to be used for MQTT, e.g. in mosquitto, to simplify the set-up and avoid having to
+go through the whole CA, cert, and private key process.
+
+A pre-shared key is a binary string of up to 32 bytes and is commonly represented in hex form. In
+addition to the key, clients can also present an id and typically the server allows a different key
+to be associated with each client id. In effect this is very similar to username and password pairs,
+except that unlike a password the key is not directly transmitted to the server, thus a connection to a
+malicious server does not divulge the password. Plus the server is also authenticated to the client.
+
+To use PSK:
+- Generate a random hex string (generating an MD5 or SHA for some file is one way to do this)
+- Come up with a string id for your client and configure your server to accept the id/key pair
+- In NetworkClientSecure use setPreSharedKey (or the appropriate connect method) to
+  set the id/key combo
+- When NetworkClientSecure connects to the target server it uses the id/key combo to authenticate the
+  server (it must prove that it has the key too), authenticate the client and then negotiate
+  encryption for the connection
+
+Please see the NetworkClientPSK example.
+
+Specifying the ALPN Protocol
+----------------------------
+
+Application-Layer Protocol Negotiation (ALPN) is a Transport Layer Security (TLS) extension that allows 
+the application layer to negotiate which protocol should be performed over a secure connection in a manner 
+that avoids additional round trips and which is independent of the application-layer protocols.
+
+For example, this is used with AWS IoT Custom Authorizers where an MQTT client must set the ALPN protocol to ```mqtt```: 
+
+```
+const char *aws_protos[] = {"mqtt", NULL};
+...
+wiFiClient.setAlpnProtocols(aws_protos);
+```
+
+Examples
+--------
+#### NetworkClientInsecure
+Demonstrates usage of insecure connection using `NetworkClientSecure::setInsecure()`
+#### NetworkClientPSK
+Wifi secure connection example for ESP32 using a pre-shared key (PSK)
+This is useful with MQTT servers instead of using a self-signed cert, tested with mosquitto.
+Running on TLS 1.2 using mbedTLS
+#### NetworkClientSecure
+Wifi secure connection example for ESP32
+Running on TLS 1.2 using mbedTLS
+#### NetworkClientSecureEnterprise
+This example demonstrates a secure connection to a WiFi network using WPA/WPA2 Enterprise (for example eduroam),
+and establishing a secure HTTPS connection with an external server (for example arduino.php5.sk) using the defined anonymous identity, user identity, and password.
+
+.. note::
+  This example is outdated and might not work. For more examples see [https://github.com/martinius96/ESP32-eduroam](https://github.com/martinius96/ESP32-eduroam)
+
+#### NetworkClientShowPeerCredentials
+Example of a establishing a secure connection and then showing the fingerprint of the certificate.
+This can be useful in an IoT setting to know for sure that you are connecting to the right server.
+Especially in situations where you cannot hardcode a trusted root certificate for long
+periods of time (as they tend to get replaced more often than the lifecycle of IoT hardware).

libraries/NetworkClientSecure/examples/WiFiClientInsecure/.skip.esp32h2

@@ -0,0 +1,70 @@
+#include <NetworkClientSecure.h>
+#include <WiFi.h>
+
+/* This is a very INSECURE approach.
+ * If for some reason the secure, proper example NetworkClientSecure
+ * does not work for you; then you may want to check the 
+ * NetworkClientTrustOnFirstUse example first. It is less secure than 
+ * NetworkClientSecure, but a lot better than this totally insecure
+ * approach shown below.
+ */
+
+const char* ssid     = "your-ssid";     // your network SSID (name of wifi network)
+const char* password = "your-password"; // your network password
+
+const char*  server = "www.howsmyssl.com";  // Server URL
+
+NetworkClientSecure client;
+
+void setup() {
+  //Initialize serial and wait for port to open:
+  Serial.begin(115200);
+  delay(100);
+
+  Serial.print("Attempting to connect to SSID: ");
+  Serial.println(ssid);
+  WiFi.begin(ssid, password);
+
+  // attempt to connect to Wifi network:
+  while (WiFi.status() != WL_CONNECTED) {
+    Serial.print(".");
+    // wait 1 second for re-trying
+    delay(1000);
+  }
+
+  Serial.print("Connected to ");
+  Serial.println(ssid);
+
+  Serial.println("\nStarting connection to server...");
+  client.setInsecure();//skip verification
+  if (!client.connect(server, 443))
+    Serial.println("Connection failed!");
+  else {
+    Serial.println("Connected to server!");
+    // Make a HTTP request:
+    client.println("GET https://www.howsmyssl.com/a/check HTTP/1.0");
+    client.println("Host: www.howsmyssl.com");
+    client.println("Connection: close");
+    client.println();
+
+    while (client.connected()) {
+      String line = client.readStringUntil('\n');
+      if (line == "\r") {
+        Serial.println("headers received");
+        break;
+      }
+    }
+    // if there are incoming bytes available
+    // from the server, read them and print them:
+    while (client.available()) {
+      char c = client.read();
+      Serial.write(c);
+    }
+
+    client.stop();
+  }
+}
+
+void loop() {
+  // do nothing
+}

libraries/NetworkClientSecure/examples/WiFiClientInsecure/WiFiClientInsecure.ino

@@ -0,0 +1,86 @@
+/*
+  Wifi secure connection example for ESP32 using a pre-shared key (PSK)
+  This is useful with MQTT servers instead of using a self-signed cert, tested with mosquitto.
+  Running on TLS 1.2 using mbedTLS
+
+  To test run a test server using: openssl s_server -accept 8443 -psk 1a2b3c4d -nocert
+  It will show the http request made, but there's no easy way to send a reply back...
+
+  2017 - Evandro Copercini - Apache 2.0 License.
+  2018 - Adapted for PSK by Thorsten von Eicken
+*/
+
+#include <NetworkClientSecure.h>
+#include <WiFi.h>
+
+#if 0
+const char* ssid     = "your-ssid";     // your network SSID (name of wifi network)
+const char* password = "your-password"; // your network password
+#else
+const char* ssid     = "test";     // your network SSID (name of wifi network)
+const char* password = "securetest"; // your network password
+#endif
+
+//const char*  server = "server.local";  // Server hostname
+const IPAddress server = IPAddress(192, 168, 0, 14);  // Server IP address
+const int    port = 8443; // server's port (8883 for MQTT)
+
+const char*  pskIdent = "Client_identity"; // PSK identity (sometimes called key hint)
+const char*  psKey = "1a2b3c4d"; // PSK Key (must be hex string without 0x)
+
+NetworkClientSecure client;
+
+void setup() {
+  //Initialize serial and wait for port to open:
+  Serial.begin(115200);
+  delay(100);
+
+  Serial.print("Attempting to connect to SSID: ");
+  Serial.println(ssid);
+  WiFi.begin(ssid, password);
+
+  // attempt to connect to Wifi network:
+  while (WiFi.status() != WL_CONNECTED) {
+    Serial.print(".");
+    // wait 1 second for re-trying
+    delay(1000);
+  }
+
+  Serial.print("Connected to ");
+  Serial.println(ssid);
+
+  client.setPreSharedKey(pskIdent, psKey);
+
+  Serial.println("\nStarting connection to server...");
+  if (!client.connect(server, port))
+    Serial.println("Connection failed!");
+  else {
+    Serial.println("Connected to server!");
+    // Make a HTTP request:
+    client.println("GET /a/check HTTP/1.0");
+    client.print("Host: ");
+    client.println(server);
+    client.println("Connection: close");
+    client.println();
+
+    while (client.connected()) {
+      String line = client.readStringUntil('\n');
+      if (line == "\r") {
+        Serial.println("headers received");
+        break;
+      }
+    }
+    // if there are incoming bytes available
+    // from the server, read them and print them:
+    while (client.available()) {
+      char c = client.read();
+      Serial.write(c);
+    }
+
+    client.stop();
+  }
+}
+
+void loop() {
+  // do nothing
+}