Replace MD5 with Blake2 (#1550)

Author: SharonHart

CHANGELOG.md

@@ -8,9 +8,9 @@ All notable changes to this project will be documented in this file.
 - Fixed: Updated URL regex pattern to correctly exclude trailing single (') and double (") quotes from matched URLs.
 
 ### Anonymizer
+- Changed: Deprecate `MD5` hash type option, defaulting into `sha256`. Added `blake2b` hash type instead, as a new option.
 
 ### Image Redactor
-
 - Changed: Updated the return type annotation of `ocr_bboxes` in `verify_dicom_instance()` from `dict` to `list`.  
 
 ### Presidio Structured

docs/anonymizer/index.md

@@ -236,16 +236,16 @@ of the AnalyzerEngine"
 
 ## Built-in operators
 
-| Operator type | Operator name | Description | Parameters |
-| --- | --- | --- | --- |
-| Anonymize | replace | Replace the PII with desired value | `new_value`: replaces existing text with the given value.<br> If `new_value` is not supplied or empty, default behavior will be: <entity_type\> e.g: <PHONE_NUMBER\> |
-| Anonymize | redact | Remove the PII completely from text | None |
-| Anonymize | hash | Hashes the PII text | `hash_type`: sets the type of hashing. Can be either `sha256`, `sha512` or `md5`. <br> The default hash type is `sha256`. |
-| Anonymize | mask | Replace the PII with a given character | `chars_to_mask`: the amount of characters out of the PII that should be replaced. <br> `masking_char`: the character to be replaced with. <br> `from_end`: Whether to mask the PII from it's end. |
-| Anonymize | encrypt | Encrypt the PII using a given key | `key`: a cryptographic key used for the encryption. |
-| Anonymize | custom | Replace the PII with the result of the function executed on the PII | `lambda`: lambda to execute on the PII data. The lambda return type must be a string. |
-| Anonymize | keep | Preserver the PII unmodified | None |
-| Deanonymize | decrypt | Decrypt the encrypted PII in the text using the encryption key | `key`: a cryptographic key used for the encryption is also used for the decryption. |
+| Operator type | Operator name | Description                                                         | Parameters                                                                                                                                                                                        |
+|---------------|---------------|---------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| Anonymize     | replace       | Replace the PII with desired value                                  | `new_value`: replaces existing text with the given value.<br> If `new_value` is not supplied or empty, default behavior will be: <entity_type\> e.g: <PHONE_NUMBER\>                              |
+| Anonymize     | redact        | Remove the PII completely from text                                 | None                                                                                                                                                                                              |
+| Anonymize     | hash          | Hashes the PII text                                                 | `hash_type`: sets the type of hashing. Can be either `sha256`, `sha512` or `blake2b`. <br> The default hash type is `sha256`.                                                                       |
+| Anonymize     | mask          | Replace the PII with a given character                              | `chars_to_mask`: the amount of characters out of the PII that should be replaced. <br> `masking_char`: the character to be replaced with. <br> `from_end`: Whether to mask the PII from it's end. |
+| Anonymize     | encrypt       | Encrypt the PII using a given key                                   | `key`: a cryptographic key used for the encryption.                                                                                                                                               |
+| Anonymize     | custom        | Replace the PII with the result of the function executed on the PII | `lambda`: lambda to execute on the PII data. The lambda return type must be a string.                                                                                                             |
+| Anonymize     | keep          | Preserver the PII unmodified                                        | None                                                                                                                                                                                              |
+| Deanonymize   | decrypt       | Decrypt the encrypted PII in the text using the encryption key      | `key`: a cryptographic key used for the encryption is also used for the decryption.                                                                                                               |
 
 !!! note "Note"
     When performing anonymization, if anonymizers map is empty or "DEFAULT" key is not stated, the default

docs/api-docs/api-docs.yml

@@ -666,11 +666,11 @@ components:
           type: string
           description: "The hashing algorithm"
           enum:
-            - md5
+            - blake2b
             - sha256
             - sha512
-          example: md5
-          default: md5
+          example: blake2b
+          default: blake2b
 
     Encrypt:
       title: Encrypt

presidio-anonymizer/README.md

@@ -30,10 +30,10 @@ Presidio anonymizer comes by default with the following anonymizers:
 
 -   **Redact**: Removes the PII completely from text.
     -   Parameters: None
--   **Hash**: Hashes the PII using either sha256, sha512 or md5. 
+-   **Hash**: Hashes the PII using either sha256, sha512 or blake2b. 
     -   Parameters:
         - `hash_type`: Sets the type of hashing. 
-          Can be either `sha256`, `sha512` or `md5`.
+          Can be either `sha256`, `sha512` or `blake2b` (`md5` is deprecated as of version 2.2.358).
           The default hash type is `sha256`.
 -   **Mask**: Replaces the PII with a sequence of a given character.
     -   Parameters:

presidio-anonymizer/presidio_anonymizer/operators/hash.py

@@ -1,19 +1,19 @@
 """Hashes the PII text entity."""
 
-from hashlib import md5, sha256, sha512
+from hashlib import blake2b, sha256, sha512
 from typing import Dict
 
 from presidio_anonymizer.operators import Operator, OperatorType
 from presidio_anonymizer.services.validators import validate_parameter_in_range
 
 
 class Hash(Operator):
-    """Hash given text with sha256/sha512/md5 algorithm."""
+    """Hash given text with sha256/sha512/blake2b algorithm."""
 
     HASH_TYPE = "hash_type"
     SHA256 = "sha256"
     SHA512 = "sha512"
-    MD5 = "md5"
+    BLAKE2B = "blake2b"
 
     def operate(self, text: str = None, params: Dict = None) -> str:
         """
@@ -25,14 +25,14 @@ def operate(self, text: str = None, params: Dict = None) -> str:
         hash_switcher = {
             self.SHA256: lambda s: sha256(s),
             self.SHA512: lambda s: sha512(s),
-            self.MD5: lambda s: md5(s),
+            self.BLAKE2B: lambda s: blake2b(s, digest_size=20),
         }
         return hash_switcher.get(hash_type)(text.encode()).hexdigest()
 
     def validate(self, params: Dict = None) -> None:
         """Validate the hash type is string and in range of allowed hash types."""
         validate_parameter_in_range(
-            [self.SHA256, self.SHA512, self.MD5],
+            [self.SHA256, self.SHA512, self.BLAKE2B],
             self._get_hash_type_or_default(params),
             self.HASH_TYPE,
             str,

presidio-anonymizer/tests/integration/test_anonymize_engine.py

@@ -158,13 +158,13 @@ def test_given_intersecting_the_same_entities_then_we_anonymize_correctly():
     # fmt: off
     "hash_type,result",
     [
-        ("md5",
-         '{"text": "hello world, my name is 1c272047233576d77a9b9a1acfdf741c. '
-         'My number is: e7706047f07bf68a5dd73e8c47db3a30", "items": [{"start": 72, '
-         '"end": 104, "entity_type": "PHONE_NUMBER", "text": '
-         '"e7706047f07bf68a5dd73e8c47db3a30", "operator": "hash"}, '
-         '{"start": 24, "end": 56, "entity_type": "NAME", "text": '
-         '"1c272047233576d77a9b9a1acfdf741c", "operator": "hash"}]}'),
+        ("blake2b",
+         '{"text": "hello world, my name is 9784349bcc3a48a6fe6e344c0701d31ee9ec1bd4. '
+         'My number is: 4c43f855362f7ab55108b2a83d50aae7cac3acec", "items": [{"start": 80, '
+         '"end": 120, "entity_type": "PHONE_NUMBER", "text": '
+         '"4c43f855362f7ab55108b2a83d50aae7cac3acec", "operator": "hash"}, '
+         '{"start": 24, "end": 64, "entity_type": "NAME", "text": '
+         '"9784349bcc3a48a6fe6e344c0701d31ee9ec1bd4", "operator": "hash"}]}'),
         ("sha256",
          '{"text": "hello world, my name is '
          '01332c876518a793b7c1b8dfaf6d4b404ff5db09b21c6627ca59710cc24f696a. '

presidio-anonymizer/tests/operators/test_hash.py

@@ -70,14 +70,14 @@ def test_when_given_valid_value_without_hash_type_then_expected_sha256_string_re
             "8500ce5af27e4db23f533e54c8c1ad74de62e93ca77c05e8de90e9eb27c7abe155"
             "e01d47868eded3106ccf6ac1f5c33bbaa95d55d40e9d89091c3d4617cc6d60",
         ),  # Sha512  Hash 'Unicode EmojiSources' character
-        ("123456", "md5", "e10adc3949ba59abbe56e057f20f883e"),  # MD5 Hash 123456
-        ("54321", "md5", "01cfcd4f6b8770febfb40cb906715822"),  # MD5 Hash 54321
+        ("123456", "blake2b", "a0f92ddfdea4892ff18a48f7e0f9fcffc55745f5"),  # blake2b Hash 123456
+        ("54321", "blake2b", "d0aaf44602cea8280a6bbdc62edc32762028183c"),  # blake2b Hash 54321
         (
             "😈😈😈😈",
-            "md5",
-            "5bf45eeaade2060ac6cdd532e1c35eef",
+            "blake2b",
+            "70d6bc6f072a94ddb04ed81a04d65f2a7304b021",
         ),
-        # MD5 Hash 'Unicode EmojiSources' character
+        # blake2b Hash 'Unicode EmojiSources' character
         # fmt: on
     ],
 )
@@ -99,7 +99,7 @@ def test_when_hash_type_not_in_range_then_ipe_raised():
     with pytest.raises(
         InvalidParamError,
         match="Parameter hash_type value not_a_hash is not in range of values"
-        " \\['sha256', 'sha512', 'md5'\\]",
+        " \\['sha256', 'sha512', 'blake2b'\\]",
     ):
         Hash().validate(params)
 
@@ -118,7 +118,7 @@ def test_when_hash_type_is_empty_string_then_ipe_raised():
     with pytest.raises(
         InvalidParamError,
         match="Parameter hash_type value  is not in range of values"
-        " \\['sha256', 'sha512', 'md5'\\]",
+        " \\['sha256', 'sha512', 'blake2b'\\]",
     ):
         Hash().validate(params)