Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

bpo-36778: cp65001 encoding becomes an alias to utf_8 #13230

Merged
merged 1 commit into from
May 10, 2019
Merged

bpo-36778: cp65001 encoding becomes an alias to utf_8 #13230

merged 1 commit into from
May 10, 2019

Conversation

vstinner
Copy link
Member

@vstinner vstinner commented May 9, 2019 •
edited by bedevere-bot
Loading

@vstinner
Copy link
Member Author

vstinner commented May 9, 2019

I reproduced #13110 (comment) benchmark:

Mean +- std dev: [ref] 156 ns +- 3 ns -> [remove] 105 ns +- 3 ns: 1.48x faster (-32%)

@vstinner vstinner requested a review from methane May 9, 2019 23:12
@vstinner
Copy link
Member Author

vstinner commented May 9, 2019

@methane: Are you ok to simply remove cp65001.py?

@paulmon
Copy link
Contributor

paulmon commented May 10, 2019

I can verify that this change fixes the issue with test_startup_imports I found on Windows IoT Core ARM32 as expected.

Also, all test_site and test_codec tests pass on Windows IoT Core with a default codepage of 65001

methane
methane approved these changes May 10, 2019
View reviewed changes
Copy link
Member

@methane methane left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@methane: Are you ok to simply remove cp65001.py?

Yes.

@vstinner vstinner merged commit d267ac2 into python:master May 10, 2019
@vstinner vstinner deleted the remove_cp65001 branch May 10, 2019 01:19
This was referenced May 10, 2019
Closed
Closed
@paulmon paulmon mentioned this pull request May 10, 2019
Closed
serhiy-storchaka
serhiy-storchaka reviewed May 10, 2019
View reviewed changes
Doc/library/codecs.rst
@@ -1106,8 +1106,7 @@ particular, the following variants typically exist:
+-----------------+--------------------------------+--------------------------------+
| cp1258 | windows-1258 | Vietnamese |
+-----------------+--------------------------------+--------------------------------+
| cp65001 | | Windows only: Windows UTF-8 |
| | | (``CP_UTF8``) |
| cp65001 | | Alias to ``utf_8`` encoding |
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add the versionchanged directive.

I think it is better to remove this row add add cp65001 to the list of utf-8 aliases.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I created PR #13240 for your doc change proposal.

Lib/test/test_codecs.py
@@ -875,95 +875,6 @@ def test_surrogatepass_handler(self):
b"abc\xed\xa0z".decode(self.encoding, "surrogatepass")


@unittest.skipUnless(sys.platform == 'win32',
'cp65001 is a Windows-only codec')
class CP65001Test(ReadTest, unittest.TestCase):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does utf-8 pass these tests?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I confirmed utf-8 passes this test.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does utf-8 pass these tests?

Sorry, I forgot to specify before I merged my PR that yes: I tested on my Windows 10 and the test still passed. But CP65001Test is now redundant with UTF8Ttest.

I confirmed utf-8 passes this test.

Thanks for checking :-)

@taleinat
Copy link
Contributor

taleinat commented May 29, 2019 •
edited
Loading

Shouldn't this comment in CodePageTest in Lib/test/test_codecs.py also be removed? It no longer makes any sense.

class CodePageTest(unittest.TestCase):
    # CP_UTF8 is already tested by CP65001Test
    CP_UTF8 = 65001

@vstinner
Copy link
Member Author

vstinner commented Jun 4, 2019

Shouldn't this comment in CodePageTest in Lib/test/test_codecs.py also be removed? It no longer makes any sense.

Right. I wrote PR #13807 to remove the comment.

icanhasmath added a commit to ActiveState/cpython that referenced this pull request Aug 9, 2024
@icanhasmath
bpo-36778: cp65001 encoding becomes an alias to utf_8 (pythonGH-13230)
81eafa7
icanhasmath added a commit to ActiveState/cpython that referenced this pull request Aug 9, 2024
@icanhasmath
bpo-36778: cp65001 encoding becomes an alias to utf_8 (pythonGH-13230)
3efa80d
@icanhasmath icanhasmath mentioned this pull request Aug 12, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@methane methane methane approved these changes

@serhiy-storchaka serhiy-storchaka serhiy-storchaka left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
7 participants
@vstinner @paulmon @taleinat @methane @serhiy-storchaka @the-knights-who-say-ni @bedevere-bot