Request UTF-32 Character Encoding

[PsyTae]

• Version: v5.4.0
• Platform: Windows 7 Professional 64 bit
• Subsystem: buffer

Character encoding for buffer.

Why are utf-8 and utf-16 included for Buffer Encoding but UTF-32 is not? UTF-32 is just as prevalent and even more useful than the first two including more options for characters.

[Fishrock123]

@PsyTae IIRC it's because of what JavaScript/v8 supports.

cc @trevnorris

[mscdex]

I wouldn't say that UTF-32 is more useful than UTF-8, but no matter what you can always use the iconv or iconv-lite modules to do necessary conversions.

[jasnell]

This one may actually make sense to get added, although it's likely fairly non-trivial. I'm willing to take this on to investigate.

[trevnorris]

One-byte (i.e. 'binary', which is actually latin1), Two-Byte ('utf16-le') and Utf8 ('utf8') are all built in support. 'ascii' exists for legacy reasons, back when it was natively supported by v8. 'hex' and 'base64', I believe, both exist as requirements for other modules.

There have been requests in the past (see #2835) to support extending supported Buffer encodings, but doing this is tricky. For example, the call to setEncoding() won't work. Which means the user would still need to run chunk.toString(<encoding>) anyway. Also easily enough accomplished by just passing the chunk to a module function.

So I'd say iconv-lite is an excellent module to use for more than just the basic encodings provided by core.

[PsyTae]

That chunk.toString(<encoding>) is actually one of the features I am looking forward to the most for the utf-32 character set. that and taking a string and breaking it back out to a utf-32 buffer.

I also don't want to be dependent on Python to build the native code for iconv.

[mscdex]

@PsyTae You might submit an issue at iconv-lite requesting UTF-32 support and just use that module.

[jasnell]

Given the responses, going to close this as it's not likely to happen in core.