Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Missing Features #9

Open
5 tasks
WebReflection opened this issue Jan 28, 2025 · 1 comment
Open
5 tasks

Missing Features #9

WebReflection opened this issue Jan 28, 2025 · 1 comment

Comments

@WebReflection
Copy link
Owner

WebReflection commented Jan 28, 2025

This is a follow up for #2 as that issue is before most changes landed already.

  • recursion should be based on bitwise operations while encoding and likely be in or out while decoding ... there are too many things that could be optimized in a way or another while streaming, sending database rows, and what not ... the number story is way different now and storing those feels like unnecessary overhead for any Map operation. Once refactored, all, some, and none could still be used to hint previous behavior ... this time based on bitwise operations
  • for JS only interactions, the number to typed conversion might not be convenient. Because any typed conversion works though, there is no harm in letting users define their number conversion preference ... I think u8 (for length and short strings) + f64 would cover all cases + i64 for bigints cases. This should speed up most common JS only use cases, keeping compatibility with other PLs regardless
  • there is a toJSON like mechanism: const toBufferedClone = Symbol.for('buffered-clone') but I think there should be a fromBufferedClone logic too but the more I think about it, the more I believe that should be reserved only users passed a callback while decoding, and that should also work only for object and array types: nothing else as I see little point in invoking that callback for strings, numbers, or other values that didn't implement a toBufferedClone and didn't return an object or array within that utility.
  • streaming examples or an easy to use implementation out of this module would be awesome! While encoding, streaming should "push" data to the underlying writer while decoding should receive data types and values as it goes ... objects, arrays, maps, sets or similar will receive key/value pairs or just values as soon as these are parsed
  • an AssemblyScript that compiles to WASM should be relatively straight forward to implement because the code base has zero closures around + it's fully typed (beside a couple of ignores that could be improved, I was just bored there)
@WebReflection
Copy link
Owner Author

WebReflection commented Feb 22, 2025

in retrospective, I should've known better: buffered clone had tons of things already explored, made standard and done right, through MessagePack but I wasn't aware of it ... buffered-clone + "extension-pack" feels like a much better project than one that reinvents the wheel

Accordingly, before even tackling above concerns I'd like to re-think this entirely and follow as close as possible MessagePack specs with the only exception that recursion will be backed in by default, options to recurse on strings too made available, API similar to that one except the register will simply be a Map as that's basically what that register is.

On top of that, using a DataView to decode the whole thing feels just right so I want to use somehow the same DataView approach to encode too, except it has to work with SharedArrayBuffer and growable / pre-allocated buffers, although the latter one feels like an awkward use case due "guestimate" aroud how much data could result into final packed outcome.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant