Update sketches #2965
Assignees
Labels
dependencies
Updates/changes to Maven or other dependencies
enhancement
Improvement to existing functionality/feature
Milestone
Comments
|
Once we decide how to update |
t92549
added a commit
that referenced
this issue
Jun 5, 2023
* Update sketches to apache datasketches * Checkstyle * Fix javadoc * Created historic serialisation tests for sketches * Spotless apply * Changed examples to use HllSketch and updated tests * Spotless * Improve JsonDeserialisation of HllSketch * Spotless and checkstyle * Deprecate usage of HyperLogLogPlus * Spotless apply * Review comments * Added javadoc * Javadoc fix * Corrected ReservoirItemsSketchSerialiser equals method
GCHQDeveloper314
added
the
dependencies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Describe the new feature you'd like
Building on from #2825, we should encourage not using the clearspring
HyperLogLogPlus. The associated aggregator and serialisers should be deprecated, and we should encourage users to use datasketches'HllSketchinstead. This should be done in code (as tracked by this ticket), as well as in our docs, tracked by gchq/gaffer-doc#93.An additional complexity is that the datasketches code has moved from
com.yahoo.datasketches:sketches-coretoorg.apache.datasketches:datasketches-javaand had many version updates. We should figure out if the new module is compatible with the old module and our aggregator/serialisers. If so I suggest we just update the dependency, if not we should provide another set of classes to interact with the new apache datasketches and deprecate the yahoo ones.Why do you want this feature?
As described in gchq/gaffer-doc#93, datasketches'
HllSketchis more performant. It is also actively maintained, whereas clearspring'sHyperLogLogPlusis not.The text was updated successfully, but these errors were encountered: