Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add return information for write #275

Open
lionelmazeyrat opened this issue Jan 24, 2017 · 8 comments
Open

Add return information for write #275

lionelmazeyrat opened this issue Jan 24, 2017 · 8 comments

Comments

@lionelmazeyrat
Copy link

When we use a write operation like : influxDB.write(batchPoints);
It will be very usefull to have a return object with the list of status (good/bad) for each write points operation.

@skovzhaw
Copy link

Must have feature for us too...

@majst01
Copy link
Collaborator

majst01 commented Jan 31, 2017

I am happy to review PR's

@gcyu
Copy link

gcyu commented Apr 26, 2017

So why influxdb's java client doesn't provide the function of returning a status code for influxDB.write(batchPoints)? And in current cases, how does client know whether the batchPoints is successfully written or not?

@majst01
Copy link
Collaborator

majst01 commented Apr 29, 2017

What should the status code look like ?

  • Number of Points written ?

To have this information the write method must wait for all batchpoints to be written and will therefore be synchronous which is a disaster from performance perspective.

Again if someone is willing to provide a PR im happy to review.

@lionelmazeyrat
Copy link
Author

To avoid to reduce write performance, perhaps the write method could implement an interface to be asynchronously notified of the write completion with a list of status for each points ?

@gcyu
Copy link

gcyu commented May 4, 2017

i think the the status code should be provided in write(batchpoints) method, such as whether the batch was successfully written or not instead of "Number of Points written". Thus the client could decide if it needs to retry/rewrite this batch or not. Note that InfluxDB's ability to write points by batch is strong(See Influxdb - Batch), so it won't be considered as a disaster in synchronous write pattern.

From higher view, this influxdb-java API provides two ways of points writing: write(batchpoints) and enablebatch, but they didn't show significant difference. It will be better that write(batchpoints) should be designed as a synchronous method for those clients who need to ensure data accuracy and enablebatch should be designed for those who ask for high write performance with tolerance of data loss. I think this API will look much better in this perspective.

Just some personal idea :-)

@fmachado
Copy link
Contributor

fmachado commented May 4, 2017

Just for reference

HBase API allows you to batch async actions[1] and, by using an array passed as parameter to the batch call, get or inspect the result of every async call. As you can see here[2], every async call may result with:

  • a null value (if the action is not supposed to return anything);
  • an instance of Throwable;
  • an instance of org.apache.hadoop.hbase.client.AsyncProcess.ReplicaResultState (if the async call is still in progress);
  • a primitive wrapper class or String (e.g. the value stored in HBase).

[1] - https://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/Table.html#batch-java.util.List-java.lang.Object:A-
[2] - https://github.com/apache/hbase/blob/master/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncRequestFutureImpl.java#L270

@majst01
Copy link
Collaborator

majst01 commented May 5, 2017

Please look at #318, i think this simple approach is usable for most users and keep this code free of a lot of logic.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

5 participants