Correct format gG precision handling #91
Open
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The current implementation does not adhere to the Python documentation and forward the precision directly to
__pfmt_f()
and__pfmt_e()
, this means the only time it is correct is for numbers in the range of0.1 <= x < 1.0
.Additionally the Python spec makes sure the printed number will never have trailing zeros by switching to scientific notation when number of integer digits exceeds precision. The current implementation simply hard code it to
x < 1E6
.The exact wording is as follows
Since I am new here I don't know if taking log directly will affect performance, and if floating point imprecision will make it 1 less than the actual exponent, but it currently has the correct behavior.
This will resolve #88