WIP: Further improving fast date parsing #19519

omus · 2016-12-07T03:02:28Z

Iterating on #18000 and #19545. Speed up is currently 140x.

before:

julia> @benchmark DateTime("2010-10-10T10:10:10.10")
BenchmarkTools.Trial: 
  memory estimate:  3.11 kb
  allocs estimate:  85
  --------------
  minimum time:     16.768 μs (0.00% GC)
  median time:      17.943 μs (0.00% GC)
  mean time:        20.198 μs (1.78% GC)
  maximum time:     3.734 ms (96.53% GC)
  --------------
  samples:          10000
  evals/sample:     1
  time tolerance:   5.00%
  memory tolerance: 1.00%

Shashi's update #19545

julia> @benchmark DateTime("2010-10-10T10:10:10.10")
BenchmarkTools.Trial: 
  memory estimate:  0.00 bytes
  allocs estimate:  0
  --------------
  minimum time:     138.344 ns (0.00% GC)
  median time:      140.283 ns (0.00% GC)
  mean time:        157.004 ns (0.00% GC)
  maximum time:     1.683 μs (0.00% GC)
  --------------
  samples:          10000
  evals/sample:     861
  time tolerance:   5.00%
  memory tolerance: 1.00%

This PR:

julia> @benchmark DateTime("2010-10-10T10:10:10.10")
BenchmarkTools.Trial: 
  memory estimate:  0.00 bytes
  allocs estimate:  0
  --------------
  minimum time:     118.559 ns (0.00% GC)
  median time:      125.402 ns (0.00% GC)
  mean time:        138.223 ns (0.00% GC)
  maximum time:     2.867 μs (0.00% GC)
  --------------
  samples:          10000
  evals/sample:     907
  time tolerance:   5.00%
  memory tolerance: 1.00%

fixes #15888, #13644
supersedes #18000, #19545

omus · 2016-12-07T03:05:03Z

I'm also experimenting with some ideas to apply this performance gain to all date formats. Feel free to merge in the mean time.

simonbyrne · 2016-12-07T12:26:01Z

@shashi?

omus · 2016-12-14T23:06:28Z

Modified direction of the PR to work off of #19545. Mostly I am cleaning up the code without losing the speedup. I will probably also address #16928 here and may also include some custom code for specific formats.

shashi

@omus were you able to accomodate the changes required for TimeZones? I see that things still deal with NTupl{7,Int}s...

I have a new commit, should try and merge it here.

shashi · 2016-12-15T03:08:28Z

base/dates/io-util.jl

-    j = 1
-    while (maxchars <= 0 || j <= maxchars) && i <= len
+    max_pos = maxchars <= 0 ? len : min(chr2ind(str, ind2chr(str,i) + maxchars - 1), len)
+    while i <= max_pos
        c, ii = next(str, i)


you can actually also put @inbounds on this line and pretty much everywhere next is called to squeeze out a few more nanoseconds!

Ah I see you have it for parsing numbers

shashi · 2016-12-15T03:36:59Z

I think this can be done by something like adding a parameter to DatePart{char, T} where T is the output type, in this case the type of the result in _tryparse can still be determined... You might have other ideas.

omus · 2016-12-15T14:15:43Z

@shashi I haven't yet dealt with parsing support for TimeZones. I have some ideas including what you suggested but I'll need to do more experimentation to find out what works best.

shashi · 2016-12-15T05:52:34Z

base/dates/io-util.jl


        @label error
-        raise && _raiseexception(t, err_idx, state)
+        raise && throw(gen_exception(t, err_idx, state))
        return R()
    end
 end


I think we need a tryparsenext in Base... Something like CSV.jl could use it.

And this tryparse_internal should become tryparsenext and return the state as the second argument.

But of course that's a project of its own, changing all tryparses in Base to work that way.

shashi · 2016-12-15T08:59:24Z

base/dates/io-util.jl

-    j = 1
-    while (maxchars <= 0 || j <= maxchars) && i <= len
+    max_pos = maxchars <= 0 ? len : min(chr2ind(str, ind2chr(str,i) + maxchars - 1), len)
+    while i <= max_pos
        c, ii = next(str, i)


Ah I see you have it for parsing numbers

omus · 2016-12-16T18:12:18Z

Added fundamentals of adding custom slots to TimeType parsing (supports parsing of ZonedDateTime type). Changes don't impact performance of Date or DateTime

shashi · 2016-12-17T12:20:32Z

Nice! :)

Is ZonedDateTime parsing itself fast?

omus · 2016-12-17T18:44:58Z

ZonedDateTime parsing is about 10x faster. There are probably some additional things I can do outside of base to improve the performance more.

omus · 2016-12-17T18:46:05Z

I've got one more change I'd like to do which will be done on Monday.

@benchmark

```julia julia> s = "2010-10-11T12:13:14.15" "2010-10-11T12:13:14.15" julia> df_default = Base.Dates.DateFormat("yyyy-mm-ddTHH:MM:SS.s") dateformat"yyyy-mm-ddTHH:MM:SS.s" julia> df_custom = Base.Dates.DateFormat("yyyy-mm-dd\\THH:MM:SS.s") dateformat"yyyy-mm-ddTHH:MM:SS.s" julia> @benchmark parse(DateTime,$s,$df_default) BenchmarkTools.Trial: memory estimate: 0.00 bytes allocs estimate: 0 -------------- minimum time: 127.267 ns (0.00% GC) median time: 135.160 ns (0.00% GC) mean time: 147.785 ns (0.00% GC) maximum time: 2.450 μs (0.00% GC) -------------- samples: 10000 evals/sample: 886 time tolerance: 5.00% memory tolerance: 1.00% julia> @benchmark parse(DateTime,$s,$df_custom) BenchmarkTools.Trial: memory estimate: 0.00 bytes allocs estimate: 0 -------------- minimum time: 71.947 ns (0.00% GC) median time: 74.662 ns (0.00% GC) mean time: 82.548 ns (0.00% GC) maximum time: 2.529 μs (0.00% GC) -------------- samples: 10000 evals/sample: 976 time tolerance: 5.00% memory tolerance: 1.00% ```

omus · 2016-12-19T15:17:15Z

I've added support for custom parsers which now makes DateTime parsing using ISODateTimeFormat 230x faster.

julia> @benchmark DateTime("2010-10-10T10:10:10.10")
BenchmarkTools.Trial: 
  memory estimate:  0.00 bytes
  allocs estimate:  0
  --------------
  minimum time:     70.023 ns (0.00% GC)
  median time:      77.467 ns (0.00% GC)
  mean time:        109.659 ns (0.00% GC)
  maximum time:     9.145 μs (0.00% GC)
  --------------
  samples:          10000
  evals/sample:     976
  time tolerance:   5.00%
  memory tolerance: 1.00%

omus · 2016-12-19T15:18:37Z

@shashi feel free to merge in my changes again. Code should be ready for review.

tkelman · 2016-12-19T15:27:11Z

should we change the target branch of this pr?

omus · 2016-12-19T15:29:22Z

Sure. Do I need to remove the "fixes" / "supersedes" notes before merging?

tkelman · 2016-12-19T15:35:29Z

good question, don't think so - i believe github only auto closes things that merge to master

tkelman · 2016-12-19T15:42:57Z

test/dates/io.jl

@@ -66,11 +66,15 @@ b2 = "96/Feb/1"
 # Here we've specifed a text month name, but given a number
 b3 = "96/2/15"
 @test_throws ArgumentError Dates.DateTime(b3,f)
-let err = try DateTime("2012-02-20T09:09:3.43i9") catch err; err end
+try
+    Dates.tryparse_internal(DateTime, "2012-02-20T09:09:3.43i9", Dates.ISODateTimeFormat, true)


add a @test false here if the exception should happen all the time, or if it doesn't throw then this would pass silently

shashi · 2016-12-20T07:01:40Z

👍

kshyatt added performance Must go faster dates Dates, times, and the Dates stdlib module labels Dec 7, 2016

omus mentioned this pull request Dec 14, 2016

speed up date time parsing #19545

Merged

omus force-pushed the cv/dates-fast-parsing branch 2 times, most recently from 1072d8e to 91c5976 Compare December 14, 2016 22:57

omus changed the title ~~Further improving on #18000~~ WIP: Further improving fast date parsing Dec 14, 2016

shashi reviewed Dec 15, 2016

View reviewed changes

Refactoring some tests

301063c

omus added 3 commits December 19, 2016 09:00

Minor refactor to DateFormat constructor

74888e6

Update testcases

0818229

omus changed the base branch from master to sh/fast_date_parsing December 19, 2016 15:36

tkelman reviewed Dec 19, 2016

View reviewed changes

omus mentioned this pull request Dec 19, 2016

Add more DateTime parsing benchmarks JuliaCI/BaseBenchmarks.jl#50

Merged

Avoid passing test silently if no exception thrown

f7f2aee

shashi merged commit fdeef5d into JuliaLang:sh/fast_date_parsing Dec 20, 2016

omus deleted the cv/dates-fast-parsing branch December 20, 2016 14:49

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

WIP: Further improving fast date parsing #19519

WIP: Further improving fast date parsing #19519

omus commented Dec 7, 2016 •

edited

Loading

omus commented Dec 7, 2016

simonbyrne commented Dec 7, 2016

omus commented Dec 14, 2016

shashi left a comment

shashi Dec 15, 2016

shashi Dec 15, 2016

shashi commented Dec 15, 2016

omus commented Dec 15, 2016

shashi Dec 15, 2016

shashi Dec 15, 2016

shashi Dec 15, 2016

shashi Dec 15, 2016

omus commented Dec 16, 2016

shashi commented Dec 17, 2016

omus commented Dec 17, 2016

omus commented Dec 17, 2016

omus commented Dec 19, 2016

omus commented Dec 19, 2016

tkelman commented Dec 19, 2016

omus commented Dec 19, 2016

tkelman commented Dec 19, 2016

tkelman Dec 19, 2016

shashi commented Dec 20, 2016

WIP: Further improving fast date parsing #19519

WIP: Further improving fast date parsing #19519

Conversation

omus commented Dec 7, 2016 • edited Loading

omus commented Dec 7, 2016

simonbyrne commented Dec 7, 2016

omus commented Dec 14, 2016

shashi left a comment

Choose a reason for hiding this comment

shashi Dec 15, 2016

Choose a reason for hiding this comment

shashi Dec 15, 2016

Choose a reason for hiding this comment

shashi commented Dec 15, 2016

omus commented Dec 15, 2016

shashi Dec 15, 2016

Choose a reason for hiding this comment

shashi Dec 15, 2016

Choose a reason for hiding this comment

shashi Dec 15, 2016

Choose a reason for hiding this comment

shashi Dec 15, 2016

Choose a reason for hiding this comment

omus commented Dec 16, 2016

shashi commented Dec 17, 2016

omus commented Dec 17, 2016

omus commented Dec 17, 2016

omus commented Dec 19, 2016

omus commented Dec 19, 2016

tkelman commented Dec 19, 2016

omus commented Dec 19, 2016

tkelman commented Dec 19, 2016

tkelman Dec 19, 2016

Choose a reason for hiding this comment

shashi commented Dec 20, 2016

omus commented Dec 7, 2016 •

edited

Loading