Bump spark version to 2.4.0

[dieu]

Hey,

to support algerbird-spark #669 module on scala 2.12.x, we upgrade spark dependency to 2.4.0.

also, we need to disable -opt:l:inline and -opt-inline-from:com.twitter.algebird.** to workround issue scala/bug#11247.

also, upgrade sbt script as the side effect.

[dieu]

tests are passed, but mimaReport is falling:

[error] sbt.librarymanagement.ResolveException: unresolved dependency: com.twitter#algebird-core_2.11;0.13.5-SNAPSHOT: not found
[error] unresolved dependency: com.twitter#algebird-test_2.11;0.13.5-SNAPSHOT: not found
[error] 	at sbt.internal.librarymanagement.IvyActions$.resolveAndRetrieve(IvyActions.scala:331)
[error] 	at sbt.internal.librarymanagement.IvyActions$.$anonfun$updateEither$1(IvyActions.scala:205)
[error] 	at sbt.internal.librarymanagement.IvySbt$Module.$anonfun$withModule$1(Ivy.scala:229)
[error] 	at sbt.internal.librarymanagement.IvySbt.$anonfun$withIvy$1(Ivy.scala:190)
[error] 	at sbt.internal.librarymanagement.IvySbt.sbt$internal$librarymanagement$IvySbt$$action$1(Ivy.scala:70)
[error] 	at sbt.internal.librarymanagement.IvySbt$$anon$3.call(Ivy.scala:77)
[error] 	at xsbt.boot.Locks$GlobalLock.withChannel$1(Locks.scala:93)
[error] 	at xsbt.boot.Locks$GlobalLock.xsbt$boot$Locks$GlobalLock$$withChannelRetries$1(Locks.scala:78)
[error] 	at xsbt.boot.Locks$GlobalLock$$anonfun$withFileLock$1.apply(Locks.scala:97)
[error] 	at xsbt.boot.Using$.withResource(Using.scala:10)
[error] 	at xsbt.boot.Using$.apply(Using.scala:9)
[error] 	at xsbt.boot.Locks$GlobalLock.ignoringDeadlockAvoided(Locks.scala:58)
[�[0merror] 	at xsbt.boot.Locks$GlobalLock.withLock(Locks.scala:48)
[error] 	at xsbt.boot.Locks$.apply0(Locks.scala:31)
[error] 	at xsbt.boot.Locks$.apply(Locks.scala:28)
[error] 	at sbt.internal.librarymanagement.IvySbt.withDefaultLogger(Ivy.scala:77)
[error] 	at sbt.internal.librarymanagement.IvySbt.withIvy(Ivy.scala:185)
[error] 	at sbt.internal.librarymanagement.IvySbt.withIvy(Ivy.scala:182)
[error] 	at sbt.internal.librarymanagement.IvySbt$Module.withModule(Ivy.scala:228)
[error] 	at sbt.internal.librarymanagement.IvyActions$.updateEither(IvyActions.scala:190)
[error] 	at sbt.librarymanagement.ivy.IvyDependencyResolution.update(IvyDependencyResolution.scala:20)
[error] 	at sbt.librarymanagement.DependencyResolution.update(DependencyResolution.scala:56)
[error] 	at sbt.internal.LibraryManagement$.resolve$1(LibraryManagement.scala:38)
[error] 	at sbt.internal.LibraryManagement$.$anonfun$cachedUpdate$12(LibraryManagement.scala:91)
[error] 	at sbt.util.Tracked$.$anonfun$lastOutput$1(Tracked.scala:68)
[error] 	at sbt.internal.LibraryManagement$.$anonfun$cachedUpdate$19(LibraryManagement.scala:104)
[error] 	at scala.util.control.Exception$Catch.apply(Exception.scala:224)
[error] 	at sbt.internal.LibraryManagement$.$anonfun$cachedUpdate$11(LibraryManagement.scala:104)
[error] 	at sbt.internal.LibraryManagement$.$anonfun$cachedUpdate$11$adapted(LibraryManagement.scala:87)
[error] 	at sbt.util.Tracked$.$anonfun$inputChanged$1(Tracked.scala:149)
[error] 	at sbt.internal.LibraryManagement$.cachedUpdate(LibraryManagement.scala:118)
[error] 	at sbt.Classpaths$.$anonfun$updateTask$5(Defaults.scala:2353)
[error] 	at scala.Function1.$anonfun$compose$1(Function1.scala:44)
[error] 	at sbt.internal.util.$tilde$greater.$anonfun$$u2219$1(TypeFunctions.scala:42)
[error] 	at sbt.std.Transform$$anon$4.work(System.scala:64)
[error] 	at sbt.Execute.$anonfun$submit$2(Execute.scala:257)
[error] 	at sbt.internal.util.ErrorHandling$.wideConvert(ErrorHandling.scala:16)
[error] 	at sbt.Execute.work(Execute.scala:266)
[error] 	at sbt.Execute.$anonfun$submit$1(Execute.scala:257)
[error] 	at sbt.ConcurrentRestrictions$$anon$4.$anonfun$submitValid$1(ConcurrentRestrictions.scala:167)
[error] 	at sbt.CompletionService$$anon$2.call(CompletionService.scala:32)
[error] 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[error] 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[error] 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[error] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[error] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[error] 	at java.lang.Thread.run(Thread.java:748)
[error] (algebird-spark/*:update) sbt.librarymanagement.ResolveException: unresolved dependency: com.twitter#algebird-core_2.11;0.13.5-SNAPSHOT: not found
[error] unresolved dependency: com.twitter#algebird-test_2.11;0.13.5-SNAPSHOT: not found

looks like is not related to this PR.

[dieu]

@johnynek still falling with same error:

[error] sbt.librarymanagement.ResolveException: unresolved dependency: com.twitter#algebird-core_2.11;0.13.5-SNAPSHOT: not found
[error] unresolved dependency: com.twitter#algebird-test_2.11;0.13.5-SNAPSHOT: not found

not sure, why it is looking for snapshot artifact.

[error] sbt.librarymanagement.ResolveException: unresolved dependency: com.twitter#algebird-spark_2.12;0.13.4: not found

we defently didn't release them for scala 2.12

[error] sbt.librarymanagement.ResolveException: unresolved dependency: com.twitter#algebird-spark_2.11;0.13.4: not found

did we release 0.13.4 version of algebird-spark?

[johnynek]

we did not release algebird-spark for 2.12 because there was no spark for 2.12 until I think today.

[dieu]

we did not release algebird-spark for 2.12 because there was no spark for 2.12 until I think today.

yes, that's error we can ignore, what's do you think about others?

[johnynek]

I think adding spark here: https://github.com/twitter/algebird/pull/670/files#diff-fdc3abdfd754eeb24090dbd90aeec2ceR159 then remove it after we publish the next version is probably the best fix as long as we are passing mima on 2.11.

[johnynek]

actually, maybe I didn't deploy at all because I couldn't figure out the sbt tricks since 2.12 wasn't supported, I may have just commented it out due to frustration.

[dieu]

actually, maybe I didn't deploy at all because I couldn't figure out the sbt tricks since 2.12 wasn't supported, I may have just commented it out due to frustration.

we actually hit sbt/sbt#4181 :) added workaround.