Skip to content

Add Gantt chart using Mux and Luxor #169

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 17 commits into from
Apr 24, 2021
Merged

Add Gantt chart using Mux and Luxor #169

merged 17 commits into from
Apr 24, 2021
+831 −137

Conversation

jpsamaroo
Copy link
Member

@jpsamaroo jpsamaroo commented Nov 30, 2020 •
edited
Loading

Todo:

  • Add scheduler_init vertical red line
  • Get profiling working well
  • Require FFMPEG to allow offline video generation
  • "Bunch up" multiple proc lines
  • Docs
  • Try to test this

Sorry, something went wrong.

@jpsamaroo jpsamaroo added the ui label Nov 30, 2020
@jpsamaroo
Copy link
Member Author

Closes #159

@jpsamaroo jpsamaroo force-pushed the jps/gantt-ui branch 2 times, most recently from 884cac5 to 40bbe2d Compare January 16, 2021 23:49
@jpsamaroo jpsamaroo mentioned this pull request Jan 16, 2021
Closed
@jpsamaroo
Copy link
Member Author

I've merged in the benchmark changes here because as I developed them they became very intertwined, such that the benchmarking script now also generates the Gantt chart and Profile flamegraph offline by default and stores them with the results .jld for analysis.

@vchuravy
Copy link
Member

Looking at https://mathdugre.me/paper-big-data-engines/dask-delayed-inc-baseline.html as an example they use different colours to differentiate between "compute", "read", "write", but keep the the colours between tasks constant.

For us I think it would make sense to have "schedule", "communication", "computation", "checkpointing", as categories.
The example above comes from dask and they are using Bokeh, which allows them to attach meta information and display that on hover.

jpsamaroo added 13 commits April 24, 2021 11:46
@jpsamaroo
Add locking to LocalEventLog
f722bbd
@jpsamaroo
Pass kwargs in Context ctor
7cd1b5c
@jpsamaroo
Added Gantt chart with Mux and Luxor
f591253 
Add Profile rendering with ProfileSVG
Optionally render Gantt and Profile data offline
@jpsamaroo
Update benchmarking script
d9bf80b 
Judge on trial means
Use envvar to specify benchmarking project
Use envvar to specify nprocs:nthreads
Use envvar to connect to other servers for benchmarking
Render Gantt and Profile data to video files
Disable old array suite, add NNMF suite
Add serial comparison
Add benchmark plotting with UnicodePlots
@jpsamaroo
Add get_tls/set_tls! helpers
93e2212
@jpsamaroo
Add CUDA support to benchmarks
7d962e6
@jpsamaroo
Allow rendering to fail
3e453c7
@jpsamaroo
Improve benchmarking suite and UI
53ec1f4
@jpsamaroo
Add mapchunks
33397ab
@jpsamaroo
Report move errors in do_task
e215052
@jpsamaroo
Various changes/fixes
64724d1 
Fix benchmark specification
Add output format selection
@jpsamaroo
Fix CUDA benchmark init
96812f2
@jpsamaroo
Print more benchmarking info
f9e2c12
@jpsamaroo
Copy link
Member Author

I'm going to merge this without tests for right now, because this entire infrastructure is pretty experimental, and doesn't yet cover all the cases that the original benchmarking script used to cover (although this one actually works). I'd especially appreciate any artistic input (w.r.t Gantt chart, flamechart, and benchmark plots visualization) as issues/PRs going forward!

vchuravy and others added 3 commits April 24, 2021 12:29
@vchuravy @jpsamaroo
add analysis script for strong and weak scaling
d6c512f
@jpsamaroo
Merge pull request #190 from JuliaParallel/vc/vis

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Verified
Learn about vigilant mode
69b3564 
add analysis script for strong and weak scaling
@jpsamaroo
Bump to 0.11.2
5f72d51
@jpsamaroo jpsamaroo merged commit 9ce4de1 into master Apr 24, 2021
@jpsamaroo jpsamaroo deleted the jps/gantt-ui branch April 24, 2021 16:36
@jpsamaroo
Copy link
Member Author

Thanks @vchuravy for all the help with directing this work!

@jpsamaroo jpsamaroo mentioned this pull request Sep 9, 2021
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
ui
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@jpsamaroo @vchuravy