Feature request: global step slider for images

[pkern90]

I believe it would be handy to have a global slider for visualizing images from different steps. At the moment, when I try to compare different image inputs and outputs, I have to slide each image individually.

[wchargin]

cc @chihuahua.

Could gauge support from Ian. If you implement this, you might ask RJ whether he'd want the same thing for audio.

[chihuahua]

Some internal folks on the image search team (among others) have also asked for a global slider. The challenge is that the steps might differ per image, although the steps are often the same within a run such as train or eval (unless a user for some reason decides to avoid merging summaries and only collect certain summaries at certain steps).

What if we tried this out. #471 should get us started. Maybe we can let the user select a global step for all the tags within each run, and if the user later changes the step for a specific image, that global slider is overriden (and the step it shows is grayed out).

FYI, @goodfeli and @rryan. And yeah, per @wchargin, this seems applicable to audio too.

[goodfeli]

SGTM

[wchargin]

@pkern90 @goodfeli Are you looking for a single slider that uniformly sets the step for all runs, or an assortment of all sliders one for each run (as opposed to one for each run–tag combination)?

[pkern90]

@wchargin for me personally a global slider would be more important. Having said that, I also believe that additional run based sliders which override the global slider could come in handy. I guess the problem would be the placement of the sliders, so it doesn't blow up the UI to much.

[chihuahua]

What's the use case for comparing images between runs? A run is like train or eval, right? Thanks!

[pkern90]

Different runs could be different models/parameters (experiments) for the same problem. Since not all models train with the same speed (number of steps) you might want to compare the outputs of different experiments from different steps.

[chihuahua]

OK, new proposal: Lets try out 1 global slider for all runs. The advantage is that we can conveniently specify 1 step cap for all runs. The disadvantage is that runs can differ in sampling rate of summaries as well as max step, so if we rely on 1 global slider alone, some steps might become inaccessible to certain runs.

Imagine a run A at step 10K and another run B at step 1K. TensorBoard samples for 10 events per run (or tag really, but assume all tags share the same steps here). With 1 global slider sampled at the rate of run A, we can access all sampled images of run A, but only 2 of run B.

Hence, how about we try 1 global slider but also maintain the individual slider per image. @wchargin, thoughts?

[pkern90]

One global slider for all runs sounds good to me.

[tachim]

Any updates here?

[nfelt]

@tachim we are stretched pretty thin right now so we can make no guarantees on when we might be able to work on this. I'll mark it contributions welcome though.

[tachim]

Maybe we can take a crack at this, what's the most similar PR we can look at? This one seems a bit sparse https://github.com/tensorflow/tensorboard/pull/471/files

[nfelt]

@chihuahua would probably know best. I think the starting point would likely be the PR curves dashboard which has per-run sliders (in #471) which could perhaps be adapted into a global slider for the images dashboard, and the logic that hooks those sliders up to the individual charts would look similar.

I think perhaps more important than the actual implementation is defining the behavior of the user controls, things like:

• how does the global slider interact w/ individual per-chart sliders? (one simple option is that moving the global slider resets all per-chart sliders to that step value, but then moving a per-chart slider overrides it)
• how should it work when individual image charts go off-screen and then return, e.g. via pagination or when a run for that chart is selected, then deselected? should the slider value persist, or return to some other value?
• what happens if the global slider is set to a value at which a given chart doesn't have a sample?
• should there be a way to reset to the current default behavior of "show the last step for each chart?" note that if different charts have different last steps, then with a global slider there's no way to express that directly, so it would either need to be a separate toggle/button or a special "last step" position on the global slider, or something similar

[chihuahua]

Yeah, I agree with @nfelt's listing of design decisions to be made. To elaborate a bit on the first point (the behaviors of chart sliders with regards to the global one), tags within a run (such as train or eval) often share the same steps because we often log, say, eval or train data at the same frequencies. However, steps may differ across runs. One might write train summaries every 5 steps, but eval summaries every 100 steps. Perhaps when the global slider changes, chart sliders could round down to the nearest step to the global one (and the UI could make that clear).

I unfortunately also can't find a precedent PR for introducing a global slider. Indeed, #471 (introducing one slider per run for PR curves) is probably most relevant.

Each image loader loads one image. Currently, the chart sliders alter this private _stepIndex property for each loader. The global slider would bind to that property (that would then be made public).

tensorboard/tensorboard/plugins/image/tf_image_dashboard/tf-image-loader.html

Line 235 in d46f60f

_stepIndex: {

[bhack]

Any progress here? It is one of the workday routine most useful feature.

[mateuszbuda]

As a workaround, I downloaded images for a specific step using wget:

for i in `seq 0 99`; do wget http://127.0.0.1:6006/data/plugin/images/individualImage?run=test\&tag=image/$i\&index=50 -O $i.png ; done

You just need to know run, tag, and step index.

[omoindrot]

Would be great to have this global slider !
Any update since last year?

[erickee]

The solution posted by mateuszbuda does not work. That only allows you to access images that you can already access via the sliders in the tensorboard.

It is impossible to compare multiple images at the same timestep if tensorboard does not expose those steps to the user. I've logged 4 different images all at the same step, and it is not helpful to see them individually at different steps. This is a major oversight for tensorboard.

[bhack]

Do you have plan to put this in the roadmap?