Neptune Logger improvement

[jakubczakon]

🚀 Feature

I think we could improve user experience for people logging experiment
data with neptune if we let people use the logger after the training is finished.

An example would be:

neptune_logger = NeptuneLogger(..., close_after_train=False)
trainer = Trainer(logger=neptune_logger)
trainer.fit(CoolSystem())

• log some external validation metric

auc = get_external_validation_auc()
neptune_logger.experiment.log_metric('roc_auc_score', auc)

• log performance chart like roc auc

fig, ax = plt.subplots(figsize=(16, 12))
plot_roc(y, y_pred, ax=ax)
neptune_logger.experiment.log_image('roc_curve', fig)

• log final model

torch.save(model.state_dict(), 'final_model.pth')
neptune_logger.experiment.log_artifact('final_model.pth')

I think the only thing that needs changing is this:

    @rank_zero_only
    def finalize(self, status):
        if self.close_after_train:
             self.experiment.stop()

Additional context

I would be happy to create a PR

[Borda]

@jakubczakon great suggestions, PR is really welcome!
Btw, would you like extend this kind of feature also for other loggers?

[jakubczakon]

Sure thing!

[Borda]

@jakubczakon how is it going? ^^

[jakubczakon]

Oh, I must have miscommunicated.
I wrote in the PR:

@Borda
I haven't added this option to other loggers as I saw people did some work there already (comet).
I can add this close_after_fit option to other loggers but I just don't want to step into other folks territory (where they may not appreciate me being) :).
It's your call.

And since you haven't answered there I assumed we would just stick with NeptuneLogger.
Would you like me to play with those other ones?
I'd definitely feel ok improving/updating TestTubeLogger and the open-source ones but I don't feel good about touching the "commercial" ones.

What do you think @Borda ?

[Borda]

that sounds fair, for TestTubeLogger it is different story/repo..
so closing this issue though 🤖