calling Trainer.test(model) will perform test twice

[pableeto]

🐛 Bug

When calling Trainer.test(model) after training separately, the Test routine will be called twice.

To Reproduce

Steps to reproduce the behavior:

model = my_lightning_model(some_hparams)
trainer = Trainer()
trainer.fit(model)
trainer.test(model)     #the test routine will be performed twice

Expected behavior

The test should be only performed once via trainer.test(model).

Environment

PyTorch version: 1.4.0
Is debug build: No
CUDA used to build PyTorch: 10.1

OS: Ubuntu 18.04.3 LTS
GCC version: (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0
CMake version: version 3.10.2

Python version: 3.6
Is CUDA available: Yes
CUDA runtime version: 10.1.243
GPU models and configuration:
GPU 0: GeForce GTX 1080 Ti
GPU 1: GeForce GTX 1080 Ti

Nvidia driver version: 430.14
cuDNN version: Could not collect

Versions of relevant libraries:
[pip3] numpy==1.18.1
[pip3] pytorch-lightning==0.6.1.dev0
[pip3] torch==1.4.0
[pip3] torchvision==0.5.0
[conda] Could not collect

Potential reason causing this bug

By dive into pl's source code, I found the reason might be related to pytorch_lightning/trainer/trainer.py at line 1104 & line 1177:
line 1104:

        # when testing requested only run test and return
        if self.testing:
            # only load test dataloader for testing
            self.reset_test_dataloader(ref_model)
            self.run_evaluation(test_mode=True)
            return

line 1177:

        self.testing = True
        if model is not None:
            self.fit(model)
        self.run_evaluation(test_mode=True)

When calling trainer.test(model) at line 1177, self.testing will be set to True; since model is not None, self.fit(model) will be called; then in line 1104 at fit(), self.testing is True, so the self.run_evaluation will be called first time. After that, the self.run_evaluation in test() (as in line 1180) will be called again, thus result in twice test evulation.

Potential fix

change the code snippet in test() to

        self.testing = True
        if model is not None:
            #as self.testing = True, self.fit(model) will only perform test
            self.fit(model)   
        else: 
             #if model is not None, then the test has been already performed in fit()
             self.run_evaluation(test_mode=True)

might fix the problem?

P.S. if possible, I'd be happy to submit a PR on this :)

[github-actions]

Hi! thanks for your contribution!, great first issue!

[ghost]

This is fixed in #770 along with my PR on the other features. Still waiting for it to be approved tho.

[kenenbek]

Updated to pytorch-lightning-1.4.9, but still found this issue.