Skip to content

TypeError occurred when train models with the CLI #440

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wangzai-zfw opened this issue Sep 30, 2022 · 3 comments · Fixed by #442
Closed

TypeError occurred when train models with the CLI #440

wangzai-zfw opened this issue Sep 30, 2022 · 3 comments · Fixed by #442

Comments

@wangzai-zfw
Copy link

Hi SchNetPack development team,

I'm currently learning the SchNet. Since I can't figure out how to add early stopping in python code, I attempt to modify the .yaml files in schnetpack/src/schnetpack/configs in order to train schnet model with early stopping. Could you please let me know how to add early stopping in python code if it's not too much trouble?

When I use CLI to train models, a TypeError occurred. The process I used to train a SchNet model with CLI is as follows:

I created two .yaml files in the two folders respectively. The names of the two folders are data and experiment in schnetpack/src/schnetpack/configs. The content of the following two TXT files is the .yaml file I created for the training model
test_in_data.txt
test_in_experiment.txt

The command I use is as follows:
spktrain --config-dir=~/git/schnetpack/src/schnetpack/configs experiment=test_in_experiment.yaml model/representation=schnet > output_test_moldel_task.txt

Then, a TypeError occurred.
image

The complete error output is in the following TXT file.
error_output.txt

Thank you in advance for any help you might be able to provide.

Best regards,
Wangzai

@ktschuett
Copy link
Contributor

I tried your experiment config and it works for me (had to switch to a different data config thought, since I don't have your data). The error indicates that schnetpack tries to load an existing model. Maybe you have trained a model previously in the same directory and the checkpoints are not compatible?

As for your question about early stopping in code: since we are using pytorch lightning as a training framework now, you can apply their early stopping callback as described here .

@sssaennn
Copy link

sssaennn commented Oct 2, 2022

Hi SchNetPack development team,

I'm currently learning the SchNet. Since I can't figure out how to add early stopping in python code, I attempt to modify the .yaml files in schnetpack/src/schnetpack/configs in order to train schnet model with early stopping. Could you please let me know how to add early stopping in python code if it's not too much trouble?

When I use CLI to train models, a TypeError occurred. The process I used to train a SchNet model with CLI is as follows:

I created two .yaml files in the two folders respectively. The names of the two folders are data and experiment in schnetpack/src/schnetpack/configs. The content of the following two TXT files is the .yaml file I created for the training model test_in_data.txt test_in_experiment.txt

The command I use is as follows: spktrain --config-dir=~/git/schnetpack/src/schnetpack/configs experiment=test_in_experiment.yaml model/representation=schnet > output_test_moldel_task.txt

Then, a TypeError occurred. image

The complete error output is in the following TXT file. error_output.txt

Thank you in advance for any help you might be able to provide.

Best regards, Wangzai

Maybe you can try editing self.save_hyperparameters(ignore=["model"]) to self.save_hyperparameters() in task.py

@ktschuett ktschuett reopened this Oct 4, 2022
@ktschuett
Copy link
Contributor

Ok, there seems to be a problem with the hyperparameter storage of pytorch lightning: If ignore=["model"] is set, saving/loading of models does not work properly, but if it is not set then logging crashes when not using the CLI. I will try to find a workaround...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants