Run_qa crashes because of parser = HfArgumentParser((ModelArguments, DataTrainingArguments, TrainingArguments))

[spacemanidol]

Environment info

• transformers version: 4.3.3
• Platform: linux
• Python version:3.7, 3.8, 3.9 reproed across all three
• PyTorch version (GPU?): 1.7, tried 1.8 with same behavior
• Tensorflow version (GPU?):N/A
• Using GPU in script?: yes
• Using distributed or parallel set-up in script?: Yes 2 gpu

Who can help

@sgugger, @patil-suraj

Information

Model I am using (Bert, XLNet ...): bert-base-uncased

The problem arises when using:

• [ X] the official example scripts: (give details below)
• my own modified scripts: (give details below)

The tasks I am working on is:

• [ X] an official GLUE/SQUaD task: (give the name)
• my own task or dataset: (give details below)
  SQUAD 1.0

To reproduce

Steps to reproduce the behavior:

1. Install clean transformers environment
2. run the run_qa.py script with instructions as specified
3. crash
   If you go ahead and create a new environment and install the most recent version of the transformer and try to run the run_qa.py script(SQUAD) it crashes because of a parser issue.

python run_qa.py --model_name_or_path bert-base-uncased --dataset_name squad --do_train --per_device_train_batch_size 8 --learning_rate 3e-5 --max_seq_length 384 --doc_stride 128 --output_dir output --overwrite_output_dir --cache_dir cache --preprocessing_num_workers 4 --seed 42 --num_train_epochs 1
Traceback (most recent call last):
File "run_qa.py", line 1095, in
main()
File "run_qa.py", line 902, in main
parser = HfArgumentParser((ModelArguments, DataTrainingArguments, TrainingArguments))
File "/home/spacemanidol/miniconda3/envs/sparseml/lib/python3.7/site-packages/transformers/hf_argparser.py", line 52, in init
self._add_dataclass_arguments(dtype)
File "/home/spacemanidol/miniconda3/envs/sparseml/lib/python3.7/site-packages/transformers/hf_argparser.py", line 93, in _add_dataclass_arguments
elif hasattr(field.type, "origin") and issubclass(field.type.origin, List):
File "/home/spacemanidol/miniconda3/envs/sparseml/lib/python3.7/typing.py", line 721, in subclasscheck
return issubclass(cls, self.origin)
TypeError: issubclass() arg 1 must be a clas

Expected behavior

Run and produce a BERT-QA model

[sgugger]

This is weird and linked to your environment somehow.
@stas00 Was this the error you encountered when dataclasses is installed in Python 3.7 or was it a different one?

[stas00]

no, that was not that error. I tested run_qa.py w/ dataclasses on py38 and it didn't fail.

the datasets error was: AttributeError: module 'typing' has no attribute '_ClassVar'

#8638

[spacemanidol]

I just tried this on 2 new servers with a fresh conda environment and reproduced behavior.
Steps.

conda create -n test python=3.8
conda activate test
pip install transformers datasets torch
python run_qa.py   --model_name_or_path bert-base-uncased  --dataset_name squad  --do_train  --per_device_train_batch_size 8  --learning_rate 3e-5  --max_seq_length 384  --doc_stride 128  --output_dir bert-base-uncased-qa/  --overwrite_output_dir  --cache_dir cache  --preprocessing_num_workers 4  --seed 42  --num_train_epochs 1

[spacemanidol]

I have also reproed with venv and regular environment on multiple machines

[sgugger]

The suggested commands work fine on my side, so can't reproduce the issue.

[sgugger]

I have pushed a fix (on master by mistake but it's pretty harmless) a tentative fix to remove the line that caused you problem and replace it by a regex. Let me know if it fixes your issue or not (I can't confirm myself since I can't reproduce).

[stas00]

FWIW, I followed your new conda env steps and couldn't reproduce the problem.

@spacemanidol, fyi I edited your comment to fix the conda create line as it had the commands reversed.

[spacemanidol]

Can confirm this works.