Add files via upload

Author: zhaominyiz

logs/40_EPiDA_Offline_EDA_Final.out

@@ -0,0 +1,15 @@
+2021-08-21 06:53:05.699761: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
+Warming up PyWSD (takes ~10 secs)... Loading the LM will be faster if you build a binary file.
+took 5.4483466148376465 secs.
+Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForSequenceClassification: ['cls.predictions.transform.LayerNorm.bias', 'cls.seq_relationship.weight', 'cls.predictions.transform.LayerNorm.weight', 'cls.predictions.bias', 'cls.predictions.transform.dense.bias', 'cls.predictions.decoder.weight', 'cls.seq_relationship.bias', 'cls.predictions.transform.dense.weight']
+- This IS expected if you are initializing BertForSequenceClassification from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
+- This IS NOT expected if you are initializing BertForSequenceClassification from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
+Some weights of BertForSequenceClassification were not initialized from the model checkpoint at bert-base-uncased and are newly initialized: ['classifier.weight', 'classifier.bias']
+You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
+Simple Test
+['So cute the small baby is crying!', 'So cute the little baby is crying!']
+test speed!
+EDA cost 188.4038935779322
+CWE cost 30.74567894718709
+EPDA EDA cost 43.84778790666555
+EPDA + CWE cost 10.050052534914453

logs/EPiDA_CWE_SST.out

@@ -0,0 +1,31 @@
+Warming up PyWSD (takes ~10 secs)... Loading the LM will be faster if you build a binary file.
+Reading /remote-home/***/Code/NLP/EPDA/lms/offense.arpa
+----5---10---15---20---25---30---35---40---45---50---55---60---65---70---75---80---85---90---95--100
+****************************************************************************************************
+Simple Test
+['thusly cute the little baby is crying!', 'So cute the little baby is crying!']
+LR= 5e-05
+Start to read:  new_data/sentiment/train_1.txt
+Load Over, Find:  153  datas.
+Start to read:  new_data/sentiment/test.txt
+Load Over, Find:  3027  datas.
+took 5.79877781867981 secs.
+Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForSequenceClassification: ['cls.predictions.transform.LayerNorm.bias', 'cls.predictions.transform.dense.bias', 'cls.predictions.bias', 'cls.predictions.decoder.weight', 'cls.predictions.transform.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight', 'cls.predictions.transform.LayerNorm.weight']
+- This IS expected if you are initializing BertForSequenceClassification from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
+- This IS NOT expected if you are initializing BertForSequenceClassification from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
+Some weights of BertForSequenceClassification were not initialized from the model checkpoint at bert-base-uncased and are newly initialized: ['classifier.weight', 'classifier.bias']
+You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
+Bert Tokenizer
+Update EPOCHES to 16
+Start to update Dataset
+?? 2295 2295 2295
+Before 153
+Start Update Dataset, Find  153 datas.
+Update Dataset Finish, Find  2295 datas.
+< Update Done.
+After 2295
+start to extract something
+torch.Size([2295, 3072]) 2295
+Error Rate EPida0.0153 EDA0.0305 CEM0.0065 REM0.0675
+Distance EPida0.0078 EDA0.0054 CEM0.0025 REM0.0121
+> Done. Model Training

logs/EPiDA_EDA_SST.out

@@ -0,0 +1,25 @@
+Warming up PyWSD (takes ~10 secs)... Loading the LM will be faster if you build a binary file.
+Reading /remote-home/***/Code/NLP/EPDA/lms/irony.arpa
+----5---10---15---20---25---30---35---40---45---50---55---60---65---70---75---80---85---90---95--100
+****************************************************************************************************
+Simple Test
+['So cunning the little baby is crying!', 'So cute the little baby is crying!']
+LR= 2e-05
+Start to read:  new_data/irony/train_10.txt
+Load Over, Find:  328  datas.
+Start to read:  new_data/irony/test.txt
+Load Over, Find:  656  datas.
+took 5.60038685798645 secs.
+Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForSequenceClassification: ['cls.predictions.decoder.weight', 'cls.predictions.transform.dense.weight', 'cls.seq_relationship.bias', 'cls.predictions.transform.LayerNorm.weight', 'cls.seq_relationship.weight', 'cls.predictions.transform.dense.bias', 'cls.predictions.bias', 'cls.predictions.transform.LayerNorm.bias']
+- This IS expected if you are initializing BertForSequenceClassification from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
+- This IS NOT expected if you are initializing BertForSequenceClassification from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
+Some weights of BertForSequenceClassification were not initialized from the model checkpoint at bert-base-uncased and are newly initialized: ['classifier.bias', 'classifier.weight']
+You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
+Loading the LM will be faster if you build a binary file.
+Reading /remote-home/***/Code/NLP/EPDA/lms/irony.arpa
+----5---10---15---20---25---30---35---40---45---50---55---60---65---70---75---80---85---90---95--100
+****************************************************************************************************
+Update EPOCHES to 80
+Start to update Dataset
+Start to calculate PPL Score.
+PPL Score 12.11044971795114

logs/Speed_Results.out

@@ -0,0 +1,25 @@
+Warming up PyWSD (takes ~10 secs)... Loading the LM will be faster if you build a binary file.
+Reading /remote-home/***/Code/NLP/EPDA/lms/offense.arpa
+----5---10---15---20---25---30---35---40---45---50---55---60---65---70---75---80---85---90---95--100
+****************************************************************************************************
+Simple Test
+['So cunning the little baby is crying!', 'So cute the little baby is crying!']
+LR= 5e-05
+Start to read:  new_data/offense/train_10.txt
+Load Over, Find:  8844  datas.
+Start to read:  new_data/offense/test.txt
+Load Over, Find:  17679  datas.
+took 5.584129810333252 secs.
+Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForSequenceClassification: ['cls.seq_relationship.bias', 'cls.predictions.transform.dense.bias', 'cls.predictions.transform.LayerNorm.bias', 'cls.predictions.transform.LayerNorm.weight', 'cls.predictions.decoder.weight', 'cls.predictions.transform.dense.weight', 'cls.predictions.bias', 'cls.seq_relationship.weight']
+- This IS expected if you are initializing BertForSequenceClassification from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
+- This IS NOT expected if you are initializing BertForSequenceClassification from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
+Some weights of BertForSequenceClassification were not initialized from the model checkpoint at bert-base-uncased and are newly initialized: ['classifier.bias', 'classifier.weight']
+You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
+Loading the LM will be faster if you build a binary file.
+Reading /remote-home/***/Code/NLP/EPDA/lms/offense.arpa
+----5---10---15---20---25---30---35---40---45---50---55---60---65---70---75---80---85---90---95--100
+****************************************************************************************************
+Update EPOCHES to 4
+Start to update Dataset
+Start to calculate PPL Score.
+PPL Score 8.570172684665541

logs/div_qua.out

@@ -0,0 +1,25 @@
+Warming up PyWSD (takes ~10 secs)... Loading the LM will be faster if you build a binary file.
+Reading /remote-home/***/Code/NLP/EPDA/lms/sentiment.arpa
+----5---10---15---20---25---30---35---40---45---50---55---60---65---70---75---80---85---90---95--100
+****************************************************************************************************
+Simple Test
+['So cunning the little baby is crying!', 'So cute the little baby is crying!']
+LR= 5e-05
+Start to read:  new_data/sentiment/train_10.txt
+Load Over, Find:  1514  datas.
+Start to read:  new_data/sentiment/test.txt
+Load Over, Find:  3027  datas.
+took 6.0170722007751465 secs.
+Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForSequenceClassification: ['cls.predictions.transform.dense.weight', 'cls.seq_relationship.weight', 'cls.predictions.transform.dense.bias', 'cls.predictions.transform.LayerNorm.weight', 'cls.predictions.bias', 'cls.predictions.decoder.weight', 'cls.predictions.transform.LayerNorm.bias', 'cls.seq_relationship.bias']
+- This IS expected if you are initializing BertForSequenceClassification from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
+- This IS NOT expected if you are initializing BertForSequenceClassification from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
+Some weights of BertForSequenceClassification were not initialized from the model checkpoint at bert-base-uncased and are newly initialized: ['classifier.bias', 'classifier.weight']
+You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
+Loading the LM will be faster if you build a binary file.
+Reading /remote-home/***/Code/NLP/EPDA/lms/sentiment.arpa
+----5---10---15---20---25---30---35---40---45---50---55---60---65---70---75---80---85---90---95--100
+****************************************************************************************************
+Update EPOCHES to 16
+Start to update Dataset
+Start to calculate PPL Score.
+PPL Score 8.096230467979732

logs/ppl/CEM_Irony_PPL.out

@@ -0,0 +1,25 @@
+Warming up PyWSD (takes ~10 secs)... Loading the LM will be faster if you build a binary file.
+Reading /remote-home/***/Code/NLP/EPDA/lms/irony.arpa
+----5---10---15---20---25---30---35---40---45---50---55---60---65---70---75---80---85---90---95--100
+****************************************************************************************************
+Simple Test
+['So cute the small baby is crying!', 'So cute the little baby is crying!']
+LR= 2e-05
+Start to read:  new_data/irony/train_10.txt
+Load Over, Find:  328  datas.
+Start to read:  new_data/irony/test.txt
+Load Over, Find:  656  datas.
+took 6.335399627685547 secs.
+Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForSequenceClassification: ['cls.seq_relationship.weight', 'cls.seq_relationship.bias', 'cls.predictions.transform.LayerNorm.weight', 'cls.predictions.transform.dense.weight', 'cls.predictions.bias', 'cls.predictions.transform.dense.bias', 'cls.predictions.decoder.weight', 'cls.predictions.transform.LayerNorm.bias']
+- This IS expected if you are initializing BertForSequenceClassification from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
+- This IS NOT expected if you are initializing BertForSequenceClassification from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
+Some weights of BertForSequenceClassification were not initialized from the model checkpoint at bert-base-uncased and are newly initialized: ['classifier.bias', 'classifier.weight']
+You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
+Loading the LM will be faster if you build a binary file.
+Reading /remote-home/***/Code/NLP/EPDA/lms/irony.arpa
+----5---10---15---20---25---30---35---40---45---50---55---60---65---70---75---80---85---90---95--100
+****************************************************************************************************
+Update EPOCHES to 80
+Start to update Dataset
+Start to calculate PPL Score.
+PPL Score 77.08645492508874

logs/ppl/CEM_Offense_PPL.out

@@ -0,0 +1,25 @@
+Warming up PyWSD (takes ~10 secs)... Loading the LM will be faster if you build a binary file.
+Reading /remote-home/***/Code/NLP/EPDA/lms/offense.arpa
+----5---10---15---20---25---30---35---40---45---50---55---60---65---70---75---80---85---90---95--100
+****************************************************************************************************
+Simple Test
+['thusly cute the little baby is crying!', 'So cute the little baby is crying!']
+LR= 5e-05
+Start to read:  new_data/offense/train_10.txt
+Load Over, Find:  8844  datas.
+Start to read:  new_data/offense/test.txt
+Load Over, Find:  17679  datas.
+took 5.67768120765686 secs.
+Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForSequenceClassification: ['cls.predictions.transform.dense.bias', 'cls.predictions.transform.dense.weight', 'cls.seq_relationship.weight', 'cls.predictions.transform.LayerNorm.bias', 'cls.predictions.bias', 'cls.seq_relationship.bias', 'cls.predictions.transform.LayerNorm.weight', 'cls.predictions.decoder.weight']
+- This IS expected if you are initializing BertForSequenceClassification from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
+- This IS NOT expected if you are initializing BertForSequenceClassification from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
+Some weights of BertForSequenceClassification were not initialized from the model checkpoint at bert-base-uncased and are newly initialized: ['classifier.weight', 'classifier.bias']
+You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
+Loading the LM will be faster if you build a binary file.
+Reading /remote-home/***/Code/NLP/EPDA/lms/offense.arpa
+----5---10---15---20---25---30---35---40---45---50---55---60---65---70---75---80---85---90---95--100
+****************************************************************************************************
+Update EPOCHES to 4
+Start to update Dataset
+Start to calculate PPL Score.
+PPL Score 81.13078085129892

logs/ppl/CEM_Sentiment_PPL.out

@@ -0,0 +1,25 @@
+Warming up PyWSD (takes ~10 secs)... Loading the LM will be faster if you build a binary file.
+Reading /remote-home/***/Code/NLP/EPDA/lms/sentiment.arpa
+----5---10---15---20---25---30---35---40---45---50---55---60---65---70---75---80---85---90---95--100
+****************************************************************************************************
+Simple Test
+['thusly cute the little baby is crying!', 'So cute the little baby is crying!']
+LR= 5e-05
+Start to read:  new_data/sentiment/train_10.txt
+Load Over, Find:  1514  datas.
+Start to read:  new_data/sentiment/test.txt
+Load Over, Find:  3027  datas.
+took 6.893878936767578 secs.
+Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForSequenceClassification: ['cls.predictions.transform.LayerNorm.bias', 'cls.predictions.decoder.weight', 'cls.predictions.bias', 'cls.seq_relationship.weight', 'cls.seq_relationship.bias', 'cls.predictions.transform.dense.weight', 'cls.predictions.transform.LayerNorm.weight', 'cls.predictions.transform.dense.bias']
+- This IS expected if you are initializing BertForSequenceClassification from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
+- This IS NOT expected if you are initializing BertForSequenceClassification from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
+Some weights of BertForSequenceClassification were not initialized from the model checkpoint at bert-base-uncased and are newly initialized: ['classifier.bias', 'classifier.weight']
+You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
+Loading the LM will be faster if you build a binary file.
+Reading /remote-home/***/Code/NLP/EPDA/lms/sentiment.arpa
+----5---10---15---20---25---30---35---40---45---50---55---60---65---70---75---80---85---90---95--100
+****************************************************************************************************
+Update EPOCHES to 16
+Start to update Dataset
+Start to calculate PPL Score.
+PPL Score 66.86463311930764