...
| Code Block |
|---|
bash scripts/run_squad.sh 10 5e-6 fp16 true 8 384 128 base 1.1 data/download/google_pretrained_weights/uncased_L-12_H-768_A-12/bert_model.ckpt 1.1
|
For SQuAD 1.1 FP16 training with XLA using (4) T4 16GB GPU's run:
| Code Block |
|---|
bash scripts/run_squad.sh 10 5e-6 fp16 true 4 384 128 base 1.1 data/download/google_pretrained_weights/uncased_L-12_H-768_A-12/bert_model.ckpt 1.1
|
1.2.6 Verify results
| Code Block |
|---|
INFO:tensorflow:-----------------------------
I0326 01:25:43.144953 140630939256640 run_squad.py:1127] -----------------------------
INFO:tensorflow:Total Inference Time = 88.62 for Sentences = 10840
I0326 01:25:43.145423 140630939256640 run_squad.py:1129] Total Inference Time = 88.62 for Sentences = 10840
INFO:tensorflow:Total Inference Time W/O Overhead = 75.86 for Sentences = 10824
I0326 01:25:43.145554 140630939256640 run_squad.py:1131] Total Inference Time W/O Overhead = 75.86 for Sentences = 10824
INFO:tensorflow:Summary Inference Statistics
I0326 01:25:43.145649 140630939256640 run_squad.py:1132] Summary Inference Statistics
INFO:tensorflow:Batch size = 8
I0326 01:25:43.145738 140630939256640 run_squad.py:1133] Batch size = 8
INFO:tensorflow:Sequence Length = 384
I0326 01:25:43.145867 140630939256640 run_squad.py:1134] Sequence Length = 384
INFO:tensorflow:Precision = fp16
I0326 01:25:43.145962 140630939256640 run_squad.py:1135] Precision = fp16
INFO:tensorflow:Latency Confidence Level 50 (ms) = 55.79
I0326 01:25:43.146052 140630939256640 run_squad.py:1136] Latency Confidence Level 50 (ms) = 55.79
INFO:tensorflow:Latency Confidence Level 90 (ms) = 57.03
I0326 01:25:43.146145 140630939256640 run_squad.py:1137] Latency Confidence Level 90 (ms) = 57.03
INFO:tensorflow:Latency Confidence Level 95 (ms) = 57.29
I0326 01:25:43.146225 140630939256640 run_squad.py:1138] Latency Confidence Level 95 (ms) = 57.29
INFO:tensorflow:Latency Confidence Level 99 (ms) = 58.62
I0326 01:25:43.146308 140630939256640 run_squad.py:1139] Latency Confidence Level 99 (ms) = 58.62
INFO:tensorflow:Latency Confidence Level 100 (ms) = 286.80
I0326 01:25:43.146387 140630939256640 run_squad.py:1140] Latency Confidence Level 100 (ms) = 286.80
INFO:tensorflow:Latency Average (ms) = 56.07
I0326 01:25:43.146471 140630939256640 run_squad.py:1141] Latency Average (ms) = 56.07
INFO:tensorflow:Throughput Average (sentences/sec) = 142.68
I0326 01:25:43.146564 140630939256640 run_squad.py:1142] Throughput Average (sentences/sec) = 142.68
INFO:tensorflow:-----------------------------
I0326 01:25:43.146645 140630939256640 run_squad.py:1143] -----------------------------
INFO:tensorflow:Writing predictions to: /results/tf_bert_finetuning_squad_base_fp16_gbs40_200326010711/predictions.json
I0326 01:25:43.146801 140630939256640 run_squad.py:431] Writing predictions to: /results/tf_bert_finetuning_squad_base_fp16_gbs40_200326010711/predictions.json
INFO:tensorflow:Writing nbest to: /results/tf_bert_finetuning_squad_base_fp16_gbs40_200326010711/nbest_predictions.json
I0326 01:25:43.146886 140630939256640 run_squad.py:432] Writing nbest to: /results/tf_bert_finetuning_squad_base_fp16_gbs40_200326010711/nbest_predictions.json
{"exact_match": 78.0321665089877, "f1": 86.34229152935384}
|
...
| Code Block |
|---|
root@tessa002:/workspace/lambdal/bert# python3 run_squad.py --vocab_file=/workspace/nvidia-examples/bert/data/download/google_pretrained_weights/uncased_L-12_H-768_A-12/vocab.txt --bert_config_file=/workspace/nvidia-examples/bert/data/download/google_pretrained_weights/uncased_L-12_H-768_A-12/bert_config.json --init_checkpoint=/results/lambdal/squad1/squad_base/model.ckpt-3649 --do_train=False --max_query_length=30 --do_predict=True --predict_file=test_input.json --predict_batch_size=16 --max_seq_length=384 --doc_stride=128 --output_dir=/results/lambdal/squad1/squad_test/
|
1.2.11 You should see similar output below :
...