unknown error while training

[akhaled89]

I got repeated unknown errors while running the RGBN training using the ‘train_faster_rcnn.sh’ patch file.
The machine is a google cloud instance with Ubuntu 16 as OS and here are the specs
Machine type :n1-standard-4 (4 vCPUs, 15 GB memory)
CPU platform: Intel Haswell
GPUs : 1 x NVIDIA Tesla T4
Zone :us-central1-a

After following the instructions in the readme file,
Cuda 8.0.44 cudnn 5.1 have been installed from Nvidia official website and Conda environment has been created with python 3.5, and the requirements that include TensorFlow GPU 0.12.1 have been installed.

The pre-trained resnet101 has been installed from this site
http://download.tensorflow.org/models/resnet_v1_101_2016_08_28.tar.gz

The dataset (train & test) mentioned in the readme file has been used also https://drive.google.com/open?id=1vIAFsftjmHg2J5lJgO92C1Xmyw539p_B

And here is the command we used to start training after modifying the train_faster_rcnn.sh ,factory.py
./train_faster_rcnn.sh 0 coco res101 EXP_DIR coco_flip_0001_bilinear_new

And here are the logs
unknown error
unknown error
unknown error
iter: 20 / 110000, total loss: 0.448014

rpn_loss_cls: 0.072341
rpn_loss_box: 0.004598
loss_cls: 0.371075
loss_box: 0.000000
lr: 0.001000
speed: 49.008s / iter
unknown error
unknown error
unknown error
logs.log