The Loss decrease to 0 quickly when it trains during Epoch2 on my dataset

[unanan]

Hi, there comes a silly question to me.
I use this repo to train my dataset, which is a 2 classes dataset on Semantic Segmentation.
There is everything ok during Epoch1(I use the Epoch1.pth to predict pictures, and the result is ok. The Segmentation performs good at some place.)
And things go strange at Epoch2. The Loss decreases quickly to 0 at about in 100 steps.
I'm confused of it. Maybe because the num of classes of my dataset is too small? How can I modify it?
Thank you in advance.

[nader93k]

It may be too late to help you. But anyway, if you could upload your code and a description of your dataset (num of training images, validation images, crop-size, actual image resolutions, and some of the images in the dataset) maybe we can trace your problem.