Brain hemorrhage is a life-threatening problem that happens by bleeding inside human head. In this study, computed tomography (CT) scan images have been used to classify whether the case is hemorrhage or non-hemorrhage. Different convolutional neural network (CNN) models have been observed along with some pre-trained deep learning models such as VGG16, VGG19, ResNet150, ResNet152 and InceptionV3. Pre-trained models have performed well on the dataset but all of them are heavyweight architectures in terms of number of total parameters. But the proposed model is a lightweight architecture as well as a well performing one. After evaluating the model performance, it has been observed that the proposed model gave 96.67% accuracy, 97.08% sensitivity and 96.25% specificity which is the best among other custom CNN models.