Orientation Aware Firearms Detection




Automatic detection of firearms is important for enhancing the security and safety of people, however, it is a challenging task owing to the wide variations in shape, size and appearance of firearms. Also, most of the generic object detectors process axis-aligned rectangular areas though, a thin and long rifle may actually cover only a small percentage of that area and the rest may contain irrelevant details suppressing the required object signatures. To handle these challenges, we propose a weakly supervised Orientation Aware Object Detection (OAOD) algorithm which learns to detect oriented object bounding boxes (OBB) while using Axis-Aligned Bounding Boxes (AABB) for training. The proposed OAOD is different from the existing oriented object detectors which strictly require OBB during training which may not always be present. The goal of training on AABB and detection of OBB is achieved by employing a multistage scheme, with Stage-1 predicting the AABB and Stage-2 predicting OBB. In-between the two stages, the oriented proposal generation module along with the object aligned RoI pooling is designed to extract features based on the predicted orientation and to make these features orientation invariant. A diverse and challenging dataset consisting of eleven thousand images is also proposed for firearm detection which is manually annotated for firearm classification and localization. The proposed ITU Firearm dataset (ITUF) contains a wide range of guns and rifles. The OAOD algorithm is evaluated on the ITUF dataset and compared with current state-of-the-art object detectors, including fully supervised oriented object detectors. OAOD has outperformed both types of object detectors with a significant margin. The experimental results (mAP: 88.3 on AABB & mAP: 77.5 on OBB) demonstrate effectiveness of the proposed algorithm for firearm detection..


Key Idea:

One limitation of the existing methods is the use of axis-aligned windows for object detection. Most classifiers decide the presence of an object by analyzing the features in that window. The physically thin and elongated structure of most of the rifles and small size of most guns, make these axis-aligned windows inefficient due to low signal to noise ratio where the signal is the firearm signature and noise is everything else in the window. In the case of firearms being carried by a person, the window will tend to contain substantial information belonging to the background or non-firearm objects, like the person himself. This mixture of the information makes it difficult for classifiers to learn to separate the required information or signal from the other objects acting as noise.


Key idea proposed to handle this issue is illustrated below. By having the the maximum area rectangle and making oriented region proposal (ORP) containing only firearms.


Proposed OAOD Overview:

Most of the current object detectors employ axis-aligned bounding boxes which may incur noise and clutter due to uncorrelated background objects. Such noise may adversely effect the performance of object detection. To overcome this issue, we propose Orientation Aware Object Detection (OAOD) algorithm which consists of a single pipe-lined network consisting of a cascade of two stages shown below. In the Stage-1, the orientation of the object is predicted along with the classification score and offset to the region proposal. In the Stage-2, the updated object proposals are warped according to the predicted orientation such that the object becomes axis aligned. The maximum area rectangle contained within the rotated proposal is cropped and the final Oriented Region Proposal is used for further classification and offset regression. Thus the redundant information contained within the region proposals is significantly reduced for the case of non-axis aligned firearms. The classifier in Stage-2 is trained to predict classification scores on these oriented region proposals to achieve better performance. The proposed network takes an entire image as input and detects and localizes two types of firearms including rifles and guns.



Dataset (DATASET is available upon request [Google Form])

Orientation wise dataset distribution and sample images from dataset.



ArXiv ( Leveraging Orientation for Weakly Supervised Object Detection with Application to Firearm Localization )

Neurocomputing (Leveraging Orientation for Weakly Supervised Object Detection with Application to Firearm Localization)

Github (OAOD with Application to Firearms)



Object Detection, Orientation Aware Object Detection, Oriented Object Detection, , Firearms Detection, Orientation Aware Firearms Detection OAOD, Deep Learning and Computer Vision for object detection, Orientation Aware Object Detection with Application to Firearms



    author = {Javed Iqbal and Muhammad Akhtar Munir and Arif Mahmood and Afsheen Rafaqat Ali and Mohsen Ali},
    title = {Leveraging Orientation for Weakly Supervised Object Detection with Application to Firearm Localization},
    journal = {Neurocomputing},
    year = {2021},
    issn = {0925-2312}
    url = {https://www.sciencedirect.com/science/article/pii/S092523122100151X}}