VANET topology is highly dynamic, wherein the vehicles frequently move across locations. Due to the continually changing topology and lack of security infrastructure, routing protocols in VANET are vulnerable to several attacks. In this paper, we focus on the black hole and the gray hole attacks due to its severity. In black hole and gray hole attacks, the attacker gains access to the wireless network and drops the received packets fully/selectively that impacts on the safety applications of VANET. This paper presents a novel security approach called Smart Blackhole and Gray hole Mitigation (SBGM) to detect and mitigate both black hole and gray hole nodes in VANET using a time series analysis of the dropped packets of each node. The computation of the packet drop distance threshold based on Dynamic Time Warping improves the detection accuracy in SBGM. We assess the performance of SBGM using AODV and OLSR routing protocols under low-dense and high-dense traffic scenarios in terms of Packet Delivery Ratio, Throughput, Average End-to-End Delay, and Packet Drop percentage. From the experimental results, it is evident that the proposed SBGM outperforms the existing techniques in detecting the black and gray hole attacks. The proposed SBGM achieves a detection rate of 99.87% in highway scenarios and 99.68% in urban scenarios.