International Research Journal of Engineering and Technology (IRJET) e-ISSN:2395-0056
Volume: 11 Issue: 10 | Oct 2024 www.irjet.net p-ISSN:2395-0072
IMAGE RECOGNITION AUTONOMOUS VEHICLES USING DEEP LEARNING
Radhika D1 , Devadharshini S2 , Kiruthiga Sri K3 , Logeshwari N4 ,Anisha A5
1Assistant Professor, Vivekanandha College of Engineering for Women, Tiruchengode, Tamilnadu (India), 2,3,4,5 Student, Vivekanandha, College of Engineering for Women, Tiruchengode, Tamilnadu (India).
Abstract - Therapidadvancementofautonomousdriving technology has necessitated the development of more sophisticated methods to ensure the safety and effectiveness of these systems. This project report focuses on the implementation and evaluation of an autonomous vehicle using the deep learningbased image recognition system powered by YOLOv8 (You Only Look Once version 8) algorithm. As the latest iteration in the YOLO series, YOLOv8 is acclaimed for its real-time object detection capabilities. It is meticulously designed for high efficiency, achieving an optimal balance between detection of speed and accuracy. YOLOv8 has been trained on an extensive dataset that covers a wide array of driving scenarios, significantly enhancing its ability to recognize various objects on the road, such as vehicles, pedestrians, and cyclists. The report provides an in-depth analysis of the data preparation procedures, the architectural design of the model, and comprehensive training methodology. This process includes model optimization, validation strategies to ensure the model performance in real-world autonomousdrivingscenarios.Theprojectemphasizesthe scalabilityofYOLOv8,demonstratingitscompatibilitywith a wide spectrum of computational platforms and its adaptabilitytovariousvehiclemodels,rangingfromluxury to economy classes, ensuring versatile deployment in the automotiveindustry.
Keywords YOLOv8, driving scenarios, methodology, pedestrians, and cyclists, pedestrians, and cyclists.
I. INDRODUCTION
The advent of autonomous vehicles heralds a transformativeshiftintransportation,promisingsaferand moreefficientjourneysonourroads.Objectrecognitionin a real-world environment represents a critical component of autonomous vehicle technology, enabling vehicles to identify and classify various objects encountered on the road, such as vehicles, pedestrians, cyclists, traffic signs, andobstacles.Traditionalapproachestoobjectrecognition in autonomous vehicles have relied on a combination of sensordata,suchasLiDAR,radar,andcameras,alongwith handcrafted features and rule- based algorithms. While these methods have made significant strides in enhancing vehicleperception,theyareoftenlimited by their inability togeneralizetodiverseandcomplexreal-worldscenarios. The emergence of deep learning methodologies has opened up new avenues for addressing
thesechallengesandrevolutionizingthefieldofcomputer vision. Deep learning algorithms, inspired by the structure and function of the human brain, have demonstrated remarkable capabilities in learning complex patterns and features directly from raw data, withouttheneedforexplicitfeatureengineering.Thishas led to significant advancements in various domains, including computer vision, natural language processing, and autonomous driving. In the context of autonomous vehicles, deep learning techniques offer the promise of morerobustandadaptiveobjectrecognitionsystemsthat can effectively handle the variability and complexity of real-world driving environments. One of the key challengesinapplyingdeeplearningtoobjectrecognition inautonomousvehiclesliesinthedevelopmentofmodels that can operate in real-time and generalize well to diverse object categories and environmental conditions. The You Only Look Once (YOLO) algorithm, particularly its latest version, YOLOv8, has emerged as a state-of-theart solution for real-time object detection tasks. Known for its high accuracy and efficiency, YOLOv8 excels in detecting multiple objects within a scene with minimal computational overhead, making it an ideal choice for autonomousdrivingapplications.Inthisresearchproject, we aim to leverage the power of deep learning, specifically the YOLOv8 algorithm, for object recognition in a real-world environment for autonomousvehicles.By providingreal-worldvideostreams as input and utilizing the comprehensive COCO (Common Objects in Context) dataset, which encompasses a wide range of object categories, we seek to develop a robust and efficient system for accurately detecting and classifying objects encounteredbyautonomousvehiclesontheroad.
II. LITERATURE REVIEW
1. 1) You Only Look Once: Unified, Real-Time Object Detection • Publication Year 2015 • Authors: Joseph Redmon, SantoshDivvala,RossGirshick,AliFarhadi•Summary: YOLO introduced a groundbreaking algorithm for real- time object detection by dividing images into a grid, predicting bounding boxes, and class probabilities directly. This approach ensured high accuracy and speed, making it ideal for applications like autonomous vehicles, where timely and precise object detection is crucial for safe navigation and decisionmaking.
International Research Journal of Engineering and Technology (IRJET) e-ISSN:2395-0056
Volume: 11 Issue: 10 | Oct 2024 www.irjet.net p-ISSN:2395-0072
2 ) YOLOv3: An Increment Improvement • Publication year: 2018
Authors:JosephRedmon,AliFarhadi•Summary:YOLOv3 improved upon the original framework with enhancements suchas multi-scaleprediction and feature pyramid networks. These improvements boosted detection accuracy and extended support for objects of various sizes, enhancing the model's robustness and versatility in handling diverse real- world scenarios encounteredinautonomousdriving.
3. YOLOv4: Optimal Speed and Accuracy of Object Detection • Publication year: 2020
Authors: Alexey Bochkovskiy, Chien-Yao Wang, Hong-yuan Mark Liao • Summary: YOLOv4 presented significant advancements in both speed and accuracy by introducing novel techniques such as CSPDarknet53 backbone and PANet path-aggregation neck. These innovationsenabledthemodeltoachievestate-of-the-art performance in object detection tasks, including those critical for autonomous vehicles, where precise and efficient detection is essential for safe and reliable navigation.
4. EfficientDet: Scalable and Efficient Object Detection • Publication year: 2019
Authors: Mingxing Tan, Ruoming Pang, Quoc V. Le • Summary: EfficientDet proposed a scalable and efficient object detection model that achieved top performance with fewer parameters and computations compared to previous methods. By leveraging compound scaling, the model effectively balanced between size and accuracy, making it well-suited for resource-constrained environments like autonomous vehicles, where computational efficiency is paramount for real-time operationanddeployment.
5. ) Multi-camera 3D Object Detection for Autonomous Driving Using Deep Learning and Self-Attention Mechanism.
• Publication year: 2023
Author:Hazarika,A.,Vyas,A.,Rahmati,M.andWang,Y.,
• Summary: The document proposes a multi-camera perception solution to predict the 3D properties of vehicles using aggregated information from multiple staticinfrastructure-installedcameras.Itleveragesadeep learningbased framework to map 21) bounding boxes to their3Dcounterparts,providingaclearunderstandingof theobject'sorientationanddimensions.Additionally,The proposed solution also includes a weighted fusion algorithmtoconsolidatetheindependentlygenerated3D bounding boxes from multiple cameras into a single, definitive bounding box with the highest possible accuracy.
6. ) YOLO with adaptive frame control for real-time object detection applications. • Publication year: 2022
Authors: Lee, J. and Hwang, K.I., • Summary: The document discusses the challenges of real-time object
detectionusingtheYOLO(YouOnlyLookOnce)software in intelligent video applications, particularly with network cameras. It highlights the need for consistent processingofvariousinputs,guaranteedmaximumframe rate, and real-time processing capabilities. The proposed solution is a novel YOLO architecture with Adaptive Frame Control (AFC) to efficiently address these challenges.ItproposesAFCasasolution, and provides experimental evidence of its effectiveness in improving real-time processing capabilities while maintainingobjectdetectionperformance.
7. YOLOv7-RAR for Urban Vehicle Detection. • Publication year:2023
Authors: Zhang, Y., Sun, Y. , Wang, Z. and Jiang, Y., • Summary: The document discusses the YOLOv7-RAR algorithm,animprovedversionoftheYOLOv7algorithm, designedtoaddressthechallengesofvehicledetectionon urban roads. The improvements were validated through ablation experiments, showing enhanced performance compared to the original algorithm. The dataset used for evaluation was UADETRAC, a large-scale dataset for vehicledetectionandtracking. The practical implications of the YOLOv7-RAR algorithm indicate its applicability in addressingreal-worldchallengesrelatedtourbantraffic.
8. ) Multi-view 3d object detection network for autonomous driving. Publication year: 2017.
Authors: Chen, X., Ma, H., wan, J., Li, B. and Xia, T., Summary:TheauthorproposesaMulti-View3Dnetwork (MV3D) for highaccuracy 3D object detection in autonomous driving scenarios. The network takes both LIDAR point cloudand RGBimages asinputandpredicts oriented 3D bounding boxes. It consists of two subnetworks: a 3D proposal network and a region-based fusion network. The proposal network efficiently generates 3D candidate boxes from the bird's eye view representation of the 3D point cloud, while the fusion network combines region-wise features from multiple viewstoenableinteractionsbetweenintermediatelayers ofdifferentpaths.
9. DC-YOLOv8: small-size object detection algorithm based on camera sensor. Publication year: 2023
Authors: Lou, H., Duan, X., Guo, J., Liu, H., Gu, J., Bi, L. and Chen, H., Summary: This paper that introduces a smallsize object detection algorithm specifically designed for camera sensors. It focuses on improving the detection accuracy of small objects, which are challenging for traditional algorithms. The algorithm combines elements from YOLOv3 and EfficientDet, utilizing depthwise separableconvolutionsandafeaturepyramidnetworkto enhance performance. Through experiments, the paper demonstrates that DC-YOLOv8 achieves superior detectionaccuracyforsmallobjectscomparedtoexisting methods while maintaining efficiency in terms of speed andmodelsize.
International Research Journal of Engineering and Technology (IRJET) e-ISSN:2395-0056
Volume: 11 Issue: 10 | Oct 2024 www.irjet.net p-ISSN:2395-0072
10. Autonomous vehicles perception (avp) using deep learning : Modeling, assessment, and challenges. Publication year: 2022
Author: Jebamikyous, H.H. and Kashef, R., Summary: The authors emphasize the critical role of perception in decisionmaking for autonomous driving systems, emphasizing the use of deep learning models for semantic segmentation and object detection tasks. They review various deep learning methods, including the use of Convolutional Neural Networks (CNN) and Reinforcement Learning, for AVP. The paper provides a detailed analysis of active and passive sensors, such as LiDAR, cameras, and radar, and their role in AVP. The authors also discuss benchmark datasets used for training and evaluating deep learning models, including theKITTI,Cityscapes,andBDDIOOKdatasets.
III. SYSTEM ANALYSIS
3.1 EXISTING SYSTEM
TheexistingsystemusestheYOLOv7algorithmforimage recognition in realworld environments for autonomous vehicles.Thissysteminvolvesusingasingleshot detector thatprocessesanentireimageina singlepass,makingit computationally efficient. However, YOLOv7 can be less accurate than other methods and less effective in detectingsmallobjects.Itisalsoatwo-stagedetectorthat uses two passes of the input image to make predictions about the presence and location of objects, which makes it more accurate but also more computationally expensive.
3.2 DISADVANTAGES
YOLOv7islessaccurateindetectingsmallobjects.
It is also two-stage detector, it can be more computationallyexpensivethansingle-shotdetectorslike YOLOv8.
It is not flexible in adapting to different architectures. Because, it has a more rigid architecture thatmaynotbeasadaptable.
YOLOv7modelwithafixeddetectionframemayface challenges in accurately detecting multiple objects with varyingsizesinanimage.
3.3
PROPOSED SYSTEM
Theproposedsystemaimstoaddressthechallenges of recognizing objects and obstacles in real-world environments by leveraging advanced deep learning techniques, specifically YOLOv8 (You Only Look Once version 8). Unlike traditional methods that depend on manual featureengineering, ourproposedsystemadopts a data-driven approach, enabling the model to automatically learn discriminative features directly from raw video data captured by the vehicle's cameras. By utilizing the YOLOv8 architecture, which offers real-time object detection capabilities with high accuracy, the proposed system can efficiently identify and localize
variousobjectsandobstaclesontheroad,suchasvehicles, pedestrians, and traffic signs, in real-time. This enables the autonomous vehicle to make informed decisions and navigate safely through complex environments. Additionally, the proposed system integrates advanced preprocessing techniques and data augmentation strategies to enhance the quality of input video frames and improvetherobustnessofthemodeltovariabilityin lighting conditions, weather, and object orientations across different video frames. Overall, the proposed system aims to provide a more efficient, scalable, and accurate solution for object recognition in real-world environments using video input, facilitating safer and morereliableautonomousdrivingexperiences.
3.4 ADVANTAGES
•Real-time object detection: The YOLOv8 algorithm is designed for real-time object detection, which makes it idealforautonomousvehiclesthatneedtoquicklyrespond totheirenvironment.
•Highaccuracy:YOLOv8hasbeenshowntoachievehigh accuracy in object detection tasks, which can help improve the safety and reliability of autonomous vehicles.
•Flexible architecture: YOLOv8 has a flexible architecture that can be easily adapted to different requirements.
•Anchor-free architecture: By eliminating the need for predefined anchor boxes, Yolov8 adapts to different object sizes and appearances, enhancing the overall performanceofthesystem.
IV. SYSTEM DESIGN
Systemdesignis"theprocessofstudyingaprocedureor business in order to identify its goals, purposes and create systemsandproceduresthatwillachievethemin anefficientway".Anotherviewseessystemanalysisasa problem- solving technique that breaks down a system intoitscomponentpiecesforthepurposeofthestudying how well those component parts work and interact to accomplish their purpose. DESIGN NOTATION Design notationsareused when planningandshould beable to communicatethepurposeofaprogramwithouttheneed for formal code. Commonly used design notations are: DFDv/ERD
5.1 UML DIAGRAM
The UML diagram for the autonomous vehicle image recognition system encompasses several key elements, including use case diagrams, class diagrams, and sequence diagrams. The use case diagram depicts the system's functionality from the perspective of its users, illustratingthe various tasks and interactions that users
International Research Journal of Engineering and Technology (IRJET) e-ISSN:2395-0056
Volume: 11 Issue: 10 | Oct 2024 www.irjet.net
can perform. This includes actors such as vehicle operators, system administrators, and developers, along with their associated use cases such as "Provide Video Feed,""DetectObjects,"and"DisplayResults.”
The system aims to recognize and identify objects in real- time video feeds to enhance the perception capabilities of autonomous vehicles. Leveraging the YOLOv8 deep learning algorithm, the system processes the video input feed and identifies various objects such as vehicles, pedestrians, and traffic signs with high accuracy and efficiency. The use case diagram helps to identify system requirements, define user roles, and prioritize features based on user needs. By focusing on real-world image recognition, the system aims to improvethesafetyandreliabilityofautonomousvehicle.
DATA FLOW DIAGRAM
The Data Flow Diagram (DFD) illustrates the flow of informationandprocessingsteps withintheautonomous vehicle image recognition system, providing a visual representation of how data moves through the various modulesandsubsystems.
At the highest level, the DFD consists of four main components: data sources, processes, data stores, and data destinations. The diagram begins with the acquisitionofvideofeeddatafromexternalsourcessuch asonboardcamerasorexternalsensors.
Thisrawvideoservesasinputdataforthesystemandis passed to the Video Input Processing module for initial processingandenhancement.
The preprocessed video feed is then fed into the Object Detection module, where the YOLOv8 deep learning algorithm performs inference to identify and classify various objectswithinthevideoframes,suchasvehicles, pedestrians, and traffic signs. The detection results, including bounding boxes, object labels, and confidence scores, are generated and passed to the Post processing moduleforrefinementandvalidation.
p-ISSN:2395-0072
Throughout this process, the DFD illustrates the flow of data frominputsourcesthroughvariousprocessingsteps tooutputdestinations,providingaclearunderstandingof thesystem'sfunctionalityandinteractions.
It serves as a valuable tool for system designers, developers, and users to visualize and comprehend the flowofinformationwithintheautonomousvehicleimage recognitionsystem.
3. USE CASE DIAGRAM
The Use Case Diagram for the object recognition system in real-world environments for autonomous vehicles provides a comprehensive overview of the system's functionalities and the interactions between its various actors and components. At the center of the diagram is the "Vehicle" actor, representing autonomous vehicles responsible for using the system to detect and analyze objectsinvideofeeds.Theprimaryusecasesidentifiedin the diagram include "Capture Frame," "Predict Objects," and"VisualizeResults."
The "Capture Frame" use case depicts the process by which autonomous vehicles capture video frames from cameras mounted on the vehicle for analysis.. Update EnvironmentalModel:
Thesystemcontinuouslyupdatesaninternalmodelofthe environmentwiththedetectedobjects.Thismodelassists in contextual understanding and improves the performanceoftheobjectrecognitionsystemovertime.
4. SEQUENCE DIAGRAM
The Sequence Diagram within the system architecture outlines the sequential flow of interactions between various components and modules involved in the object recognition process. At its core, the sequence diagram provides a visual representation of how different entities withinthesystemcommunicateandcollaboratetoachieve theoverarchinggoal of object detection. In the context of theobjectrecognitionsystemforautonomousvehicles,the sequencediagramtypicallybeginswiththeinitiationofthe recognitionprocess, triggered by the vehicle's request to analyze a set of videoframes captured by the camera. This module retrieves the captured video frames and forwards them to the Object Detection (YOLOv8) module, where the YOLOv8 algorithm is applied to predict and post-process thedetectedobjects.
Figure 5.1 UML DIAGRAM
International Research Journal of Engineering and Technology (IRJET) e-ISSN:2395-0056
Volume: 11 Issue: 10 | Oct 2024 www.irjet.net
5.2 Sequence Diagram
In the context of the object recognition system for autonomous vehicles, the sequence diagram typically begins with the initiation of the recognition process, triggeredbythevehicle'srequesttoanalyzeasetofvideo framescapturedby the camera. This request is typically initiated through the "Capture Frame" use case in the Video Processing module. Upon receiving the vehicle's request, the system orchestrates a series of interactions between different modules, starting with the Video Processing module. At its core, the sequence diagram providesavisualrepresentationofhowdifferententities within the system communicate and collaborate to achieve the overarching goal of object detection. In the contextoftheobjectrecognitionsystemforautonomous vehicles, the sequence diagram typically begins with the initiation of the recognition process, triggered by the vehicle's request to analyze a set of video frames captured by the camera. This module retrieves the captured video frames and forwards them to the Object Detection (YOLOv8) module, where the YOLOv8 algorithm is applied to predict and post- process the detectedobject.
V. SYSTEM IMPLEMENTATION
6.1 MODULE DESCRIPTION
6.1.1 Data Collection:
Objective:
To gather high-quality and diverse image datasets representing the real-world environment for autonomous vehicles. Data collection is a foundational step in building an effective image recognition system for autonomous vehicles. The quality and diversity of the dataset directly influence the performance and accuracy of the machine learning models, such as the YOLOv8 algorithm, used for object detection. Sources of Data Onboard cameras from autonomous vehicles Public datasets like COCO, ImageNet, and KITTI Simulated environments using platforms like Unreal Engine or CARLA
6.1.2 Annotation Process
p-ISSN:2395-0072
Imagesinthedatasetareannotatedwithboundingboxes, servingasgroundtruthlabelsfortrainingandevaluation oftheobjectdetectionmodel.
6.1.3 Data Quality and Diversity
• Quality: High-resolution images with accurate annotations.
• Diversity: Include various objects, environments, lighting, and weather conditions to ensure model robustness.
6.1.4 Data Storage and Management
Efficient storage and management of the collected data are essential for easy access, retrieval, and processing. Proper indexing, categorization, and backup strategies should be implemented to maintain data integrity and availability.
6.2 DATA PREPROCESSING
Objective:
Topreprocesstherawimagedatatoenhancequalityand consistencyfortrainingthe YOLOv8 model.
Data preprocessing is essential topreparethe rawimage datainaformatcompatiblewiththedeeplearningmodel andconducivetooptimaltrainingperformance.
Image Resizing
Resizing the images to a uniform dimension ensures consistent input data and reduces computational complexity and memory requirements during training andinference.
Image Normalization
Normalizing the pixel values of the images to a specific range standardizes the input data, improving the convergencespeedandstabilityofthetrainingprocess.
Data Augmentation
Data augmentation techniques, such as rotation, flipping, zooming, cropping, and brightness/contrast adjustments, are applied to increase the dataset's diversity and robustness, improving the model's generalization capabilityandperformance.
6.3
DATA AUGUMENTATION
Objective:
Toartificiallyincreasethediversityandrobustnessofthe training dataset through various image transformation techniques. Data augmentation increases the diversity and robustness of the training dataset by applying various image transformation techniques, improving the model's generalization capability and resilience to variations in the real-world environment. Rotation Randomly rotating the images by a certain angle improves the model's adaptability to various viewing anglesandperspectives.
Figure
Flipping
International Research Journal of Engineering and Technology (IRJET) e-ISSN:2395-0056
Volume: 11 Issue: 10 | Oct 2024 www.irjet.net
Horizontal or vertical flipping of the images increases the dataset's diversity and trains the model to recognize objectsinmirroredorinvertedorientations.
Zooming and Cropping
Randomlyzoominginorcroppingtheimagestofocuson specific regions of interest improves the model's ability todetectobjectsatdifferentscalesanddistances.
Brightness and Contrast Adjustment
Randomly adjusting the brightness and contrast of the imagessimulatesdifferentlightingconditions,enhancing themodel'sresiliencetovariationsinillumination.
Noise Injection
Adding random noise to the images simulates imperfections and artifactsen countered in real-world images, improvingthe model's robustness and handling ofnoisyandlow-qualityimages.
6.4 YOLOv8 IMPLEMENTATION
YOLO, as a one-stage object detection algorithm with high computational efficiency, has become a prominent approachinthefield.Intherealmofdeeplearning,YOLO hasgainedpopularityduetoitsrobustness,validity,and rapiddetectioncapabilities.TheadvantagesoftheYOLO algorithm include its speed, ease of configuration, opensource nature, compatibility with various frameworks andlibraries,andhighaccuracy.Overthepastfewyears, the YOLO algorithm has undergone several iterations, includingYOLOv2,YOLOv3, YOLOv4, YOLOv5, YOLOv6, and YOLOv7.
These iterations represent the evolution and improvement of the YOLO algorithm over time. YOLOv8 capabilities and improvements in a computer vision model used for tasks such as object detection, classification, and segmentation. It mentions that YOLOv8 is easy to use and can be trained on large datasets In the realm of deep learning, YOLO has gained popularity due to its robustness, validity, and rapid detection capabilities. The advantages of the YOLO algorithm include its speed, ease of configuration, opensource nature, compatibility with various frameworks andlibraries,andhighaccuracy.
The implementation of YOLOv8 marks a significant advancement in the realm of real-time object detection, combining high accuracy with exceptional processing speed. Building upon the strengths of its predecessors, YOLOv8 leverages a refined architecture and innovative techniquesthatenhanceitsabilitytoidentifyandclassify objects across various scales. This model is particularly notable for its flexibility, allowing seamless integration into diverse applications such as autonomous driving, surveillance,androbotics. Witha user-friendlyinterface and robust performance metrics, YOLOv8 facilitates not only efficient inference on static images and video
p-ISSN:2395-0072
streams but also supports training on custom datasets, thereby broadening its accessibility to researchers and developers alike. The advancements in YOLOv8 underscore the ongoing evolution of deep learning models,highlightingtheirtransformativeimpactonrealworldapplications.
Furthermore, YOLOv8's user-friendly interface and robust performance metrics facilitate efficient inference on static images and video streams, while its support for training on custom datasets broadens its accessibility to researchers and developers alike. This capability enables practitioners to fine- tune the model for specific tasks, enhancingitsutilityinniche applications, such as wildlife monitoring or industrial automation. The advancements in YOLOv8 not onlydemonstrate the ongoing evolution of deep learning models but also highlight their transformative impact on real-world scenarios, paving the way for smarter, moreresponsive systemsinvariousindustries.Asthedemandforreal-time processing continues to rise, YOLOv8 stands at the forefront of innovation, offering promisingsolutions for enhancing safety, efficiency, and productivity in a rapidly changing technologicallandscape.
The components of YOLOv8, such as the backbone and neck. The Backbone component of YOLOv8 is fundamentally similar to YOLOv5. The backbone consists of modules like C2f and SPPF,whichextractfeaturesand ensure accuracy at different scales. The neck module incorporates feature fusion using PAN-FPN and incorporatestheideaofseparatingtheheadforimproved accuracy. Additionally, the C2f module enhances detectionaccuracybycombininghigh-level featureswith contextual information. At the end of the backbone, the SPPFmoduleisstillutilizedinYOLOv8toextractfeatures through three pooling operations, aiming to enhance the network'sreceptivefield.
SPPF is an element in YOLOv8's architecture. It uses pooling operations to extract features across various scales, enhancing the model's ability to capture context and spatial information effectively. SPPF is employed at the end of the backbone, improving the receptive field of the network. By aggregating information from different scales, SPPF ensures that YOLOv8 can accurately detect objectsofvaryingsizesandcomplexities.TheC2fmodule involves dividing the feature maps of the base layer into two parts, merging them hierarchically, and using two convolutions.
FIG 6.4 YOLOv8 architecture
International Research Journal of Engineering and Technology (IRJET) e-ISSN:2395-0056
Volume: 11 Issue: 10 | Oct 2024 www.irjet.net
This approach optimizes computation by reducing gradient repetition and enhances the model's feature extraction capability. The C2f module plays a significant role in YOLOv8's backbone and contributes to its overall accuracy. It uses pooling operations to extract features across variousscales, enhancing the model's ability to capture context and spatial information effectively. SPPF is employed attheendofthebackbone,improving the receptive field of the network. By aggregating information from different scales, SPPF ensures that YOLOv8 can accurately detect objects of varying sizes andcomplexities.
CBS, which forms the backbone of YOLOv8, employs cross- stage hierarchy by splitting and merging feature maps, improving gradient flow optimization. This innovative approach reduces computational burden while maintaining or enhancing accuracy. YOLOv8 integrates CBS-inspired C2f modules, enhancing the performanceofitsbackboneanddrivingefficientfeature extraction.
6.5 INFERENCE AND REAL-TIME OBJECT DETECTION
Objective:
TodeploythetrainedYOLOv8modelforreal-timeobject detection in autonomous driving scenarios, ensuring accurateandefficientdetectionofobjectsindiverseand dynamic environments. Inference and real-time object detection are crucial components of the image recognitionpipeline,wherethetrainedYOLOv8modelis deployed to analyze and interpret the input images captured by the onboard cameras of the autonomous vehicles.
The goal is to accurately detect and classify objects in real- time, enabling the autonomous vehicles to make informed decisions and navigate safely in complex and dynamic environments. Inference and real-time object detection are crucial components of the image recognitionpipeline,wherethetrainedYOLOv8modelis deployed to analyze and interpret the input images captured by the onboard cameras of the autonomous vehicles.
Model Deployment
Once the YOLOv8 model is trained and optimized, it is deployed on the onboard computer system of the autonomous vehicle, where it will process the incoming imagesfromtheonboardcamerasandperformreal-time objectdetection.
Input Image Processing
Before feeding the input images into the YOLOv8 model for inference, the images undergo the same preprocessing steps as during the training phase, including resizing, normalization, and data augmentation. This ensures that the input data is
p-ISSN:2395-0072
consistent and compatible with the model's input requirements.
Object Detection and Classification
The YOLOv8 model processes the input images and predicts the bounding boxes, objective scores, and class probabilities for each detected object. The model is capable of detecting multiple objects of different classes simultaneously in a single pass, making it highly efficient forreal-timeobjectdetectionindynamicenvironments.
Post-Processing and Filtering
The raw detections generated by the YOLOv8 model may contain false positives and redundant bounding boxes.
Therefore,apost-processingstepisappliedtofilteroutthe detections below a certain confidence threshold and perform nonmaximum suppression to remove redundant bounding boxes and retain only the most accurate andrelevantdetections.
Object Tracking and Localization
In addition to object detection, object tracking and localization techniques can be employed to track the detected objects over time and predict their future trajectoriesandmovements.Thisisparticularlyusefulfor maintaining awareness of the surrounding objects and predicting potential collision risks or interaction scenarios.
Integration with Autonomous Vehicle Control System
Thedetectedandclassifiedobjectsareintegratedintothe autonomousvehicle'scontrolsystemtoinformandguide the vehicle's decision-making processes, such as navigation,path planning, obstacle avoidance, and traffic rule compliance. The object detection results are used to adjust the vehicle's speed, direction, and behavior to ensure safe and efficient navigation in complex and dynamicenvironments.
Real-Time Performance Optimization
To achieve real-time object detection with minimal latency and high throughput, the YOLOv8 model and the overall image recognition pipeline are optimized for performance, leveraging techniques such as model quantization, pruning, and acceleration using hardware accelerators (e.g., GPUs, TPUs) and optimized software libraries(e.g.,TensorRT,OpenVINO).
Monitoring and Logging
To ensure the robustness and reliability of the object detection system, comprehensivemonitoring and logging mechanisms are implemented to continuously monitor the system's performance, detect anomalies, and log the detectedobjects,theirclassifications,andtheirattributes foranalysis,debugging,andimprovementpurposes
International Research Journal of Engineering and Technology (IRJET) e-ISSN:2395-0056
Volume: 11 Issue: 10 | Oct 2024 www.irjet.net
VII. CONCLUSION & FUTURE WORK
8.1 CONCLUSION
The project successfully integrates the YOLOv8 algorithm within autonomous vehicles, utilizing libraries such as Ultralytics, NumPy, and OpenCV to effectively process visual data for real-time object detection and tracking.
This integration has resulted in a measurable improvement in navigation safety and efficiency, with YOLOv8's rapid and accurate data analysis from vehiclemounted cameras providing critical situational awareness. This capability ensures that autonomous vehicles can react quickly to changes, reducing the likelihoodofaccidents.
Despite these advancements, our system currently lacks multi-modal sensor fusion. Future enhancements will explore the incorporation of additional sensing methods, suchas radar and LIDAR, to improve the robustness and reliability of our detection system under diverse conditions. This advancement will continue to drive the progress of autonomous vehicle systems, contributing to saferandmorereliabletransportation.
8.2 FUTURE WORK
Future work will focus on optimizing the algorithm, improving the training dataset, and implementing realtime processing optimizations to make it suitable for onboard processing in autonomous vehicles, thereby advancing the development and deployment of autonomousdrivingtechnologiesandcontributingtothe realization of safer and more efficient autonomous transportation systems. Additional efforts will be directed towards enhancing the algorithm's object recognition capabilities, reducing false positives, and improvingthedetectionofsmallerobjectssuch as traffic lightsandroadsigns.
This will involve further refinement and optimization of the YOLOv8 algorithm, as well as the development of more comprehensive and diverse training datasets to improve the algorithm's generalization capabilities and ensure consistent and reliable performance across diverse driving scenarios. Moreover, the computational complexityassociatedwithreal-timeprocessingofvideo feeds will be addressed by implementing advanced optimization techniques and leveraging the capabilities of modern hardware platforms to reduce computational overheadandimproveprocessingspeedandefficiency.
This will enable the efficient implementation and operation of the image recognition system onboard autonomous vehicles, ensuring smooth and reliable performance in realworld driving conditions. While our project has made significant strides in utilizing YOLOv8 for visual object detection, future enhancements could explore the integration of multi-modal sensor fusion. This would combine the strengths of various sensing
p-ISSN:2395-0072
technologies to create a more robust and fault-tolerant system, ensuring reliable vehicle operation even in challengingenvironmentalconditions.
Figure 9.1 Image Detection
Figure 9.2 Image detection
Figure 9.3 Video detection
Figure 9.4 Video Detection
International Research Journal of Engineering and Technology (IRJET) e-ISSN:2395-0056
Volume: 11 Issue: 10 | Oct 2024 www.irjet.net p-ISSN:2395-0072
REFERENCES
1. Redmon, J., Divvala, S., Girshick, R. and Farhadi, A. , 2016. You only look once: Unified, real-time object detection. In Proceedings of the IEEE conference on computervisionandpatternrecognition(pp.779-788).
2. Redmon, J. and Farhadi, A., 2018. Yolov3: An incremental improvement. arXiv preprint arXiv: 1804.02767.
3. Bochkovskiy, A. , Wang, C.Y. and Liao, H.Y.M., 2020. Yolov4: Optimal speed and accuracy of object detection. arXivpreprintarXiv:2004.10934.
4. Tan, M. , Pang, R. and Le, Q.V., 2020. Efficientdet: Scalable and efficient object detection. In Proceedings of theIEEE/CVFconferenceoncomputervisionandpattern recognition(pp.10781-10790).
5. Hazarika,A.,Vyas,A.,Rahmati,M.andWang,Y.,2023. Multicamera3DObjectDetectionforAutonomousDriving Using Deep Learning and SelfAttention Mechanism. IEEE Access.
6. Lee,J.andHwang,K.I.,2022.YOLOwithadaptiveframe control for real-time object detection applications. Multimedia Tools and Applications, 81 (25), pp.3637536396.
7. Zhang,Y.,sun,Y.,Wang,Z.andJiang,Y.,2023.YOLOv7RARforUrbanVehicleDetection.Sensors,23(4),p.1801.
8. Chen, X., Ma, H., wan, J., Li, B. and Xia, T., 2017. Multiview3dobjectdetectionnetworkforautonomousdriving. InProceedingsoftheIEEEconferenceonComputerVision andPatternRecognition(pp.1907-1915).
9. Lou,H.,Duan,X.,Guo,J.,Liu, Fl.,Gu,J.,Bi,L.andChen, H., 2023. DCYOLOv8: small-size object detection algorithm based on camera sensor. Electronics, 12(10), p.2323.
10.Jebamikyous, H.H. and Kashef, R. , 2022. Autonomous vehicles perception (avp) using deep learning: Modeling, assessment, and challenges. IEEE Access, 10, pp.1052310535.
Figure 9.5 Live Detection
Figure 9.6 Live detection