Parametric Study of Autonomous Vehicle Model with Traffic Simulations by CART Analysis

Coleman Ferrell, Matthew Carroll, and Jinkun Lee

Abstract

Autonomous vehicles (AV) are becoming more prevalent on our roadways, and the saf ety of driving alongside AVs is a growing public concern. AV developers tend to program a conservative self-driving algorithm in order to ensure public saf ety and avoid legal problems. However, a conservative position can signif icantly worsen traffic f low when AVs are introduced into the traffic network. Therefore, there is a need f or a public discussion and agreement so developers can take a more aggressive approach to make AVs drive seamlessly alongside human drivers, while also maintaining saf ety. With the objective of preventing vehicle collisions while maximizing traffic efficiency, the control parameters of AV models are identif ied, analyzed, and designed inside a traffic network simulation within the Greenville, NC network. Specifically, machine learning is utilized to create classification and regressions trees (CART), which assist in determining the values of the parameters that achieve the target variables. The outcome of this work, the set of optimal control parameters of AVs determined f rom this research, will be the decision basis when traf fic-related authorities, AV developers, and public representatives are trying to reach a consensus and discuss regulations f or all AVs.

Introduction

The quantity of AVs with driving automation is rapidly increasing, thus traffic efficiency and perf ormance will drastically change in the near f uture. The Society of Automotive Engineers (SAE) establishes distinct levels of automation in AVs. In general, an AV is considered a vehicle with a driving automation level 3 or above, which is def ined by SAE as any vehicle where the driver is not driving when the automated driving f eatures are active (SAE International, 2021). Investigation needed to be conducted urgently to prepare for the integration of AVs onto the roadways. The inclusions of AVs and the ef fect on traffic dynamics can be modeled using traffic simulations. This research uses the traffic simulation platf orm, Simulation of Urban MObility (SUMO), to simulate a mix of conventional and AVs on the Greenville, NC network. SUMO (Lopez et al., 2018) allows f or the microscopic simulation of traffic, meaning individual vehicles are simulated

in the model. Additionally, SUMO can output the macroscopic properties of the network, which is critical in evaluating traffic performance and safety. The network data generated in SUMO is collected f rom the geographic database OpenStreetMap (OpenStreetMap, 2021). Figure 1 displays the Greenville, NC road network generated for traf fic simulation in SUMO.

Figure 1: Greenville, NC Road Network in SUMO

AV developers determine specific parameter values in the self -driving algorithm, such as distance to maintain between vehicles or allowable maximum vehicle acceleration/deceleration. The default values of these parameters f or AV models are basically provided by SUMO, but these values can be modif ied. If any parameter value in the AV algorithm is selected only f or the traffic performance, it may induce collisions between AVs and other vehicles that may catastrophically result in a temporal network disruption. Any network disruption results in enormous personal and public costs directly or indirectly. Therefore, no collisions need to be set as a hard constraint when trying to decide AV parameter values to improve the traffic network. Therefore, the AV developers tend to take a conservative position and select collision avoiding parameters to prevent legal issues, leading to deteriorating traffic flow rather than conf orming to it. However, if the parameter values f or no collisions are overemphasized, they may decrease the traffic performance. Think about a road network full of novice drivers who keep too much gap between vehicles and decelerate too of ten to maintain the distance. Decreased traffic performance means increased travel time on the roads. Additionally, longer travel time may also lead to more pollutant emissions, which is another important issue regarding the environmental sustainability. Therefore, we aim to set the boundaries or limits of specif ic parameter values that are related to collisions. These boundaries will def ine a f easible region when we search optimal parameter set values to achieve best overall traffic performance. The prospected outcome of this study, a comprehensive understanding of the behaviors of AVs mixed

with human drivers, will be the steppingstone f or AV developers and traffic related authorities to decide the allowable design parameter limit when developing or regulating the AV driving algorithm.

Methods

There are numerous model parameters in the vehicle models used within SUMO that can be adjusted, but particular parameters have more inf luence on the overall traffic perf ormance than others. In order to achieve better traffic performance, we f irst identify the most inf luential parameters based on the Pareto rule then search an optimal set of parameters by using optimization algorithm, such as genetic algorithm (GA). Before searching the optimal parameter sets, we need to f ind the boundaries of parameters that are related to collisions, which is the goal of this study. A summary of the model input parameters and output measures along with their def initions described in SUMO are denoted in Table 1.

Table 1: Selected Inputs and Outputs with their Definitions from SUMO

We use a machine learning algorithm, genetic algorithm (GA), to minimize the density while maximizing the traffic’s speed and f low. In order to f ind an optimal parameter set, the algorithm modifies and tests the values of parameters per each simulation, records the outputs, and iterates this process until it converges. First, it f inds the parameter values in one of the simulations that optimize the objective output, then it runs another iteration and repeats the process until the optimal parameters are f ound. Although the GA searches a set of optimal parameter values, it does not deduce the threshold of the values that could achieve specific outputs or satisfy any hard constraint. Therefore, identif ying such thresholds, that is boundary values, is a critical step because it not only helps us avoid any violation of the hard constraint, but also it makes the search process ef ficient by reducing the f easible domain. For example, we may search candidate parameter sets that improve traffic performance while avoiding any single collision. In

doing so, a CART analysis can be instrumental in discovering the acceptable boundaries f or such parameters (Breiman, Friedman, Olshen, & Stone, 2017). We perf orm the CART analysis using the CART Regression tool in the statistical sof tware Minitab (Minitab, LLC, 2021). CART involves separating the data into similar groups based on correlations between dif ferent inputs and outputs. The CART tool executes the best split and repeats the process until the f inal collection of groups is f ormed. Multiple decision trees are generated with rules that def ine how the groups are split. The relative variable importance of each variable is signif icant to the classif ication of the groups. In this study, the relative importance is the impact each parameter has on the various traf fic outputs. The relative importance can also reveal the ef fect the variables have on the outputs. Thus, the CART analysis will tell which parameters have the most inf luence on the traffic performance outputs and reveal the value of the parameters that cause the outputs.

Analysis and Discussion

In the traf fic simulation, no collisions are a hard constraint because any optimal AV model parameter set will not be adopted if public saf ety is not guaranteed. Therefore, we f irst investigate the collision output. The training and test data set f or CART is f rom multiple simulations with randomly selected parameter values within the appropriate range f rom their default values. In this study, we used 1,550 parameter sets. To determine which input parameters heavily inf luence the number of collisions, we evaluate the relative variable importance of the variables. The plot of the R-squared values versus the number of nodes on the decision tree is in Figure 2. The optimal

Figure 2: Plot of R-squared Value based on Number of Nodes in the Decision Tree

decision tree in Figure 2 is def ined as the minimum number of nodes within one sigma f rom the maximum R-squared value. Although the optimal decision tree can suf ficiently describe the data, the decision tree with a maximum R-squared will describe the

parameter impacts more completely. So, we compare the results we discover f rom the optimal decision tree to the results f rom the maximum R-squared decision tree. Figure 3 displays the relative variable importance of the optimal decision tree and the maximum R-squared decision tree. The most inf luential variable in each tree turns out to be tau, indicating that the variability of tau has the most signif icant impact on the number of collisions. The variables that are the second most important have only slightly inf luenced collisions and are signif icantly less important than that of tau, so the f ocus will remain on tau. The graph of relative importance only indicates the magnitude of the impact, it does not show the direction of correlation between the variables.

Figure 3: Relative Importance of Input Parameters

(a) Optimal Tree

(b) Maximum R-squared Tree

We utilize the scatter plot in Figure 4 to learn the relationship between tau and collisions. The scatter plot reveals a nonlinear, negative correlation between collisions

and tau. Thus, the larger the tau parameter, we can expect f ewer collisions to occur. But a large tau value will lead to poor traffic performance, since it denotes a large time gap between vehicles. The point at which tau results in zero collisions needs to be determined.

Figure 4: Scatter Plot of collisions versus tau

The optimal decision tree we generate in Figure 5 assists in displaying this point f or tau. As seen at node 1, when the tau value is greater than 0.683, the average number of collisions reduces to under one collision. The average number of collisions is f urther reduced to 0.241 at terminal node 4 when tau takes on values greater than 0.799. Although, this terminal has a sizeable standard deviation of 3.619, indicating the potential f or the collisions to be more than one. This occurrence is rare, however, and the grouping in the decision tree can still be helpf ul in f inding a value of tau. Therefore, the tau value of 0.799 will still be used as the lower boundary when minimizing the number of collisions. From Table 1, tau is def ined as the driver’s desired time headway between the vehicle preceding and their vehicle. Therefore, a tau value of 0.799 seconds means the f ollowing vehicle should maintain a distance f rom the leading vehicle so that the f ollowing vehicle will come to a stop at the leading vehicle’s location in 0.799 seconds.

Figure 5 is the optimal decision tree because the tree has the smallest number of nodes while also staying within one standard deviation of the maximum R-squared value (Minitab, LLC, 2021).

Figure 5: Optimal Tree for Grouping with respect to Collisions

The maximum R-squared decision tree is in Figure 6. The terminal node with the minimum mean number of collisions is identical to node terminal 4 in Figure 5. Additionally, tau is the determining parameter, and the threshold value is 0.799. This indicates that the simplif ied, optimal decision tree contains accurate groupings and threshold values.

Figure 6: Maximum R-squared Tree for Grouping with respect to Collisions

Now, we explore the effect the input parameters have on the perf ormance-related outputs. When looking at the relative variable importance, the accel parameter was the most inf luential f or the performance-related outputs. However, unlike with the collisions, the second important variable had a substantial impact on the outcome. The parameters we f ound to be the second most important are seen in Table 2. tau was the most common secondary important variable, meaning tau also has a notable impact on the perf ormance-related outputs. This can be f urther illustrated in Figure 7, where the graph of relative importance of the output variable f low shows a large relative importance f or tau.

Table 2: Second Most Influential Variable for Each Output

Output Density sampledSeconds waitingtime Occupancy timeLoss speed entered flow

Variable tau tau minGap tau tau tau delta tau

Figure 7: Relative Importance of Input Parameters on flow

A scatter plot f or the tau values versus the output f low, shown in Figure 8. There is a weak, negative linear correlation between tau and f low. This f inding signif ies that lower values of tau improve traffic efficiency. Hence, the minimum value of tau discovered to avoid collisions, 0.73 seconds, will be the parameter value that optimizes traffic perf ormance while also ensuring public saf ety.

Figure 8: Scatter Plot of flow versus tau

Conclusion

Self-driving parameters of AVs will become a critical determinant of public saf ety as well as traf fic performance. The anticipated increasing occupancy of AVs on roadways needs a proactive investigation of their parameters because these parameters will determine their behaviors in the mixed traffic network. We run numerous simulations with SUMO and collected data with various input parameters and their corresponding outputs. Based on this data set, we perf ormed CART analysis to determine the most inf luential parameter, which turns out to be tau, to guarantee no collisions. The regression trees in the CART analysis assisted in pinpointing the threshold of tau f or no collisions. The f indings indicate that 0.799 seconds time gap should be maintained between vehicles. This outcome will support the decision of traffic-related authorities, AV developers, and public representatives to enact AV regulations that improve mixed traffic performance while ensuring public saf ety. 

References

Breiman, L., Friedman, J. H., Olshen, R. A., & Stone, C. J. (2017). Classification and regression trees. Routledge. doi: 10.1201/9781315139470 Lopez, P. A., Behrisch, M., Bieker-Walz, L., Erdmann, J., Flotter¨od, Y.-P., Hilbrich, R., Wiessner, E. (2018). Microscopic traffic simulation using sumo. In 2018 21st International Conference on Intelligent Transportation Systems (ITSC) (pp. 2575–2582). (ISSN: 2153-0017) doi: 10.1109/ITSC.2018.8569938 Minitab, LLC. (2021). Minitab 20.2. Retrieved f rom https://www.minitab.com OpenStreetMap. (2021). Planet dump. retrieved f rom https://planet.osm.org. https:// www.openstreetmap.org. SAE International. (2021). Taxonomy and Definitions f or Terms Related to Driving Automation Systems f or On Road Motor Vehicles. Retrieved f rom https://www.sae.org/standards/content/j3016202104/