Yuqiao
Prof. Dr.-Ing. Frank Petzold
Ivan Bratoev, Frank Petzold
Yuqiao Sun, Qingyu Xue
03751835, 03752794
TopicAs society develops, people‘s demand for quality urban space gradually increases, as it is closely related to a sense of security and well-being in life. People have the impression that a comfortable neighbourhood is often quiet, tree-lined and tidy, while dense traffic and noisy people are uncomfortable. This evaluation has important implications for neighbourhood transformation and for comfort evaluation on a larger scale. Much of the current research is based on manual photographs and manual assessments, which are time and labour intensive and have two disadvantages due to the complexity of defining neighbourhood amenity: firstly, the amenity of the same street often varies for different subjects. Secondly, differences in the perceptions of different evaluators can also reduce the accuracy of the results.
In recent years, street view data, represented by Google Maps, has made it possible to obtain street view images efficiently and on a large scale. At the same time, machine learning algorithms using deep convolutional neural network frameworks have been able to accurately process street view images in depth, effectively identifying the sky, pavements, lanes, buildings, greenery and other elements in the images. Building on the streetscape database and unified evaluation criteria, artificial intelligence will have the opportunity to play a role in the field of urban spatial evaluation.
In this paper, we take the evaluation of the spatial comfort of urban streets as the object of our research, firstly by scientifically formulating evaluation criteria and quantifying them as a quantitative assessment of the detected objects, secondly by realising the recognition of target objects based on the yolov5 algorithm, and finally by drawing a heat map of comfort in an urban area in Munich as an example and giving suggestions for quality improvement based on the results.
The scope of the study of street comfort in this paper is the area around the Technical University of Munich in the centre of Munich for two reasons: Firstly, people themselves already have comfort perceptions for familiar spaces and can make judgements about the accuracy of the AI artificially. Secondly, the area is rich in building types and street scales, and the accuracy can be judged based on different scenarios.
There is already a large body of literature that proposes criteria and methods for evaluating the quality of public spaces, and for our research we refer to a paper called Evaluating Public Space. Two of the points are instructive: the authors refine the quality of public space into aspects such as inclusiveness, support for meaningful activities and safety, so that the evaluation reflects the spatial characteristics and they have the same overall score, which means that they have an equal impact on the quality of the space. At the same time, in the questionnaire on spatial quality, the authors have given different weights to the scores of each influencing factor, with greater weights for important objects, so that the influence of these factors is unequal.
Taking into account the current urban situation in Munich, comfort was divided into safety, walkability and street dynamics and an evaluation form was developed. Quantifying the evaluation criteria as a detection of the number of objects will help AI recognition, so we identified the study objects as: Person, Chair, Bus, Truck, Traffic Light, Bench and Tree.
We selected a total of 102 points in the centre of the street on Google Maps and extracted the corresponding street view photos.
The paper discusses the evaluation and influencing factors on the quality of urban public space, proposing criteria and methods. Although there are differences in the subjects studied, the development of the evaluation system is worthy of reference.
The public space index in the questionnaire is based on: variables, weights, scores and measures. With the same base score, things that have a stronger impact on the quality of public space are weighted more heavily. To make it simpler for the reader to make an assessment, the scoring takes the same grading.
There are three tiers of the rating system, with the Comfort Index covering Safety, Walkability and Street Vitality, which in turn include Person, Chair, Bus, Truck, Traffic Light, Bench and Tree.
All three areas are scored out of 30, with the object scores being weighted to take account of differences in impact. As the AI directly corresponds the number to the scoring, the weights do not need to be written out separately.
The red dots are the locations where data was collected in the Street View map. The city streets through which vehicles can pass have been selected and the red dots are located in the centre of each street, with a total of 102 data.
What we wanted was the comfort of a typical street in Munich. This means that it will satisfy several preconditions:
1.Excluding the square, the inner space of the block and other spaces.
2.The street scale is similar - that is, there will be no special buildings such as skyscrapers, urban expressways, highways, etc.
3.The so-called typical space is an area similar to the old town of Munich
4.Factors such as seasons are not taken into account
5.Special holidays, memorial buildings, etc. will not be considered
On the premise that these conditions are met, we will randomly select Google Street View for data processing within the scope of the Technical University of Munich.
In addition, these street scenes are all street scenes in the direction of the vehicle, so as to ensure the similarity of the samples.
Before doing object detection, we need to „teach“ the software how to recognize trees, because Yolov5 has no function for tree detection. Therefore, we selected the street scenes of several large cities in Germany, took the same angle, manually selected the shape of the tree, and then gave the software data recognition.
After obtaining the identification data (weights), we conduct a detection study on the building. First, weights files cannot be merged, so we need to identify buildings twice. We merged the two files, and then handed it over to grasshopper for image processing to get the image we needed.
The first is the workflow for training the recognition tree Then there is the workflow of identifying images and processing the data using the trained model and known models
The core of the entire data processing is Yolov5-centered artificial intelligence. This is an artificial intelligence algorithm based on CNN (Convolutional neural network). An image is composed of pixels of different colors, where each pixel contains a pixel value and three channels (RGB). If image recognition is required, the computer needs to know the pixel distribution of an image. But a picture has a lot of pixels, so we can use the method of segmenting the image to process. Then we can multiply each segmented pixel with a matrix, and then operate layer by layer to find the mathematical distribution we need.
After this, we need a large number of samples for training, so that the computer will know what kind of pixel distribution we need.
Our group uses a Yolov5 model based on this technique. This model has been developed over many generations so that it can not only perform image processing, but also perform fast image recognition on video. We are using version 6.0 of it. We found the version we need from GITHUB. But running such a computer program requires a specific runtime environment. We use pytorch to run this software and install a bunch of virtual environment files. In this process we encountered many difficulties, especially the compatibility of versions. We need to make repeated changes to the virtual environment to correct all bugs. In addition, in the existing WEIGHTS file, the identification of the „tree“ we need is missing. So we need to train ourselves.
It‘s worth noting, however, that we don‘t need computers to recognize almost any object that meets the biological definition of „tree“ on a fairly broad world scale, but rather typical European street trees (large shrubs and trees).
When we operate this system, the most important thing is to install various libraries, find suitable data, and find or train the weights file by ourselves. For the various values obtained, as well as the various values and data that appear in the operation, we can add various data processing codes for data processing by ourselves. For example, after obtaining the calculated data, we will introduce the processing code for the data later.
CNN is not a system that can be learned quickly, but its core code is not very large, but can be written in a few hundred lines (or even less).
An AI project prototype like ours is mainly divided into four parts.
The first part is to collect the data, including the required training data and finally the Munich Street View data for analysis. We found specialized datasets on the web that included street view data for major cities in Germany. The data used for Street View training should theoretically be in the thousands. At the same time, even with a high-performance RTX graphics card, we need a lot of time for training. However, in this project, because of the high similarity of the street views and the similarity of the content to be identified, the data for our training is greatly reduced, and at the same time, a very high degree of confidence can be obtained.
The second part is training. To train the model, you need to manually add model information, that is, use labelimg to select the training data set, and select the trees one by one. Then create an identical folder corresponding to the pictures folder. In this way, the information of the picture and the tree in it can be in one-to-one correspondence. It should be noted that the graphics card can be used for computing acceleration, that is, installing the CUDA file package. When the graphics card is used for acceleration, its computing training speed will be significantly faster.
After the training, we will get the recognized image, and two training models. Through these two models, we can proceed to the next step of detection.
The third part is the detection of the model. Through the parameter control in the code, we can set basic settings such as probability filtering, output results, and detection targets for the pictures we need to detect.
Identify Trees
Identify Cars, Pedestrians, Trucks, Benches,
The fourth part is data processing. In this part, we will visualize the data in the obtained data (excel) table, so that we can get the final comfortable distribution picture.
The data processing code can be roughly divided into the structure shown on the right: we first import the txt file, and then select the first column of data we need. Then use the count function to calculate the number of objects we have identified. such as cars, pedestrians, etc. After obtaining the data we need, we use a function to classify and weight each data. We divided all data into three categories, namely street vitality, street safety, and pedestrian comfort mentioned above. Then add the three values to get the final comfort.
After we get comfort, we can derive comfort. Then into grasshopper for processing.
Safety score = Car score + Bus score + Truck score + Traffic light score
Vitality score = Person score + Chair score
For this prototype, we mainly need to pay attention to the following four aspects:
The first is the installation of the environment. When setting up the environment, we need to use the virtual environment, that is, to integrate all the libraries and plug-ins required for the code to run. It should be noted that the compatibility of various software versions, once a version of the plug-in conflicts with several others, then you can consider replacing this version of the plug-in. In addition, the computer storage space required by the virtual environment is not small, so users are expected to pay attention in advance. Do not install virtual environments where storage space is too small.
The second is that the computer will crash due to insufficient memory during training. In this case, we need to adjust the batch-size - that is, make it smaller. This way, especially laptops won‘t crash or run programs due to insufficient memory. Another problem in the training process is the location of the image folder. Due to the problem of python language translation, we need to add r to its absolute path to tell the computer that it is an absolute path.
We encountered a problem when recognizing images - the recognition model (weights) files cannot be merged, which requires us to perform two detect. The first time identifies the content of the tree in the analysis file, the second time identifies people, vehicles, benches, etc., and then manually merges the two data files. We haven‘t found a better way yet.
Ultimately it‘s a matter of definition about comfort, walkability, safety. Computer science is quantitative science, but when it comes to this kind of executive feeling, it‘s hard to quantify. Therefore we refer to a paper and the quantized weights in it. But it is worth noting that some results should be changeable, that is, with changes in the results of the questionnaire or changes in people‘s living habits. Therefore, we believe that there is still room for improvement in the definition of the comfort function.
During the installation and modification of the yolov5 algorithm model, the running, modification and training of the program were the main problems encountered. The program has requirements for the virtual environment, the Python version, in addition to the fact that the program cannot directly integrate the weights of the different recognition objects, resulting in us having to output the results and then merge them, and finally, training the model takes a long time and our current recognition objects still need to be improved.
We trained the model on a recognition tree with a database of 100 Munich street images and other objects that came with the original yolo model. The results show that the model is good for significant objects, with similar accuracy for objects of different light and distance, but needs to be improved for small targets or targets that are obscured from each other, such as tables and chairs in front of a restaurant, benches partially obscured by objects, etc. The heat map drawn based on the current experimental data gives a more realistic picture of the street, but the state of individual streets is not universal due to the timeliness of the street photos, and the data needs to be adjusted
the current research and methodology still has some limitations. Firstly, the street view image data is obtained by using street view collection vehicles as the basis. Due to privacy issues, some of the streets are obscured on Google Maps, thus lacking the full street image data to reflect the real situation, which may affect the analysis results to some extent. This may affect the analysis results to a certain extent.
Supplementing this part of the images with manual collection would improve the accuracy. Secondly, in this study, the height of the streetscape data is the height of the vehicle-based streetscape collection system, which is slightly higher than the height of most people‘s eyesight, causing the recognition of the scene to differ from the real one, and the numerical weights of the system will be adjusted accordingly. In addition, based on the three existing aspects, we will increase the richness of the recognition objects, which on the one hand can improve the accuracy of the system, and on the other hand bring more application possibilities, such as planning the most suitable route for cycling according to the density of vehicles and comfort, recommending the visiting route for tourists according to the vitality of the street, and even adding the time variable so that we can analyse the change of comfort of the same street within a day.
Our research was divided into five steps: first we reviewed papers and developed comfort evaluation criteria, while going to websites such as github to learn about existing models, comparing them with our needs, identifying available programs and the parts we needed to train ourselves. The virtual environment was then configured in the computer and the yolov5 program was run. After successfully recognising objects using the existing models, the unwanted objects were removed according to our needs, while the Munich streetscape database was downloaded from the cityscape website and 100 of them were selected to train the tree recognition. For subsequent analysis we exported the recognition results as a txt file. Based on the evaluation system, we then wrote a part of the program for scoring the comfort of the corresponding locations according to the number of objects, inputting the previous results file and entering the walkability, safety, vibrancy and comfort scores of each location into an Excel sheet for the next step of the heat mapping process. We then built the grasshopper model, mapped the colours to the scores and created a heat map to represent the data visually. Finally we gave recommendations for improvements in some areas based on the results of the study.
