Performance Based Design: simplyCity by architekturinformatik

Student(s) Name(s) Chair for Architectural Department of Architecture Technichal University of simplyCityMunich Bausch, K., Clark Koth, S., Ficht, R., Kargruber, J. of Architectural Technical University of Munich

Informatics

Project Title

2 simplyCity Chair of Architectural Informatics Prof. Dr.-Ing. Frank Petzold Performance Based Design Gerhard Schubert Nick 03677494036482470367621603723047Förster

3 3432302928222016141210864 ProblemPrototypeConceptHypothesisResearchIntroduction1- Data Creating Dataset Labeling the Data Problem ReferencesContactCollaborationOutlookDiscussion2 Table of Contents

4 Introduction

In today’s world urban growth is not only an inevitable trend worldwide, but it also poses a key factor to sustainable development. As the human population is on a constant rise and globalization is shaping our daily life, urbanization is explosively emerging especially in newly industrialized countries. The United Nations projects that two-thirds of the world population will live in urban areas by 2050 [1]. While it supports economic growth and fosters education, progress and innovation [2], it has and will pose many challenges. Physical and mental health [3] as well as crime [4] are surely among the more obvious, but sustainable challenges have to be addressed equally. part in the equation. Fig. 1 illustrates 25 cities that have seen major growth over the last 25 years but that have also experienced a loss in urban density rethinking infrastructures. The last decades have seen all kind of concepts to create greener, healthier, smarter and more livable cities. [6]

While all these concepts are attempts to solving current problems, they all struggle with the same factor. The uncertainty of predicting the future. Making an educated guess on which urban design will succeed and what problems the realm of our long-term urban development. Will the city grow or stagnate? Will it densify or spread? In which areas of the city do we expect the most growth and what infrastructures are necessary to allow for this growth? All these and many more uncertainties make it nearly impossible to think and plan proactively.

5 Fig 1 Growth and sprawl of various cities mapped. Own illustration based on [5]

6 Research

Many projects try to grasp the complexity and uncertainty of urban development. But while urban development is a long process that can take decades to go from planning to completion, urban problems often occur and require solving in a matter of months. may be seen in the mobility infrastructure of nearly all urban areas worldwide, especially so in North America. While making the cities car-friendly and face a reality in which, due to climate change, we are rethinking if that same infrastructure is sustainable after all. As this is not a new discovery but in in changing this very infrastructure and the long way we still have to go, to Another example of how planed urban developments don’t meet the quick morphological requirements of cities is addressed in [10]. The paper examines and simulates the highly temporal dynamics of urban areas related to poverty, as example of an unplanned, impulsive development that reacts to real conditions and problems instead of constructed urban patterns. This rapid and unplanned urban development doesn’t only trigger social unrest and crime rates to skyrocket but also lays the very foundation of widening the inequality gap within the city, as [11] point out. them, a further problem of these reactions lays in the reasoning of the policy itself.

In his paper on Slow Growth and Urban Development Policy, Christopher Leo, as some others before him also, suggests that growth may be the single most important and prominent factor in the policy of North American cities. Therefore, he further argues that big as well as medium sized (population > 750,000 people) urban areas chronically view the absence of growth or even slow growth as undesirable condition. This unreasonable and obsessive actual behavior but imaginary growth development. The paper concludes a a cities will better serve the health and sustainability of that urban area then drastically spending money on infrastructure or economic development in the wish for growth that can neither be detected nor should be hoped for. [12] urban planning as a prolonged process and the ii) blindness and ignorance to underlying truths or at least a void and missing of information and guidance to establish a more rational policy making.

While this is an easy way to simplify the information, in the case of [13], the Not only is this a time-consuming task it is also highly challenging to recognize and simplify patterns in the urban complexity.

Many urban development research papers nowadays, use satellite images to detect urban and landscape changes of the past decades, to analyze growth patterns of cities. As [13] explains: By identifying the past urbanization patterns, policy makers and planners can gain better insight into the contributing factors that have resulted in the most problematic development patterns now and into the future. [...] to highlight the underlying process by utilizing satellite images between 1986 and 2002.

This doesn‘t have to only apply for urban growth alone. Evaluating growth in every aspect and scale and predicting growth patterns can always inform us about a possible future, and that always leads to a head start on reacting to problems that have not occurred yet or proactively making policies.

Hypothesis

If we had a better understanding of the future urban growth and development, we could plan more sustainable cities. Meaning, if we were to simplify the learning algorithms to help us grasp the complexity and make an educated prediction, we could focus less on understanding the past and focus more time on discussing the future.

Before formulating the general idea resulting from these challenges, it is important to state that it is generally possible to conduct complex spatio-

authors backgrounds and intention to inform the building sector, this of architecture, urbanistic, landscape architecture, civil engineering, housing market, and all respective consulting work in the realm of sustainable planning and building. A tool that needs no knowledge in detecting or understanding urban growth patterns and trends, but that predicts urban growth with the aid Furthermore,simplyCity.

Fig 2 Example of a spatio-temporal prediction [14]

9 In [14], the authors introduce a spatio-temporal neural network that is able to forecast several time series of spatial processes, for example wind speeds water temperature (left) is compared to the prediction (right) using the neural network developed by the authors. It is clearly evident that there is a great correlation between reality and prediction. Similar functioning concepts of spatio-temporal predictions via machine learning can be found in [15], [16] and Given[17].the

addressing the mentioned user groups, we chose the interface that are further addressed in the outlook, this paper also shows the limits of environment.

10 ConceptImaginedScenarioSateliteDataImageseriesofcitygrowthofthepast.OwnDatasetImageseriesofslime-moldgrowth. LabelingLabelingoftheimagesintopixel-list.Input Images Image series as training data for growth recognition. Later only one input image. Adjusted Scenario JPEG

11 Labeled Pixel List Values between 0 and 1, as input for the Neural Network. Neural transitionalFakeSpatio-temporalNetworkPredictionrandompredictionwhileresultsfromtheneural PredictedPredictionfuture as labeled pixel list translated into a visual depiction. Output byHighlightedFuturechanges,comparingtheoutputtotheinputimage.Fig3 Concept diagram

1Prototype required a work around solution, as illustrated in the concept diagram. that could serve as training data for our neural network. In general, gathering of large amounts of data is possible, as there are many sources that provide satellite images with high resolutions. The major limitation with these data, in order to be useful for our project, is that they do not represent continuous timelines of city growth, which is essential for the intended prediction task. Furthermore, the satellite images that we could manage to acquire, were not yet labeled. As the labeling itself turned out to be a lot more work than we originally expected and would have been a task for itself, in terms of image processing as well as understanding of urban structures, our focus shifted away from satellite images. We decided to on the one hand create our own dataset and on the other to minimize the complexity of factors that weigh into the urban morphology.

The following discusses both problems further and shows our workarounds as temporary solutions, that would have to be improved in a further continuation of the project.

13 2 it was tried to use the Grasshopper plugin Crow to make a spatio-temporal prediction. Due to general limitations of Crow regarding computation power, networks to model spatio-temporal dependencies, a prediction could not be obtained. After further research and gaining a broader understanding of neural network in Python based on an approach explicitly created for spatiotemporal forecasting. The result of this approach seems to represent a still focusing on the advantages of outsourcing the neural network in Python and using Grasshopper only as a tool of communication between the input/output and the neural network as well as a tool for visualizing the prediction.

Problem 1 - Data pieces. Therefore, they might be able to recognize similar patterns in data not presented to them before. In this context, a large enough data set is seen as a prerequisite to create reasonable results. In our case, reasonable predictions of Today,growth.there are plenty of sources for present-day satellite images publicly toolbox for accessing and tweaking combined multispectral satellite imagery from multiple earth observation missions of the entire globe. Satellite images in general contain a variety of multispectral images. To create training data from that, these dimensions and therefore complexity needs to be reduced. The color values should therefore be broken down to only one dimension, the grey-scale-value of the pixel ranging from 0 to 1. In this way, non-urban can be labeling, but unfortunately only for present day datasets and not for a historic timeline of images. To create our training datasets, we would have needed to create this labeling by our owns. This would need the implementation of a complex clustering algorithms, like for example the one proposed by [19]. Having a labeling algorithm set, we would need historic satellite imagery. One of the longest available consistent datasets for satellite images is the data from the Landsat 5 missions which date back from 1975 to 2013. Unfortunately, due to quite long revisit times, for most places there is only about one picture per year available. To train a neural network, this is not close to being enough.

Fig 4 Physarum Polycephalum before growth.

Usually, thousands of datasets are necessary to train such a network.

As conclusion of our research about the processing of satellite images, we to being prioritized. Before someone puts energy in creating a realistic representative training dataset.

When thinking about what to use as a sample dataset we wanted to use We came across the slime mold physarum polycephalum, seen in Fig.4. Physarum is commonly used as a study object in many disciplines. Cultivated on a culture medium in a petri dish, Physarum tries to optimize its mold body in order to obtain food sources and large as wide as possible. The estonishing part is, that it can learn and adapt over time to work as highly where it dosen‘t.

Outline

To gain a large enough and complete data collection, it was important to set up a stationary system that would continuously take pictures of the slime mold growth without interruption. To ensure a homogenous dataset and at least 1000 images depending on the speed of growth. The data collection was carried out 3 times, before getting the expected result, due to various problems with the lighting, right moisture level and image time interval.

Methods

The set up took place in a completely dark testing laboratory at the TUM create an evenly distributed and scattered indirect light, that wouldn’t throw shadows, as seen in Fig. 5. The LED lighting was furthermore set to contain a high blue value of roughly 7000K. This was so the image would display a high contrast between the yellowish colors of the slime mold and the otherwise white surfaces, as seen in Fig. 6. We used a Canon EOS 90D with a lens set to 40mm focal length (Fig. 7). Further settings included a light sensitivity of ISO400, an aperture of F11 and a shutter speed of 3.2/1s. The alternative to these settings would have been brighter light conditions and therefore quicker shutter speed, higher F and lower ISO values. While this would have produced a better quality in images we chose not to go with those settings as the slime mold is very light reactive and limits the growing speed when exposed to bright conditions. The camera

Creating Dataset

17 was placed on a tripod with a remote timer for the shutter. While choosing a collections, as the ideal interval of capturing the growth.

Fig 5 Setup lighting. Fig 6 Data example. Fig 7 Equiptment.

The slime mold was contained in a circular glass petri dish of 90mm diameter to contain water for as long as possible before moisturizing it again. All slime molds except the last, dried out at some point of the data collection, bringing the growth to an end promptly. To activate the slime mold, we added few water droplets on to the mold itself

18 Outcome 2,686 homogeneous images. Critic Throughout the data collection we recognized that the locations of the nutrients play a vital role in determining the speed and pattern of the mold growth. When working with the images as a substitute for the satellite images, we have to remember that these points of interest do not equal real world points of interest in the growth of urban areas. While this poses an interesting without reference. Any other placement would have most likely resulted real world growth patterns of cities. When compared to urban growth the While it strengthens its infrastructure with every further nutrient it reaches it completely abandons the core center point it originally started at. The circular scaling, we see in urban growth from a center point outward is poorly to not at all resembled through the slime mold growth. It is merely used as a tool of abstraction for growth patterns.

19 Fig 8 Datacollection.

Labeling the data

In the Grasshopper tool, the individual images are imported in sequence and divided into the RGB channels and split into two clusters using Owl‘s K-Means Clustering component. This component uses the K-Means algorithm, which divides a similar group into K non-overlapping clusters [20]. Since we want to distinguish between slime mold and non-slime mold, K in our case is 2.

To analyze the growth of the slime mold using the Neural Network, the Problem 1. For this purpose, we created a grasshopper tool that converts our color images into dichromatic black and white images. However, to reduce the noise in the image and convert the images more accurately, we cropped the images to the necessary boundary of the petri dish, color corrected the images and increased the contrast with Photoshop, as seen in Fig. 9. This step is not necessarily considered to be integrated in any future script but a tool become sharper and noise in surrounding pixels doesn’t translate as easily.

Subsequently, the generated clusters are statistically evaluated, whereby the number 0.01 (which corresponds to the color black) is assigned to the respective larger cluster and the number 0.99 (which corresponds to the color white) to the smaller one, as seen in Fig. 10. In our case, the non-slime mold always represents the larger cluster. By labeling them with the number 0.01, i.e., black, the average of the input is closer to 0, which accelerates learning according to [21]. However, it should be noted, that this cluster size ratio may change over a longer observation time series. If the slime mold grows to such an extent that it takes up more than 50% of the image, it is no longer the

21 background but the slime mold that is assigned the number 0.01 and thus Finally,black. the pixel data of the generated dichromatic images are saved in an the Neural Network for the growth analysis. Fig 9 Color correction. Fig 10 Labeled image.

Initially, although presented with possible approaches how to perform spatiotemporal predictions in [14], [15], [16] and [17], it was decided to use the Grasshopper plugin Crow to create a neural network structure. This was due to from the separately generated dataset of slime mold growth. Grasshopper in this case should serve as a platform, to ensure a user-friendliness towards professions that already know and use it, in the realm of architecture and urban planning, as mentioned earlier.

Problem 2

For example, the training input data can consist of a list of pixel-values that represents the output. The more examples the neural network can train on, of the neural network, for example the amount and typology of layers and networks using the backpropagation algorithm and consisting of multiple solve complicated learning tasks.

Crow can perform supervised machine learning via the backpropagation algorithm. As described in [22], supervised learning describes a machine learning procedure in which the neural network is presented with exemplary

These capabilities of Crow led to the misconception that it can produce reasonable predictions on spatio-temporal scales. With this in mind, the concept of input and output structure in Crow was thought of as follows: input and output. In this case, each input represents the pixel-representation (9409 pixel-values) of the slime mold at time t. Since the output of the network should also show an image, each output represents the pixel-representation of the slime mold at time t + timestep. The last timestep is left out from training the network, in order to use it for testing. This network structure did not create any meaningful results. When looking neurons), testing the network with an unknown image from the end of the time series (Fig. 11) results in an image (Fig. 12) that seems to approximate the average of all images used for training (Fig. 13). It is in no way depicting the real situation that should have resulted (Fig. 14).

24 Fig 13 Average of all images. Fig 11 Input image. Fig 12 Prediction Crow. Fig 14 Actual growth.

On the Keras website, a code example [25] was found that is based exactly on this “Convolutional LSTM” approach for predicting the next frame of an image sequence. Keras [26] is a deep learning framework acting as an API (application programming interface) for Python. In this way, the machine learning is no longer connected to Grasshopper and therefore performs way faster. With small changes to the code example, the labeled dichromatic images could be processed and used as an input for the spatio-temporal prediction model. Out of the 2686 images, 2600 were used to train and validate the model (90% training and 10% validation). 86 images were used for testing the trained Duemodel.to the long calculation time, only a small number of cycles (100) within (Fig. 15) as input for the trained model, Fig. 16 results as an output. As to be seen, a shape is distinguishable. Even though the contrast between

The attempt of increasing the network accuracy by increasing the number exceeded the capacity of Grasshopper. As a result, a prediction could not be task and the computational limitations of Grasshopper. The neural network architectures that helped predict spatio-temporal patterns in [14], [15], [16] and [17] could not be transferred to the setting of this project, as the necessary knowledge of ANN in order to fully reproduce the frameworks described in these studies was missing. Nevertheless, they served as a starting point of further research regarding spatio-temporal prediction. In [24], the authors propose an approach of predicting spatio-temporal relationships via a so-called “Convolutional LSTM”. As described in the study, it is combining the advantages of LSTM (Long Short-Term Memory) networks that are able to detect temporal correlations and convolutional networks that on the other hand can distinguish spatial relationships within data.

In an attempt to increase training accuracy and reduce calculation time, the dichromatic images were scaled down to 50x50 pixels, resulting in only a fourth of pixel values (2500 instead of nearly 10000). Due to the reduced information content, it was possible to increase the number of training cycles to 250 with a simultaneous reduction of simulation time. The output for the same test image as above (but scaled down, Fig. 18) can be seen in Fig. 19. The result generally seems slightly more “sharpened” and with the actual next timestep (Fig. 20). All that indicates a higher, but still limited prediction power. Fig 16 Prediction output 100x100 pixel.Fig 15 Input image 100x100 pixel. Fig 17 Actual growth 100x100 pixel.

26 the pixels is very low, the result resembles roughly, at least shape-wise, the actual image that should have been predicted (Fig. 17). It seems like the trained model in some sort recognizes time series patterns. Nevertheless, it becomes obvious that the predictive power is quite low.

Fig 19 Prediction output 50x50 pixel.Fig 18 Input image 50x50 pixel. Fig 20 Actual growth 50x50 pixel. Another limitation of the model lies in its way to process sequence data. The model is created in a way that it demands a large number of small sequences (chunks of a few frames), instead of a whole long sequence to make a prediction. The reason for that characteristic is unknown but may lie Additionally, this study wasn’t able to extract multi-timestep predictions from the model, given the equally limited knowledge of ANN within the Keras timeframework.toset up a more elaborate „Convolutional LSTM” model, it should be possible to create a more accurate spatio-temporal prediction considering long-term dependencies, ideally reaching not only one but several timesteps into the future.

ML algorithms not using the whole computational power of the machine they better on these issues. Therefore, Python can act as a basis for creating neural scriptsnetworks.within Grasshopper. Unfortunately, this so called GhPython is limited to the IronPython framework, which has a quite limited functionality. In order to implement ANNs, one must use a full-format python distribution.

Grasshopper could then still analyze the outputs, compare the prediction with the status quo and illustrate the results. This would bring together the user friendliness of grasshopper with the computational power of present-day machine learning algorithms.

28 Discussion

Grasshopper has the advantage of being user friendly and broadly distributed, thereby we suggest keeping the grasshopper scripting as it is in our prototyping and only outsource the Machine learning part. Using sockets in Python would allow the python script to communicate with the grasshopper script. Simple implementations of UDP sockets for Grasshopper are available as GhOwl on Food4Rhino or can quite easily be implemented in a GhPython script.

30 Outlook

While a functioning prototype still seems far-fetched, the approach described in this paper can serve as a model to demonstrate the general concept.

Fig 21 Theoretical prediction. Fig 22 Focus areas of interest.

dichromatic time series of images represented by pixel-values between 0 and 1. Further, a theoretical prediction image (Fig. 21) as well as a focus image (Fig. 22) can be displayed, representing the deviation of the last time step and the prediction.

This would require even more research in spatio-temporal forecasting precise insight into the temporal relationships within urban areas, on large scales of entire cities as well as on small scales of enclosed neighborhoods.

As described in chapter ‚Prototype‘ and ‚Creating Dataset‘, the slime mold is not used as an equal depiction of city development, but as an abstracted way to model growth patterns in general. The complexity of city growth inheriting data, only representing if the mold is spatially existent or not.

When willing to create a broader and more diverse prediction of changing cities, it is necessary to implement the multidimensionality within urban environments into the machine learning frameworks. Instead of only stating the binary condition (city/not city), one could for example implement temporal data about population densities, building height, concentration of amenities, etc. into each pixel of a city. As a result, one would perform a prediction not only for one but a variety of dimensions.

Collaboration

We want to thank the following people for the support we recieved throughout this project: Lukas Liebel (TUM) For his knowledgeable consulting and advice on data availability and image labeling. Cemal Koray Bingöl (Bits n‘ Bricks) For his insights into supervised learning and CycleGan algorithms, his advise on switching environments and his help in spatio-temporal neural networks. Tobias Heimig (RWTH) and for showing and explaining to us, its limitations.

Contact Konstantin Bausch, konstantin.bausch@tum.deB.Eng.

Sebastian Clark Koth, sebastian.koth@tum.deB.A.

Roman Ficht, B.Sc. Johanna Kargruber, johanna.kargruber@tum.deB.Sc.

[9] Rodrigue, J-P. (2016). Transportation, Sustainability and Decarbonization. In Rodrigue, J-P., Comtois, C., Slack, B. The Geography of Transport Systems. Chapter 4.4. Doi: transformations10.4324/9781315618159.acrossthe globe using Earth observation data. Cities, Vol. 107. Doi: 10.1016/j. cities.2020.102905

[3] Adli, M., Berger, M., Brakemeier, E. L. (2017). Neurourbanism: towards a new discipline. The Lancet Psychiatry, 4, 183-185.

[6] Anabtawi, S. (2018). The Compact City: Utopian Vision or Practical Solution to Medium-sized Cities in Developing Countries - The Case of Jenin/State of Palestine. Dissertation at Technical University Munich.

2021 at urbanization-prospects.htmlhttps://www.un.org/development/desa/en/news/population/2018-revision-of-world-

[11] Zurich Insurance Group Ltd. (2015). The risks of rapid urbanization in developing countries. Accessed on 23. November 2021 at a%20city,urban%20violence%20and%20social%20unrest.the-risks-of-rapid-urbanization-in-developing-countries#:~:text=While%20moving%20to%20https://www.zurich.com/en/knowledge/topics/global-risks/ References

[5] Vivid Maps. 25 Years of Urban Growth and Density Change in 66 Global Cities. Accessed on 23. November 2021 at https://vividmaps.com/25-years-urban-growth-density-change-66global-cities/

[7] Chester, M., et al. (2013). Infrastructure and automobile shifts: positioning transit to reduce lifecycle environmental impacts for urban sustainability goals. IOP Publishing Ltd Environmental Research Letters, Vol. 8, 1. Doi: 10.1088/1748-9326/8/1/015041.

[8] Gallo, M., Marinelli, M. (2020). Sustainable Mobility: A Review of Possible Actions and Policies. Sustainability. Vol. 12, 18. https://doi.org/10.3390/su12187499

[4] McCollister, K.E., French, M.T., Fang, H. (2010). The cost of crime to society: new crime109. Doi: 10.1016/j.drugalcdep.2009.12.002.

[2] Vandecasteele, I., et al. The future of cities. European Commission. Accessed on 23. November 2021 at https://urban.jrc.ec.europa.eu/thefutureofcities/cities-as-innovation-hubs#the-chapter

[17] Zhang, Z., Dong, Y. (2020). Temperature Forecasting via Convolutional Recurrent Neural Networks Based on Time-Series Data. Complexity, Vol. 2020, pp. 1 - 8. University Press, pp. 30 - 32.

[12] Leo, C. & Brown, W. (2000). Slow Growth and Urban Development Policy. Journal of Urban

[15] Vlachas, P. R., Pathak, J., Hunt, B. R., Sapsis, T. P., Girvan, M., Ott, E., Koumoutsakos, P. (2020). Backpropagation Algorithms and Reservoir Computing in Recurrent Neural Networks for the Forecasting of Complex Spatiotemporal Dynamics. Neural Networks, Vol. 126, pp. 191217.

195Tricks197.ofthe Trade. Lecture Notes in Computer Science, Vol. 7700, pp. 9 - 48.

[13] Esbah, H. et al. (2012). Understanding urban growth patterns: A landscape ecology point of view. Accessed on 23. November 2021 at _A_LANDSCAPE_ECOLOGY_POINT_OF_VIEW.publication/228898456_UNDERSTANDING_URBAN_GROWTH_PATTERNShttps://www.researchgate.net/

[24] Shi, X., Chen, Z., Wang, H., Yeung, D. Y. (2015). Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting [25] Keras. Next-frame prediction with Conv-LSTM. Accessed on 23. November 2021 at https:// [26] Keras. Accessed on 23. November 2021 at https://keras.io/

[14] Ziat, A., Delasalles, E., Denoyer, L., Gallinari, P. (2017). Spatio-Temporal Neural Networks for Space-Time Series Forecasting and Relations Discovery. IEEE International Conference on Data Mining (ICDM), p. 705 - 714.

Proceedings of the IEEE, Vol. 86, No. 11, pp. 2278 - 2324.

keras.io/examples/vision/conv_lstm/

[20] Wu, J. (2012). Advances in k-means clustering: A data mining thinking. ProQuest Ebook Central. Accessed on 23. November 2021 at https://ebookcentral-proquest-com.eaccess. documentub.tum.de.recognition.

[16] Chattopadhyay, A., Hassanzadeh, P., Pasha, S. (2020). Predicting clustered weather patterns: A test case for applications of convolutional neural networks to spatio-temporal climate data.

[19] Zhu, X. X. et al. (2017). Deep learning in remote sensing. A comprehensive review and list of resources. IEEE Geoscience and remote sensing magazine.