Decoding Archives: Mediating The Third Skin by Shahdad Badiehneshin

Decoding Archives Mediating The Third Skin

Promotors: Corneel Cannaerts + Michiel Helbig By: Shahdad Badiehneshin 2021-2022

“Special thanks to my promotors Corneel and Michiel for being endless sources of inspiration, my wife for her help and support, my friends who helped me along the way, and my family who made this journey possible.” Shahdad Badiehneshin June 2022

Table of Contents Introduction

Scale 1: The Layers that We Live on - in - under The Second Skin The Third Skin

8 8 9

Archives of the City The Physical Archives The Digital Archives

10 11 12

Scale 2: City Decoding Archives: Working with the Online Datasets Archive of Archives

14 15 20

Scale 3: Vicinity of the Body Two Physical Walks Tasker Isovists

24 25 26 27

Four Perspectives The Digital Perspective The Machine Perspective The Mediated Perspective

34 36 37 41

Conclusion

References

Introduction Nowadays more than half of the human population have a way to connect to the Internet (ourworldindata.org, n.d.), either with a computer, or at least a smartphone. These devices connect us to each other and databases around the world, helping the human collective to share data and experiences with each other. They also have changed the scale of perceivable space that we can comprehend. The old few millimeters to few kilometers have become microns to light years. These devices, in a way, have become a digital extension of our bodies, organs which connect us to the invisible layer of digital data. “Architects have a contract with society. This contract suggests that architects have a mediating role in the way humans use the world surrounding their bodies.” (Raoul Bunschoten, 2001, p. 23) Through sharing of information with mobile devices, often containing metadata such as geotags, a new layer of digital media ecology has formed on top of our physical surroundings. (Taffel, 2019) While this layer corresponds to the physical and social layout of the city, it has an agency of its own, and increasingly starts to steer and affect how we inhibit, construct, and maintain cities. With the screens shaping the space around human bodies, and the digital world becoming increasingly real, the role of the architect as a mediator will inevitably change. 6

The problem regarding the digital layer is, however, it is rarely mediated by architects and urban designers, the group of individuals who are specialized to optimize the spaces for the wellbeing of humans. The digital layer is mostly designed and operated by big tech companies that run the information uploaded by users into their algorithms and mostly design their interfaces in a way to yield the most profit for their shareholders. For instance, the emergence of a platform economy, where companies like Airbnb or Uber do not actually own the material houses or cars, but extract value from it. Their technologies are mostly closed source and is presented to users in a “Blackbox” manner, meaning their mechanisms and algorithms are hidden from the user. This shift of focus from benefitting the society to the profit of companies has caused controversy around this layer and opened the discussion about how architects and urban designers can keep their role as mediators. (Bridle, 2018) (Mattern, A City Is Not a Computer, 2017) With the emergence of this new digital layer, different collections of information also came into existence. Information that either manually or automatically, for the purpose of using a certain digital service, were uploaded. This collection of information, or digital archives, contain valuable information about the processes that happen in the physical layer of the city. The goal of this project is to interact with and mediate some of these digital archives as an architect living in this era, with the help of different tools and technics, to materialize some of the “invisible” aspects of digital processes. The digital archives of the city are then investigated on three scales. On the first scale, different layers that create the urban fabric will be briefly discussed. The meaning of archives, their various kinds and their role in the project are explained in this chapter. On the second scale, the focus shifts to the city of Ghent in Belgium. Ghent is a vibrant city, and much of its population are young (citypopulation.de, 2021), thus the usage of online services is popular. Ghent also actively positions itself as a digital innovator with its startup scene and initiatives like Digi polis, District09 and Gent M. (Digi Tech | Stad Gent, n.d.) (District09 - Digitaal gedreven door Gent | District09, n.d.) (Gent M | Facebook, n.d.) In addition, the municipality of Ghent has an open data policy, which means that their data is accessible for the public on their website. By mapping various aspects of the digital representation of this city and superimposing them with each other, new hotspots of the urban data are discovered. On the third scale, some of these hotspots are visited in the physical Ghent and its digital twin, and input from the physical layer is recorded via the digital tools to create new digital archives. Later in the chapter, in addition to the recorded footage, three additional perspectives are investigated: The digital perspective, the machine perspective and the mediated perspective. These perspectives will then be put together to visualize what information creeps from the physical layer to the digital layer during the routines of our daily lives. 7

Scale 1: The Layers that We Live on - in - under Our environment can be understood to consist out of several layers. According to Bunschoten, the first and most important layer that every other layer depends on and is essential for the existence of natural life is the Earth’s crust, or the first skin. It is a thin layer of the Earth that we live and build on. Our cities roughly follow the shapes of its hills and valleys and the natural conditions are dictated by it. (Raoul Bunschoten, 2001) Not all the layers are material. For example, the political borders that humans respect and live in have no significance for other species. With the rapid advancement of technology in the 21st century, many layers were added to the already complex society of humans.

The Second Skin Town, city, urban layer or the second skin are a few of the terms that Bunschoten uses in his book Urban Flotsam to describe the infrastructure that is built on top of the Earth’s crust by us and is crucial to the progress of modern human civilization. It is the layer that humans interact with their physical bodies. (Raoul Bunschoten, 2001) The urban layer itself acts like a living body also; it has intake, metabolism, and output. It also reflects the emotions of its inhabitants with street art, spontaneous mini gardens in front of houses and stores, broken windows of abandoned houses and unique styles of architecture. It is constantly changing and evolving. The area covered by the urban layer is not a major part of the area that is inhabited by the entire human species. However, since the density of population in it is remarkably high and most of the technological advancements happen in this layer, it possesses a massive importance. The urban fabric is scattered all over the land part of our planet and down to its microscopic parts, it affects its underlying layer. 8

The Third Skin Based on Bunschoten terms, this new digital urban layer which sits on top of the physical urban layer, follows its landscape with geotags and provides spaces for humans to have an alternate existance there, can be considered the third skin, which is the focus of this project. But what is digital urban skin, and how does the notion of it relate to the notion of The Stack by Benjamin Bratton? (Bratton, 2019) The digital layer, the internet, the world wide web, and the third skin are all part of the immaterial layer that connects more than half of humans around the world. This layer predates the digital era. Earlier tangible inventions like the telephone network, the communication hubs, television and radio stations and the lines that moved this information, as well as non-tangible ones like flow of capital are all predigital era forms that belong to this layer. (Castells, 2004) With the invention of internet and the rapid sharing of knowledge around the world, which is the main reason for the rapid advancement of technology, this layer is rapidly becoming more involved in our daily routines and humans are becoming more reliant on it by each passing day. (D’Haeseleer, 2000) The third skin, which is part of the digital layer, uses the data that is provided by the sensors present in the satellites, the urban layer, and our daily-used devices to visualize, map and model the second skin and to “optimize” it. It is always presented to the public that these sensors are for the optimization of the quality of the digital services, but often the information collected by the same sensors are used to simulate the behavioral patterns of its users. In a way, humans have become the sensor and the sensed. (Bratton, 2019) Critics believe that not all the necessary laws that overlook the practices in the digital layer exist yet, and the people responsible for the design of this layer do not possess the incentive to do so. (Antoine, 2015) (Cox, 2019) The result of this difference of interest shows itself in the representation of a city in its digital layer. Anything that generates the most profit finds its way to the top of the list of social media and anything that generates less is buried in the ever-growing database of scrolling pages, regardless of their true value and their benefit for the humans. This layer is also affecting our daily lives. We are turning into cyborgs that need this connection and feel amputated while offline and make us unable to navigate our surroundings without it. The discussion about laws of design will become even more important when the upcoming technologies are integrated into our bodies. For the reasons mentioned above, it is crucial for architects to understand this layer, which has become the dominant space around human bodies, and to be able to interact with it in a positive and constructive way. Because of the vast scale of this layer, and the depth that it affects our civilization, it is impossible to fit all its characteristics in a single research or project. Hence the focus of this project is around the navigation part of this layer and parts that are related to the urban life. 9

Archives of the City To proceed with this project, first we need to collect the required information related to the urban fabric, also known as the archives of the city, and then to understand and process them. But what is an archive? Archive, a repository or collection especially of information, according to Merriam Webster dictionary, can consist of either material or immaterial objects. Archives of the city, therefore, refers to any collection that carries some sort of information about the city or its processes. Archive of historical maps of Ghent, archive of the location of points of interest, or archive of beers brewed in Belgium belong to this category. All archives are valuable and contain information about the country, city, its people, and its culture. In addition, the city itself is an ever-updating archive which gives us information such as construction materials, past ambitions, cultural identities, disasters, and conflicts. (Mattern, Code and Clay, Data and Dirt: Five Thousand Years of Urban Media, 2017) 10

The Physical Archives Physical archives which are present in the urban layer, may consist of but not limited to photos, writings, maps, or objects that carry information. Museums are a clear example of places that hold valuable archives about our history and technologies. One of the most important archives of architecture of Belgium is CIVA in Brussels that holds an extensive collection of items related to architecture, landscape, and urban planning of Belgium. It also contains some items related to the fine arts and fashion category. The items in CIVA range from conventional architecture archives like blueprints, maquettes, and photos to pieces of railings, furniture and posters related to Art Nouveau style.

CIVA archive of models, furniture, photos, documents and more related to architecture. Photos taken by the Author

The Digital Archives With the emergence of the third skin, the required framework for the storage of information in non-physical databases was introduced. Of course, the data in the digital layer is on a hard drive in a data center at some point of the physical world, but the way they work is very much integrated with the immediate space around us. With the assistance of smartphones, archives of digital nature have become accessible for internet users anywhere with an internet coverage. On the other hand, this has also enabled companies to track their users thoroughly using geolocation technology. According to Foursquare, known to most people as a mobile application to find activities around their locations, “By harnessing the power of geolocation technology, companies create better experiences for us as individuals by gaining deeper insight into our behavioral patterns in aggregate. This helps them make smarter business decisions as well. In addition to identifying a device’s location in a given moment, geolocation technology can show how devices move over time, offering visibility into people’s daily schedules, brand loyalties, path to purchase, and much more.” (Geolocation 101 | Foursquare - Independant Location Data Platform, n.d.) Some of the digital archives in the third skin are updated daily thanks to their users updating them daily. They hold up to date valuable, reliable information about the daily routines and urban flows that happen in the second skin. So, the natural course of action for businesses is to buy these services to have an edge and to rise above their competitors, and to have a digital archive that corresponds to their physical archives. Clear example of this is the online shops that use geofencing in an area around their rivals to show advertisements of their own items and redirect costumers to their own stores. The potential that these archives present can be used for the benefit of the public also. Tactical Tech is one of the organizations that works with this invisible layer by engaging citizens and civil-society organizations to explore and mitigate the impacts of technology on society. (Tactical Tech, 2020) Constant is also another non-profit, artist-run organization based in Brussels since 1997 and active in the fields of art, media, and technology. They regularly interact with digital archives with the help of software and hackable devices to create works of art. (Constant, n.d.) Since archives are optimized for a certain task and are usually collected by big tech companies to increase profit margin, they are meant for the use of a select group. Since demand drives supply, the use of archives by groups other than the targeted audience, will lead to richer future archives, with information that benefits a larger portion of the society. An example of this matter is the case of the CG (computer graphics) software Blender. Blender started as a paid application but became open source after the hard decision that its founder, Ton Roosendaal, had to make to save the company. It then grew in quality exponentially because of the engagement of its users around the world. Today it has one of the largest archives of free tutorials among the 3D CG software and is enabling artists and architects to have access to a free tool to create their imagination. (Guru, 2018) (blender.org - Home of the Blender project - Free and Open 3D Creation Software, n.d.) 12

Architects and urban designers, responsible for designing the space around humans, must engage with the archives to create the demand that is meant for the well-being of the population. The archives will then serve people from all levels of society to experiment, design and create with the help of the rich urban database. For these reasons, the digital urban archives are the basis of this project.

Scale 2: City The city of Ghent is in the East Flanders region of Belgium. This city was founded in medieval times, and several of its historical buildings are preserved. It has a rich character and history. Nowadays Ghent houses a large population of students each year and it is a cultural center for the creation of knowledge and experimentation. These factors lead to the popularity of online services, and most of the services and businesses that have activity in Ghent have embraced the digital layer. Furthermore, both the Flemish region and the city of Ghent share their online databases on their websites, and they are publicly accessible without any restrictions. (Geopunt Vlaanderen, n.d.) (Explore - Open Data Portaal Stad Gent, n.d.) The reasons mentioned above, and the fact that I live in Ghent and have easy access to it for regular experiments, makes Ghent the perfect case for this project. Using various tactics of mapping, several aspects of the digital representation of the city of Ghent are brought to light, to reveal a part of the hidden layer.

Decoding Archives: Working with the Online Datasets There are a couple of websites that provide information about the city of Ghent. The website of the government of Flanders provides the Large-scale reference file or “Base Map of Flanders (GRB)” in different formats. Ghent being in Flanders also benefits from this and so this project will use maps that are based on the GRB map. In addition, the city of Ghent also has an extensive archive of maps and databases which provides them in various formats. This project will use a selection of comma separated value (CSV) files from this website. To visualize the gathered CSV data, which is essentially a massive text file of all the points, routes and areas of interest, a script was needed. For this purpose, the Processing platform was used. Processing is a flexible software sketchbook and a language for learning how to code within the context of the visual arts. (Welcome to Processing! / Processing.org, n.d.) It is a free coding platform with built-in libraries for designers. The next step was to select a fixed region of Gent to be able to align and overlay different layers of data precisely. To achieve this goal, the coordinates from Open Street Map (OSM) was used. This system uses WGS-84 coordinate system, as do most GPS units. (OpenStreetMap, n.d.) The below coordinates were selected which include the R40 ring plus the surrounding areas: Longitude: Latitude:

3.68 E 51.025 N

to to

3.76 E 51.075 N

Open Street Map provides exports with exact coordinates. Photos taken from www.openstreetmap.org

A crucial step before using the datasets is to sort and format the CSV files according to the Processing standards. The datasets from the municipality each have a different formatting style with different value names, and instead of using comma, they use semicolon for the separation of values, so that comma can be used in text entries. However, Processing does not accept semicolon as a separator. To make these files compatible, which meant bulk-formatting the texts in the datasets, a free software called Notepad++ was used. (Ho, n.d.) By using the find and replace in all opened document’s function, the datasets were reformatted according to the Processing standard. The idea behind the next step was simple, parsing the CSV files and storing their values to a class. Essentially building the database which can be used later. The class for points and lines only needs to store the geometry data. But the class for sound levels should be able to store the average intensity for each area also. By having these coordinates and using the map() function, all the geometry from datasets can be mapped correctly on their respective spots. Last step of the procedure was to visualize the outputs, one dataset at a time. Three distinct functions were created for points, lines, and surfaces. Based on the importance of each data type, a certain pixel width was selected. 40 pixels for the public buildings, 20 pixels for the routes, 10 pixels for the trees, and 5 pixels for the illegal trash dumps. For the sound levels, the brightness of the color of the areas was used to determine the pixel value. collection of the available data, the datasets that were selected and sorted are the following: a. Suggested wandering route, Gandrien route, touristic route, safari route b. Locations of libraries, cinemas, cafes, museums, film theaters, sport facilities and commercial centers c. Trees of Gent d. Illegal Trash Dumping of 2021 e. Average sound levels during daytime

Drawing points and lines with the map() function Photos taken from Processing environment

HIGH

Low

Public Buildings Trees Illegal Trash Dumping Sound Level During Day

The selected datasets from data.stad.gent website. Each category of data has a certain pixel width. Made with Processing and Photoshop by the Author

Gandrien Route Safari Route Touristic Route Wandering Route

The selected datasets from data.stad.gent website. Each category of data has a certain pixel width. Made with Processing and Photoshop by the Author

Archive of Archives One way of working with various digital datasets and mapping them on physical layer is to combine and intersect them and generate a so called Heatmap. Silent London by Simon Elvins is an example of the use of the heatmap of sound levels to locate the quiet areas of London. (dpr-barcelona, 2010) The work of Erica Fischer called Locals and Tourists is another example that uses Twitter’s digital database to generate heatmaps that show the locations of local inhabitants and tourists by mapping their tweet locations. (Fischer, n.d.) In heatmaps we can find out the hot spots immediately, and it will give us a good basis for the next steps of the project. After creating the first database, the next step was to import all the images from the Processing script output to Adobe Photoshop. The steps of the procedure are as follows: 1. All the route layers were multiplied and merged, and the same was applied to all the public buildings. 2. All the remaining layers and layers from step 1 were then converted into grayscale images with equal starting gray pixel values. 3. All the layers were multiplied and merged. This step essentially provided the most and least active points of the city, with the darker pixels showing the more concentration of data points 4. Levels of the resulting image were adjusted to 0-255 5. A gradient map was added The resulting map is called a heat map, which shows the concentration of a specific type of data. In our case, it shows the places which have the most data points in the chosen datasets. The overall map of the selected datasets, where black areas have the least activity, and yellow spots show a lot of data overlay. Made with Photoshop and Processing by the Author

However, to really grasp the digital landscape of a city, a 3D rendition of the heatmap can help us. Metacity Datatown project by MVRDV and Stockholm – 19 project by Senseable City Lab are notable examples of 3D visualization of a certain urban database. (MVRDV, 1999) (Senseable City Lab, 2019) Ghent Theft Auto by Lucas Selfslagh is also another project of creating a virtual landscape by using the city of Ghent database. (Selfslagh, 2020) To achieve the 3D representation, a displacement mosaic map with the pixel size of 30 pixels generated from the heat map and was imported into Blender to apply to a plane with the texture of the heat map. The result shows a digital landscape of the city of Gent, where high rises depict the hotspots of the datasets and flat lands suggest less density of data points. This archive, which was made from the archives of the city, is a composite database, and can now be used to provide new points of interest for the next steps of this project. 22

Digital landscape of Ghent, where black areas have the least activity, and yellow spots show a lot of data overlay. Made with Photoshop and Blender by the Author

Scale 3: Vicinity of the Body With the previous step showing the hot spots, we have a basis to work with. But what do these FRONT CAMERA spaces, which are the result of interacting with the digital archives of the city, look like in the urban layer? To answer this question, first a few similar projects are investigated. In project constraint city, Gordan Savičić uses a custom-made vest which can interact with the nearby wifi signals, tightning and loosening the vest. By walking in different cities, he maps his interactions with the digital layer. (Savičić, 2007-2010) Team Constant also designs custom footwear for users to walk and interact with the city in the Routes + Routines project. (Routes + Routines - Constant, 2006-2011) In both projects, the interaction with the digital layer was made by walking in the city with the help of digital tools and building their digital archive from the input they received from the city. Therefore, a physical visit to the hot spots, or even better, a walk that can visit multiple points will yield us our required digital archive. 24

SIDE CAMERA

GPS DETECTION BLUETOOTH LOGGER

Two Physical Walks From the eight hot spots located in the inner inside the R40 loop, two random points, one towards the South side and one towards the North side were randomly chosen: a. The intersection of Nederkouter and Bagattenstraat b. The intersection of Steendam and Sint-Jansdreef

The points with the highest density of data, based on the heat map from the last chapter Made with Photoshop by the Author

These two points are located on opposite sides of the city center, 20 minutes apart by foot. A walk between them will pass through other points as well. To navigate between the points a and b, the two popular navigation apps, Google maps and Apple maps were used. Additionally, an android app called TASKER was used to automatically log nearby bluetooth devices. The suggested routes were slightly different. But more interestingly, along the path the businesses that were visible on the digital map were also slightly different. For example, on Apple maps, the restaurant called Revue was visible from the beginning, from the overview screen, but it was hidden on the Google maps and only became visible after zooming in. The mentioned findings led to a mapping of mixed urban and digital spaces. In this mapping, the buildings along the path that were visible were tracked, and the businesses that were among those buildings, with their respective visibility zoom level were tagged. 25

Some businesses are not visible until the zoom is at a certain level on the digital map Picture taken from www.google.com/maps

Tasker Bluetooth is a recent technology that was invented in 1998, mainly used for short range (maximum 10 meters) wireless data transfer between electronic devices on request. Nowadays almost every mobile phone and electronic gadget has the updated version, Bluetooth 4.0 or BLE (Bluetooth Low Energy) technology integrated in them, which has increased range and reduced power consumption. The low energy part of the recent update allows it to be always on with minimal power consumption. With the popularity of smartphones among city dwellers, this technology is emitted wherever they go. Consequently, if a device can log a detected Bluetooth signal within its range with the geotag, it essentially maps the person that uses the Bluetooth device. Corona Alert mobile application is the perfect example that uses this technology to track infections around Belgium. (Sciensano, n.d.) To proceed with this strategy, a ONE A2003 with the android version 6.0.1 was used. To be able to continuously scan for nearby Bluetooth devices and then log them, the application Tasker was used. Tasker is an automation application which can listen for certain events and trigger a certain action automatically. 26

Isovists To make the mappings of Google and Apple walk, the method of drawing the isovists were used. A single isovist is the volume of space visible from a given point in space, together with a specification of the location of that point. It is a geometric concept coined by Clifford Tandy in 1967 and further refined by the architect Michael Benedikt. (Harris & Jenkin, 2011) (Wagman & Blau, 2020) To draw this space, the isovist node in Grasshopper of Rhino can do the trick. The steps of visualizing the isovists are as follows: 1. First, the 2D GRB map of Gent was imported to Rhino, in the top view 2. The path of the walk was drawn using polyline tool 3. The script was made in Grasshopper. Note: To map the entire volume of the space visible from a given path, which essentially is a line with infinite points, we need to divide the path line into a finite number. The reasoning behind this is that we are limited by the compute power. So, 200 points gives us enough data to visualize the map as if there were infinite points 4. The resulting points and their respective isovist surfaces were baked into the map 5. The building blocks which had no interaction with our points were removed

The walk path

This section takes the division points and visualizes them into a path made of circles

Division number of the path, the higher the number, the more realistic the path, but requires more compute power

The clipping distance of the vision set at 1.5 km

The building boundries in the walking area

This section takes the calculated points from the IsoVist node and turns it into surfaces

The maximum sides the vision polygons can have. Higher number results in smoother shape, but takes more time to calculate

The above Grasshopper algorithm generates the isovists according to the Made with Rhino, and Grasshopper by the Author

Dulle Griet Cafe Ventura

Cafe Afsnis

Het Keozershof Da Adriano Carlos Quinto Ocean sushi De Rave

Jack Premium Burgers

Gigi 6161

Hotel NH Gent Belfort Mosquito Coast 1898 The Post The Cobbler

Frituur Tartaar 8tea5 Ghent

ZARA

Novotel Amadeus Ghent 2 Surtoe Passion

Ibis Take Five Espressobar

B&B Hotel Gent Centrum

Cafe Rene Bocca Patrick Foley’s Irish Pub

Brasserie Café des Arts

Cafe Theatre

Revue Food & Drink Vintage Factory

Think Twice

Visible urban fabric and buisinesses during the walk made using Apple Maps Made with AutoCAD, Rhino, Grasshopper, and Photoshop by the Author

STEK

Bali Nachtwinkel

NOON Sandwich Rommelmarkt Gent Dulle Griet SPAR Joremaaie Trollekelder

Charlatan Gigi

Hotel NH Gent Belfort HEMA

Horn OK Please Hotel Cathedral

De Post Albert Heijn

Centre Ville Bier Central Gent

Bar Bidon

8tea5 Ghent

ZARA

Ibis B&B Hotel Gent Centrum

Fnac INNO GENT Café René

Bocca Patrick Foley’s Irish Pub Restaurant Bar Cafe Theatre

Standaard Boekhandel Worlds’ End Comics & Games Center Revue

Ibis Luv l’Oeuf

Greenway Gent

STEK

Visible urban fabric and buisinesses during the walk made using Google Maps Made with AutoCAD, Rhino, Grasshopper, and Photoshop by the Author

Bluetooth devices detected during the walk made using Apple Maps at 20:00 293 devices detected Signal strength controls the size and opacity of the circles Higher signal = smaller circle (closer device) and more opacity Made with Tasker(Andriod), Processing, and Photoshop by the Author

Bluetooth devices detected during the walk made using Google Maps at 13:00 323 devices detected Signal strength controls the size and opacity of the circles Lower signal = Larger circle (farther device) and more transparency Made with Tasker(Andriod), Processing, and Photoshop by the Author

Bluetooth devices detected during the walk made using Apple Maps at 20:00 293 devices detected Sorted according to signal strength (distance from the logger device) Device count in each range controls ring color Higher count = Brighter ring Made with Tasker(Andriod), Processing, and Photoshop by the Author

Bluetooth devices detected during the walk made using Google Maps at 13:00 323 devices detected Sorted according to signal strength (distance from the logger device) Device count in each range controls ring color Lower count = Darker ring Made with Tasker(Andriod), Processing, and Photoshop by the Author

Four Perspectives The exercise of the isovist method and the navigation applications mentioned in the last section, was a good practice to show how the digital layer affects what buildings we see during our commutes, and what businesses we find during our daily walks. However, we navigate our cities in different manners, and not all of them belong to the physical layer. In addition to us, the machine also sees our cities in diverse ways, and gathers data by analyzing the videos and pictures we take during our strolls in the city. In this section, different perspectives are created and compared with each other, to understand what forms the representation of the urban layer takes in the digital layer, and what parts of the physical layer bleeds into the digital one. During the walk from point a to b, two Go Pro Hero 7 cameras were used to record the surrounding for this purpose. The resulting video footages were then fed into two different processes: Object Detection and Photogrammetry. In addition, the walk was also experienced in the Google Earth Studio environment. 34

The Digital Perspective We live in a time which it is possible to virtually visit another country kilometers away, or even other planets by using Google Maps or similar services. (Marquardt, 2017) These services provide 2D and 3D realistic models of the physical layer, which are made from high quality imagery and computer algorithms, and have become a staple part of our daily navigation, Some use these services to avoid traffic, some to discover new places and to plan their next trip, and some to find their way. Therefore, this viewpoint is as important as the one experienced in the physical layer.

To recreate the walk for this perspective, Google Earth Studio service was used. This new service contains the highest quality models and textures available to public and packs a variety of animation tools and services. (Introduction – Google Earth Studio, n.d.) The process of creating the digital perspective are as follows: 1. At the date of writing this document, this service is at its beta stage, and asks for an application to grant access. The application consists of a form which asks about the manner this service is used 2. After gaining access, a new blank project is created 3. After setting up the animation resolution, length, movement, and other settings, it is rendered. There are two options for this step: a. First is to use the user’s computer for rendering, which saves each frame on the hard drive, and has no frame limitation b. The second option is to render the animation on the cloud, which has a certain frame count limitation, but provides the final video for download after the process is complete

Google Earth Studio is made to provide the highest quality of Google imagery for the purpose of animation Photo taken from Google Earth Studio Environment

The Machine Perspective Object detection is a computer technology related to computer vision and image processing that deals with detecting instances of semantic objects of a certain class (such as humans, buildings, or cars) in digital images and videos. (Dasiopoulou, 2005) It is essentially an AI that can be trained on dataset of prepared images to learn how certain objects look like, and saves it on a trained model. It can then detect those class of objects on images that it has not seen before, by making predictions and comparing them with its trained model. From the moment we start to record video footage or take pictures and feed them into this process, the machine starts to see the physical world and interact with it. Therefore, it becomes another perspective that looks at the city. The apparatus made by Joris Putteneers in the project Ugly Stupid Honest, uses object detection as an input to decide its next destination. (Putteneers, 2020) Dries Depoorter in the project The Flemish Scrollers also uses object detection to detect distracted politicians that are being streamed on a YouTube channel. (Depoorter, 2021) Artificial Landscapes: Machines Point of View and Beyond the Grid: Territories of Resolution are also notable examples that visualize this point of view. (Vankerckhoven, Kaura, Van Wesemael, & Uyttenhove, 2021) (Safieddine, Dong, Luo, & Liu, 2021) Before the start of this procedure, we need to first understand how this technology works. Object detection consists of three distinct parts: 1. The code that is related to the neural network of the AI, which controls the methodology of the Deep Learning process 2. The dataset that we want the AI to train on, which can be virtually anything. It can be highly specialized footage like pictures of cancerous cells, or perfectly common like pictures of persons, cars or bags, or even fictional beings like orcs, elves, or unicorns. The larger the dataset of different images of those objects, the more accurate the AI becomes 3. The code that we write, which communicates with the AI, feeds the footage to the AI, receives the detection from the AI then and visualizes the outcome of the process

The library that was used in this project is Detectron2. Detectron2 is Facebook AI Research's next generation library that provides state-of-the-art detection and segmentation algorithms. (Wu, Kirillov, Massa, Lo, & Girshick, 2019) To visualize the detected object, image segmentation method of PointRend was used. This method adds an extra stage to the prediction stage of Mask R-CNN, resulting in much more accurate outputs. (Kirillov, Wu, He, & Girshick, 2019) One of the most used image datasets is COCO (Common Objects in Context), which contains 2.5 million labeled instances of 80 of the most used classes like person, car and bicycle marked in 328k pictures, ready for the training process. Many object detection programmers train their AI on this dataset and release their pre-trained models for the use of the commons. (Lin, et al., 2014) This project used a pre-trained model of PointRend on COCO dataset to detect the objects in the captured footage of the walk in Ghent. The last step is the one that feeds the input to the AI and then takes the result of the detection, and can either visualize it, or use it to trigger another process. In this stage the project controls the opacity, color and scale of the mask and the font and adds these items to the input image and saves it at a pre-defined location.

Input photo taken from the entrance of CIVA by the Author Output photo made with Python, Detectron2, and PointRend library by the Author

Input photo by Fons Heijnsbroek on Unsplash

Output photo made with Python, Detectron2, and PointRend library by the Author

Input photo by billow926 on Unsplash

Output photo made with Python, Detectron2, and PointRend library by the Author

The Mediated Perspective Designer’s way of conveying a design, a representation of their interpretation of a subject, can take many different shapes and forms, with many different techniques to create. The famous painting The Scream by Van Gogh, the Statue of Liberty by Frédéric Auguste Bartholdi and Gustave Eiffel, or even a 3D CG representation of a building before it is built, are all mediated forms that try to convey a certain idea or design. In Where the City Can’t See, by using laser scanning technology, Liam Young and Maughan show us the Chinese owned and controlled Detroit Economic Zone in a point cloud format. (Young & Maughan, 2016) In Metaphysical Greenhouse, Michiel Swerts shows us a blended reality, a place which the borders of perspectives are barely recognizable. (Swerts, 2021) To make the mediated perspective of the walk, the technique of Photogrammetry was used to recreate the space seen by the camera. Photogrammetry is a technology that uses an album of photographs taken from a certain object to recreate its mesh form in the CG environment. To achieve the best results for this process however, a lot of photos must be taken from all sorts of angles. The more photos are taken, the better the results will be, with a lot of details. In other words, the more attention is spent on a certain subject, the better it gets saved. This is also true for the human mind, the more we give attention to a certain subject, the better we remember it. 41

The walk in the city, however, is a stroll, is a routine. Some things that we see during a walk become blurry, while some things stay vivid. To achieve this result in the model of mediated perspective, a method opposite of the suggested Photogrammetry principle was used: 1. Objects were not scanned thoroughly; the speed of the footage is based on a normal walk from a strolling pedestrian 2. Only five frames were extracted from 60 frames per second of the walk 3. The resulting images were then fed into Agisoft Metashape software (Agisoft Metashape, n.d.) 4. The resulting mesh was roughly cleaned from the blobs and noise artefacts 5. The cleaned mesh was then exported as in OBJ format and imported into Blender 6. The available 360 photos in Google maps in the walk area were downloaded using Street View Download 360 software (Orlita, n.d.) 7. Sphere objects were put inside the model in Blender in the spot of the 360 photos, then their texture was changed into the 360 photos 8. Scales, lighting, and other rendering options of the project were corrected and then the final form was rendered This mediated version of the walk shows the city with all its imperfections, lack of attentions, missing information and its vivid 360 views. It’s a blend of captured footage of the physical layer and what is found on the digital layer. 42

The mediated perspective shows the city as a blurry space that blends the digital and the physical layer Made with Metashape, Belnder and Street View Download 360 by the Author

Conclusion The interaction with the digital urban layer happened in three different scales on this project. On the first scale, some of the layers that humans interact with were briefly discussed. The Earth’s crust, which everything is built on, or the first skin, the urban fabric that houses the majority of human population, or the second skin, and the digital urban layer that has come to existence with the advancement of technology and connects the human population around the globe, or the third skin, are interacting with humans on a daily basis. The physical archives which hold valuable information about our history and the flows of the physical layer, and the digital archives, which hold up to date data about our daily behaviors and routines were discussed. Moreover, the importance of interacting with the digital archives to create the demand for public benefit-oriented archives were talked about. At second scale, the digital archives of the city of Ghent were interacted with to create an overall heatmap which shows the hot spots of the digital data immediately and creates a starting point for the next steps of the project. On the third scale, the hot spots were visited in person and in the digital layer. The walks happened using two different navigation applications. The slight difference of them led to slightly various parts of the city being visible to us. Moreover, each application was promoting different businesses on their interfaces. The video footage of a part of this walk was recorded with digital tools to create yet another digital archive to work with. The four different perspectives that interact with the city were investigated in the closing chapter: The recorded perspective, the digital perspective, the machine perspective, and the mediated perspective. The digital perspective chapter talked about the importance of the digital navigation services in our daily routines. On the machine perspective, the view of the computer algorithms of us and our cities were briefly investigated. (Leech, 2011) Lastly, on the mediated perspective chapter, a new way of mediating the layers that we interact with was exercised. All these perspectives briefly show how we (and our creations, the AI) experience, look at and interact with the city. They briefly show the processes that happen daily on the invisible layer without making us aware. They show the space surrounding the average person and tell us how important it for architects is to interact with the digital archives to mediate the city in this new age for the benefit of its citizens. They also briefly show the potential of these techniques, like machine learning based solely on architecture, or digital tools that can manipulate the urban fabric.

Watch “Four Perspectives” video on YouTube: https://youtu.be/38uIB858R8Y

References (n.d.). From OpenStreetMap: www.openstreetmap.org Agisoft Metashape. (n.d.). From Agisoft: https://www.agisoft.com/ Antoine, P. (2015). Smart Cities: A Spatialised Intelligence. John Wiley & Sons. blender.org - Home of the Blender project - Free and Open 3D Creation Software. (n.d.). From Blender: https://www. blender.org/ Bratton, B. H. (2019). The Stack: On Software and Sovereignty. Cambridge: MIT Press. Bridle, J. (2018). New dark age : Technology and the End of the Future. London ; Brooklyn, NY: Verso. Castells, M. (2004). An Introduction to the Information Age. The Information Society Reader, 138-49. citypopulation.de. (2021, June 22). Gent (Gent, Oost-Vlaanderen, Belgium) with population statistics, charts, map, location, weather and web information. From City Population: https://www.citypopulation.de/en/belgium/oostvlaanderen/gent/44021__gent/ Constant. (n.d.). From Constant: https://constantvzw.org/site/ Cox, K. (2019, July 30). Proposed US law would ban infinite scroll, autoplaying video | Ars Technica. From Ars Technica: https://arstechnica.com/tech-policy/2019/07/proposed-us-law-would-ban-infinite-scroll-autoplaying-video/ Dasiopoulou, S. e. (2005). Knowledge-assisted semantic video object detection. IEEE Transactions on Circuits and Systems for Video Technology, 1210-1224. Depoorter, D. (2021). The Flemish Scrollers, 2021-2022 &#8211; Dries Depoorter. From Dries Depoorter: https://driesdepoorter.be/theflemishscrollers/ D’Haeseleer, K. (2000). File | www.li-ma.nl. From li-ma: https://www.li-ma.nl/lima/catalogue/art/kurt-d-haeseleer/ file/6406 Digi Tech | Stad Gent. (n.d.). From Stad Gent: https://stad.gent/en/invest-ghent/future-tech/digi-tech District09 - Digitaal gedreven door Gent | District09. (n.d.). From District09: https://district09.gent/nl dpr-barcelona. (2010, April 9). SILENT LONDON | Simon Elvins | arkinet. From arkinet: https://arkinetblog.wordpress. com/2010/04/09/silent-london-simon-elvins/ Explore - Open Data Portaal - Stad Gent. (n.d.). From https://data.stad.gent/ Fischer, E. (n.d.). Mapbox: Locals & Tourists. From Mapbox: https://labs.mapbox.com/labs/twitter-gnip/locals/#12/35.6845/139.6161 Gent M | Facebook. (n.d.). From Facebook: https://www.facebook.com/GentMeMe Geolocation 101 | Foursquare - Independant Location Data Platform. (n.d.). From https://foursquare.com/article/geolocation-101/ Geopunt Vlaanderen. (n.d.). From https://www.geopunt.be Guru, B. (2018, February 26). “Money doesn’t interest me” - Ton Roosendaal interview - YouTube. From YouTube: https:// www.youtube.com/watch?v=qJEWOTZnFeg Harris, L. R., & Jenkin, M. R. (2011). Vision in 3D Environments. Cambridge, UK: Cambridge University Press, 311. Ho, D. (n.d.). From Notepad++: https://notepad-plus-plus.org/ Introduction – Google Earth Studio. (n.d.). From Google Earth Studio: https://earth.google.com/studio/docs/ Kirillov, A., Wu, Y., He, K., & Girshick, R. (2019). detectron2/README.md at main · facebookresearch/detectron2. From Github: https://github.com/facebookresearch/detectron2/blob/main/projects/PointRend/README.md Leech, G. (2011, December 5). James Bridle - Waving at the Machines - Web Directions. From Web Directions: https:// webdirections.org/resources/james-bridle-waving-at-the-machines/ Lin, T.-Y. L., Maire, M., Belongie, S., Bourdev, L., Girshick, R., Hays, J., . . . Dollár, P. (2014). Microsoft COCO: Common Objects in Context. From arxiv: https://arxiv.org/abs/1405.0312 Marquardt, S. (2017, October 16). Space out with planets in Google Maps. From Google Blog: https://blog.google/products/maps/space-out-planets-google-maps/ Mattern, S. (2017). A City Is Not a Computer. From Places Journal: https://placesjournal.org/article/a-city-is-not-a-computer/ Mattern, S. (2017). Code and Clay, Data and Dirt: Five Thousand Years of Urban Media. Minneapolis: University Of Minnesota Press. MVRDV. (1999). MVRDV - Metacity / Datatown. From MVRDV: https://www.mvrdv.nl/projects/147/metacity--datatown-

Orlita, T. (n.d.). Street View Download 360. From Street View Download 360: https://svd360.istreetview.com/ ourworldindata.org. (n.d.). Internet - Our World in Data. From https://ourworldindata.org/internet Putteneers, J. (2020). Ugly Stupid Honest. From putteneersjoris.xyz: https://putteneersjoris.xyz/projects/Ugly%20Stupid%20Honest/ugly_stupid_honest.html Raoul Bunschoten, H. B. (2001). Urban Flotsam: Stirring the City. 010 Publishers. Routes + Routines - Constant. (2006-2011). From Constant: https://constantvzw.org/site/-Routes-Routines,5-.html Safieddine, C., Dong, J., Luo, K., & Liu, Y. (2021). Bartlett B-Pro 2021 UD RC19 - Beyond the Grid: Territories of Resolution on Vimeo. From Vimeo: https://vimeo.com/629924776 Savičić, G. (2007-2010). constraint city. From yugo: https://www.yugo.at/equilibre/ Sciensano. (n.d.). How does it work? - Coronalert. From Corona Alert: https://coronalert.be/en/how-does-it-work/ Selfslagh, L. (2020, May 1). Making Ghent Theft Auto, with heightmaps. | by Lucas Selfslagh | Ghent Theft Auto | Medium. From Medium: https://medium.com/ghent-theft-auto/making-ghent-theft-auto-with-height-maps5b7400f6ce56 Senseable City Lab. (2019). Stockholm - 19. From senseable city lab: https://senseable.mit.edu/stockholm-19/ Swerts, M. (2021). Metaphysical Greenhouse on Vimeo. From Vimeo: https://vimeo.com/531380704?embedded=true&source=vimeo_logo&owner=103205157 Tactical Tech. (2020). Exposing the Invisible. From Exposing the Invisible: https://exposingtheinvisible.org/ Taffel, S. (2019). Digital Media Ecologies: Entanglements of Content, Code and Hardware. Bloomsbury Academic.

June 2022