BARC0141: Built Environment Dissertation "3D Spatial layouts using 3D GANS A case-study and computational framework for creating datasets for learning and generating voxel grid-based houses."
by
Shuyao Dai 18124837 September 2020 Word count: 9337
Dissertation submitted in part fulfilment of the Degree of Master MSc Architectural Computation Bartlett School Architecture University College London
UCL FACULTY OF THE BUILT ENVIRONMENT BARTLETT SCHOOL OF ARCHITECTURE MSC/MRES AC
MSc/MRes Architectural Computation Dissertation Submission Form A signed and dated copy of the following MUST be inserted after the title page of your dissertation. If you fail to submit this statement duly signed and dated, your submission will not be accepted for marking.
1. DECLARATION OF AUTHORSHIP I confirm that I have read and understood the guidelines on plagiarism, that I understand the meaning of plagiarism and that I may be penalised for submitting work that has been plagiarised. I certify that the work submitted is my own and that it has been also submitted electronically and that this can be checked detection service, TurnitinÂŽ. I declare that all material presented in the accompanying work is entirely my own work except where explicitly and individually indicated and that all sources used in its preparation and all quotations are clearly cited. Should this statement prove to be untrue, I recognise the right of the Board of Examiners to recommend what action should be taken in line with UCL's regulations. 2. COPYRIGHT The copyright of this report remains with me as its author. However, I understand that a copy may be given to my funding body (alongside limited feedback on my academic performance). A copy may also be given to any organisation which has given me access to data and maps (if requested and if appropriate). I also understand that a digital copy may be deposited in the UCL public access repository and copies may be available on the UCL library bookshelves.
Please write your initials in the box if you DO NOT want this report to be made available publicly either electronically or in hard copy.
Name: _____________________________________________________ Signed: _____________________________________________________ Date:
_____________________________________________________
Abstract With the maturity of statistical-based artificial intelligence (AI) technology, it plays crucial roles as an analytical tool or core intelligent algorithm in many industrial fields, academic research, and other applications, and the design field is not an exception. As an imperative research trend, AI technology offers unprecedented possibilities in improving design awareness and automated design direction. However, in the process of studying applying AI into the design domain, it faces a difficult problem, which is how to translate a design problem into a machine learning problem. This thesis explores a solution by concatenating generally segmented workflows of architecture design with the machine learning procedure. The main contributions are three-fold. First, a prototype of a toolset for dataset preparation; Second, four datasets with two formats. Third, a three-dimensional deep convolutional generative networks (3DDCGANs) model written in Python language using the PyTorch framework.
Keywords: machine learning, deep learning, convolutional neural network, GANs, DCGANs, 3D GANs, 3D DCGANs, dataset, artificial intelligent in design
1
Acknowledgement I would like to firstly thank Vishu Bhooshan and Marcin Kosicki, my supervisors, for their patience, support, and continuous insight. This year is unique. Because of the global pandemic of new coronary pneumonia, the whole process of the paper is challenging. Online communication has created significant obstacles. Staying alone in a closed environment for a long time has inevitably caused my mental state to be unstable, leaving me in a long-term nervous and depressed mood. I would like to thank my family and friends who did not hesitate to lend me a helping hand during this most difficult time. Their support allowed me to persist through this challenging journey. Among them, I would like to especially thank Yang Li, who has supported me through the difficulties. I would also like to thank a friend Derick from Imperial College, for his patient explanation on machine learning.
2
Contents Abstract ........................................................................................................................................................ 1 Acknowledgement ................................................................................................................................... 2 Table of figures .......................................................................................................................................... 5 Introduction................................................................................................................................................ 7 Chapter 1: Literature review ............................................................................................................... 9 Dataset and its relevant tools research ....................................................................................10 Neural network research ...............................................................................................................10 Design Application research .........................................................................................................11 Identify academic gap ......................................................................................................................12 Chapter 2: Method..................................................................................................................................14 Organization ........................................................................................................................................15 Field trip ...........................................................................................................................................16 Data collection................................................................................................................................16 Database ...........................................................................................................................................17 From database to a dataset .......................................................................................................17 From dataset to learning ...........................................................................................................17 Computational framework ............................................................................................................18 Dataset preparation Toolset ....................................................................................................18 Excel property table.....................................................................................................................19 CAD representation .....................................................................................................................20 Geometry preparation ................................................................................................................20 Data clean.........................................................................................................................................22 3D model generation ...................................................................................................................24 3D model representation for machine learning ...............................................................26 Data augmentation .......................................................................................................................29 Batch processing ...........................................................................................................................31
3
Formatting the data .....................................................................................................................36 Training framework_3D DCGANs...........................................................................................37 Chapter 3: Experiments and result .................................................................................................44 Data preparation toolset test ...................................................................................................44 3D DCGANs Reliability test on an existing dataset .........................................................45 3D DCGANs test on the customized dataset ......................................................................47 Chapter 4: Discussion and conclusion ...........................................................................................50 Dataset preparation tool and dataset........................................................................................50 3D DCGANs ...........................................................................................................................................50 Future work .........................................................................................................................................52 References .................................................................................................................................................54 Appendix ....................................................................................................................................................56
4
Table of figures Figure 1 Organization and the relationship between current research and previous research ......................................................................................................................................................15 Figure 2 The whole procedure of data processing ...................................................................19 Figure 3 Example of a property table for one house ...............................................................19 Figure 4 CAD single line representation of one house ............................................................20 Figure 5 Pseudocode for closing boundary of each space block .........................................21 Figure 6 Table data clean procedure ..............................................................................................23 Figure 7 3D model generation based on the CAD plan model .............................................25 Figure 8 3D model of one village houses in one 3D geometry file (in Rhino format) 25 Figure 9 Binary encoding ....................................................................................................................27 Figure 10 Fix uniform voxel grid with changing volume .......................................................27 Figure 11 The resolution of final representation of the same geometry in a uniform voxel grid with changing volume .....................................................................................................28 Figure 12 Multiple values encoding................................................................................................29 Figure 13 Augmentation: rotation...................................................................................................30 Figure 14 Augmentation: transposition........................................................................................30 Figure 15 Augmentation: scale change .........................................................................................30 Figure 16 Dynamic voxelization process for one house geometries.................................31 Figure 17 Batch processing ................................................................................................................32 Figure 18 The whole process of data augmentation................................................................33 Figure 19 Augmentation process and internalize database .................................................34 Figure 20 Binary file and multiple value file output ................................................................35 Figure 21 Octave pseudocode ...........................................................................................................36 Figure 22 Architecture of a GANs model ......................................................................................37 Figure 23 3D and 2D CNNs kernels.................................................................................................38 Figure 24 3D DCGANs structure .......................................................................................................39 Figure 25 Generator architecture ....................................................................................................40 Figure 26 Current application strategies: the binary dataset trains the left one,........41 Figure 27 Training procedure ...........................................................................................................42 Figure 28 All houses of one village generated directly from database ............................44 Figure 29 3D model examples of houses in one village and these examples are exaggerated in hight ..............................................................................................................................44
5
Figure 30 Overall shape examples of houses ..............................................................................45 Figure 31 Reliability test on existing dataset .............................................................................46 Figure 32 Results of chairs generation ..........................................................................................46 Figure 33 Test on a binary custom dataset and results show several critical features of the dataset. ...........................................................................................................................................47 Figure 34 Results of village house generation ...........................................................................48 Figure 35 Multiple value encoding tests .......................................................................................49 Figure 36 Future training strategy of using 3D conditional GANs based on the research of this thesis. ..........................................................................................................................52
6
Introduction Artificial Intelligence (AI), with its recent improvements have forayed into the research field of business, economics, bioscience etc. It may because of a considerable breakthrough in computer science in recent years. Architecture and design fields are in the moment, expecting insights coming from this wave of revolution. The building is a dynamic and complex system. Therefore, for cognition, learning, and even design such a scenario, science-based technical methods are re-emphasized. Data support is considered a significant and meaningful way to recognize, learn, and provide a basis for decision-making. It can make people aware of many things beyond intuition. Although some scholars believe that design is not science, we cannot deny that the premise of the design is still cognition. More comprehensive and accurate awareness is an essential part of the completion of the appropriate solution to the design problem. In many data-based disciplines, like economics and data science, data-aided analysis has been proven to help with more precise and comprehensive cognition and application. So there are reasons to believe that data-aided methodology can effectively support design to some extent (Bachman 2012, 116). Designers can use these insights derived from data to design when confronting wicked complexity of problem. However, a more attractive question is whether these data-derived perceptions can be learned directly by computers, and the designing process can be more involved by computers. Machine learning is a computer algorithm that imitates human learning progression and further apply the process of induction and deduction. Because of its ability to utilize data to do analysis and prediction it has been naturally considered as a potential way to deeply embed AI into the design field. However, the approach is difficult, and generally, there are three fundamental issues needed to address: 1. How to transform a design problem into a problem that a machine can identify;
7
2. How to transform design-related data into objects that machines can learn; 3. How to choose a neural network to be suitable for a specific design problem. This thesis explores one complete one approach towards these problems by concatenating several fragmented workflows together, which contains from fieldwork to databases, from a database to a dataset, and ultimately from dataset to machine learning. It intends to explore the deep integration of AI in design while creating several tools for further exploration. In chapter one, some pieces of literature will be reviewed; they are about the current research trend on machine learning and relevant studies on design applications. It gives a general overview of what categories it has when studying AI in the design field. In chapter two, precedent studies will be mentioned, and how it relates to this current research will be talked about. After that the methodology used in this thesis will be discussed. In chapter three, the experiments will show the implementation results of the methods described in the previous chapter, and the results will be critically evaluated as well. Last chapter will be the conclusion and future research. The contributions of this thesis are：
1， Providing an approach to transform a design problem into machine learning one. By using it, currently conducted works like field survey, data collection and established database can be integrated into a new workflow. 2， Providing a toolset including a C# library, a python script, and two examples of implementation files in rhino grasshopper, which is compatible with most of the current design processes. 3， Exploring a new 3D voxel dataset format used for machine learning, which is an experimental method to label voxel data for a learning purpose. 4， Providing two voxel-model-based datasets for the exploration of design purpose machine learning. One is an ancient village binary dataset including 115 houses with courtyards, the other is a multiple value dataset of the same village. 5， Providing a 3D-DCGAN model to process the 3D voxel datasets and critically evaluating the reliability and effectiveness of this model are also tested.
8
Chapter 1: Literature review Many pieces of research show the significant potential and interesting in applying AI in a design context, especially when design can rely more on a computational algorithm and more supports from highly developed technical domains like software engineering and hardware engineering. The algorithm of design automation is not a novel topic, but from a historical perspective, it also shows an exploration trend of bottom-up approaches (Haviland 2019, 17). As a branch of AI, statistic-based methods are introduced mostly from the computer vision domain. The techniques studied in that area are mainly related to some essential works, including learning, knowledge representation, and inference, as well as demonstration programs in language understanding, translation, theorem proving, associative memory, and knowledge-based systems (Bruce G. Buchanan 2005, 1). AI is a machine which can mimic some cognitive and deductive functions of a brain like human-like learning and problem solving (Russell and Norvig 2010, 2). Machine learning is the core part of AI inventing for making the computer smarter to solve complex problems. Basically, it is an algorithm that has a sequence of executive instructions for a computer to process data from input to output (Alpaydin 2014, 2). Machine learning is useful when encountering some problems that human has no algorithm for, in which it is expected to extract an appropriate algorithm (Alpaydin 2014, 2). Usually, in the design domain, applied algorithm mostly coming from a designer, but machine learning can provide another approach to extract algorithm automatically. Roughly, when it comes to applying AI into a specific domain, there are three research directions, which are dataset research, neural network architecture research, and application research. The first type of research is the basis of the latter two, and the last one largely depends on the maturity of the first two types of research.
9
Dataset and its relevant tools research Dataset research focuses on how data can be standardly collected, organized, stored, cleaned, represented, and augmented. It can be considered as the foundation of neural network architecture research and specialized application research. Suitable datasets can support not only the exploration of structures of neural networks for different tasks but also the improvement of existing networks (Wichmann, Agoub, and Kada 2018, 1). As for the application, the same dataset cooperates with neural networks with different architecture it can perform different task, for instance, the generative adversarial networks (GANS) can generate objects look like training data, but Cycle-GANs can transform between two different types of object. The three-dimensional domain, even in the computer vision field, is relatively new, it may be due to the limitation of current hardware, which leads to the expensive computational cost and time-consuming. RoofN3D is a point cloud dataset for 3D buildings deep learning, like reconstruction, segmentation, and classification, and the purpose of this research is to address the issue of lacking appropriate training dataset (Wichmann, Agoub, and Kada 2018, 1). ModelNet is another large-scale 3D CAD model dataset providing a variety of 3D furniture-scale objects, such as sofa, table, dresser (Z. Wu et al. 2015, 1).
Neural network research A neural network is an approach to do machine learning. Research of it usually focuses on the effectiveness and reliability of different neural network architectures for the same task. The more foundational research is about establishing or optimizing the technical framework of learning and the mathematic model of the learning algorithm. In this kind of research, most of them choose to use reliable existing datasets.
10
GANs is a generative model designed by Goodfellow et al. in 2014. It uses deep learning techniques like convolutional neural networks (CNNs) (Goodfellow et al. 2014, 1). This new framework research bases on several famous datasets, including a digital handwriting dataset (MNIST), a Toronto Faces Dataset (TFD), and a Canadian Institute For Advanced Research dataset (CIFAR-10) (Goodfellow et al. 2014, 6). As extension research of CNNs, threedimensional convolutional neural networks (3D CNNs) firstly appears in the context of videos learning, but it is not limited to videos but can also apply to other three-dimensional inputs, such as point clouds (Sedaghat et al. 2017, 3). Another research introduces a voxel-based learning strategy by using 3D GANs for 3D furniture generation, which bases on the ModelNet dataset (J. Wu et al. 2016). And a mesh-based learning framework MeshGAN has been used for facial expression mesh generation (Cheng et al. 2019, 1). In comparison to other machine learning algorithm, designers are more likely to choose a GANs model, since it can generate completely new data after training, which match the need of the creative process in the design.
Design Application research Application-related research generally bases on the above two types of research. It focuses on how to design learning strategies by utilizing or concatenating the existing neural network and dataset to achieve a particular application. In this field of research, it rarely collects and labels data by themselves, but there is an exception. In a recognition and classification research of architectural elements, such as the door, the window, the chair, and the table, the training dataset comes from web scrapper and then labelled manually by volunteers (Eltarabishy 2017, 19–23). In this field, it also explores the possibility of re-using the existing deep learning model from the computation vision domain into a design context (Eltarabishy 2017, 13). Another recent research from GSD explores how to use another successful GANs model, named pix2pix, to automatically generate floor plans or transform plan styles (Haviland 2019).
11
However, the design relevant pieces of research are mainly on the application level, the limitations are apparent. Firstly, the types of available dataset, recently, most of them are provided for image-based machine vision task. Secondly, lacking variety types of reliable neural networks, and also most are designed for classical computer vision tasks (object identification, objects classification). Above reasons may have been cause that overconcentration of imagebased learning in current applied research. Fortunately, the highly developed and gradually mature techniques seem to stimulate the expansion of research, like the new learning framework PyTorch with its growing library of models. Furthermore, the architectures of neural networks are also gradually getting more affluent and more diverse, like deep convolutional GANs (DCGANs), conditional GANs (C_GANs), cycle GANs, which can enlarge the range of applications. The incompatibility between the objective and technical framework is one reason that teaching a machine to implement a design task is so challenging. Because in order to do it, firstly, it is necessary to convert the data into the form that the current learning framework can use and, secondly, identify a neural network that can perform a similar task within design domain and its original domain.
Identify academic gap At present, in the field of 3D data learning, since most of them are used for robot vision, the primary purpose of model training is 3d objects recognition, shape retrieval, and classification (Ahmed et al. 2018). Thus most of the datasets allowed to use are mainly collected for this training purpose, like ShapeNet dataset, SHREC'13 dataset. Therefore, the lack of available datasets has become the first problem in the exploration of the application of 3D machine learning in design. Secondly, because there is less research in this field than in the 2D field, dataset preparation tools are relatively lacking. There is also a lack of tools compatible with platforms commonly used in the design field.
12
Furthermore, there are relatively few network architecture explorations suitable for 3D spatial data learning, and these explorations are aimed at different 3D data types (Ahmed et al. 2018). Although the rapid development of deep neural networks for various tasks has been witnessed by the computer vision community (J. Wu et al. 2016, 2), there are still relatively few use-ready learning models available. Finally, for applications, most of the previous cases were based on 2D pictures. Upgrading of hardware, and the gradual maturity of 3D neural networks, the diversity of learning data will also expand. 3D data learning can fully consider the information of three-dimensional objects and their relevance. Although they are currently computationally expensive, its advantages over two-dimensional data are worthy of further mining and exploration.
13
Chapter 2: Method This research introduces an approach that explores and develops several tools to utilize data from a general architecture research framework for machine learning. This exploration indicates two folds methods. First, it mainly focusses on how to utilize the existing database constructed though a general investigation path to generate 3D-learning-ready datasets, and it also provides another approach for using a general existing 3D model to prepare datasets. Second, it also provides a 3D deep convolutional generative adversarial networks (DCGANs) migrated from the computer vision field to perform design tasks. In many architecture research methods, anthropological perspective and sociological perspective are always embedded. It is inevitable to use mix investigation method, including several subjects. The methods contain field trips, questionnaires, observation, photo-taking, interviews, measuring, and so on. Commonly, data on paper collected through these approaches is achieved as raw data. Then well-trained professionals will take responsibility to digitalize the raw data to consist of a digital database for research purposes. Usually, Excel is the common platform for questionnaire and interview data storage, and the measuring data is commonly stored as CAD floor plan model files. In order to use the data for machine learning and explore the first step of the possibility of 3D machine learning in a design context, six problems need to be addressed. •
The entries of properties stored in Excel tables are not connected to CAD geometries files.
•
The original data has flaws and errors.
•
There are no 3D representations of the investigated houses available to use.
14
•
The general 3D representation, such as mesh or NURBS, in a design field that can not be easily used in a current learning framework, but mesh GANs is under development.
•
The size limitations of the real-world data.
•
The migration of 3D DCGANs from the computer vision domain to this specific leaning task.
Organization The objective of this research is to establish a framework for 3D spatial information learning while providing a toolset for data preparation. It includes several sequential strategies from data collection to machine learning (Figure 1).
Figure 1 Organization and the relationship between current research and previous research
A mixed research method has been conducted in previous research of digital ancient villages, which is supported and conducted by the International Research Center of Computational Design (IRCCD) of Architectural Design and Research Institute of Tsinghua University (THAD) CO.LTD.; it is a program funded by The National Natural Science Foundation of China (Chang et al. 2015). Field trip, data collection and database establishment were conducted in the previous study leaded by me and my colleague BoChao Zhang, and this research bases on it and extends it into a case of information utilization, dataset preparation, and machine learning. The investigation and methodology details was published in an journal
15
article "Data Science Aided Design: An Effective Method and Supporting Tool for Architectural Programming" (Chang et al. 2015).
Field trip The example sites are the historical villages in the west and north China. The government expects the investigation and analysis can contribute to several aspects of policy, urban or village planning, and architecture design. As an experimental project, the research team are allowed to establish a novel interdisciplinary research approach based on methods from sociology, data science, and architecture. The final database has two levels of information, the first is the village level, the second is the house level, and the latter is the data will be used as an example in this research. One of the first steps is the fieldwork of local stylized houses. It includes a questionnaire, interview, drawing, and measuring, which is trying to map the backgrounds and behaviours of users with the physical spaces.
Data collection The data collection is mainly done by some students led by a professional field research team. Each survey team entering the house is divided into two groups, one responsible for interviewing and filling out questionnaires, and the other for drawing and measuring. Surveyors first assign a family ID (Family ID) to the subjects surveyed according to the time at which the survey began and the name of the head of household. Then they need to use paper and pen to record the house planning map, and to assign a serial number to each space block as the space ID (Room ID), and corresponding to each space ID annotation function, the building age, construction materials, height and other related properties.
16
Database The original data is archived in paper form. After review, the data entry personnel will enter the data. The plan is saved in the form of CAD. Each space is marked with Room ID, and each household is marked with Family ID. The attributes associated with space are stored in Excel. In the CAD model, the space interval is described in the form of a single line. The doors and windows are also stored in the form of a single line in other layers. Each space has its number (Room ID), and each number corresponds to the attributes in Excel. In order to retain as much as possible the original information obtained during research and surveying.
From database to a dataset For machine learning, database data cannot be used directly for learning. There will be different data formats and needs for different purposes and learning strategies. The main exploration in this research is the learning of three-dimensional spatial data. For machine learning, The current learning framework cannot directly use the 3D model, so the target information must be converted into a format that can be recognized by the machine. The voxel-based model has naturally become a continuation of the pixel. Converting the excel and its corresponding space in the database into a solid model and properly processing it has become an inevitable problem. Using geometric generation rules referencing to tables properties, it allows generating 3D models directly from database.
From dataset to learning For diverse learning goals, one can choose different neural networks, and even for the same learning goals, one can also choose different networks. The same neural network will train completely different mathematical models according to different datasets. Therefore, data sets produced for a specific purpose cannot be simply transferred to other applications. Therefore, producing a data set of sufficient quality that matches the goals of the design task is a critical step in machine learning.
17
The information in the same database can be converted into different types of datasets for different learning purposes.
Computational framework The previous study bases on a database provided by the Institute of Architectural Design of Tsinghua University, and it included 12 ancient Chinese villages. Investigation team has more than 100 people coming from the University of Hong Kong and Tsinghua University, and the program was conducted from 2013 to 2016. The following is an example of data from one village to prepare the dataset that machine learning will use.
Dataset preparation Toolset Figure 2 shows the whole procedure of the data processing. The original database contains information which is not all relevant to the house. Thus the preparation step is to extract the most relevant raw data from the database. The property table of house and original measuring drawings are the essential ones. Approximately, each village contains an average of 60 to 100 houses. This data and attributes are evaluated before and after the survey. The effectiveness is evaluated according to how the data is collected and how well it performs. The original data related to the house include electronic questionnaires and hand-drawn house mapping drawings, which will first verify the information of the hand-drawn brief drawings to ensure the validity of the data. These include (1) the reliability of the encoded value (FID uniqueness); (2) the integrity of the information recorded; (3) the reliability of the recorded information, and (4) the evaluation of the effectiveness, reliability and accuracy of the information. In order to facilitate operation and do not add additional tools, the Microsoft Excel platform was chosen to record the data.
18
Figure 2 The whole procedure of data processing
Excel property table The properties included in the previous research are the date of investigation, the time of the investigation, name of surveyor, Family ID (FID), the house owner, Room ID (RID), floor level, height, original function, the function of using, user behaviours, core furniture, the material of the wall, the material of ceiling, roof-style, room user, building year. The properties may vary due to the investigation needs are different, but most of the properties are maintained the same. One house example is shown in Figure 3.
Figure 3 Example of a property table for one house
19
CAD representation In order to do further analysis, the original drawings need to be represented by single lines model of CAD (Figure 4). The standard of representation using in this research has been shown as below:
Figure 4 CAD single line representation of one house
All the elements represented in the CAD need to be separated into different layers. One important thing is to make sure that the left bottom corner of each text object should be included in the space that the text object indicates. This process will further ensure the feature matching in the next step.
Geometry preparation The primary purpose of this step is to associate the properties of each space block stored in excel with the model in CAD and to generate a three-dimensional spatial model for learning based on different attributes. There are two approaches, the first one, only CAD plan model
20
is provided. In this case, it has to match the properties to CAD plan model then based on the property to generate the 3D model; the second one, the 3D model is already provided, the only thing need to conduct is to match the property to the targeted model segments. The first approach: In order to generate a 3D model based on features in the table, the boundary geometry of each space should be generated, and it should have a unique and right ID, here the format is FID+RID, to match the table. Since the number of geometry and label is vast, thus one optimization way is to, firstly, rename all elements within an auxiliary housing boundary with the right FID, then further to check the RID label with the same name of FID. The whole procedure is shown in Figure 5:
Figure 5 Pseudocode for closing boundary of each space block
21
The second approach: The second approach is appropriate in a scenario that 3D models already exist. In comparison to the above approach, the requirement is to treat the layers insides of Rhino as the property labels and distribute geometries into the correct layer before further processing. For instance, if the function of space is the label, then create enough layers for all function types, then put 3D geometry that represents one specific function into the layer with the same name. As for the preparation of batch processing, the groups of geometries need to marked manually by changing the name of the geometries. For example, several houses of geometries can use consecutive integer, like 1, 2, 3, as the names.
Data clean Data clean is a significant step to ensure the reliability and correctness of the data. It mainly indicates two-folds in this research. First, is the CAD drawing, which should have no errors in FID, RID and the locations of them, all the spatial divisional lines should totally separate the space. Some samples of houses are identically irregular that should be removed. Second, because the properties in the table should be original as close to the raw recorded data as possible, thus nulls, redundancy and errors in data are inevitable. Clean the CAD plan The former question is relatively easy to solve; the solution is to check all drawings and identify the irregularity based on the experience then remove them manually. Generally, the experience is an intuitive sense coming from the knowledge that has learned from a long time investigation, including a sense of extracting the classic house prototype locally.
22
Clean the features of the table The latter problem needs a further procedure to manage that is shown in Figure 6. Generally, the feature contains three types of irregularity. (1) The invalid data, it should be replaced by a null value or an acceptable default value; (2) The same meaning data but has different values, in which a unified value can substitute it. (3) The redundant data; it does not matter so much since the unmatched data will not be used.
Figure 6 Table data clean procedure
23
All of the above irregularities can be managed by creating a replace table, in which firstly statistically checking the uniqueness of all values in one feature then output the table. Secondly, manually check and considering the appropriate value of a replacement. By using the original function as an example, statistically, the property contains 63 different values, after the clean replacement process, it has been successfully reduced to 13 values which are considered more useful and reliable. For instance, in the 63 values, there are values like "bedroom", "Bedroom"," bed" and "BedRoom". They are the same, so all of them can be replaced with one value like "BedRoom".
3D model generation Here the 3D model is a representation fo data in 3D space. The generation of a 3D model is a process of projecting the properties of a spatial area into 3D space. It can be considered as a spatial representation of attribute information in a 3D space, not only the building model. If the 3D models of the building already exist, the property can be used as the labels. In this case, a rhino file can use layers as the representation of the property. After the previous step, the basic geometry, which is the spatial block boundary, with correct ID attached, has been generated. Figure 7 shows the process, and by using this ID, the properties associated with it can be accessed. Some properties are directly used as parameters like the height. Some are used as the instruction of model generation and others used as the property of new geometry.
24
Figure 7 3D model generation based on the CAD plan model
For instance, the original function is used as the basis for the three-dimensional model generation rules, and also as the layered basis for the new model. Floor properties are used to determine where the model is generated. Set different model generation rules for different original functions in property, such as a bedroom on the first floor, that extrude blocks directly based on height. A second-floor space block needs to first move to the correct position before extruding the block geometry. Generating rules here are relatively simple, but they can be extended to more complex ones as needed. Figure 8 is the 3D models of one village generated through this procedure.
Figure 8 3D model of one village houses in one 3D geometry file (in Rhino format)
25
3D model representation for machine learning In the field of machine vision, 3D model learning has increasingly become a separate branch field, and the formats and representation methods of 3D models are also numerous. Different representation methods are closely related to different learning frameworks, and different networks and technical frameworks have different technical paths for different threedimensional data representation methods. For instance, volumetric representations like voxel and octree data can not be used in a learning framework which takes mesh or point clouds as inputs. The three-dimensional data represented by the voxel model is selected in this study, which is related to the choice of deep learning neural network based on 3D convolution. Since the current framework of machine learning technology cannot sufficiently support the direct learning of 3D models of Mesh and NURBS, the models need to transform. Here, the voxel is selected as the basic model representation for 3D model learning. Voxelization is one approach among several different 3D model representations. However, the voxel model itself has some shortcomings, such as occupying more memory, and under the current hardware technical conditions, it can only represent a model with a lower level of detail considered as a low-resolution data. Moreover, the generated model of 3D voxels needs to transform back to a more useful format like mesh. Binary encoding for the whole shape of a house The binary encoding format is used in the first deep 3D learning model, ShapeNet, which tried to explore the full geometry of the object represented as voxels (11). The input data of learning is a 30x30x30 binary tensor that represents the shape of a 3D object by indicating the occupied region as 1 and the non-occupied region as 0. Usually, the occupied region is defined as the target 3D objects that represent the shape of geometry (Figure 9).
26
Figure 9 Binary encoding
Converting from other geometry formats to a binary voxel grid has several different methods, the method used in this research is based on a uniform voxel grid with a changing volume (Figure 10).
Figure 10 Fix uniform voxel grid with changing volume
The density of the grid, which determined by the ratio of the occupied and non-occupied region,
mostly determines the resolution of the converted geometry, since the final
representation is the result of the cells' centre points that are included in the closed geometric regions (Figure 11).
27
Figure 11 The resolution of final representation of the same geometry in a uniform voxel grid with changing volume
And for the reason of the same size of all cells, this grid is named as uniform voxel grid, this even distribution of resolution ignores the needs of high detail parts on geometry. Although it makes the learning and transforming a learning result back easier, due to the low effectiveness of representation, the better approach of a non-uniform voxel grid may be a more useful method to explore.
Multiple values encoding In theory, based on the above binary three-dimensional data format, we can hypothesize that using different values to refer to different parts of the model may be a feasible strategy, which means that numbers can be a labelling system.
28
Figure 12 Multiple values encoding
Instead of only representing the geometry shape, it can extend the presentation more effective with different labels. Thus, if this method is feasible, the voxel model will not only refer to the shape of the object but also refer to different model contents. It will allow final prediction, including shapes with labels (Figure 12). For example, in the previous step, the overall shape of the house is calibrated to 1. Different original functions can be further calibrated to the consecutive integers 2, 3 and so on after 1.
Data augmentation Data augmentation is a technique used to increase the data size on a limited dataset. In many machine learning cases with limited data size, it increases the amount of data for training while preventing overfitting. Typically, in a two-dimensional environment, it is achieved by transforming, rotating, cropping, reversing, zooming in and out. 3D data can theoretically be enhanced similarly. In order to increment the data size, on the one hand, the above operations can be randomly combined; on the other hand, the suitable change parameters can be randomly selected in a certain range.
29
Figure 13 Augmentation: rotation
Figure 14 Augmentation: transposition
Figure 15 Augmentation: scale change
Compare to 2D image augmentation methods, and some operations may change the features that are specialities considered in a design domain. For instance, rotation operation makes machine learning to identify the shape of the object, ignoring the rotated situations. Flipping operation makes machine learning the content no matter it is flipped or not. However, these two operations ignore the important meanings of orientation and layout of architectural spaces. Considerably, the tool provided in this research select rotation, scale change and transposition as the main methods for augmentation (Figure 13, Figure 14, Figure 15).
30
By combining all strategies, a dynamic augmentation method is introduced thereafter (Figure 16), it takes random parameters for transposition and scale change while rotating the grouped geometries. The parameter of rotation is a fixed degree enquires to 360 degrees divided by a needed step amount. The voxel grid keeps changing while dynamically generating the final representation in both two formats of files, which is the binary file and the multiple value file.
Figure 16 Dynamic voxelization process for one house geometries
Batch processing In order to optimize the speed and convenience of the whole process, a batch processing strategy has been developed (Figure 17). The amount of data is not small, and each house requires a complete data augmentation process as above mentioned. Thus putting all geometric
31
objects of all houses in a file makes trouble for augmenting houses one by one. The proposed method is shown in Figure 18.
Figure 17 Batch processing
32
The specific implementation has several steps (Figure 18). First, using consecutive positive integers to encode every single house as the primary code, such as 1, 2, 3, etc. Second, automatically selecting all geometries of the same house each time according to the integer code. Third, generating a subordinate name under this code, such as 1_1. Then, after a set of data processing, two single-dimensional vector files are output including a binary one and a multiple values one, and both of them are stored at a specified path in the "txt" format with the secondary name generated ahead. After that, a new set of change parameters is generated, and a new secondary name is generated, such as 1_2. The data processing process is repeated to produce files until a certain amount of augmentation of a single house is finished. Until here, one house data augmentation process is complete, and then it will automatically move to another house group by changing the primary integer code to 2.
Figure 18 The whole process of data augmentation
During data processing, each house includes several pieces of geometry. The house group keep rotating, and because the orientation of the voxel grid is fixed, it is necessary to change its size to contain the rotated geometries dynamically completely. In this research, the orientation of voxel gird is fixed to the orientation of default world coordination in a Rhino (Software name)
33
file. The input tensor size of the neural network will be 30x30x30, which means 30dimensional voxel grid, as mentioned above.
Figure 19 Augmentation process and internalize database
The first step is to internalize entities from property table, after matching the geometric object by using ID (FID+RID) with the entities in the Excel table, an internal database of spatial blocks will be established in Rhino file. The block class stores the original piece of geometry, which is one piece of a house, with all the linked properties from the property table (Figure 19). Simultaneously, based on the geometry group, a bounding box will be calculated, and the centre point of the bottom face of this bounding box will be treated as the rotation centre. After each step of rotation, a fixed orientation but shifted size bounding box will be recalculated (Figure 19). Each geometric level change will update the 3D model stored inside the internal block database, for the reason of linking the properties. The second step is to
34
generate the cubic voxel grid based on the bounding box. By extracting the x, y and z axes of the bounding box and comparing the length, the longest one will be used as the length of the voxel grid (Figure 19).
Figure 20 Binary file and multiple value file output
Then, the next step is to check the index of the included grid voxels for each different block and finally generate the binary and multiply value file (Figure 20). First, generate a 30*30*30 length array with all values as 0. Second, take the geometry of the current block and check which cells are included inside of the geometry based on the centre point rule mentioned above. The index of the occupied voxel will be added into one list, which only represents one block. After checking all blocks of one house, there are several lists of occupied voxels. Third, in order to output a binary file with the name of a sub name, combining all lists as a replace index list and replacing the array value from 0 to 1 based on it. Parallelly, for multiple value
35
file output, except combining all lists, set different consecutive positive integers for different blocks based on the original function property. Then replacing the array value from 0 to a specific integer based on each index list, then output the array with the same sub same.
Formatting the data OCTAVE is free software, and it provides powerful mathematics-oriented syntax with builtin 2D/3D plotting and visualization tools. It is good for solving operations on vectors and matrices. Based on its built-in library, a batch data processing script has been provided (Figure 21), and it can be further extended if further operations are needed. Here it provides a reliable platform for data reshaping and checking before training with a visualized interface. All the single-dimensional vectors in "txt" are converted into three-dimensional vectors and stored in the "mat" format commonly used in learning.
Figure 21 Octave pseudocode
36
Training framework_3D DCGANs Compared to other machine learning models, GANs is a model that can generate completely new data after training. In a design context, the generation is valuable due to it can be considered as a creative and deductive process especially when the result derived from the prior design or real-world experience. That is the main reason to introduce GANs in this research. In the GAN setting, two differentiable functions represented by the neural network, the generator and the discriminator, compete against each other (Figure 22). The generator and the discriminator have different roles in this framework. The generator tries to generate data from a certain probability distribution and tries to get as close as possible to the training data to fool the discriminator.
Figure 22 Architecture of a GANs model
The discriminator behaves like a judge. It will adjust and improve the neural network by judging whether the data comes from the generator or the real training set. The generator attempts to maximize the probability that the discriminator will treat its inputs of fake data as real. And the discriminator guides the generator to produce more realistic data. DCGANs is an extended version of GANs, it was first introduced in the paper Unsupervised Representation Learning With Deep Convolutional Generative Adversarial Networks (Radford, Metz, and Chintala 2016). Based on the basic architecture of GANs, the main
37
difference is that DCGANs uses convolutional and convolutional-transpose layers in the discriminator and generator. In the DCGAN paper, the author believes that the key to training GAN is the following two techniques: (1) the all convolutional network and (2) batch normalization (BN). The first indicates stride convolutional layers instead of pooling layers, which means increasing and decreasing the spatial size of features. The second one normalizes the feature vector, which intends to solve the problems of poor weight initialization (Radford, Metz, and Chintala 2016). According to the paper, in order to establish stable Deep Convolutional GANs, the authors suggest that instead of using pooling layers, better to using strided convolutions in discriminator and fractional-stried convolutions in the generator. And removing all fully connected hidden layers if the architectures are deep. Then in both generator and discriminator, using batch norm. In generator, except for the output layer, which is suggested to use Tanh, using ReLU activation other all layers. In discriminator, using LeakyReLU activation for all layers (Radford, Metz, and Chintala 2016). The basic architecture concept of 3D GANs is the same with 2D GANs, however, the convolutional kernels are different.
Figure 23 3D and 2D CNNs kernels
The 2D convolution kernel (Figure 23) can make predictions using the context of a single twodimensional data matrix. However, since 2D convolutional neural networks (CNNs) take a single two-dimensional matrix as input, they are inherently unable to utilize the context of
38
adjacent two-dimensional matrix data. Voxel information from the adjacent matrix may be more useful for spatial data prediction. But the problem need to address is that 3D convolutional kernels (Figure 23) leveraging the context of interconnected layers result in the increased parameters used in CNNs, which further lead to a computational cost. 3D DCGANs structure The structure of generator a discriminator may vary due to the variety of specific training purpose, the structure of them used in this research is based on the official tutorial of PyTorch, which is an open-source machine learning library developed by Facebook's AI Research lab (FAIR). The framework is based on an existing 3D GANs provided by Jaesung Rim on the Github (Jaesung Rim 2017). However, the original code has several problems, including errors and missing parts, and it also lacks a specific part for multiple value data. These problems have been solved and several original pieces, including model and training procedure adjustment, data visualization and test model, have been added on (Figure 24).
Figure 24 3D DCGANs structure
39
Generator There are five convolutional layers in the generator, all of which use the ConvTranspose3d model in the Pytorch framework. The first four layers are followed by BatchNormal3D and rectified linear unit (ReLU) activation, and the last layer uses Tanh activation. It takes a Gaussian distribution initialized vector z with 200 dimensions as the input (Inkawhich 2017). Then the vector is fed into the generator, which starts a series of upsampling layers by using ConvTranspose3d (Inkawhich 2017). By referencing to one successful network of J. Wu et al. 's research, choosing to use 4x4x4 kernel and strides 2 in all generator layers (J. Wu et al. 2016, 3). Instead of the original size of tensor in each layer, it has been adjusted to 32x32x32 based tensor (Figure 25).
Figure 25 Generator architecture
Discriminator The discriminator mostly mirrors the generator, instead of enlarging the tensor, it condenses the tensor (J. Wu et al. 2016, 3). There are also five convolutional layers in the discriminator, and the Conv3D model in the PyTorch framework is used here. In the first four layers, after Con3D are followed by BatchNormal3D and rectified linear unit (LeakyReLU) as the activator, and the last layer uses Sigmoid activation without LeakyReLU (Inkawhich 2017). Current application strategy The application strategy tested in this research is directly using 3D DCGANs, trained by binary dataset, to automatically generate 3D shapes of a house, which is shown on the left side of figure 26. The right side is another experimental strategy to use multiple value dataset training
40
a modified 3D DCGANs for a house generation with different functions (Figure 26). They are separated processes, and the outputs shown on the figure are the expected results of the hypothesis.
Figure 26 Current application strategies: the binary dataset trains the left one, and the multiple value dataset trains the right one.
41
Training procedure Figure 27 shows the expanded note of 3D DCGANs in the previous figure. It indicates the details of the training process in the model of this neural network.
Figure 27 Training procedure
Both of discriminator and generator are using BCEloss as the loss function, and the loss value in discriminator is the sum of the loss of fake data and the loss of real data. The fake loss is calculated by using fake label 0 with the result coming from the discriminator feed with fake data, and the real loss came from using real label 1 with the result by feeding real data. In comparison, the loss of the generator is computed by using real label 1 with the result from
42
generator feed with fake data. Feeding real data into the discriminator and take the value bigger than 0.5 as the real accuracy of it. Then feeding fake data into it and taking the value lower than 0.5 as the fake accuracy. The accuracy of the discriminator is the sum of the real and fake accuracy. Because the discriminator developed quicker than the generator, it is suggested to set a barrier, which only updates the discriminator if the accuracy is smaller than 0.8 (J. Wu et al. 2016, 3). For the multiple value training, before feeding data, the real dataset has been remapped to a range of 0 to 1. After training, the result of the generator is denormalized back to the previous range of values. The models will be saved while training and after all training epochs finished the user can implement the test experiment to generate a 3D shape, which is still a voxelbased representation, by using the final trained model.
43
Chapter 3: Experiments and result Data preparation toolset test Figure 28 shows the 3D models of one village automatically generated from database. This village does not show a very classical prototype of house, the variety is varied. And since the accuracy of voxel grid, there is an exaggeration on height, which is as twice as original one.
Figure 28 All houses of one village generated directly from database
Figure 29 shows five typical types of house including double floor house, partial double floor house, very narrow house, small square house and large rectangle house.
Figure 29 3D model examples of houses in one village and these examples are exaggerated in hight
44
Figure 30 Overall shape examples of houses
Figure 30, to some extent, shows the typical overall shapes of this village. This true data can be used as the comparable shape to evaluate the binary training result. The implementation of augmentation and batch process is also implemented stably and reliably.
3D DCGANs Reliability test on an existing dataset Generally, it is better to test an untested neural network on a reliable existing dataset before applying it onto customized dataset. The quality of the new dataset is uncertain, so this first experiment uses a dataset of the chairs provided by ShapeNet, which includes 10688 files of several different types of chair. After training 500 epoch, the trained network can generate results showed as below (Figure 31). The networks generate several different types of the chair from subfigure 1 to 8, and the 1 is the rotated vision of 4, the 5 is the rotated version of 8. Although some chairs lack small pieces of either leg or back, most parts of the shapes are well defined. Figure 32 shows a range of data generated while training process.
45
Figure 31 Reliability test on existing dataset
Figure 32 Results of chairs generation
46
3D DCGANs test on the customized dataset Binary encoding test The customized dataset of an ancient village includes 112 houses, and after augmentation, it is a dataset including 1242 files. Figure 33 shows the test result on the customized dataset with the same number of training epoch and other parameters. Subfigures 3, 4 and 5, 7 also show that the rotation feature learned through augmentation data, and the basic form of the whole house is easy to identify. Almost all of the results indicate an inner courtyard space in the middle of the shape. Another important feature the model has learned is the entrance hole of the houses is clear in 3 and 4, but barely seen in 5 and 7 subfigures. Moreover, the feather that some houses have two floors, but others only have one is also learned, for example, 6 and 8 or 1 and 3. The ratio of length and width of the overall shape is another identical feature shown in all results. Nevertheless, it has the same problem with the previous experiment that some parts of the houses are missing, for instance, the subfigure 6 and 2. Figure 34 shows a range of data generated while training process.
Figure 33 Test on a binary custom dataset and results show several critical features of the dataset.
47
Figure 34 Results of village house generation
48
Multiple value encoding test Figure 35 shows the experiment on multiple value dataset with the same amount of epoch and parameters, but the result is unsatisfied. There is no clear shape can be identified, and there is a trend that all values were going down to zero at the end. As the representation of functions, the colors keep changing randomly, which means nothing has been learned from the dataset. Subfigure 7 and 8 seem show some points aggregation of a certain color, but it does not show any identical similarity to the original data.
Figure 35 Multiple value encoding tests
49
Chapter 4: Discussion and conclusion Dataset preparation tool and dataset Dataset preparation tool presented can stably and successfully generate 3D models of the house by using a dataset consists of CAD plan and Excel table. It can also link the property entities successfully with the 3D geometries. It also provides not only strategy and tool for data clean but also two approaches to encode, either through CAD and excel or directly from the 3D model. Meanwhile, it can successfully encode 3D model into both binary file and multiple value file. Additionally, the augmentation algorithm can successfully augment small size dataset to a larger size dataset. Finally, the batch processing strategy is also successful, as it allows all houses in one file to be implemented and augmented at once. The new format with multiple value labels is meaningful in machine identification, but it needs further research on how to learn it more appropriately. Several aspects need to improve, such as qualifying the dataset, choosing the data source. In this research, the quality of the dataset is uncertain, in which the quality means whether it appropriate for a training task if there is an additional method to qualify dataset before training, the performance would be better. When it comes to the problem of how to set the parameters for augmentation, it requires more experiments on the test of different parameters. Furthermore, the data source is another problem; real-world data has a lot of noise, and it requires more efforts on data cleaning.
3D DCGANs This neural network can stably implement a learning task and can successfully generate new 3D objects similar to the inputs, especially on the binary files. The augmentation algorithm also performs well since the experiment result includes several rotated 3D models. To some extent, this 3D DCGANs, as a learning algorithm, it can successfully learn the form of architectural 3D objects, which has an architectural meaning is to extract several features of
50
houses from a specific dataset by a statistical methodology. Thus, in a scenario that designer has no cognition of local buildings, it can contribute to a general awareness of them. Furthermore, it may contribute to the next step of house function generation, when treating it as the input of a 3D conditional GANs, it has potential to be the constrain condition that learning algorithm can learn from (Figure 36). Generally, the 3D DCGANs is very sensitive to parameters, and this 3D object generation approach does not allow the participation of designers. The generator randomly generates uncontrollable results, so it is an obstacle of using this model directly into a design context, in which participation of a designer in the decision is required. Moreover, it does not consider other design constraints, so the application is still limited. Moreover, the training test on multiple value format performed unsatisfiable, which is an experimental approach towards house function generation. It may result from several reasons: 1. Using this format is an entirely new experiment which has no prior successful references. 2. There is no similar existing dataset allowed to use as a comparable control group for testing. 3. In comparison to the Chair dataset of ShapeNet, which contains 10688 files, the size of the customized dataset is comparatively small, in which only has 1242 files. 4.
The multiple value problem is treated as a regression problem in this study, maybe it is better to be treated as a multiclassification problem.
5. As for the perspective of the neural network, there are several different combinations of parameters need further experiments, such as learning rate, using soft labelling or not. Otherwise, what is the reasons leading to this result cannot be identified.
51
Figure 36 Future training strategy of using 3D conditional GANs based on the research of this thesis.
Future work In regarding of dataset preparation tool, it is better to have a qualification function which can exam the quality of dataset for the desired training purpose. As for the dataset, it is better to prepare a comparable dataset of clean artificial data, and a more general design dataset embed with the organization knowledge of living units. The first is useful for test new real-word
52
dataset as a controlled group, and the latter is more meaningful for more general design purposes. As for 3D DCGANs, more experiments need to be conducted to profoundly exploring the relationship between results and parameters. Besides, developing vary architectures of the neural network may lead to more accurate generated results. The failed experiment on multiple value dataset insights a new approach which is shown in Figure 36. A 3D conditional DCGANs may be able to concatenate the 3D DCGANs to generate functions based on the result of 3D DCGANs.
53
References Ahmed, Eman, Alexandre Saint, Abd El Rahman Shabayek, Kseniya Cherenkova, Rig Das, Gleb Gusev, Djamila Aouada, and Bjorn Ottersten. 2018. “A Survey on Deep Learning Advances on Different 3D Data Representations” 1 (1): 1–35. http://arxiv.org/abs/1808.01462. Alpaydin, Ethem. 2014. “Introduction to Machine Learning Ethem Alpaydin.” Introduction to Machine Learning, Third Edition. https://books.google.co.uk/books?hl=en&lr=&id=tZnSDwAAQBAJ&oi=fnd&pg=P R7&dq=Introduction+to+Machine+Learning+Ethem+Alpaydin.&ots=F2_W7V9rxg &sig=mviGtMwDf2AhgSz5JFpQjsWrKU&redir_esc=y#v=onepage&q=Introduction to Machine Learning Ethem Alpaydin.&f=false. Bachman, Leonard R. 2012. Two Spheres: Physical and Strategic Design in Architecture. Two Spheres: Physical and Strategic Design in Architecture. https://doi.org/10.4324/9780203120101. Bruce G. Buchanan. 2005. “A (Very) Brief History of Arti Cial Intelligence.” AI Magazine, 53–60. https://web.archive.org/web/20070926023314/http://www.aaai.org/AITopics/assets /PDF/AIMag26-04-016.pdf. Chang, Qiang, BoChao Zhang, ShuYao Dai, and LiQun Zhao. 2015. “Data Science Aided Design:An Effective Method and Supporting Tool for Architectural Programming.” Design Community 4: 60–69. http://gb.oversea.cnki.net/KCMS/detail/detail.aspx?filename=ZUQU201504025&db code=CJFD&dbname=CJFD2015. Cheng, Shiyang, Michael Bronstein, Yuxiang Zhou, Irene Kotsia, Maja Pantic, and Stefanos Zafeiriou. 2019. “MeshGAN: Non-Linear 3D Morphable Models of Faces.” http://arxiv.org/abs/1903.10384. Eltarabishy, Sherif. 2017. “Towards Data-Driven Design : Leveraging Open Data and Deep Learning in Symbol Spotting for Furniture Layout Planning.” Goodfellow, Ian J., Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. “Generative Adversarial Nets.” Advances in Neural Information Processing Systems 3 (January): 2672–80. Haviland, William A. 2019. “Ii. Architecture.” Excavations in the West Plaza of Tikal, 3–30. https://doi.org/10.9783/9781949057027-006. Inkawhich, Nathan. 2017. “DCGAN Tutorial — PyTorch Tutorials 1.6.0 Documentation.” 2017. https://pytorch.org/tutorials/beginner/dcgan_faces_tutorial.html. Jaesung Rim. 2017. “GitHub - Rimchang/3DGAN-Pytorch: 3DGAN-Pytorch.” 2017. https://github.com/rimchang/3DGAN-Pytorch. Radford, Alec, Luke Metz, and Soumith Chintala. 2016. “Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks.” 4th International Conference on Learning Representations, ICLR 2016 - Conference Track Proceedings, 1–16.
54
Russell, Stuart, and Peter Norvig. 2010. Artificial Intelligence A Modern Approach Third Edition. Pearson. https://doi.org/10.1017/S0269888900007724. Sedaghat, Nima, Mohammadreza Zolfaghari, Ehsan Amiri, and Thomas Brox. 2017. “Orientation-Boosted Voxel Nets for 3D Object Recognition.” British Machine Vision Conference 2017, BMVC 2017, 1–18. https://doi.org/10.5244/c.31.97. Wichmann, Andreas, Amgad Agoub, and Martin Kada. 2018. “ROOFN3D: Deep Learning Training Data for 3D Building Reconstruction.” International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences - ISPRS Archives 42 (2): 1191–98. https://doi.org/10.5194/isprs-archives-XLII-2-1191-2018. Wu, Jiajun, Chengkai Zhang, Tianfan Xue, William T. Freeman, and Joshua B. Tenenbaum. 2016. “Learning a Probabilistic Latent Space of Object Shapes via 3D GenerativeAdversarial Modeling.” Advances in Neural Information Processing Systems, no. Nips: 82–90. Wu, Zhirong, Shuran Song, Aditya Khosla, Fisher Yu, Linguang Zhang, Xiaoou Tang, and Jianxiong Xiao. 2015. “3D ShapeNets: A Deep Representation for Volumetric Shapes.” Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition 07-12-June: 1912–20. https://doi.org/10.1109/CVPR.2015.7298801.
55
Appendix 3D DCGANs for binary dataset: https://drive.google.com/file/d/1YtSRO0KwdyCVyAkJGFsAo0im9Z5nvJI/view?usp=sharing 3D DCGANs for multiple value dataset: https://drive.google.com/file/d/1yOIdtygvDUjP5RoJKh5NcsK5QAz1gBfU/view?usp=sh aring Binary dataset (Augmented to a big size) https://drive.google.com/file/d/1JGaa_rAfz74ez7RPZhO2UkPdb9WYbni/view?usp=sharing Multiple value dataset (Augmented to a big size) https://drive.google.com/file/d/1EaUlSjUvzkRokdfoVdjqfJFtQLqxiYt3/view?usp=sharing Binary dataset (Augmented to a small size) https://drive.google.com/file/d/1bIUXdptDKOGom08LMfZhI2Nn_Q3j9dme/view?usp =sharing Multiple value dataset (Augmented to a small size) https://drive.google.com/file/d/1YtSRO0KwdyCVyAkJGFsAo0im9Z5nvJI/view?usp=sharing
56
Implementation of dataset to 3D geometry https://drive.google.com/file/d/1pm4AWGpYTSEuEppGdKbsdTKluPqStR6s/view?usp =sharing Implementation of dynamic voxelization https://drive.google.com/file/d/1d6tyXtZ_RU55gRTL3vOqmF7as7aMdtiU/view?usp=sha ring
57