Research topic 3D Laser Scanner Surveying and VR Production from Point Cloud Data
Subtitle ~Creation of Digital Archives for Prefectural Designated Cultural Properties and Landscape Formation Significant Buildings
Keywords DX Literacy, Curriculum development, Cultural heritage, Digital archive, Industry-Academia-Government Collaboration, Local contribution
School Name Hyogo prefectural Toban technical high school
Location 1748-1 Kanki Higashi-kanki Kakogawa Hyogo 675-0057
Author Hironori Ohtoshi, Studio Master at FabLabStudio
1. Research background and problem awareness
The modern society is rapidly changing due to the Fourth Industrial Revolution. Society 5.0, advocated by the Cabinet Office, suggests that advanced technologies such as artificial intelligence (AI), big data, Internet of Things (IoT), and robotics are being integrated into various industries and social life, leading to a dramatic change in the very nature of society, which can be described as 'discontinuous.' In the midst of such societal changes, the Ministry of Education, Culture, Sports, Science and Technology has set forth 'Education for Society 5.0,'1 stating that 'it is essential for individuals who have been engaged in manufacturing and services in various fields to expand into various fields while maximizing the use of AI and data capabilities’ 2
On the other hand, the 'Future Talent Vision'3 by the Ministry of Economy, Trade and Industry predicts a 35% decrease in employment in the construction industry by 2050 4 It underscores the urgent need for the cultivation of digital talents who can leverage advanced technologies to create new value, as the construction industry also grapples with complex and challenging societal issues and aims for sustainable societal development.
In the traditional construction industry, skills such as design, construction, and safety management have been sought after. However, in the society that students will live in, beyond these aforementioned skills, the ability to create new value by leveraging advanced technologies is also required. Therefore, even in technical high schools, in addition to traditional technical education, there is a need to cultivate students who can utilize advanced technologies to tackle problems and engage in manufacturing using AI, virtual reality (VR), augmented reality (AR), and other technologies.
The Ministry of Education, Culture, Sports, Science and Technology (MEXT) has undertaken efforts to facilitate the establishment of a conducive environment for the realization of Smart Specialized Schools, including technical high schools, in the fiscal year 2020. 5 Furthermore, in the fiscal year 2024, MEXT launched the High School DX Acceleration Promotion Project 6 In our school's architecture department, we introduced 3D laser scanners in the fiscal year 2022 to aim at nurturing students capable of engaging in manufacturing by leveraging cutting-edge technologies. However, there is currently no established effective teaching model for fostering students who can acquire advanced technological skills and concretely conceptualize how to utilize them to solve various problems.
2. The objective of the research
Building upon the aforementioned problem awareness, the objective of this research is to elucidate a teaching model aimed at nurturing students who actively engage in problem-solving utilizing advanced technologies. Furthermore, the practical aim is to foster DX literacy by comprehending the background and significance of DX, understanding the utilization of data and digital technologies, and acquiring advanced technologies applicable to real-world challenges.
3. Research methodology
This practical application will be conducted within the context of project-based learning classes, targeting 8 students from the FabLabStudio (Project Research Group).
First, we will clarify the definition of DX literacy and develop a teaching model to foster DX literacy. Next, we will implement the developed teaching model in practice and analyze the results to verify the effectiveness of the teaching model.
For the analysis, we will utilize a questionnaire incorporating the following items: 'Mindset,' 'Why,' 'What,' and 'How.' This decision is influenced by the DX Literacy framework provided by the Ministry of Economy, Trade and Industry, which sets learning goals for each of these aspects (refer to Figure 1). For each item, we will prepare a combination of open-ended questions and Yes/No format questions, and then analyze the practical results accordingly.
4. Development of teaching model (1) DX literacy
The Ministry of Economy, Trade and Industry has set a goal of ‘each business person acquiring DX literacy, enabling them to perceive DX as their own concern and take action toward transformation’ , 7 and has established the 'DX Literacy Standard' to achieve this. Furthermore, DX literacy is represented as a collective of four elements, as shown in Figure 1. 8
In other words, "Mindset" refers to the awareness and attitudes necessary to create new value amidst societal changes, "Why" pertains to understanding the background of DX, "What" involves understanding the data and technologies utilized in DX, and "How" demonstrates an understanding of the methods for utilizing data and technology. Furthermore, the Ministry of Economy, Trade and Industry has outlined the pathway to growth as a DX talent (Figure 2).
(2) Development of a teaching model to foster DX literacy
Taking inspiration from the pathway to growth as DX talent, we will develop a teaching model to foster DX literacy (see Table 1).
Specifically, the initial unit 1 will focus on the 'awareness/exploration' stage of advanced technologies in the construction industry, where students will experientially learn about the necessity of utilizing advanced technologies in the construction industry and the technologies being advanced.
The following unit 2 will represent the 'trial/use' stage for advanced technologies. Building upon the knowledge gained in unit 1, students will work on digitizing local cultural heritage using 3D laser scanning and VR technology.
The final unit 3 signifies the 'application' stage for advanced technologies, where students will present their projects to the local community and the construction industry, engaging in discussions on the utilization of 3D laser scanning and VR technology in practice, thereby exploring the possibilities of advanced technology utilization.
Regarding the acquisition of 3D laser scanner operation skills marked as (1) in Table 1, a total of 35 students from the architecture department, including 8 members of the FabLabStudio, will participate in the elective practical training.
(1) Mastery of 3D laser scanner operation (selective practice)
• Repeatedly practicing practical and theoretical aspects to acquire the basics.
• Conducting point cloud measurements, considering the school building as the site.
scanning at Motookake Residence
May to July
2. VR creation of Motookake Residence
July 25-26
(2) 3D laser scanning and point cloud data processing of the interior and exterior of the Motokake Residence
Explanation of activity progress and demonstration of 3D laser scanner at the 1st Open High School
August 5
September to October
October 11
Panel exhibition at the summer children's workshop (held in commercial facilities)
(3) VR production and practice for VR demonstration
(4) Toda Mirai Foundation activity report meeting (including VR of significant landscape formation)
November 2 (5) VR demonstration at the high school cultural festival
November 8
3. VR e xperience event and achievement p resentation meeting
November 12
November 18
December
December 13
January 19-21
February 1
(6) Explanation of activity progress and VR demonstration at the 2nd Open High School
(7) VR experience event at Kakogawa Two-Day March (City Hall Plaza)
(8) Activity presentation and VR experience event at Hyogo Industrial Education Fair
Compilation of booklet and video summarizing activities
(9) Presentation of achievements and VR experience event at the Kakogawa City Hall
(10) Architecture department exhibition (held at commercial facility plaza)
Local contribution activities and revitalization project - in-school practice presentation event
• Collaborating with the municipal board of education to conduct point cloud measurements.
• Explaining the activity to middle school students.
• Receiving instruction on surveying methods from specialized companies "Knowing / Investigating"
• Introducing the technical aspects of 3D laser scanner through demonstrations.
• Promoting activities to the local community.
• Introducing construction ICT learned in the industrial department.
• Learning knowledge, skills, and their application methods.
• Practicing VR operation firsthand for better understanding.
Presenting FabLabStudio's activities to the construction industry.
Introducing VR technology to students from other departments.
• Explaining activities to middle school students.
• Introducing architectural VR through demonstrations.
Promoting the digital archiving of cultural heritage activities to citizens.
• Promoting the VR digitization of prefecturally designated cultural assets
• Introducing VR experience events through the media.
Using DTP and video editing software.
• Donating point cloud data and VR videos.
• Participating in the municipal board of education's sponsored results presentation.
• Creating panels to promote activities.
• Exhibiting activity results externally.
• Creating a narrated activity report video.
• Presenting activity results within the school.
5. Implementation of the teaching model
(1) Mastery of 3D laser scanner operation
The 35 students from the architecture department who opted for practical training as an elective subject were divided into three groups, and each group received six hours of classes. Initially, they experienced hands-on training with the 3D laser scanner* (Figure 3) and deepened their knowledge through reports (Figure 4), repeating this process to acquire basic skills. Furthermore, they conducted surveys considering the practice building as a real site and performed data processing on PCs (Figure 5). Ultimately, using the point cloud data, they produced fly-through videos (Figure 6) of both the interior and exterior of the practice building.
(2) 3D laser scanning and point cloud data processing of the interior and exterior of the Motokake Residence
In collaboration with the Kakogawa City Cultural Heritage Survey and Research Center, the Motookake Residence, a designated tangible cultural property built in the Edo period (year 1694), was surveyed using a 3D laser scanner. Eight students from the FabLabStudio conducted approximately 10 hours of on-site surveys every Monday during the 5th and 6th periods. Laser scanning was performed at approximately 70 locations both inside and outside the building (Figure 7), resulting in the collection of approximately 300 million point cloud data points. Additionally, the students received explanations about this large-scale private house from the city's curator (Figure 8) and learned how to use laser scanner targets from a surveying company (Figure 9).
Subsequently, data processing, including point cloud synthesis, was performed on a PC (Figure 10).
* The 3D laser scanner emits lasers radially, which are then reflected upon hitting the surface of objects. This process detects the surface shape of the object and measures the coordinates of these points. By employing multiple lasers, the scanner collects multiple points from various parts of the object, arranging them in a three-dimensional coordinate system to generate data. This data, known as a point cloud, accurately reproduces the shape of the object. The point cloud includes not only the position of each point but also color information, reproducing the coloration of the object.
(3) VR production and practice for VR demonstration
Following that, the same eight students loaded the point cloud data of the Motookake Residence into the Unity** game engine using the Stipple*** plugin, and constructed a VR environment (Figure 11). Subsequently, they practiced operation by wearing head-mounted displays (Figure 12).
(4) Toda Mirai Foundation Activity Report Meeting
The construction industry organizations, companies, and schools each presented their activities. Officials from the Ministry of Land, Infrastructure, Transport and Tourism and the Ministry of Health, Labour and Welfare also attended as guests, and as representatives of the schools, students provided an online activity report for the past year, including the production of digital archives of the former Kakogawa Library / Important Cultural Property for Landscape Formation (Figures 13, 14).
(5) VR demonstration at the high school cultural festival
We conducted a VR experience session of the Motookake Residence for students, teachers, and parents from other departments who visited the demonstration area, providing guidance on how to wear the headset (Figure 15) and operate the controllers (Figure 16).
(6) 3D laser scanner and VR demonstrations at the Open High School
Students conducted explanations of the activities at the Motookake Residence (Figure 17), demonstrations of the 3D laser scanner (Figure 18), and VR experience sessions (Figures 19, 20) for 80 junior high school students considering enrollment in our school's architecture department and their parents.
(7) VR experience event at the Kakogawa Two-Day March
To broadly promote our school's activities and let the local community know about the wonders of craftsmanship, we set up a booth at the central venue in front of the city hall every year (Figure 21). At this booth, we conducted VR experience sessions for the Motookake Residence (Figure 22).
(8) Activity presentation and VR experience event at the Hyogo Industrial Education Fair
PR videos and exhibits of works by prefectural technical high schools, along with a workshop, were held at the plaza of a commercial facility in the city. The students presented the content of this activity on stage (Figure 23) and also conducted a VR experience session (Figure 24).
(9) Presentation of achievements and VR experience event at the Kakogawa City Hall
The survey data of the Motooka Residence was donated to the city of Kakogawa (Figure 25), and the results were presented. A promotional video of the same residence created in VR is publicly available on the website of the Kakogawa City Cultural Heritage Survey and Research Center (Figure 26).
(10) Architecture department exhibition
Local contribution activities and revitalization project - in-school practice presentation event
The results of the activities were exhibited and presented both within and outside the school (Figures 27, 28).
** Unity is a game engine that includes an integrated development environment
*** Stipple is a Unity plugin used for importing, editing, and visualizing point clouds.
and many
using it
6. Reflection on Classroom Practice
As outlined in the research methodology, we created a custom questionnaire focusing on each aspect of DX literacy: "Mindset," "Why," "What," and "How." The questionnaire was administered to the eight students from the FabLabStudio who participated in the developed teaching model.
Below, we present the responses, aggregated results, and further analysis for each item. Please note that only representative examples of responses for each item will be provided for the open-ended questions.
The "Mindset" is defined as the awareness, attitude, and behavior necessary to create new value. We have summarized the results of the questionnaire based on this perspective (Table 2).
Table 2: Questionnaire and Responses, Aggregate Results from the Perspective of "Mindset"
How can the knowledge gained from this year's efforts in digitizing cultural heritage be applied in the future?
When we created VR of the Edo-era house located in our local area and donated it to the city, the city officials who experienced the VR were amazed and found it "interesting". It's my pride that this VR will continue to exist in the future. However, at this point, I cannot specifically envision which cutting-edge technologies to utilize to solve local issues. Nevertheless, I hope to contribute to the local community by leveraging the cutting-edge technologies learned here. (Student A)
Have you researched the application methods and examples of advanced technologies in the construction industry outside of class?
Yes: 1 student (Student B), No: 7 students (Students A, C-H)
In the response from Student A, a positive attitude towards a practical project, involving digitizing local cultural heritage and donating it to the city, suggests an increased willingness to contribute to the community using advanced technology. However, only one student, apart from the classroom, researched the application methods and examples of advanced technology. The remaining seven students did not. This indicates that, despite students showing interest in advanced technology, they did not engage in self-directed learning. This suggests that the cultivation of "Mindset" was insufficient.
For the "Why" perspective, we established specialized questions focused on whether students recognize the effectiveness of utilizing data and digital technology to improve people's lives and address societal challenges (Table 3).
Table 3: Survey and Responses from the "Why" Perspective
Why is advanced technology necessary in the construction industry?
Advanced technology is essential for streamlining operations and addressing labor shortages in the construction industry. In particular, the 3D laser scanner used in this project enables swift and high-precision surveying, and remote operation via tablet allows measurement of hazardous areas, thereby enhancing safety on the job. (Student C)
Do you understand the necessity of advanced technology in the construction industry?
Yes: 7 students (Students B to H), No: 1 student (StudentA)
Student C's response indicates an understanding of the necessity of advanced technology in the construction industry. Particularly, they provide specific reasons for the relevance of the 3D laser scanner used in class to improve efficiency and safety on construction sites. This demonstrates the student's
comprehension of the background behind DX and suggests that they have developed an understanding from the "Why" perspective.
In the "What" perspective, we evaluated the understanding of students regarding the recognition of data and digital technology as means to promote DX. The following questions were posed for this purpose (Table 4).
Table 4: Survey and Responses from the "What" Perspective
What kind of advanced technologies are being utilized in the construction industry?
In addition to Information and Communication Technology (ICT) and Software as a Service (SaaS), there are other technologies utilized in the construction industry such asAI, deep learning, and drones. (Student C)
Do you understand the advanced technologies utilized in the construction industry?
Yes: 8 students (Ato H), No: 0 students
The fact that all students have knowledge about advanced technology and understand specific technologies in the construction industry indicates that the "What" perspective has also been fostered.
We evaluated whether students understood examples of utilizing data and digital technology and whether they could imagine practical applications from the "How" perspective (Table 5).
Table 5: Survey and Responses from the Perspective of "How"
How do you think advanced technology can be utilized in the construction industry?
At the house manufacturer where I have a job offer, it is expected that the issue of skill inheritance can be resolved by combiningAI with Mixed Reality (MR), allowing craftsmen's skills to be verified through goggles.
Do you understand how advanced technology is utilized in the construction industry?
Yes: 7 students (Students B to H), No: 1 student (StudentA)
Student D has demonstrated the ability to envision and propose specific examples of how advanced technology can be utilized in the construction industry. The student shows an understanding of how to apply data and technology, thus fostering the "How" perspective.
Analyzing these surveys, it can be inferred that among the four components constituting DX literacy, there appears to be insufficient development in fostering the "Mindset" aspect, while some achievements were noted in the other components. Students succeeded in promoting local cultural heritage through activities involving VR, receiving high praise from the local administration. This likely nurtured a mindset and attitude among students to utilize digital skills and contribute actively to society. Therefore, there seems to have been some success in developing the "Mindset" aspect as well.
However, there is a tendency that the creativity necessary to execute this is not sufficiently nurtured. Therefore, as an improvement suggestion for the teaching model, it is proposed to introduce a reflection unit at the end of the lesson plan. This would provide students with an opportunity to apply the knowledge and digital skills they have learned in class to real-world problems, thereby deepening their learning. It is expected that students will reflect on their own growth and problem-solving process, and develop specific action plans to apply to future activities.
7. Research Achievements and Challenges
The developed teaching model has been shown to significantly foster the ability to tackle challenges using advanced technology. Additionally, based on the analysis indicating insufficient development in fostering the "Mindset" aspect, there is a proposal to introduce a reflection unit at the end of the lesson plan. This is aimed at nurturing the ability of more students to adapt to societal changes, engage in self-directed learning, acquire new knowledge and digital skills, and conceive concrete solutions to problems.
Hyogo Prefecture boasts the highest number of ancient burial mounds in Japan, and there are burial mound clusters around our school. Therefore, for the next academic year, we plan to promote student exchanges beyond the scope of the industrial science department and encourage self-directed learning of Geographic Information Technology (GIT). Specifically, new students at FabLabStudio will engage in activities such as 3D laser surveying of burial mounds, soil volume calculations, contour mapping, and utilizingAR technology to inherit historical sites for future generations, fostering DX literacy.
In this project, we aim to demonstrate that by adding a unit for reflection to the teaching model, students can acquire the ability to utilize advanced technologies to address real local challenges while flexibly adapting to the rapid changes in society.
8. Acknowledgments
We would like to express our sincere gratitude to the Cultural Heritage Survey and Research Center of Kakogawa City and Kobe Seiko Co., Ltd. for their invaluable cooperation and support.
9. References
1. Ministry of Education, Culture, Sports, Science and Technology. (2018). Human Resource Education for Society 5.0, p. 2.
2. Ministry of Education, Culture, Sports, Science and Technology. (2018). same document, p. 7.
3. Ministry of Economy, Trade and Industry. (2022). Future Talent Vision.
4. Ministry of Economy, Trade and Industry. (2022). same document, p. 24.
5. Ministry of Education, Culture, Sports, Science and Technology. (2020). FY 2020 Supplementary Budget for the Ministry of Education, Culture, Sports, Science and Technology - Data Collection by Project, p. 14.
6. Ministry of Education, Culture, Sports, Science and Technology. (2023). Accelerating High School DX Promotion Project (DX High School).
7. Ministry of Economy, Trade and Industry. (2022). DX Literacy Standard, pp. 5-7.
8. Ministry of Economy, Trade and Industry. (2022). same document, pp. 5, 6, 17.
9. Ministry of Economy, Trade and Industry. (2022). same document, p. 17.
10. Ministry of Economy, Trade and Industry. (2023). Digital Skill Standards, p. 13.
