1. Introduction

The relevant definitions and scopes of Building Information Modelling (BIM) are constantly evolving over time. The American Institute of Architects (AIA) defines BIM as a multi-dimensional model presentation technology that adopts the database approach to integrate information of construction projects. In recent years, as the “Digital Twin” has been repeatedly regarded as one of the top ten global technology trends of the year by the information technology research organization [1], it could be understood that such trend is highly connected with BIM as it integrates multiple proposed applications in the technological and related fields.

In addition to implementing strategies in compliance with government policies, the BIM TASK GROUP, established under the promotion of the UK government, has its major goal of promoting the various duties of the building production and construction supply chain. However, after the government agencies having formulated various collaborative work specifications, starting from April 2016, it could be foreseen that building information models with highly collaborative functions would even become one of the basic elements of future architectures [2].

BIM plays an important role in all stages of the building life cycle, making it infrastructure of the future as one of the trends. Because BIM not only serves as a beneficial tool or database in the construction phase, but also provides various flexibility and integration functions that can offer various requirements for building O&M, and exerts powerful application integration capabilities(see Figure 1).

Figure 1: The value of documentation in the facility lifecycle
(Source: Eastman, 2008)

For the current application of building O&M, the importance of facility management (FM) and the improvement of building efficiency and functionality in facility management have gradually become prominent. Nowadays, increasingly complex building functions are making the building space and equipment more diversified and sophisticated. Building management is no longer simply checking whether it is operating normally, but finding out its unique usability based on the huge volume of building usage information generated, real-time spatial analysis, and retrieval. The changeable[3] complexity and real-time nature of modern computer technology can assist decision-making and provide optimized applied artificial intelligence (AI). For a building space involving a wide range of levels, the integration of software and hardware and the deployment of various systems, such as environmental data, the number of users, and the analysis of crowds, etc., would have a certain degree of crucial impact on building O&M. In response to such impact, this study believes that the usage status of these buildings must be actively displayed through machine learning and the application of AI technology, and such relationship of changes requires to be reflected in the O&M process. In order to solve the problem of BIM in the O&M of building facilities, this study proposes the “BIM management platform for decision-making”, which is hereby referred to as “SyncBIM”[4].

2. Related Work

In this Section, we present two popular aspects of adopting BIM in the building life cycle, which are Digital Twin and AI applications. The first lays the foundation for the cloud platform architecture, and diversified AI applications drive BIM to become the “sense” of architectures.

2.1. Application of Digital Twin to BIM

AEC includes buildings&architectures, engineering, and construction industries. At present, BIM technology has reached certain achievements under the promotion of government agencies of various countries. Meanwhile, with the evolution of information and communication technology, more application technologies such as computer vision, AI, and various sensors and environmental sensing devices such as LiDAR could be able to complement the possibility of collecting real-time and on-site data in a real environment reality, which would allow the BIM model to predict and plan the allocation of resources and transportation, and supervise the safety of the site environment. The introduction of Digital Twin at an earlier stage can further enhance the application of BIM advantages of the whole building life cycle.

An indicative example of a Digital Twin application is the “Virtual Singapore” developed by the French software company Dassault Systèmes for five years. On this virtual three-dimensional platform, one can freely browse all corners of Singapore City, regardless of the data of buildings, roads, public infrastructure, or the natural environment[5]. More importantly, it provides four main functions, namely “Virtual Experimentation”, “Virtual Test-Bedding”, “Urban Planning and Decision-Making”, and “Research and Development” that allow decision-makers of urban affairs to predict the impact of different decisions and policy directions on the city. Therefore, there would be wider application in the fields of urban disaster prevention such as logistics and environmental conservation, etc.

Figure 2 shows the UI of Virtual Singapore. With real-time information and the large volume of data provided by sectors from public departments and communities, Virtual Singapore is able to solve the problem of livability in areas with limited resources and unlimited potential. For example, after importing annual data of insolation areas, temperatures, and humidity, the system would be able to provide space decision-makers with the information for making a more comfortable and humanized environment.

Figure 2: Interface of Virtual Singapore (Source: nrf.gov.sg)

From the perspective of the “smart city”, infrastructure is the element that composes a city. The system developed by Dassault Systèmes allows people to see the digitization of urban infrastructure, which is constructed in the digital domain of computers and adds value to or optimizes the decision-making for the city to shape the vision of smart city life of the future[6]. The BIM model plays the role of an information carrier, and after importing the information, it serves as a bridge between the real and virtual worlds. In the next stage, the health and safety issues at the construction site, which has the widest impact and draws the most attention during the construction phase, will be further reviewed and discussed in the scope and application focused on by this study, and the development of the research proposed by this study will be applied to the connection of the construction site to make trigger more innovative and progressive linkage[7].

2.2. Application of Artificial Intelligence in the Building Life Cycle

The current major technology of artificial intelligence (AI) aims at analyzing and realizing professional human techniques of various fields through extensive computer science training [8]. With its quality management and supervision functions, AI could replace a large amount of work previously requiring human judgment, which would achieve accurate and effective predictive capability [9]. For example, for a construction site facing the pressure from both construction schedule and cost while requiring the high integration of the construction specifications and interface of various fields, the introduction of AI and BIM technology for the integration of various information would provide a certain degree of efficiency improvement and added value[10].

According to the research conducted by the Science & Technology Policy Research and Information Center of the National Applied Research Laboratories, The assistance provided by AI to the construction industry can be roughly divided into four categories as shown in Table 1, which are respectively known as (1) Planning and Design, (2) Safety, (3) Automation Equipment, and (4) Monitoring and Maintenance.

The following four AI application areas proposed by the Science & Technology Policy Research and Information Center of the National Applied Research Laboratories are all able to be highly connected with BIM:

• Planning and Design

Table 1: Application of AI in the Construction Industry (Source: Science & Technology Policy Research and Information Center of the National Applied Research Laboratories, 2018)

Application of AI in the Construction Industry
Categories	Contents
Planning and Design	The BIM model uses AI to automatically complete pipeline configuration, analysis, and machine learning to manage construction risks
Safety	AI with sound and image recognition system to automatically mark the construction site
Automation Equipment	Smart building management to make users feel comfortable
Monitoring and Maintenance	AI with robots and drones to improve construction efficiency

The construction site requires to consider structural, electrical and mechanical, and pipeline engineering issues, etc., which involve different contractors using different appliances and interfaces that may often cause delays and repetitive work in the final integration stage. Therefore, Building System Planning developed GenMEP by using BIM modeling software Autodesk Revit. By adopting Generative Design, the configuration of the electrical system in the building model can be automatically built using the GenMEP plug-in while also taking into account the design specifications and the spatial complexity of different buildings (see Figure 3).

Figure 3: Interface of GenMEP (Source: BuildingSP, Inc.)

• Safety

AI technology such as data analysis and machine learning would be adopted to manage construction risks. BIM 360 Project IQ launched by Autodesk is an example in which the BIM 360 Field is shaped by various images, audio, and other construction management information collected on site, which is analyzed to provide suggestions and reduce risks for relevant QA staff on site (see Figure 4).

• Automation Equipment

Jetson AI platform constructed by Nvidia and the Japanese company Komatsu is designed to perform 3D visualization and analysis of real-time interactive modes such as tracking the on-site staff, machinery, and objects, which could allow AI to take over tasks of on-site monitoring and management to provide round-the-clock protection for site safety, as well as improving the quality of construction management.

Figure 4: Interface of BIM 360 Project IQ (Source: BIM360)

• Monitoring and Maintenance

Suffolk Construction based in Boston adopts a virtual model constructed by VR to monitor the progress of on-site construction projects to cope with the rising construction costs and on-site safety management, as well as improving construction efficiency. In addition, Figure 5 shows that Doxel AI from Silicon Valley combines exploration robots and drones with photography equipment and LiDAR to scan and construct the real-time situation of the site, and adopts visualization data and deep learning algorithm processing to allow users and managers to automatically eliminate various errors and monitoring risks at the construction site by using the on-site construction progress analysis adopting AI technology.

Figure 5: Visualization of On-site Construction Information by Doxel (Source: Doxel AI)

The key points and precautions of the occupational safety inspection of each working item are clearly-defined specifications, for example, the specifications for the categorization of hazardous areas and the key points of inspection. Hence, AI could serve as a 24/7 supervisor to check items such as whether safety equipment is worn and any staff is entering the hazardous areas by adopting the image recognition technology to conduct round-the-clock monitoring at the construction site. For example, the setting of geofencing allows personnel entering the hazardous areas to be adequately alerted and protected while notifications are also provided to have managers aware of the current presence of high-level risks of the staff (see Figure 6), and the real-time information could also be fed back to the Digital Twin’s BIM environment integration platform and presented in a visualized manner. With the integration of BIM and various AI techniques, the required time and cost for the construction site could be improved in terms of management efficiency, and the occupational safety and working environment for labors could also be assured, which would achieve a win-win situation[11], [12].

Figure 6: Construction Site Management Adopting AI Image Recognition Technology (Source: Smartvid.io)

The definition and application of BIM often evolve along with various technological advancements. And the pursuit of environmental friendliness and green living has always been its core value. Building O&M and management is a complex interface integration and multiple interdisciplinary coordination. However, when carrying out various intimate and highly customized O&M processes for future space users, a large number of people would be exposed to a high degree of on-site O&M risks for achieving the establishment of a smart city or its infrastructures. If the managers and participants could improve the building O&M mode in a safer and more efficient way, it will be more complete and environmentally friendly when establishing an overall smart city, which would also be a precious gift to the fields of building O&M.

3. The Planning and Design of Decision-Making BIM-Based Cloud Platform

This Section mainly explores the system adopting BIM in the auxiliary decision-making platform. It would discuss the information processing flow and architecture of the smart building O&M system in two parts respectively known as the information transmission architecture the system operates, and the information transmission loop of the system.

3.1. Information Transmission Architecture of the System

Figure 7 describe such information transmission architecture. The architecture itself adopts the IPO model as the application basis, and in the process of data transmission, a web portal is established between the endpoints in order to make the clearly defined division of labor of the system. The scalability and integration of the web portal also provide flexibility for the equipment and sensing devices installed in each period of the system’s operation. And it is also helpful for the integration of information formats. For the transmission architecture specified in this Section, we will further describe it in detail in the following three subsections respectively titled (1) Input, (2) Process, and (3) Output.

Figure 7: Diagram of Information Transmission Architecture for the Smart O&M System

• System Input: Information Collection

The information sources the smart building O&M requires could be summarized as three scales. One is the meta data that covers various real parameters and O&M mode scripts, which transmit the three-dimensional information model to the web portal and subsequently to the dynamic database for storage, integration, and further process. The meta data includes the real parameters, the scripts and modes related to various building O&M. If it is necessary to perform appropriate O&M for a certain building, the data monitoring operation would start the deployment independently without waiting for the completion of the system installation, and the scaling of input data monitoring could be performed even under the presence of an existing system which requires only the addition of new web portals. If the building usage data required by the system is transmitted to a specific web portal, such input would be designed to operate independently, and the system only retrieves the data from the web portal to achieve the division of labor and facilitate any subsequent scaling. Furthermore, upon the occurrence of equipment failure or update, the system would not be completely shut down but only some of the functions are temporarily suspended, which would also improve efficiency in troubleshooting for the system.

• System Process: Dynamic Database

Because the various O&M data and 3D models from the input would be updated continuously over time, a dynamic database that can be updated and accessed in real time shall be established for the process. Such a dynamic database for the process would retrieve the information of the web portal, then the system would analyze and compare the usage mode. Because the information carried by the web portal has been standardized, it is sufficient for the database to complete the data connection with the web portal while allowing the system to retain the scalability, which would be helpful for the maintenance and testing of some areas of the system or the addition of O&M areas.

• System Output: Provision of Services

The dynamic database compares and delivers the O&M data to the web portal of the output. Unlike the input web portal, the output web portal saves various service-related information which would be used for building O&M. Rather than being the big data, such types of information required for O&M is integrated information, such as the parameter codes after script writing or the trigger values of execution, which are provided to the application services. The types of application services are divided into two types respectively known as “control type” and “service type”. The output of the information transmission architecture would trigger related services according to different types of parameters.

3.2. Information Transmission Loop

In the adaptive O&M system, the transmission of information is not a straight-linear process but a continuous act of constant updating and modification. Therefore, a loop of information transmission shall be established to complete the architecture required for the O&M maintenance under the constant-changing relationship. When the data of the usage pattern is established in the dynamic database and compared with the analysis characteristics, these data can be used as a basis for system adaptation, learning, and reflective practice. Also, because both people and the built environment would continue to change and influence each other over time, the usage data will continue to be imported to the database, and then the updated results would be transmitted to the end sensors, which forms an information transmission loop accordingly. The three processes of this loop formation are respectively known as (1) Data Collection, (2) Database Comparison, and (3) Database Update. Figure 8 presents the data transmission loop formed by the system.

Figure 8: System Data Transmission Loop of SyncBIM

• Data Collection

After completing the layout of the sensor network for collecting building spatial data, the data via the cloud would start to be transmitted to the designated dynamic database. The types of data refer to the open information, local information, and community information mentioned in the previous paragraph. Such three types of information would be compared and analyzed within the database.

• Database Comparison

When the volume of data within the database is sufficient, it would be available for further analysis to find the average value, critical value, or usage habits of the particular space for the establishment of a spatial usage pattern. The usage pattern upon being determined established could be subject to continuous updates besides being compared with the data. By analyzing the dynamic pattern of the data, one could infer whether there is any device functioning abnormally or abnormal spatial activities, and if so, the information visualization would display the location and time where such abnormality takes place to allow the recording by the system and update to the user mode of the database.

• Database Update

With the increase of usage data and the modification of user mode, the operation of the system will keep all usage information updated along with the building’s existence period and usage history, and update its information to various building equipment and sensors, which also means that the database would continue to be updated over time to continuously adjust the balance between people and the building environment.

4. SyncBIM that Integrates Computer Vision Applications

This Section presents the construction of a SyncBIM system based on the system architecture mentioned in the previous Section. It would discuss in two parts respectively titled “The System Interface Planning”, and “The Process of Integrating Computer Vision Applications”.

4.1. The System Interface planning of SyncBIM

The SyncBIM system is established based on the following five principles respectively known as (1) 3D Information Model Aiding Decision-making, (2) Object Orientation Design Delivering Service, (3) Dynamic Database based on Real-time Monitoring, (4) Mapping Data-driven Visualized Results, and (5) Hierarchical Integration of Contextualized Spatial Information. (see Figure 9).

• 3D Information Model Aiding Decision-making:

The spatial information presented in a three-dimensional manner allows viewers to view in an intuitive way, while also being able to efficiently be aware of the information and contextualized spatial data carried by the objects.

• Object Orientation Design Delivering Service:

The BIM model constructed with individual objects not only uses object-oriented design to guide information to the building objects, but also treats each object as a node that can provide the required services, and synchronously connect the functions and services required for the operation of the O&M system.

• Dynamic Database based on Real-time Monitoring:

When the spatial information from various usage data in real buildings is collected, the system could continuously transmit the parameterized data to the database. In addition to using the database as the source of review for system learning and reflection, such a database would also become a dynamic database for real-time display of building usage.

• Mapping Data-driven Visualized Results:

With the import of real-time data, users would be able to customize the colorized levels for visual alters and the threshold for event trigger with the system’s logical decision-making function, which would allow the decision-making process more adaptable.

• Hierarchical Integration of Contextualized Spatial Information:

The system would be presented in an adaptive hierarchical manner for different scales and service targets. In addition to allowing precise deployment of various services, users could also perceive the contextualized spatial relationship.

Figure 9: System Interface of SyncBIM

4.2. Importing SyncBIM into MATLAB for Computer Vision

      MATLAB is software designed for currently commonly available algorithms, which is mainly used for data analysis and development of algorithms and also image processing and computer vision. The IoT platform “ThingSpeak” owned by MATLAB can transmit the analyzed results and parameters to the IoT platform, and further serve as a scalable modular web portal for the dynamic database of SyncBIM. The API could be used to access the data in the designated channel of the IoT platform (see Figure 10).

Figure 10: ThingSpeak API

       The designated IoT device would be set to ensure that the information is free of overlay (see Figure 11). This step would allow the continuous recording of the on-site data in the IoT platform and transmission to the SyncBIM database.

Figure 11: IoT Device and Object Tracker

The timestamp would be generated for the collected (Figure 12), and such data would be continuously uploaded to the IoT channel shown in Figure 10. The timestamp of the data will be able to provide a future database as a basis for the decision-making of spatial events and situations.

Figure 12: Timer for Logging Data

After running such a program in the IoT device, it can continue to perform image recognition and facial recognition (Figure 13). It would not only calculate the number of people, but also record the time point, upload the data to the designated IoT platform, and continue to be available for retrieval by the SyncBIM System database. Such a device has scalability, which allows it to be installed in different spatial architecture and scaled as required. For the system itself, only the specified channel data retrieval settings are required to allow the division of labor for O&M for achieving a higher level of efficiency.

Figure 13: Top: Image Recognition Screen; Bottom: Data Collection Channel of IoT Platform

The ThingSpeak IoT platform would serve as a web portal to save the collected real-time data, while SyncBIM retrieves the data in the channel and continuously updates the dynamic database. The flexible architecture of the system allows the user preferences and spatial characteristics to be visualized for display (see Figure 14). The number and time data of user facial recognition collected in this study, combined with the building spatial information in BIM, could provide system operators with the required actions for interpreting the O&M situation, and even further adjusting the types of data collection such as various real-time information like air quality, temperature, and humidity. The adjustment for software and hardware would be feasible from simple video lenses to temperature detection cameras, and even computer vision algorithms could be modified according to events, such as detecting whether the person is wearing a mask (which is one of the major safety issues for spaces under the current global pandemic of COVID-19 and more application scenarios and services will become necessary). The spatial information and information integration in SyncBIM is the most appropriate platform to provide these derivative services. In the future building environment, it can be foreseen that such type of cloud service platform will become a necessary asset, even as a part of infrastructures. For O&M, it can also effectively improve efficiency and further enhance the mutually friendly relationship between people and the environment.

Figure 14: The system displays the facial recognition data screen. Left: The viewing panel of the 3D models. Right: The viewing panel of detailed object information, such as the time and current data of each sensor.

5. Conclusion

This study proposes a cloud platform that integrates computer vision and building information models, which is called SyncBIM system, it provides the monitoring and information visualization functions required for building O&M, and improves the passive and equipment troubleshooting-oriented building maintenance modes. In addition to being used for auxiliary O&M decision-making purposes, it can also immediately reflect the O&M state most suitable for the very moment and distinguish the relationship between the user and the spatial context, which is also a major feature of the SyncBIM system. By recording and reflecting the changes in the spatial context and usage situation to provide various instant feedback information required for O&M of buildings, it is expected that to use the SyncBIM system as the best driver for the common prosperity between people and the environment and the pursuit of well-being in the increasingly complex and multi-faceted building O&M context.

1. Q. Qi, F.Tao, Y.Zuo, D.Z.-P.Cirp, undefined2018, “Digital twin service towards smart manufacturing,” Elsevier, Sep. 2021.
2. A. Matthews, B.Ta, “Applying a National BIM Model to Vietnam’s National Implementation of BIM: Lessons Learned from the UK-Vietnam Collaboration for the Industry,” Lecture Notes in Civil Engineering, 54, 57–66, 2020, doi:10.1007/978-981-15-0802-8_6.
3. Y. T.Shen, T.Y.Wu, “Sync-BIM: The interactive BIM-based platform for controlling data-driven kinetic façade,” Communications in Computer and Information Science, 618, 445–450, 2016, doi:10.1007/978-3-319-40542-1_72.
4. C.E. Yang, Y.T.Shen, S.H.Liao, “Integration of Real-time Facial Recognition and Visualization in BIM,” 2nd IEEE Eurasia Conference on IOT, Communication and Engineering 2020, ECICE 2020, 243–245, 2020, doi:10.1109/ECICE50847.2020.9301999.
5. J. Kim, H.Kim, Y.Ham, “Mapping Local Vulnerabilities into a 3D City Model through Social Sensing and the CAVE System toward Digital Twin City,” Computing in Civil Engineering 2019: Smart Cities, Sustainability, and Resilience – Selected Papers from the ASCE International Conference on Computing in Civil Engineering 2019, 451–458, 2019, doi:10.1061/9780784482445.058.
6. F. Tao, M.Zhang, A.Nee, “Digital twin driven smart manufacturing,” 2019.
7. I. Din, M.Guizani, J.Rodrigues, …S.H.-F.G., undefined2019, “Machine learning in the Internet of Things: Designed techniques for smart cities,” Elsevier, Sep. 2021.
8. J. Chen, T.Tseng, …C.L.-2012 9th I., undefined2012, “An intelligent virtual fence security system for the detection of people invading,” Ieeexplore.Ieee.Org, 2012, doi:10.1109/UIC-ATC.2012.64.
9. M. Despotovic, D.Koch, S.Leiber, …M.D.-E. and, undefined2019, “Prediction and analysis of heating energy demand for detached houses by computer vision,” Elsevier, Sep. 2021.
10. E.I. Brunberg, K.E.Bøe, K.M.Sørheim, “Testing a new virtual fencing system on sheep,” Acta Agriculturae Scandinavica A: Animal Sciences, 65(3–4), 168–175, 2015, doi:10.1080/09064702.2015.1128478.
11. B. Akinci, M.Fischer, J.Kunz, R.Levitt, “Representing Work Spaces Generically in Construction Method Models,” Journal of Construction Engineering and Management, 128(4), 296–305, 2002, doi:10.1061/(ASCE)0733-9364(2002)128:4(296).
12. B. Akinci, M.Fischer, J.Kunz, “Automated Generation of Work Spaces Required by Construction Activities,” Journal of Construction Engineering and Management, 128(4), 306–315, 2002, doi:10.1061/(ASCE)0733-9364(2002)128:4(306).

(Click Here)