Next Article in Journal
Effect of Steel Support Cross-Section and Preloaded Axial Force on the Stability of Deep Foundation Pits
Previous Article in Journal
Spatial Syntactic Analysis and Revitalization Strategies for Rural Settlements in Ethnic Minority Areas: A Case Study of Shuanglang Town, China
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Buildings
Volume 14
Issue 8
10.3390/buildings14082533
buildings-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Integrating Multimodal Generative AI and Blockchain for Enhancing Generative Design in the Early Phase of Architectural Design Process

by
Adam Fitriawijaya
* and
Taysheng Jeng
Department of Architecture, National Cheng Kung University, Tainan City 701, Taiwan
*
Author to whom correspondence should be addressed.
Buildings 2024, 14(8), 2533; https://doi.org/10.3390/buildings14082533
Submission received: 16 July 2024 / Revised: 11 August 2024 / Accepted: 12 August 2024 / Published: 16 August 2024
(This article belongs to the Section Architectural Design, Urban Science, and Real Estate)
Browse Figures
Versions Notes

Abstract

Multimodal generative AI and generative design empower architects to create better-performing, sustainable, and efficient design solutions and explore diverse design possibilities. Blockchain technology ensures secure data management and traceability. This study aims to design and evaluate a framework that integrates blockchain into generative AI-driven design drawing processes in architectural design to enhance authenticity and traceability. We employed a scenario as an example to integrate generative AI and blockchain into architectural designs by using a generative AI tool and leveraging multimodal generative AI to enhance design creativity by combining textual and visual inputs. These images were stored on blockchain systems, where metadata were attached to each image before being converted into NFT format, which ensured secure data ownership and management. This research exemplifies the pragmatic fusion of generative AI and blockchain technology applied in architectural design for more transparent, secure, and effective results in the early stages of the architectural design process.

1. Introduction

The architectural design process has significantly transformed over the past few decades with generative design emerging as a powerful tool for exploring diverse possibilities quickly and precisely [1]. Although generative design originated in specialised software environments decades ago, it has become more accessible, even in widely used programs like Revit [2]. It enables a broader range of architects to incorporate advanced design techniques.
Nowadays, the application of AI in architectural design has introduced unprecedented opportunities for creativity and efficiency, allowing architects to explore new forms and functions with the aid of generative models [3,4]. Ben Dreith [5] highlighted AI’s potential to transform architectural and product design’s creation and conceptual stages. Generative AI applications like Midjourney, DALL-E, and Stable Diffusion, created by diverse technology firms, utilise text-to-image and image-to-image inputs to produce AI-generated images, prompting discussions about their forthcoming impact on design and architecture [6,7]. Architects can use these AI applications to explore various design possibilities, enhance their creative abilities, and receive immediate feedback for iterative improvements [8,9]. However, some argue that AI should complement and strengthen architects’ skills and intuition, not replace them. Human interpretation and critical thinking remain essential in the creative process.
Traditional architectural design processes often involve iterative conceptualisation, refinement, and implementation cycles, requiring significant time and resources [10]. In comparison, Ploenings and Berger [11] noted that generative AI has shown promise in automating aspects of design generation. The image generation process starts with collecting a varied dataset from online repositories [12]. Consequently, challenges persist in ensuring the security, transparency, and traceability of design data and transactions throughout the architectural lifecycle. Moreover, the process of human–AI generative design introduces legal risks, particularly concerning intellectual property infringement and various security concerns, including issues related to data privacy and copyright [13,14,15,16]. Despite these hurdles, leveraging generative AI for architectural design presents innovative possibilities. It focuses on job augmentation and collaborative synergy between human designers and AI systems. However, achieving this synergy demands careful consideration of copyright interests and ethical implications, requiring ongoing research and dialogue [17].
Blockchain technology offers an alternative solution for dealing with challenges in creating images through generative AI. Blockchain provides a decentralised and immutable ledger system that securely stores metadata related to image datasets, training parameters, and model outputs [18,19,20]. Blockchain technology offers a robust framework for securing digital assets, ensuring transparency, and maintaining ownership rights through non-fungible tokens (NFTs) [21,22,23,24]. Additionally, Nawari and Shriram [20] pointed out that by immutably recording transactional data, blockchain ensures that the integrity and origin of training data are protected, preventing unauthorised modifications or data tampering. Integrating generative AI and blockchain technology promises to revolutionise how the architectural design process is conceived, shared, and protected.
Despite the recognised potential of generative AI and blockchain in architecture design, a significant gap exists in understanding how these technologies can be effectively integrated to enhance design processes. The study explores the integration of multimodal generative AI and blockchain technology to address challenges in the early stages of architectural design, such as enhancing creativity, ensuring data authenticity, and improving collaboration between architects and clients. The study’s objective is to investigate how multimodal generative AI, which leverages both textual and visual inputs, can be used to enhance design creativity in architectural projects. Furthermore, it aims to explore the application of blockchain technology in converting design metadata into non-fungible tokens (NFTs), ensuring secure, authentic, and traceable data storage. By positioning the research question within the context of these emerging technologies, the study seeks to contribute to the broader discourse on how AI and blockchain can be pragmatically applied to create more transparent, secure, and effective design processes in architecture.
The significance of this research lies in its potential to redefine architectural practice by providing architects and all teams involved with tools that augment their creative capabilities and protect their intellectual property in an increasingly digital world. By integrating AI-driven generative design with blockchain-based verification, architects can achieve higher creativity and innovation while ensuring their work remains secure and traceable.

2. Materials and Methods

This study explores the integration of multimodal generative AI and blockchain technology in the architectural design process. The research followed these key stages:
  • Research design: The hypothesis posits that combining generative AI with blockchain technology enhances architectural design by ensuring secure data storage and traceability. The research followed a structured sequence, progressing from design conceptualisation to implementing blockchain technology.
  • Material selection: Generative AI tools, such as Midjourney, were selected for their ability to generate diverse designs using both text and image inputs. The Ethereum blockchain was chosen for its secure NFT-based storage capabilities. Architectural datasets were sourced from online repositories, focusing on incorporating diverse styles.
  • Procedure: In the design phase, initial sketches and text inputs were fed into AI for design generation, with iterative refinement using blending and upscaling features. For blockchain integration, design metadata were created and stored as NFTs on Ethereum, utilising a Java application for uploading and converting metadata on Firebase.
  • Result: Documented outcomes with metadata linking each AI-generated design to its NFT, verifying authenticity and traceability and evaluating supported by relevant papers, highlighting the innovative use of AI in design and the critical role of blockchain in protecting intellectual property and ensuring data integrity.
The methodology encompasses two key techniques: multimodal generative AI and data storage on the blockchain system, which are illustrated comprehensively in Figure 1. This scenario-based approach reflects the rationale behind this integration’s workflow with real-life applications, demonstrating the proposed framework for integrating multimodal Generative AI with blockchain technology in architectural design. It outlines the workflow from the initial design phase, where sketches and text inputs are processed by AI, to the final blockchain integration, where design metadata are stored as NFTs on Ethereum. This figure visually represents the structured process of creating, refining, and securing architectural designs using advanced AI and blockchain technologies.
As part of the multimodal shown in Table 1, we initiate the design process by outlining the building’s intention through an initial sketch input into a generative AI application, like Midjourney. We use the application features and add/remove building elements within generated images iteratively. In the architectural design context, several terminologies are essential for stressing and clarifying how a beautiful building looks up to what finally ends after the practical stages provide shape. It may be through factors such as building typology, which describes the kind of project and its design, such as sustainability or cultural interpretation. For the next stage, we will design to provide context for buildings with a specific style in mind (modern architecture/postmodern/renaissance, or other desired style) and contextual designs like site or location data and details of the surrounding area. Then, the rendering style is also defined, and focus points, such as interior and lighting features, are given more depth. The generative AI system is provided with design parameters such as aspect ratio, negative prompts and level of detail in the case of 2D images to guide it toward the desired output. We put together a prompt accordingly to send over the AI model and receive an image representing our architectural concept. The following design iteration analyses the generated image to be traced back to what is intended for the final design concept.
The elements integrated in generative AI applications, such as prompts, blending, upscales, variants, and remixes, are integral to the design process. These features enable a wide range of design options, offering designers a wide range of choices. Each resulting design drawing produces four variants, allowing further exploration and refinement. Alternative design options can be created by remixing the image and including additional parameters. The resulting images play a role in the architectural design process, whether they become the final result or not.
These models gain the ability to create art by discerning statistical patterns within pre-existing artistic media. The generative AI process involves training algorithms on extensive datasets comprising various art forms like paintings and photographs. These datasets are a foundation for algorithms to learn underlying patterns and stylistic elements within the artistic media. The training process includes the algorithm iteratively processing the data, refining its understanding of patterns, and gradually enhancing its ability to generate new art.
When generative AI models generate outputs based on training data, the ownership of that data can impact the generated content’s legal, ethical, and regulatory implications. The training data may contain copyrighted material, and data ownership determines liability, attribution, and potential legal consequences. Moreover, data ownership plays a significant role in fostering innovation and promoting fair competition. For instance, when individuals or organisations invest time, effort, and resources into curating and creating high-quality training datasets, they should be able to derive value from their investments. It allows individuals and organisations to protect their data, influence usage, and assert their rights over the generated outputs. It also includes control and rights over the design images produced through generative AI applications, encompassing usage, modification, distribution, and monetisation. It is essential to clarify ownership boundaries and evaluate the implications of utilising specific design features to safeguard ownership claims in architectural design. By examining the role of these features in digital ownership, designers can navigate the complex landscape of control, rights, and responsibilities. This exploration can lead to frameworks and guidelines addressing the legal and ethical aspects of digital ownership and protection in architectural design.
This section will demonstrate the prompt data that could be stored within the blockchain system. Here, we utilise NFTs to store metadata, establishing data ownership for AI-generated images in the architecture design process. The application processing is available in our dataset [25].
The process begins with generating architectural images using generative AI and curating selected images. Subsequently, metadata for each image are created and stored as a .json file. The linkage between metadata and images is established, and the data are stored in cloud storage. To facilitate this, we develop a Java uploader application to store the images and their associated metadata in Firebase Storage by Google. Finally, the metadata are transformed into NFT metadata, and the entire process is deployed and executed in Remix using the ERC721 standard presented in Figure 2 and Figure 3.

3. Results

This section presents the outcomes following the generative design procedure, which are stored as NFTs. These results encapsulate the prompt details as metadata linked to the resulting images. Table 2 displays pertinent information, including User ID, Job ID, Seed, and Timestamp. All data processing and implementation conducted as part of the pilot project are included in our research dataset [25].
The data in Table 2 indicate a successful transaction-deployed smart contract by representing unique identifiers of transactions or interactions within the Ethereum blockchain. Each Ethereum address represents an account or a contract on the Ethereum network. In this case, the contract address for this project is 0xb27A31f1b0AF2946B7F582768f03239b1eC07c2c, as shown in the references cited in Supplementary Materials. The transaction was successfully mined and executed (status 0 × 1) without generating additional output or logs. The contract deployment process is completed without errors, and the newly deployed contract is now available at the specified address on the Ethereum blockchain.
Following contract deployment, a transaction executed the creation of a token within the deployed smart contract. This action resulted in minting an NFT token, incorporating metadata (data from the generated image), with ID 1 as the identifier for this metadata event. The transaction incurred a gas cost, representing the computational expense of executing the transaction on the Ethereum blockchain with an execution cost of 183,830 gas. The gas cost is the amount of Ether (ETH) required to perform transactions or execute smart contracts, reflecting the computational effort needed. In the next step, two logs were generated as a result of this transaction: the first indicating a transfer event, transferring ownership of the newly minted token to address 0x5B38Da6a701c568545dCfcB03FcB875f56beddC4, and the second representing an update of the metadata event, signifying the update of metadata associated with the token.
Subsequently, a call was made to the token URI function within the smart contract to retrieve the URI associated with token ID 1. The decoded output of this call provided the token URI, which was a URL pointing to the metadata stored on a Firebase storage bucket. In this instance, the token URI is (https://firebasestorage.googleapis.com/v0/b/genainft-ac24b.appspot.com/o/metadata%2F1.json?alt=media, accessed on 15 July 2024).
These metadata contain relevant information about the NFT, including its attributes, properties, and provenance. The token URI is a standardised method for accessing NFT metadata, enabling owners and users to retrieve detailed information about the digital asset. Functionally, token URIs enhance interoperability by providing consistent access to metadata across different NFT platforms and applications.
The results of this study demonstrate the practical integration of generative AI and blockchain technology in the architectural design process. The methodology and implementation details have been systematically explored, highlighting the benefits and challenges of this innovative approach. Table 3 summarises the key points considered in this integration:
This approach ensures transparency, security, and efficiency in the architectural design process, paving the way for future advancements in the field.

3.1. AI Serves as a Creative Catalyst for Multimodal Design Generation

Multimodal generative AI has emerged as a groundbreaking approach in architectural design, offering the potential to revolutionise the design process through the integration of diverse modalities such as text, images, and videos [26], delve into the system implications of multimodal generation, highlighting challenges and opportunities for text-to-image (TTI) and text-to-video (TTV) models. There are two main categories of generative AI models: unimodal and multimodal [27]. Unimodal models take prompts from the same modality as the content they generate. In contrast, multimodal models can accept prompts from different modalities and produce results in multiple modalities, as shown in Figure 4. Unimodal models rely on a single type of input, such as text, to generate outputs, which may limit the richness and diversity of the designs produced. In contrast, multimodal models combine multiple types of inputs, like text and images, to produce more sophisticated and varied outputs, enabling more nuanced and comprehensive design possibilities in architectural contexts.
Reflecting on the case study presented in this paper, particularly as shown in Table 4, consider the difference between utilising a unimodal generative AI model solely from text input and integrating a multimodal approach using text and image prompts. When using a unimodal model, the architect’s ability to convey nuanced design concepts may be limited by the constraints of text-only input.
By incorporating image prompts alongside textual descriptions, architects, as users, can communicate their design intent more effectively and explore a broader spectrum of creative possibilities. This seamless transition between image and written prompts enables flexible and dynamic design exploration, ultimately generating novel and innovative design solutions. Integrating multimodal generative AI empowers architects to harness visual and textual inputs, enriching the design process and producing more robust and sophisticated design outcomes.
The multimodal AI design process involves multiple iterations and adjustments based on initial sketches and continuous modifications. This iterative nature requires significant time and effort, mainly when refining designs to meet specific architectural requirements. Consequently, project timelines may experience delays, and architects and designers may face an increased workload.
The effectiveness of multimodal generative AI highly depends on the data it is trained on. If the training data lack diversity or quality, the AI may produce repetitive or uninspired designs that do not meet the project’s unique needs. Hence, there is a limitation in the AI’s ability to generate creative and high-quality design options, potentially resulting in subpar architectural outcomes.
Based on the results and evaluation, particularly in the design process utilising multimodal generative AI, we conclude that three aspects significantly impact the architectural design process: efficiency, accuracy, and user interface.
Table 5 summarises the advantages and challenges of applying multimodal generative AI in design, as highlighted by various research studies. As the case study reflects, multimodal generative AI has significantly enhanced computational efficiency, accuracy, and user experience across multiple applications. On the other hand, all aspects also conclude some challenges in maintaining high accuracy, consistency, and reliability while optimising resource usage and ensuring adaptability across diverse contexts and environments, particularly in critical applications.
The benefits of using generative AI to promote efficiency significantly contribute to increased architects’ labour productivity and reduced design time. Firstly, generative AI saves time in the design practice. For example, while traditional methods may take several minutes to hours to render an image, generative AI can complete the rendering in seconds. Secondly, generative AI provides a non-linear interactive design process which catalyses design creativity. This flexibility allows designers to improvise on ideas in a random sequence not necessarily having to follow a stringent and sequential process. This flexibility will enable designers to explore and iterate on their ideas non-linearly, meaning they are not restricted to following a strict, sequential process. Instead, they can move back and forth between different design stages, making adjustments and refinements as needed. It becomes a dynamic and adaptive approach to optimising greater creativity, allowing designers to experiment with multiple ideas, receive instant feedback, and quickly incorporate changes. As a result, generative AI helps unlock new creative possibilities by driving innovative design solutions. Overall, generative AI is revolutionising the design process, making it an indispensable tool in architectural design and other fields.

3.2. Blockchain Provides Methods of Verifying and Tracing the Authenticity of AI–Human Generative Design

Blockchain technology ensures ownership by offering a transparent ledger that records and validates ownership transactions. This technology enables the creation and management of assets through non-fungible tokens (NFTs), which are unique tokens representing the ownership of a specific item or content piece.
Based on the case study in this research, illustrated in Figure 5, the “Transaction to execute createToken() function within NFT Smart Contract” and the subsequent processes related to minting the NFT token and updating metadata can be considered as part of the authentication process. This authentication involves verifying the transaction’s validity and ensuring that the correct function is executed within the smart contract, ultimately leading to the creation and authentication of the NFT token. It will empower users to securely generate and possess digital assets on the blockchain. Furthermore, the “Token URI: Standardised Method for Retrieving Metadata Associated with NFT” process can be considered as part of the traceability process. This step involves accessing the metadata associated with the NFT through a standardised token URI. By retrieving these metadata, users can trace and verify information about the digital asset, including its attributes, properties, and provenance.
NFTs play a pivotal role in establishing ownership rights in architectural AI images by providing a unique digital representation of the asset on the blockchain. These tokens serve as a digital certificate of authenticity, verifying the ownership and provenance of the associated digital asset. In architectural AI images, NFTs can be used to tokenise specific designs, renderings, or other creative outputs generated through generative AI algorithms. By minting these assets as NFTs, architects and creators can assert ownership over their digital creations and establish a transparent chain of ownership on the blockchain. It ensures that the creator’s rights are recognised and protected in the digital realm, enabling them to monetise their work, license it to others, or transfer ownership as desired. Additionally, NFTs can embed metadata that provide detailed information about the architectural AI image, further enhancing its value and utility for potential buyers or users. NFTs offer a secure and transparent mechanism for asserting ownership and managing intellectual property rights in architectural AI images, fostering trust and accountability in the digital ecosystem.
The case study approach provides detailed insights into real-world implementations, capturing multiple perspectives by examining these technologies and enhancing the reliability of the findings. After demonstrating the integration of generative AI and the implementation of the NFT blockchain, we concluded that three main aspects must be evaluated. In Table 6, we highlight the usage of NFTs from several papers, focusing on several key aspects: authenticity and ownership, integration in the creative process, and application in digital environments.
Integrating blockchain technology and NFTs forms a robust framework for ensuring the authenticity and ownership of digital assets within the generative AI design process. Features of generative AI, such as creating unique digital content, are enhanced by blockchain’s ability to track and verify ownership securely. Legal aspects of data ownership are addressed through the immutable records provided by blockchain, ensuring clear attribution and reducing disputes. Blockchain is also a reliable digital asset storage solution maintaining authenticity and ownership. Results from various studies demonstrate the successful implementation of these technologies, highlighting their potential to revolutionise digital asset management by providing secure, transparent, and verifiable ownership of AI-generated content.

4. Discussion

This paper has uncovered important new information about integrating multimodal generative AI with blockchain technology in architectural design. We have shown that this integration improves the AI’s ability to create design options and tackles key concerns about ownership verification and data protection using blockchain technology. These results are a new addition to the field, especially in digital design and asset management.
In preceding studies, the literature focused extensively on generative AI’s role in design or blockchain’s utility in securing digital assets. For example, Castro et al. [79] highlighted the transformative potential of AI in the conceptual stages of design, while Chalmers et al. [80] emphasised blockchain’s capability to provide secure ownership records through NFTs. Our study bridges these two domains by demonstrating how their integration can create a more robust and secure design process. This approach addresses the concerns raised by Chen et al. [81] regarding the legal and ethical challenges of AI-generated content.
The similarities between our findings and those of previous research, such as the role of AI in augmenting creativity [82,83], confirm the broader applicability of generative AI in creative industries. However, our research differs by incorporating blockchain technology to enhance security and traceability, which is a step that was not comprehensively addressed in earlier studies. This combination offers a dual advantage: fostering creativity while ensuring the creative outputs are securely managed and legally compliant.
The results of our study confirm our initial hypothesis that integrating multimodal generative AI with blockchain technology can significantly enhance both the creative and security aspects of architectural design. This integration provides a framework for addressing the challenges of data ownership and opens up new avenues for securely managing digital assets.
This study identified several challenges related to integrating multimodal generative AI and blockchain technology. Table 7 summarises our approaches to addressing these challenges and the remaining research gaps.
Future work should focus on expanding training datasets to include diverse architectural styles and exploring methods to disentangle style and organisation within generative design algorithms that should accommodate various aspects supporting the architectural design process, such as building function, architectural programming, building circulation, and environmental adaptability. Additionally, we must develop user-friendly tools and interfaces that enable architects and designers to integrate generative AI and blockchain into their design workflows easily. It could involve developing plug-ins for popular design software, such as Building Information Modelling (BIM) applications, that streamline generating AI-driven designs and recording them on the blockchain. Integrating BIM and generative AI makes generative design encompass a broader design exploration. It is not only for the architectural design process but also very possible in construction design. Moreover, numerous researchers and software developers are advancing the integration of BIM and blockchain in the AECO industry. Integrating those technologies will likely support the creation of smarter, more sustainable, more accurate, collaborative, and cost-effective buildings, ultimately benefiting all stakeholders and fields involved in the project.

5. Conclusions

This study highlights the advantages of combining generative AI and blockchain in architecture, such as improving the design process, safeguarding data ownership, and enhancing authenticity and traceability. By utilising multimodal generative AI, we explored new and innovative design forms and functions, significantly broadening the creative possibilities in architectural practices. At the same time, blockchain technology facilitated the secure storage and verification of these designs. By transforming AI-generated designs into non-fungible tokens (NFTs) and storing them on a blockchain, we ensured that the digital assets remained authentic and traceable. This method not only protected the intellectual property of the designs but also created a clear and unchangeable record of ownership, which is essential in today’s digital economy.
The results of this study support the idea that merging generative AI with blockchain can greatly improve the creative and security aspects of architectural design. This combination of technologies effectively tackles the critical issues of data ownership, authenticity, and innovation, offering a well-rounded solution that meets the changing demands of the architecture industry.

Supplementary Materials

The following supporting information can be downloaded at https://data.mendeley.com/datasets/d9zh352rf2/1.

Author Contributions

JT made corrections to improve the paper; A.F. wrote the article; T.J. reviewed the whole text and made comments and suggestions to improve it. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded National Science and Technology Council in Taiwan NSTC 113-2221-E-006-072.

Data Availability Statement

The datasets used and analysed during the current study are available from the corresponding author upon reasonable request.

Acknowledgments

The authors thank the reviewers for their valuable comments and the editors for improving the manuscript.

Conflicts of Interest

The authors declare no competing interests.

Abbreviations

The following abbreviations are used in this manuscript:
3DI3-Dimension Invariant
AECOArchitecture, Engineering, Construction, and Operation.
CGANConditional Generative Adversarial Network
CLIPContrastive Language-Image Pre-Training
dAppDecentralied Application
DdeData-driven evaluator
ERC721Ethereum Request for Comment 721
FERFacial Expression Recognition
GAIGenerative Artificial Intelligence
GAISGenerative Artificial Intelligence System
GANGenerative Adversarial Network
GenAIVAGenerative Artificial Intelligence for Virtual Avatar
GPTGenerative Pre-trained Transformer
IDIdentifier
JSONJavaScript Object Notation
JobIDJob Identifier
LLMLarge Language Model
MMAIRMultimodal Artificial Intelligence Recognition
MMLMultimodal Machine Learning
NFTnon-fungible token
NLPNatural Language Processing
U-NetU-shaped Convolutional Neural Network
URIUniform Resource Identifier
URLUniform Resource Locators
URNUniform Resource Names
UUIDUniversally Unique Identifier

References

  1. Caetano, I.; Santos, L.; Leitão, A. Computational design in architecture: Defining parametric, generative, and algorithmic design. Front. Arch. Res. 2020, 9, 287–300. [Google Scholar] [CrossRef]
  2. Ma, W.; Wang, X.; Wang, J.; Xiang, X.; Sun, J. Generative design in building information modelling (BIM): Approaches and requirements. Sensors 2021, 21, 5439. [Google Scholar] [CrossRef] [PubMed]
  3. Zheng, H.; Yuan, P.F. A generative architectural and urban design method through artificial neural networks. Build. Environ. 2021, 205, 108178. [Google Scholar] [CrossRef]
  4. Liao, W.; Lu, X.; Fei, Y.; Gu, Y.; Huang, Y. Generative AI design for building structures. Autom. Constr. 2024, 157, 105187. [Google Scholar] [CrossRef]
  5. Dreith, B. How AI Software Will Change Architecture and Design; Dezeen Limited. Available online: https://www.dezeen.com/2022/11/16/ai-design-architecture-product/ (accessed on 15 April 2024).
  6. Yıldırım, E. Comparative Analysis Of Leonardo Ai, Midjourney, And Dall-E: AI’s Perspective On Future Cities. Urbanizm 2023, 28, 82–96. [Google Scholar] [CrossRef]
  7. Agkathidis, A.; Song, Y.; Symeonidou, I. AI-Assisted Design: Utilising artificial intelligence as a generative form-finding tool in architectural design studio teaching. Presented at the eCAADe 2024, Leukosia, Cyprus, 9–13 September 2024; Available online: https://www.researchgate.net/publication/381917897_AI-Assisted_Design_Utilising_artificial_intelligence_as_a_generative_form-finding_tool_in_architectural_design_studio_teaching (accessed on 15 July 2024).
  8. Milošević, J.; Đukanović, L.; Zivkovic, M.; Žujović, M.; Gavrilović, M. Automated Compositions: Artificial Intelligence Aided Conceptual Design Explorations in Architecture. Presented at the 9th International Scientific Conference on Geometry and Graphics MONGEOMETRIJA 2023, Novi Sad, Serbia, 7–10 June 2023; pp. 103–115. Available online: https://raf.arh.bg.ac.rs/handle/123456789/1282 (accessed on 15 July 2024).
  9. Almaz, A.; El-Agouz, E.; Abdelfatah, M.; Raafat, I. The Future Role of Artificial Intelligence (AI) Design’s Integration into Architectural and Interior Design Education is to Improve Efficiency, Sustainability, and Creativity. Civ. Eng. Archit. 2024, 12, 1749–1772. [Google Scholar] [CrossRef]
  10. Gao, J.; Fischer, M. Framework and Case Studies Comparing Implementations and Impacts of 3D/4D Modeling across Projects, Citeseer. 2008. Available online: https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&doi=358d76d9a14294eb5bef59da664c5f1ed6409438 (accessed on 15 July 2024).
  11. Ploennigs, J.; Berger, M. AI art in architecture. AI Civ. Eng. 2023, 2, 8. [Google Scholar] [CrossRef]
  12. Graikos, A.; Yellapragada, S.; Le, M.-Q.; Kapse, S.; Prasanna, P.; Saltz, J.; Samaras, D. Learned representation-guided diffusion models for large-image generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition) 2024, Seattle, WA, USA, 17–21 June 2024; pp. 8532–8542. Available online: https://openaccess.thecvf.com/content/CVPR2024/papers/Graikos_Learned_Representation-Guided_Diffusion_Models_for_Large-Image_Generation_CVPR_2024_paper.pdf (accessed on 15 July 2024).
  13. Litan, A. Why Trust and Security Are Essential for the Future of Generative AI; Gartner Inc.: Stamford, CT, USA, 2023; Available online: https://www.gartner.com/en/newsroom/press-releases/2023-04-20-why-trust-and-security-are-essential-for-the-future-of-generative-ai (accessed on 15 July 2024).
  14. Reichwein, F.L. Ethical and Societal Implications of Generative AI-Models. Bachelor’s Thesis, Technische Hochschule Ingolstadt, Ingolstadt, Germany, 2024. Available online: https://opus4.kobv.de/opus4-haw/frontdoor/index/index/year/2024/docId/4694 (accessed on 15 January 2024).
  15. AI. Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile; NIST Trustworthy and Responsible AI: Gaithersburg, MD, USA, 2024. [Google Scholar] [CrossRef]
  16. Carvalko, J.R. Generative AI, Ingenuity, and Law. IEEE Trans. Technol. Soc. 2024, 5, 169–182. [Google Scholar] [CrossRef]
  17. Samuelson, P. Generative AI meets copyright. Science 2023, 381, 158–161. [Google Scholar] [CrossRef]
  18. Hosseinzadeh Kassani, S. Towards Secure and Intelligent Diagnosis: Deep Learning and Blockchain Technology for Computer-Aided Diagnosis Systems. Ph.D. Thesis, University of Saskatchewan, Saskatoon, SK, Canada, 2021. Available online: https://harvest.usask.ca/bitstream/10388/13302/1/HOSSEINZADEHKASSANI-DISSERTATION-2021.pdf (accessed on 15 January 2024).
  19. Mohsan, S.A.H.; Razzaq, A.; Ghayyur, S.A.K.; Alkahtani, H.K.; Al-Kahtani, N.; Mostafa, S.M. Decentralised patient-centric report and medical image management system based on blockchain technology and the inter-planetary file system. Int. J. Environ. Res. Public Health 2022, 19, 14641. [Google Scholar] [CrossRef]
  20. Nawari, N.O.; Ravindran, S. Blockchain technology and BIM process: Review and potential applications. J. Inf. Technol. Constr. 2019, 24, 209–238. Available online: https://www.itcon.org/paper/2019/12 (accessed on 15 January 2024).
  21. Lu, W.; Wu, L. A blockchain-based deployment framework for protecting building design intellectual property rights in collaborative digital environments. Comput. Ind. 2024, 159, 104098. [Google Scholar] [CrossRef]
  22. Kifokeris, D.; Dounas, T.; Tezel, A.; Moon, S. What is the potential value of tokens and token engineering for the architecture, engineering, and construction industry? A positional paper. In Proceedings of the 2023 European Conference on Computing in Construction (EC3 2023) co-located with the 40th International CIB (International Council for Research and Innovation in Building and Construction) conference (CIB W78 2023), Heraklion, Greece, 10–12 July 2023. [Google Scholar] [CrossRef]
  23. Dounas, T.; Jabi, W.; Lombardi, D. Non-fungible building components: Using smart contracts for a circular economy in the built environment. Blucher Des. Proc. 2021, 9, 1189–1198. [Google Scholar] [CrossRef]
  24. Naderi, H.; Ly, R.; Shojaei, A. From Data to Value: Introducing an NFT-Powered Framework for Data Exchange of Digital Twins in the AEC Industry. In Proceedings of the Construction Research Congress 2024, Des Moines, IA, USA, 20–23 March 2024; pp. 299–308. [Google Scholar] [CrossRef]
  25. Fitriawijaya, A.; Taysheng, J. Multimodal Generative AI and NFT Metadata, Version 1; National Cheng Kung University: Taipei, Taiwan, 2024. [Google Scholar] [CrossRef]
  26. Golden, A.; Hsia, S.; Sun, F.; Acun, B.; Hosmer, B.; Lee, Y.; Devito, Z.; Johnson, J.; Wei, G.-Y.; Brooks, D. Generative AI beyond LLMs: System Implications of Multimodal Generation. arXiv 2023, arXiv:2312.14385. [Google Scholar] [CrossRef]
  27. Hariri, W. Unlocking the potential of ChatGPT: A comprehensive exploration of its applications, advantages, limitations, and future directions in natural language processing. arXiv 2023, arXiv:2304.02017. [Google Scholar] [CrossRef]
  28. Del Castillo, A.P. AI: Discovering the Many Faces of a Faceless Technology; ETUI aisbl: Brussels, Belgium, 2023; Available online: https://www.etui.org/sites/default/files/2023-05/AI-Guide-discovering%20the%20many%20faces%20of%20a%20faceless%20technology-2023.pdf (accessed on 15 January 2024).
  29. Ma, S.Y. Exploring Ambiguity in Generative AI Images and Its Impact on Collaborative Design Ideation. Master’s Thesis, Industrial Engineering and Innovation Sciences, Eindhoven University of Technology, Eindhoven, The Netherlands, 2024. Available online: https://pure.tue.nl/ws/portalfiles/portal/320754968/MTP_thesis_report_Sherry_Ma.pdf (accessed on 15 January 2024).
  30. Jaruga-Rozdolska, A. Artificial intelligence as part of future practices in the architect’s work: MidJourney generative tool as part of a process of creating an architectural form. Architectus 2022, 3, 95–104. [Google Scholar] [CrossRef]
  31. Meeran, A. AI and Architecture: Image-Based Machine Learning for Early-Stage Design Conceptualisation. Doctoral Dissertation, Singapore University of Technology and Design, Singapore, 2021. [Google Scholar]
  32. Basole, R.C.; Major, T. Generative AI for Visualization: Opportunities and Challenges. IEEE Comput. Graph. Appl. 2024, 44, 55–64. [Google Scholar] [CrossRef]
  33. Harreis, H.; Koullias, T.; Roberts, R.; Te, K. Generative AI: Unlocking the Future of Fashion; McKinsey Company: New York, NY, USA, 2023; Available online: https://digital-humanai.io/wp-content/uploads/2023/03/Generative-AI-Unlocking-the-future-of-fashion.pdf (accessed on 15 January 2024).
  34. Zhong, C.; Yi’an Shi, L.H.C.; Wang, L. AI-enhanced performative building design optimisation and exploration. Presented at the 29th International Conference on Computer-Aided Architectural Design Research in Asia, CAADRIA 2024, Singapore, 23–25 April 2024; The Association for Computer-Aided Architectural Design Research in Asia (CAADRIA): Hong Kong, China, 2024; Volume 1. Available online: https://papers.cumincad.org/data/works/att/caadria2024_15.pdf (accessed on 15 January 2024).
  35. Bstieler, L.; Noble, C.H. The PDMA Handbook of Innovation and New Product Development; John Wiley & Sons: Hoboken, NJ, USA, 2023. [Google Scholar]
  36. Li, C.; Zhang, T.; Du, X.; Zhang, Y.; Xie, H. Generative AI for Architectural Design: A Literature Review. arXiv 2024, arXiv:2404.01335. [Google Scholar] [CrossRef]
  37. Zhang, K.; Cai, S.; Yang, W.; Wu, W.; Shen, H. Exploring Optimal Combinations: The Impact of Sequential Multimodal Inspirational Stimuli in Design Concepts on Creativity. In Proceedings of the 2024 ACM Designing Interactive Systems Conference, Copenhagen, Denmark, 1–5 July 2024; IT University of Copenhagen, Denmark Association for Computing Machinery: New York, NY, USA, 2024; pp. 2788–2801. [Google Scholar] [CrossRef]
  38. Ochoa, K.S. Can Artificial Intelligence Mark the Next Architectural Revolution? Design Exploration in the Realm of Generative Algorithms and Search Engines; Springer: Berlin/Heidelberg, Germany, 2024. [Google Scholar] [CrossRef]
  39. Paananen, V.; Oppenlaender, J.; Visuri, A. Using text-to-image generation for architectural design ideation. Int. J. Arch. Comput. 2023. [Google Scholar] [CrossRef]
  40. Albaghajati, Z.M.; Bettaieb, D.M.; Malek, R.B. Exploring text-to-image application in architectural design: Insights and implications, Architecture. Struct. Constr. 2023, 3, 475–497. [Google Scholar] [CrossRef]
  41. Yuan, C.; Marion, T.; Moghaddam, M. Dde-gan: Integrating a data-driven design evaluator into generative adversarial networks for desirable and diverse concept generation. J. Mech. Des. 2023, 145, 041407. [Google Scholar] [CrossRef]
  42. Salem, A.A.; Mansour, Y.; Eldaly, H. Generative vs. Non-Generative AI: Analysing the Effects of AI on the Architectural Design Process. Eng. Res. J. 2024, 53, 119–128. [Google Scholar] [CrossRef]
  43. Rane, N.; Choudhary, S.; Rane, J. Integrating ChatGPT, Bard, and leading-edge generative artificial intelligence in architectural design and engineering: Applications, framework, and challenges. Int. J. Arch. Plan. 2023, 3, 92–124. [Google Scholar] [CrossRef]
  44. Shen, Y.; Shen, Y.; Cheng, J.; Jiang, C.; Fan, M.; Wang, Z. Neural Canvas: Supporting Scenic Design Prototyping by Integrating 3D Sketching and Generative AI. In Proceedings of the CHI Conference on Human Factors in Computing Systems, Honolulu, HI, USA, 11–16 May 2024; Association for Computing Machinery: New York, NY, USA, 2024; p. 1056. [Google Scholar] [CrossRef]
  45. Makatura, L.; Foshey, M.; Wang, B.; Hähnlein, F.; Ma, P.; Deng, B.; Tjandrasuwita, M.; Spielberg, A.; Owens, C.E.; Chen, P.Y.; et al. Large Language Models for Design and Manufacturing. MIT Explor. Gener. AI 2024. [CrossRef]
  46. Maqbool, J.; Hassan, S.T.; Cheema, M.I. Application of conditional generative adversarial networks toward time-efficient and high-fidelity imaging via multimode fibers, in AI and Optical Data Sciences V. SPIE 2024, 12903, 69–73. [Google Scholar] [CrossRef]
  47. Florance, G.; Nirmala, M.; Khan, J.; Saifudeen Hisham, K.M. Envisioning the interactive convergence of Generative AI and Facial Expression Recognition. In Proceedings of the 2024 IEEE 9th International Conference for Convergence in Technology (I2CT), Pune, India, 5–7 April 2024; pp. 1–5. [Google Scholar] [CrossRef]
  48. Koga, S. Evaluating ChatGPT in pathology: Towards multimodal AI in medical imaging. J. Clin. Pathol. 2024, 77. [Google Scholar] [CrossRef]
  49. Micheal, A.A.; Prasanth, A.; Aswin, T.; Krisha, B. Advancing Educational Accessibility: The LangChain LLM Chatbot’s Impact on Multimedia Syllabus-Based Learning. Preprint 2024. [Google Scholar] [CrossRef]
  50. Dollar, O. Deep Inverse Design, Discovery, and Optimisation of Molecular Structure through 3D Invariant and Multimodal Machine Learning. Ph.D. Thesis, University of Washington, Seattle, WA, USA, 2023. Available online: http://hdl.handle.net/1773/50264 (accessed on 5 February 2024).
  51. Zou, H.; Zhao, Q.; Bariah, L.; Tian, Y.; Bennis, M.; Lasaulce, S.; Debbah, M.; Bader, F. GenAINet: Enabling Wireless Collective Intelligence via Knowledge Transfer and Reasoning. arXiv 2024, arXiv:2402.16631. [Google Scholar] [CrossRef]
  52. Zürcher, A. Developing a Chatbot for Internal Documents. Bachelor’s Thesis, Business Information Technology, Haaga-Helia University of Applied Sciences, Helsinki, Finland,, 2024. Available online: https://www.theseus.fi/bitstream/handle/10024/861594/Zurcher_Alexandre.pdf?sequence=2 (accessed on 5 February 2024).
  53. Parati, I.; Zolotova, M. Using Future Thinking as a steering tool for Generative AI creative output: A case study aiming at rethink lighting in the next future. In Proceedings of the 11th International Conference on Human Interaction & Emerging Technologies: Artificial Intelligence & Future Applications (IHIET-AI 2024), Lausanne, Switzerland, 25–27 April 2024; AHFE International Open Access: New York, NY, USA, 2024. [Google Scholar] [CrossRef]
  54. Nistler, J.; Pojeta, T. Graphical use of AI. VTEI 2023, 65, 54–56. Available online: https://www.vtei.cz/wp-content/uploads/2023/08/6575-casopis-VTEI-4-23-EN-AI.pdf (accessed on 10 March 2024).
  55. Bagnato, V.P. Artificial Intelligence for Design: The Artificial Intelligence of Objects. Interdiscip. J. Archit. Built Environ. 2023, 27, 30–35. Available online: https://www.researchgate.net/profile/Valerio-Perna/publication/379573623_FORUM_AP_27_Venturing_into_the_Age_of_AI_Insights_and_Perspectives/links/660fb14db839e05a20bd9cfb/FORUM-A-P-27-Venturing-into-the-Age-of-AI-Insights-and-Perspectives.pdf#page=31 (accessed on 10 March 2024). [CrossRef]
  56. Schraml, N.T. ‘You’ve Got All the Weapons You Need. Now Fight!’. Database Trends Appl. 2023, 47, 32. [Google Scholar]
  57. Sharma, S.K.; Dwivedi, Y.K.; Metri, B.; Lal, B.; Elbanna, A. Transfer, Diffusion and Adoption of Next-Generation Digital Technologies. In Proceedings of the IFIP WG 8.6 International Working Conference on Transfer and Diffusion of IT (TDIT 2023), Nagpur, India, 15–16 December 2023; Proceedings, Part I. Springer Nature: Berlin/Heidelberg, Germany, 2023. Available online: https://link.springer.com/book/10.1007/978-3-031-50192-0 (accessed on 10 March 2024).
  58. Tomaževič, N.; Ravšelj, D.; Aristovnik, A. Artificial Intelligence for Human-Centric Society: The Future Is Here; European Liberal Forum: Brussels, Belgium, 2023; Available online: https://liberalforum.eu/wp-content/uploads/2023/12/Artificial-Intelligence-for-human-centric-society.pdf (accessed on 24 January 2024).
  59. Mcnamara, T. Artificial intelligence and the emergence of co-creativism in contemporary art. INSAM J. Contemp. Music. Art Technol. 2023, 11, 12–38. Available online: https://www.ceeol.com/search/article-detail?id=1210421 (accessed on 10 June 2024).
  60. Zhang, B.; Chen, G.; Ooi, B.C.; Shou, M.Z.; Tan, K.L.; Tung, A.K.H.; Xiao, X.; Yip, J.W.L.; Zhang, M. Managing Metaverse Data Tsunami: Actionable Insights. IEEE Trans. Knowl. Data Eng. 2024, 1–20. [Google Scholar] [CrossRef]
  61. Guo, Y.; Liu, Q.; Chen, J.; Xue, W.; Jensen, H.; Rosas, F.; Shaw, J.; Wu, X.; Zhang, J.; Xu, J. Pathway to Future Symbiotic Creativity. arXiv 2022, arXiv:2209.02388. [Google Scholar] [CrossRef]
  62. Parra Pennefather, P. Prototyping with Generative AI. In Creative Prototyping with Generative AI: Augmenting Creative Workflows with Generative AI; Springer: Berlin/Heidelberg, Germany, 2023; pp. 109–143. [Google Scholar] [CrossRef]
  63. Ioannıdıs, S.; Kontıs, A.P. The 4 Epochs of the Metaverse. J. Metaverse 2023, 3, 152–165. [Google Scholar] [CrossRef]
  64. Kalpokas, I. Work of art in the Age of Its AI Reproduction. Philos. Soc. Crit. 2023. [CrossRef]
  65. Rudolf, I. Understanding the Influence of Artificial Intelligence Art on Transaction in the Art World. Master’s Thesis, School of Humanities, Social, Science, and Economics, International Hellenic University, Thessaloniki, Greece, 2024. Available online: https://repository.ihu.edu.gr/xmlui/bitstream/handle/11544/30356/Ion%20Rudolf.pdf?sequence=1 (accessed on 5 June 2024).
  66. Gupta, R.; Pal, S.K. Introduction to Metaverse; Springer Books: Berlin/Heidelberg, Germany, 2023. [Google Scholar] [CrossRef]
  67. Popescu, A.-D. Non-fungible tokens (nft)–innovation beyond the craze, in 5th International Conference on Innovation in Business. Econ. Mark. Res. 2021, 32, 26–30. Available online: https://www.researchgate.net/publication/353973149_Non-Fungible_Tokens_NFT_-_Innovation_beyond_the_craze#fullTextFileContent (accessed on 5 June 2024).
  68. Sahu, B.; Chandramohan Jha, A.M. NFT Marketplaces: The Future of Digital Asset Trading, International Journal of Scientific Research in Computer Science. Eng. Inf. Technol. (IJSRCSEIT) 2023, 9, 513–519. [Google Scholar] [CrossRef]
  69. Han, Y.; Wang, C.; Wang, H. Research on Blockchain Cross-Chain Model Based on “NFT + Cross-Chain Bridge”. IEEE Access 2024, 12, 77065–77078. [Google Scholar] [CrossRef]
  70. Wang, Q.; Li, R.; Wang, Q.; Chen, S. Non-fungible token (NFT): Overview, evaluation, opportunities and challenges. arXiv 2021, arXiv:2105.07447. [Google Scholar] [CrossRef]
  71. Moreaux, A. Visual content tracking, IPR management, & blockchain: From process abstraction to functional interoperability. Doctoral Thesis, Institut Polytechnique de Paris, Paris, France, 2023. Available online: https://theses.hal.science/tel-04418984/ (accessed on 5 June 2024).
  72. Truong, V.T.; Le, L.; Niyato, D. Blockchain meets metaverse and digital asset management: A comprehensive survey. IEEE Access 2023, 11, 26258–26288. [Google Scholar] [CrossRef]
  73. Park, A.; Kietzmann, J.; Pitt, L.; Dabirian, A. The evolution of non-fungible tokens: Complexity and novelty of NFT use-cases. IT Prof. 2022, 24, 9–14. [Google Scholar] [CrossRef]
  74. Huang, M.-H.; Rust, R.T. Artificial intelligence in service. J. Serv. Res. 2018, 21, 155–172. [Google Scholar] [CrossRef]
  75. Morháč, D.; Valaštín, V.; Košťál, K.; Kotuliak, I. Cross-Chain Payments on Blockchain Networks: An Apartment Booking Use-Case. In Proceedings of the 39th ACM/SIGAPP Symposium on Applied Computing, Avila, Spain, 8–12 April 2024; Association for Computing Machinery: New York, NY, USA, 2024; pp. 608–611. [Google Scholar] [CrossRef]
  76. Battah, A.; Madine, M.; Yaqoob, I.; Salah, K.; Hasan, H.R.; Jayaraman, R. Blockchain and NFTs for trusted ownership, trading, and access of AI models. IEEE Access 2022, 10, 112230–112249. [Google Scholar] [CrossRef]
  77. Bhujel, S.; Rahulamathavan, Y. A survey: Security, transparency, and scalability issues of nft’s and its marketplaces. Sensors 2022, 22, 8833. [Google Scholar] [CrossRef] [PubMed]
  78. Khalil, U.; Uddin, M.; Malik, O.A.; Hong, O.W. A Novel NFT Solution for Assets Digitization and Authentication in Cyber-Physical Systems: Blueprint and Evaluation. IEEE Open J. Comput. Soc. 2024, 5, 131–143. [Google Scholar] [CrossRef]
  79. Castro Pena, M.L.; Carballal, A.; Rodríguez-Fernández, N.; Santos, I.; Romero, J. Artificial intelligence applied to conceptual design. A review of its use in architecture. Autom. Constr. 2021, 124, 103550. [Google Scholar] [CrossRef]
  80. Chalmers, D.; Fisch, C.; Matthews, R.; Quinn, W.; Recker, J. Beyond the bubble: Will NFTs and digital proof of ownership empower creative industry entrepreneurs? J. Bus. Ventur. Insights 2022, 17, e00309. [Google Scholar] [CrossRef]
  81. Chen, C.; Fu, J.; Lyu, L. A pathway towards responsible ai generated content. arXiv 2023, arXiv:2303.01325. [Google Scholar] [CrossRef]
  82. Ramdurai, B.; Adhithya, P. The impact, advancements and applications of generative AI. Int. J. Comput. Sci. Eng. 2023, 10, 1–8. [Google Scholar] [CrossRef]
  83. Griebel, M.; Flath, C.; Friesike, S. Augmented creativity: Leveraging artificial intelligence for idea generation in the creative sphere. In Proceedings of the Twenty-Eighth European Conference on Information Systems (ECIS2020), Marrakech, Morocco, 15–17 June 2020; p. 77. Available online: https://aisel.aisnet.org/ecis2020_rip/77 (accessed on 5 June 2024).
Buildings 14 02533 g001
Figure 1. A framework for integrating multimodal generative AI and blockchain systems, created by the author.
Figure 1. A framework for integrating multimodal generative AI and blockchain systems, created by the author.
Buildings 14 02533 g001
Buildings 14 02533 g002
Figure 2. AI-generated images to NFT metadata: a streamlined process created by the author.
Figure 2. AI-generated images to NFT metadata: a streamlined process created by the author.
Buildings 14 02533 g002
Buildings 14 02533 g003
Figure 3. The screenshot depicts the deployment of NFT metadata using the ERC721 standard sourced from the dataset [25].
Figure 3. The screenshot depicts the deployment of NFT metadata using the ERC721 standard sourced from the dataset [25].
Buildings 14 02533 g003
Buildings 14 02533 g004
Figure 4. The difference between unimodal and multimodal generative AI, adapted from [27].
Figure 4. The difference between unimodal and multimodal generative AI, adapted from [27].
Buildings 14 02533 g004
Buildings 14 02533 g005
Figure 5. The process of NFT smart contract: verifying, validity and traceability.
Figure 5. The process of NFT smart contract: verifying, validity and traceability.
Buildings 14 02533 g005
Table 1. An example of the architectural design process using multimodal generative AI (Sketch, prompts and diagram created by the author; images generated by Midjourney).
Table 1. An example of the architectural design process using multimodal generative AI (Sketch, prompts and diagram created by the author; images generated by Midjourney).
InputOutput
Image OptionsSelected Images
Design Intention:
Micro-library is located in the rice field with a building concept modern house with a pyramid roof
Produce a hand sketch depicting the intended shape of the building to ensure alignment with the desired design
Buildings 14 02533 i001		Prompt:
https://s.mj.run/mzXZOLMb-Xw people walking, sunny day, architectural rendering --s 750
Buildings 14 02533 i002		Job 1 ID:
09effa33-8139-42cc-8704-feeddebf8186
Buildings 14 02533 i003
Modify the building and add the environment to adjust with design intention
Buildings 14 02533 i004		Prompt:
Modern stilt house building with long cube shape with wooden material and perforated building façade
Buildings 14 02533 i005		Job ID:
c4e63e2f-b027-4234-986b-ef2294080713
Buildings 14 02533 i006
Modify the building element
Buildings 14 02533 i007		Prompt:
trees and sky—No building—Variations (region)
Buildings 14 02533 i008		Job ID:
8084c539-22e0-4cec-8163-84eba9190947
Buildings 14 02533 i009
1 Job ID is a unique identifier (ID) assigned to each task or job that a user submits, allowing for tracking, managing, and retrieving the specific details and results of that task.
Table 2. NFT metadata, source [25].
Table 2. NFT metadata, source [25].
Generative AI Image Linked to Metadata in Firebase Storage:
Buildings 14 02533 i010
Image Linked in Firebase1 Storage:
“https://firebasestorage.googleapis.com/v0/b/genainft-ac24b.appspot.com/o/image%2F1.png?alt=media&token=0aab1e85-3c98-4e04-ac79-db175ab7c82e (accessed on 15 April 2024)”
Metadata (.json file):
{
"attributes": [
{
"username": “digicliffnotes”,
“user_id”: “1084483399319822387”,
“job_id”: “c4e63e2f-b027-4234-986b-ef2294080713”,
“creator_name”: “Adam”,
“creation_date”: “April 13th, 2024 11:26 pm”,
“creation_tool”: “Midjourney”,
“prompt”: “modern_stilt_house_building_with_long_cube_shape_with_wooden_material_and_perforated_building_facade”,
“image_link”: “https://cdn.discordapp.com/attachments/1087237286707605535/1228727456182177862/digicliffnotes_modern_stilt_house_building_with_long_cube_shape_c4e63e2f-b027-4234-986b-ef2294080713.png?ex=662d189e&is=661aa39e&hm=e6931926dc71ae46f1f1966444fa2b5f344916e0a168a2b74e71688c1f69a478&”
}
],
“image”:”https://firebasestorage.googleapis.com/v0/b/genainft-ac24b.appspot.com/o/image%2F1.png?alt=media&token=0aab1e85-3c98-4e04-ac79-db175ab7c82e”,
“name”: “Design_option #1”
}
Link of Metadata in Firebase Storage:
https://firebasestorage.googleapis.com/v0/b/genainft-ac24b.appspot.com/o/metadata%2F1.json?alt=media&token=c08b907d-b6f4-4653-9ceb-c8966e48e659 (accessed on 15 April 2024)
Deploy and Transaction the Metadata using Smart Contract:
The contract address: 0xb27A31f1b0AF2946B7F582768f03239b1eC07c2c
Token address 1: 0x5B38Da6a701c568545dCfcB03FcB875f56beddC4
Token URI 2: “https://firebasestorage.googleapis.com/v0/b/genainft-ac24b.appspot.com/o/metadata%2F1.json?alt=media (accessed on 15th April 2024)”
1 In transaction metadata using a smart contract, a token address is the unique identifier of a token contract on the blockchain, specifying which token is being interacted with; 2 Token URI is a unique identifier that points to the metadata of a specific token on the blockchain, often providing details like the token’s name, image, description, and other attributes. URI (Uniform Resource Identifier) is a string of characters used to identify a resource on the internet, encompassing both URLs (Uniform Resource Locators) and URNs (Uniform Resource Names). Token refers to a unit of value or utility that is created and managed within a specific blockchain ecosystem.
Table 3. Key points of scenario.
Table 3. Key points of scenario.
Key PointsDescription
Framework for integrationCombines generative AI and blockchain for architectural design using a hypothesis scenario framework to outline the design process flow and real-life applications.
Generative AI design processInvolves initial sketches, prompt engineering, and iterative refinement to generate accurate AI outputs. Key elements include building typology, site details, materials, spatial layout, and rendering style.
Features of generative AIUtilises variants, upscale, blends, remixes, and prompts to provide multiple design possibilities. AI models are trained on large datasets to identify artistic trends and stylistic components.
Data ownership and legal aspectsEmphasises the importance of data ownership in AI training datasets, impacting legal, moral, and regulatory implications. It includes rights to the usage, modification, distribution, and monetisation of AI-generated outputs.
Blockchain for data storageUses blockchain to store prompt data and AI-generated images as NFTs, ensuring secure data ownership. The process includes generating images, producing metadata, storing data in Google Firebase, and converting metadata into the NFT format.
Results and implementationShowcases the outcomes of the generative design process with metadata linked to final images. Provides examples of metadata, storage links, and smart contract details for NFT deployment, demonstrating the practical application of the method.
Table 4. Comparing unimodal and multimodal generative AI in the architectural design phase.
Table 4. Comparing unimodal and multimodal generative AI in the architectural design phase.
Unimodal
InputOutput (Generated Image by Generative AI)
Image OptionsSelected Image
Design
objective		Design iteration: Find the reference building with writing the prompt to develop the building shape suitable with the design intentionJob ID:
7c006bd2-07ad-4e8d-b9ef-47f00abc4732
Buildings 14 02533 i011		Job ID:
f918bc31-bcff-4a1c-a49a-d98959652c05
Buildings 14 02533 i012
Image-
PromptsCreate a prompt as a trigger to draw the environment:
[“micro-library, incorporating vernacular and contemporary architecture, combination of perforated metal panel and transparent muted glass wall as facade, mir rendering, perspective view, located in the rice field near the village in Taiwan, natural light”]
Multimodal
Design objectiveDesign iteration: Combine the building to obtain wider range design options using blendJob ID:
f4e81d0e-b2e2-4057-8ee3-42a59f97551c
seed 2798360210
Buildings 14 02533 i015		Job ID:
8ede5825-a45c-452b-ba8e-bb22e6c2d14a
seed 2798360210
Buildings 14 02533 i016
ImageBuildings 14 02533 i013

Buildings 14 02533 i014
PromptsNo prompt
Table 5. Overview of the use of generative AI.
Table 5. Overview of the use of generative AI.
AspectResumeSpecific Generative AI Applications or
Technology—Source
Computational efficiencyAI enables rapid design generation, exploration, and iteration. 1 MidJourney—[28,29,30];
NS 2—[31,32]; Dall-E 3—[33].
Designers can quickly produce, evaluate, and refine multiple options, leading to more innovative and optimised solutions.NLP 4 and MMAIR 5—[34]
NS—[35,36,37,38];
Dall-E—[39,40];
Dde 6—GAN 7—[41];
CLIP 8—[42].
Generative AI tools for architecture need high computational power and complex algorithms.NS—[4,36]; ChatGPT 9—[43]; Bard AI 10—[43];
Neural Canvas 11—[44].
Ensuring efficiency and accessibility for all firms is challenging due to large datasets, diverse inputs, and multiple design constraints.NS—[4,36]; ChatGPT—[43]; Bard AI—[43];
NS—[34,35,36,37,38];
LLMs 12—[45]
These demands can slow down processing and increase resource consumption.NS—[4,36]; Neural Canvas [44].
AccuracySignificant improvement in imaging accuracy ensures high-fidelity imaging for precise applications.CGANs 13—[46]; U-Net Arch 14—[46].
Enhances reliability of multimodal communication and AI diagnostic processes.GenAIVA 15 and FER 16—[47];
ChatGPT—[48];
Improves accuracy and transparency with visual explanations and textual analysis.ChatGPT—[48]
Maintaining high accuracy while optimising resource usage and ensuring adaptability across diverse contexts is challenging.LangChain LLM—[49];
Ensuring consistent and reliable accuracy, generalisability, and efficient knowledge transfer in resource-limited environments is crucial.3DI 17—[50]; MML 18—[50];
GenAINet 19—[51].
Making visual explanations and textual analyses both accurate and comprehensible is challenging.ChatGPT—[48].
User
Experience (UX)		AI tools, like chatbots, improve adaptability, responsiveness, and user interaction by managing tasks and information efficiently.ChatGPT—[52];
Enhanced visualisation and engagement build trust in AI systemsMidJourney—[53,54];
Integrating text, image, and voice modalities into one tool is technically complex.NS—[55].
Generative AI tools require new skills and workflows, causing potential frustration and reduced productivity.Dall-E—[53]; OpenAI—[56]
Interoperability issues and variable AI output quality may need refinement.MidJourney—[53];
Dall-E—[53].
Limited customisation can constrain designers’ creativity.NS—[55].
Building user trust is challenging due to past unreliable performance and data privacy and security concerns.20 GAIS (IBM Watson)—[57]
1 Generative AI program to generate images using natural language descriptions; 2 Non-specific; 3 Generative AI model developed by OpenAI; 4 Natural Language Processing; 5 Multimodal AI recognition; 6 Dde: Data-driven evaluator; 7 Generative Adversarial Network; 8 Contrastive Language-Image Pre-Training; 9 Generative Pre-trained Transformer; 10 Google’ AI Chatbot; 11 AI Comic Generator; 12 Large Language Models; 13 Conditional Generative Adversarial Network; 14 Convolutional neural network; 15 Generative AI for Virtual Avatar; 16 FER: Facial Expression Recognition; 17 3D Invariant; 18 Multimodal Machine Learning; 19 Generative AI networks; 20 Generative AI system.
Table 6. Overview of the aspect of using NFT and its integration.
Table 6. Overview of the aspect of using NFT and its integration.
AspectResumeSource
Authenticity, certification and ownershipNFTs provide a robust method for certifying the authenticity of digital assets through blockchain technology.[28,58,59,60,61,62,63,64]
Blockchain’s immutable nature ensures that the ownership records of NFTs remain tamper-proof and verifiable, thus guaranteeing the authenticity of AI-generated content.[64,65,66]
Proving ownership and authenticity in a decentralised NFT market can be complex.[67,68]
Ensuring the security of blockchain and NFTs against hacking, fraud, and other malicious activities is a significant concern that can impact the reliability of authenticity and ownership records.[59,63,64]
Integration in the creative processNFTs facilitate creating, owning, and distributing collaborative AI–human creations. This integration supports a new dimension of creativity where digital assets are co-created by humans and AI.[59,61,65]
Interoperability issues between different blockchain platforms can hinder seamless integration and data exchange. [69,70,71]
Complexity integrating NFT-based.[21,72,73]
Scalability and traceability processBlockchain-based NFTs simplify the registration, verification, and tracing of financial transactions related to digital assets, thus enhancing these transactions’ overall security and transparency.[60,63,66,74,75]
NFTs offer a transparent and secure means to track and verify ownership of digital assets. [76,77]
Implementing blockchain solutions on a large scale can be complex and costly, limiting their practicality for verifying and tracking digital assets.[21,72,78]
Table 7. Overview of the research challenges.
Table 7. Overview of the research challenges.
AspectChallengesAddressing
Challenges		Proposed SolutionLimitations
Authenticity and traceabilityEnsuring the authenticity and traceability of AI-generated imagesDeveloping a blockchain-integrated framework that ensures the authenticity and traceability of the generative AI process.A blockchain system can be used to store AI-generated images and their metadata as NFTs, ensuring secure and traceable data.Scalability and performance of integrated systems in large-scale applications
Integration of technologiesIntegrating multimodal generative AI and blockchain technologies seamlesslyDevelop a structured framework for integration.Combine multimodal generative AI and blockchain technology in a streamlined workflow for architectural design.

Potential integration with architecture design software.		Interoperability between different generative AI tools and blockchain platforms.
Data ownership and legal issuesManaging data ownership for AI-generated contentAddressing data ownership and regulatory issues by ensuring proper attribution and legal compliance through blockchain records.Store AI-generated images and metadata in a blockchain system, ensuring data ownership and legal compliance through NFTs.Comprehensive studies on legal and regulatory frameworks required to govern the use of AI and blockchain in architectural design.
User experience and
interaction		Improving design efficiency, accuracy, and user interactionUtilising detailed prompt engineering to ensure accurate and relevant AI-generated images that align with the intended architectural designs.Use generative AI applications to create and refine architectural designs, ensuring user-friendly interaction and high-quality outputs.User acceptance and trust in AI-generated designs and blockchain-based data management.
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Fitriawijaya, A.; Jeng, T. Integrating Multimodal Generative AI and Blockchain for Enhancing Generative Design in the Early Phase of Architectural Design Process. Buildings 2024, 14, 2533. https://doi.org/10.3390/buildings14082533

AMA Style

Fitriawijaya A, Jeng T. Integrating Multimodal Generative AI and Blockchain for Enhancing Generative Design in the Early Phase of Architectural Design Process. Buildings. 2024; 14(8):2533. https://doi.org/10.3390/buildings14082533

Chicago/Turabian Style

Fitriawijaya, Adam, and Taysheng Jeng. 2024. "Integrating Multimodal Generative AI and Blockchain for Enhancing Generative Design in the Early Phase of Architectural Design Process" Buildings 14, no. 8: 2533. https://doi.org/10.3390/buildings14082533

APA Style

Fitriawijaya, A., & Jeng, T. (2024). Integrating Multimodal Generative AI and Blockchain for Enhancing Generative Design in the Early Phase of Architectural Design Process. Buildings, 14(8), 2533. https://doi.org/10.3390/buildings14082533

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop