Applying a Deep-Learning-Based Keypoint Detection in Analyzing Surface Nanostructures
Abstract
:1. Introduction
2. Discussion and Results
3. Conclusions
Supplementary Materials
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Di Giovannantonio, M.; Fasel, R. On-Surface Synthesis and Atomic Scale Characterization of Unprotected Indenofluorene Polymers. J. Polym. Sci. 2022, 60, 1814–1826. [Google Scholar] [CrossRef]
- Wang, J.; Niu, K.; Xu, C.; Zhu, H.; Ding, H.; Han, D.; Zheng, Y.; Xi, J.; You, S.; Deng, C.; et al. Influence of Molecular Configurations on the Desulfonylation Reactions on Metal Surfaces. J. Am. Chem. Soc. 2022, 144, 21596–21605. [Google Scholar] [CrossRef] [PubMed]
- Kang, F.; Sun, L.; Gao, W.; Sun, Q.; Xu, W. On-Surface Synthesis of a Carbon Nanoribbon Composed of 4–5–6–8-Membered Rings. ACS Nano 2023, 17, 8717–8722. [Google Scholar] [CrossRef]
- Kinikar, A.; Di Giovannantonio, M.; Urgel, J.I.; Eimre, K.; Qiu, Z.; Gu, Y.; Jin, E.; Narita, A.; Wang, X.-Y.; Müllen, K.; et al. On-Surface Polyarylene Synthesis by Cycloaromatization of Isopropyl Substituents. Nat. Synth. 2022, 1, 289–296. [Google Scholar] [CrossRef]
- Liu, L.; Klaasen, H.; Witteler, M.C.; Schulze Lammers, B.; Timmer, A.; Kong, H.; Moenig, H.; Gao, H.Y.; Neugebauer, J.; Fuchs, H.; et al. Polymerization of Silanes through Dehydrogenative Si–Si Bond Formation on Metal Surfaces. Nat. Chem. 2021, 13, 350–357. [Google Scholar] [CrossRef] [PubMed]
- Mallada, B.; de la Torre, B.; Mendieta-Moreno, J.I.; Nachtigallová, D.; Matěj, A.; Matoušek, M.; Mutombo, P.; Brabec, J.; Veis, L.; Cadart, T.; et al. On-Surface Strain-Driven Synthesis of Nonalternant Non-Benzenoid Aromatic Compounds Containing Four- to Eight-Membered Rings. J. Am. Chem. Soc. 2021, 143, 14694–14702. [Google Scholar] [CrossRef] [PubMed]
- Sun, Q.; Yan, Y.; Yao, X.; Müllen, K.; Narita, A.; Fasel, R.; Ruffieux, P. Evolution of the Topological Energy Band in Graphene Nanoribbons. J. Phys. Chem. Lett. 2021, 12, 8679–8684. [Google Scholar] [CrossRef]
- Zhu, X.; Liu, Y.; Pu, W.; Liu, F.-Z.; Xue, Z.; Sun, Z.; Yan, K.; Yu, P. On-Surface Synthesis of C144 Hexagonal Coronoid with Zigzag Edges. ACS Nano 2022, 16, 10600–10607. [Google Scholar] [CrossRef]
- Jung, T.A.; Schlittler, R.R.; Gimzewski, J.K. Conformational Identification of Individual Adsorbed Molecules with the STM. Nature 1997, 386, 696–698. [Google Scholar] [CrossRef]
- Wyrick, J.; Wang, X.; Namboodiri, P.; Kashid, R.V.; Fei, F.; Fox, J.; Silver, R. Enhanced Atomic Precision Fabrication by Adsorption of Phosphine into Engineered Dangling Bonds on H–Si Using STM and DFT. ACS Nano 2022, 16, 19114–19123. [Google Scholar] [CrossRef]
- Wang, L.; Xia, Y.; Ho, W. Atomic-Scale Quantum Sensing Based on the Ultrafast Coherence of an H2 Molecule in an STM Cavity. Science 2022, 376, 401–405. [Google Scholar] [CrossRef] [PubMed]
- Meng, T.; Lu, Y.; Lei, P.; Li, S.; Deng, K.; Xiao, X.; Ogino, K.; Zeng, Q. Self-Assembly of Triphenylamine Macrocycles and Co-Assembly with Guest Molecules at the Liquid–Solid Interface Studied by STM: Influence of Different Side Chains on Host–Guest Interaction. Langmuir 2022, 38, 3568–3574. [Google Scholar] [CrossRef] [PubMed]
- Moreno, D.; Parreiras, S.O.; Urgel, J.I.; Muñiz-Cano, B.; Martín-Fuentes, C.; Lauwaet, K.; Valvidares, M.; Valbuena, M.A.; Gallego, J.M.; Martínez, J.I. Engineering Periodic Dinuclear Lanthanide-Directed Networks Featuring Tunable Energy Level Alignment and Magnetic Anisotropy by Metal Exchange. Small 2022, 18, 2107073. [Google Scholar] [CrossRef] [PubMed]
- Lyu, C.-K.; Gao, Y.-F.; Gao, Z.-A.; Mo, S.-Y.; Hua, M.-Q.; Li, E.; Fu, S.-Q.; Chen, J.-Y.; Liu, P.-N.; Huang, L.; et al. Synthesis of Single-Layer Two-Dimensional Metal–Organic Frameworks M3(HAT)2 (M = Ni, Fe, Co, HAT = 1,4,5,8,9,12-Hexaazatriphenylene) Using an On-Surface Reaction. Angew. Chem. 2022, 134, e202204528. [Google Scholar] [CrossRef]
- Liu, J.; Li, J.; Xu, Z.; Zhou, X.; Xue, Q.; Wu, T.; Zhong, M.; Li, R.; Sun, R.; Shen, Z.; et al. On-Surface Preparation of Coordinated Lanthanide-Transition-Metal Clusters. Nat. Commun. 2021, 12, 1619. [Google Scholar] [CrossRef] [PubMed]
- Jiang, H.; Lu, J.; Zheng, F.; Zhu, Z.; Yan, Y.; Sun, Q. Steering On-Surface Polymerization through Coordination with a Bidentate Ligand. Chem. Commun. 2023, 59, 8067–8070. [Google Scholar] [CrossRef]
- Zhu, Z.; Lu, J.; Zheng, F.; Chen, C.; Lv, Y.; Jiang, H.; Yan, Y.; Narita, A.; Müllen, K.; Wang, X.; et al. A Deep-Learning Framework for the Automated Recognition of Molecules in Scanning-Probe-Microscopy Images. Angew. Chem. Int. Ed. 2022, 61, e202213503. [Google Scholar] [CrossRef]
- Milošević, D.; Vodanović, M.; Galić, I.; Subašić, M. Automated Estimation of Chronological Age from Panoramic Dental X-ray Images Using Deep Learning. Expert Syst. Appl. 2022, 189, 116038. [Google Scholar] [CrossRef]
- Zheng, F.; Zhu, Z.; Lu, J.; Yan, Y.; Jiang, H.; Sun, Q. Predicting the HOMO-LUMO Gap of Benzenoid Polycyclic Hydrocarbons via Interpretable Machine Learning. Chem. Phys. Lett. 2023, 814, 140358. [Google Scholar] [CrossRef]
- Krull, A.; Hirsch, P.; Rother, C.; Schiffrin, A.; Krull, C. Artificial-Intelligence-Driven Scanning Probe Microscopy. Commun. Phys. 2020, 3, 54. [Google Scholar] [CrossRef] [Green Version]
- Alldritt, B.; Hapala, P.; Oinonen, N.; Urtev, F.; Krejci, O.; Federici Canova, F.; Kannala, J.; Schulz, F.; Liljeroth, P.; Foster, A.S. Automated Structure Discovery in Atomic Force Microscopy. Sci. Adv. 2020, 6, eaay6913. [Google Scholar] [CrossRef] [Green Version]
- Hellerstedt, J.; Cahlík, A.; Švec, M.; Stetsovych, O.; Hennen, T. Counting Molecules: Python Based Scheme for Automated Enumeration and Categorization of Molecules in Scanning Tunneling Microscopy Images. Softw. Impacts 2022, 12, 100301. [Google Scholar] [CrossRef]
- Li, J.; Telychko, M.; Yin, J.; Zhu, Y.; Li, G.; Song, S.; Yang, H.; Li, J.; Wu, J.; Lu, J.; et al. Machine Vision Automated Chiral Molecule Detection and Classification in Molecular Imaging. J. Am. Chem. Soc. 2021, 27, 10177–10188. [Google Scholar] [CrossRef] [PubMed]
- Gordon, O.M.; Hodgkinson, J.E.; Farley, S.M.; Hunsicker, E.L.; Moriarty, P.J. Automated Searching and Identification of Self-Organized Nanostructures. Nano Lett. 2022, 20, 7688–7693. [Google Scholar] [CrossRef]
- Yan, Y.; Zheng, F.; Qie, B.; Lu, J.; Jiang, H.; Zhu, Z.; Sun, Q. Triangle Counting Rule: An Approach to Forecast the Magnetic Properties of Benzenoid Polycyclic Hydrocarbons. J. Phys. Chem. Lett. 2023, 14, 3193–3198. [Google Scholar] [CrossRef]
- Kang, J.; Yoo, Y.J.; Park, J.-H.; Ko, J.H.; Kim, S.; Stanciu, S.G.; Stenmark, H.A.; Lee, J.; Mahmud, A.A.; Jeon, H.-G.; et al. Deepgt: Deep Learning-Based Quantification of Nanosized Bioparticles in Bright-Field Micrographs of Gires-Tournois Biosensor. NANOTODAY-D-23-00370. Available online: https://ssrn.com/abstract=4428599 (accessed on 10 July 2023).
- Faraz, K.; Grenier, T.; Ducottet, C.; Epicier, T. Deep Learning Detection of Nanoparticles and Multiple Object Tracking of Their Dynamic Evolution during in Situ ETEM Studies. Sci. Rep. 2022, 12, 2484. [Google Scholar] [CrossRef]
- Newby, J.M.; Schaefer, A.M.; Lee, P.T.; Forest, M.G.; Lai, S.K. Convolutional Neural Networks Automate Detection for Tracking of Submicron-Scale Particles in 2D and 3D. Proc. Natl. Acad. Sci. USA 2018, 115, 9026–9031. [Google Scholar] [CrossRef] [Green Version]
- Nartova, A.V.; Mashukov, M.Y.; Astakhov, R.R.; Kudinov, V.Y.; Matveev, A.V.; Okunev, A.G. Particle Recognition on Transmission Electron Microscopy Images Using Computer Vision and Deep Learning for Catalytic Applications. Catalysts 2022, 12, 135. [Google Scholar] [CrossRef]
- Choudhary, K.; DeCost, B.; Chen, C.; Jain, A.; Tavazza, F.; Cohn, R.; Park, C.W.; Choudhary, A.; Agrawal, A.; Billinge, S.J.; et al. Recent Advances and Applications of Deep Learning Methods in Materials Science. Npj Comput. Mater. 2022, 8, 59. [Google Scholar] [CrossRef]
- Wang, C.-Y.; Bochkovskiy, A.; Liao, H.-Y.M. YOLOv7: Trainable Bag-of-Freebies Sets New State-of-the-Art for Real-Time Object Detectors. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada, 18–22 June 2023; pp. 7464–7475. [Google Scholar]
- Okunev, A.G.; Nartova, A.V.; Matveev, A.V. Recognition of Nanoparticles on Scanning Probe Microscopy Images Using Computer Vision and Deep Machine Learning. In Proceedings of the 2019 International Multi-Conference on Engineering, Computer and Information Sciences (SIBIRCON), Novosibirsk, Russia, 21–27 October 2019; pp. 0940–0943. [Google Scholar]
- Ullah, M.B. CPU Based YOLO: A Real Time Object Detection Algorithm. In Proceedings of the 2020 IEEE Region 10 Symposium (TENSYMP), Dhaka, Bangladesh, 5–7 June 2020; pp. 552–555. [Google Scholar]
- Feng, H.; Mu, G.; Zhong, S.; Zhang, P.; Yuan, T. Benchmark Analysis of YOLO Performance on Edge Intelligence Devices. Cryptography 2022, 6, 16. [Google Scholar] [CrossRef]
- Zhou, S.; Cai, K.; Feng, Y.; Tang, X.; Pang, H.; He, J.; Shi, X. An Accurate Detection Model of Takifugu Rubripes Using an Improved YOLO-V7 Network. J. Mar. Sci. Eng. 2023, 11, 1051. [Google Scholar] [CrossRef]
- Lu, J.; Jiang, H.; Yan, Y.; Zhu, Z.; Zheng, F.; Sun, Q. High-Throughput Preparation of Supramolecular Nanostructures on Metal Surfaces. ACS Nano 2022, 16, 13160–13167. [Google Scholar] [CrossRef] [PubMed]
- Ding, K.; Xu, Z.; Tong, H.; Liu, H. Data Augmentation for Deep Graph Learning: A Survey. SIGKDD Explor. Newsl. 2022, 24, 61–77. [Google Scholar] [CrossRef]
- Perez, L.; Wang, J. The Effectiveness of Data Augmentation in Image Classification Using Deep Learning. arXiv 2017, arXiv:1712.04621. [Google Scholar]
- Maji, D.; Nagori, S.; Mathew, M.; Poddar, D. YOLO-Pose: Enhancing YOLO for Multi Person Pose Estimation Using Object Keypoint Similarity Loss. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA, 18–24 June 2022; pp. 2637–2646. [Google Scholar]
- Le, V.-H. Automatic 3D Hand Pose Estimation Based on YOLOv7 and HandFoldingNet from Egocentric Videos. In Proceedings of the 2022 RIVF International Conference on Computing and Communication Technologies (RIVF), Ho Chi Minh City, Vietnam, 20–22 December 2022; pp. 161–166. [Google Scholar]
- Roth, K.; Pemula, L.; Zepeda, J.; Schölkopf, B.; Brox, T.; Gehler, P. Towards Total Recall in Industrial Anomaly Detection. arXiv 2022, arXiv:2106.08265. [Google Scholar]
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Yuan, S.; Zhu, Z.; Lu, J.; Zheng, F.; Jiang, H.; Sun, Q. Applying a Deep-Learning-Based Keypoint Detection in Analyzing Surface Nanostructures. Molecules 2023, 28, 5387. https://doi.org/10.3390/molecules28145387
Yuan S, Zhu Z, Lu J, Zheng F, Jiang H, Sun Q. Applying a Deep-Learning-Based Keypoint Detection in Analyzing Surface Nanostructures. Molecules. 2023; 28(14):5387. https://doi.org/10.3390/molecules28145387
Chicago/Turabian StyleYuan, Shaoxuan, Zhiwen Zhu, Jiayi Lu, Fengru Zheng, Hao Jiang, and Qiang Sun. 2023. "Applying a Deep-Learning-Based Keypoint Detection in Analyzing Surface Nanostructures" Molecules 28, no. 14: 5387. https://doi.org/10.3390/molecules28145387