Real-Time Home Automation System Using BCI Technology
Abstract
:1. Introduction
2. Materials and Methods
2.1. Material Used for Printing
2.2. The Technology Used
2.2.1. Technology Used to Design and Print the 3D Door and Frame
2.2.2. The Neural Headset Used as a Tool to Control the Implemented and Simulated Systems
2.2.3. Technology Used in the Video Simulation of the System Operation
2.3. Methods
2.3.1. System Implementation
2.3.2. System Control
Algorithm 1 Cortex API updates in Python |
1: Define class ConnectionManager 2: Define initialisation 3: Set active_connections as an empty list of WebSockets 4: 5: Define method Broadcast 6: If the time since last_sent exceeds 3 s 7: Send a text message to all active_connections 8: Update last_sent_time 9: Log message sent 10: Else 11: Log message sending skipped due to cooldown 12: 13: Define WebSocket Endpoint “/ws” 14: Connect incoming WebSocket 15: Continuously receive text from WebSocket 16: On exception, log and disconnect WebSocket 17: 18: Define Background Task to Listen for Commands 19: Continuously 20: Retrieve data from the Cortex instance 21: If valid command data received with sufficient power 22: Process and broadcast command 23: Handle exceptions with error logging and retry with delays 24: 25: Define Command Processing 26: Depending on the command 27: Set descriptions for recognised commands 28: Convert command and description to JSON 29: Broadcast JSON message 30: 31: Define Application Startup Event 32: Initialise and configure the Cortex instance 33: Connect and authorise with Cortex 34: Create a session and subscribe to the mental commands data stream 35: Start background task for listening to commands |
Algorithm 2 Functions developed in Flutter, to be run on the phone |
1: FUNCTION connectToWebSocket 2: SET channel TO IOWebSocketChannel.connect(Uri.parse(‘ws://$ipAddress:8000/ws’)) 3: SET channel.stream LISTEN processMessage 4: END FUNCTION 5: 6: FUNCTION processMessage(message AS STRING) 7: DECLARE decodedMessage AS OBJECT 8: SET decodedMessage TO jsonDecode(message) 9: DECLARE command AS STRING 10: SET command TO decodedMessage[‘command’] 11: 12: IF command EQUALS ‘door action corresponding word’ THEN 13: INCREMENT pushCount BY 1 14: SET displayMessage TO IF pushCount MOD 2 EQUALS 1 THEN ‘Door Opened’ ELSE ‘Door Closed’ 15: SET backgroundColor TO IF pushCount MOD 2 EQUALS 1 THEN Colors.green ELSE Colors.red 16: ELSE IF command EQUALS ‘light action corresponding word’ THEN 17: INCREMENT pullCount BY 1 18: SET displayMessage TO IF pullCount MOD 2 EQUALS 1 THEN ‘Light On’ ELSE ‘Light Off’ 19: SET backgroundColor TO IF pullCount MOD 2 EQUALS 1 THEN Colors.yellow ELSE Colors.blueGrey 20: END IF 21: CALL setState 22: END FUNCTION |
2.3.3. Demonstrative Real-Time Unity Simulation of Controlling Home Automation System Using BCI
Algorithm 3 WebSocketController class in C# |
1: CLASS WebSocketController EXTENDS MonoBehaviour 2: 3: FUNCTION Start 4: CREATE WebSocket to “ws://localhost:8000/ws” 5: SET event handlers for WebSocket (OnOpen, OnError, OnClose, OnMessage) 6: CONNECT to WebSocket 7: END FUNCTION 8: 9: FUNCTION Update 10: IF NOT running in WebGL OR running in Unity Editor THEN 11: DISPATCH message queue from WebSocket 12: END IF 13: END FUNCTION 14: 15: FUNCTION OnApplicationQuit 16: CLOSE WebSocket 17: END FUNCTION 18: 19: FUNCTION OnMessage(bytes AS Byte Array) 20: CONVERT bytes to messageJson STRING 21: PARSE messageJson to CommandMessage 22: IF mental command is “‘door action corresponding word’ “ THEN 23: START Flashing to toggle door based on door’s current state 24: ELSE IF mental command is “light action corresponding word’ “ THEN 25: START Flashing to toggle lights 26: END IF 27: END FUNCTION 28: 29: FUNCTION Reconnect 30: CONNECT to WebSocket again 31: END FUNCTION 32: 33: END CLASS |
3. Results and Discussion
4. Limitation
5. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Data Availability Statement
Conflicts of Interest
Correction Statement
References
- Vidal, J.J. Toward direct brain-computer communication. Annu. Rev. Biophys. Bioeng. 1973, 2, 157–180. [Google Scholar] [CrossRef] [PubMed]
- Kaplan, A.Y. Neurophysiological foundations and practical realizations of the brain–machine interfaces in the technology in neurological rehabilitation. Hum. Physiol. 2016, 42, 103–110. [Google Scholar] [CrossRef]
- Bobrova, E.V.; Reshetnikova, V.V.; Frolov, A.A.; Gerasimenko, Y.P. Use of imaginary lower limb movements to control brain–computer interface systems. Neurosci. Behav. Physiol. 2020, 50, 585–592. [Google Scholar] [CrossRef]
- Engdahl, S.M.; Christie, B.P.; Kelly, B.; Davis, A.; Chestek, C.A.; Gates, D.H. Surveying the interest of individuals with upper limb loss in novel prosthetic control techniques. J. Neuroeng. Rehabil. 2015, 12, 1–11. [Google Scholar] [CrossRef] [PubMed]
- Beyrouthy, T.; Al Kork, S.K.; Korbane, J.A.; Abdulmonem, A. EEG mind controlled smart prosthetic arm. In Proceedings of the 2016 IEEE International Conference on Emerging Technologies and Innovative Business Practices for the Transformation of Societies (EmergiTech), Balaclava, Mauritius, 3–6 August 2016; pp. 404–409. [Google Scholar]
- Madakam, S.; Lake, V.; Lake, V.; Lake, V. Internet of Things (IoT): A literature review. J. Comput. Commun. 2015, 3, 164. [Google Scholar] [CrossRef]
- Türk, Ö.; Özerdem, M.S. Epilepsy detection by using scalogram based convolutional neural network from EEG signals. Brain Sci. 2019, 9, 115. [Google Scholar] [CrossRef] [PubMed]
- Drăgoi, M.-V.; Hadăr, A.; Goga, N.; Grigore, L.; Ștefan, A.; Ali, H.A. Design and implementation of an eeg-based bci prosthetic lower limb using raspberry Pi 4. UPB Bul.-Ser. C 2023, 3, 14. Available online: https://scholar.google.com/citations?view_op=view_citation&hl=ro&user=s2TXlaEAAAAJ&citation_for_view=s2TXlaEAAAAJ:UeHWp8X0CEIC (accessed on 20 September 2024).
- Drăgoi, M.-V.; Hadăr, A.; Goga, N.; Baciu, F.; Ștefan, A.; Grigore, L.Ș.; Gorgoteanu, D.; Molder, C.; Oncioiu, I. Contributions to the Dynamic Regime Behavior of a Bionic Leg Prosthesis. Biomimetics 2023, 8, 414. [Google Scholar] [CrossRef] [PubMed]
- Ali, H.A.; Popescu, D.; Hadar, A.; Vasilateanu, A.; Popa, R.C.; Goga, N.; Hussam, H.A.D.Q. EEG-based Brain Computer Interface Prosthetic Hand using Raspberry Pi 4. Int. J. Adv. Comput. Sci. Appl. 2021, 12, 44–49. [Google Scholar] [CrossRef]
- Boord, P.; Craig, A.; Tran, Y.; Nguyen, H. Discrimination of left and right leg motor imagery for brain–computer interfaces. Med. Biol. Eng. Comput. 2010, 48, 343–350. [Google Scholar] [CrossRef] [PubMed]
- Xu, L.D. Internet of Things (IoT): An Introduction. In Wiley Encyclopedia of Electrical and Electronics Engineering; Wiley: Hoboken, NJ, USA, 2021. [Google Scholar] [CrossRef]
- Jia, X.; Feng, Q.; Fan, T.; Lei, Q. RFID technology and its applications in Internet of Things (IoT). In Proceedings of the 2012 2nd international conference on consumer electronics, communications and networks (CECNet), Yichang, China, 21–23 April 2012; pp. 1282–1285. [Google Scholar]
- Landaluce, H.; Arjona, L.; Perallos, A.; Falcone, F.; Angulo, I.; Muralter, F. A review of IoT sensing applications and challenges using RFID and wireless sensor networks. Sensors 2020, 20, 2495. [Google Scholar] [CrossRef] [PubMed]
- Domingo, M.C. An overview of the Internet of Things for people with disabilities. J. Netw. Comput. Appl. 2012, 35, 584–596. [Google Scholar] [CrossRef]
- Gao, Q.; Zhao, X.; Yu, X.; Song, Y.; Wang, Z. Controlling of smart home system based on brain-computer interface. Technol. Health Care 2018, 26, 769–783. [Google Scholar] [CrossRef] [PubMed]
- Babu, K.M.C.; Vardhini, P.A.H. Brain Computer Interface based Arduino Home Automation System for Physically Challenged. In Proceedings of the 2020 3rd International Conference on Intelligent Sustainable Systems (ICISS), Thoothukudi, India, 3–5 December 2020; pp. 125–130. [Google Scholar]
- Lanka, L.; Dhulipalla, D.K.; Karumuru, S.B.; Yarramaneni, N.S.; Dhulipalla, V.R. Electroencephalography (EEG) based Home Automation for Physically Challenged People using Brain Computer Interface (BCI). In Proceedings of the 2022 International Conference on Inventive Computation Technologies (ICICT), Lalitpur, Nepal, 20–22 July 2022; pp. 683–687. [Google Scholar]
- Al-Masri, E.; Singh, A.; Souri, A. IoBCT: A Brain Computer Interface using EEG Signals for Controlling IoT Devices. In Proceedings of the 2022 IEEE 5th International Conference on Knowledge Innovation and Invention (ICKII), Hualien, Taiwan, 22–24 July 2022; pp. 18–23. [Google Scholar]
- Ahmed, D.; Dillshad, V.; Danish, A.S.; Jahangir, F.; Kashif, H.; Shahbaz, T. Enhancing Home Automation through Brain-Computer Interface Technology. J. Xi’an Shiyou Univ. Nat. Sci. Ed. 2023, 19, 1–7. [Google Scholar]
- Ali, H.A.; Ali, L.A.; Vasilateanu, A.; Goga, N.; Popa, R.C. Towards Design and Implementation of an EEG-Based BCI TV Remote Control. Int. J. Online Biomed. Eng. 2023, 19, 118–139. [Google Scholar] [CrossRef]
- Farah, S.; Anderson, D.G.; Langer, R. Physical and mechanical properties of PLA, and their functions in widespread applications—A comprehensive review. Adv. Drug Deliv. Rev. 2016, 107, 367–392. [Google Scholar] [CrossRef] [PubMed]
- Lokesh, N.; Praveena, B.A.; Reddy, J.S.; Vasu, V.K.; Vijaykumar, S. Evaluation on effect of printing process parameter through Taguchi approach on mechanical properties of 3D printed PLA specimens using FDM at constant printing temperature. Mater. Today Proc. 2022, 52, 1288–1293. [Google Scholar] [CrossRef]
- EMOTIV Insight—5 Channel Wireless EEG Headset. 2019. Available online: https://www.emotiv.com/products/insight (accessed on 26 August 2024).
- Unity User Manual 2022.3. Available online: https://docs.unity3d.com/Manual/ (accessed on 18 March 2024).
- Bosmos, F.; Tzallas, A.T.; Tsipouras, M.G.; Glavas, E.; Giannakeas, N. Virtual and Augmented Experience in Virtual Learning Tours. Information 2023, 14, 294. [Google Scholar] [CrossRef]
- Vukić, M.; Grgić, B.; Dinčir, D.; Kostelac, L.; Marković, I. Unity based urban environment simulation for autonomous vehicle stereo vision evaluation. In Proceedings of the 2019 42nd International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO), Opatija, Croatia, 20–24 May 2019; pp. 949–954. [Google Scholar]
- Li, F.; Chen, J.; Zhou, Z.; Xie, J.; Gao, Z.; Xiao, Y.; Dai, P.; Xu, C.; Wang, X.; Zhou, Y. Lightweight soft robotic glove with whole-hand finger motion tracking for hand rehabilitation in virtual reality. Biomimetics 2023, 8, 425. [Google Scholar] [CrossRef] [PubMed]
- R. P. (Trading) Ltd. Raspberry Pi Zero 2 W Datasheet. 2021. Available online: https://datasheets.raspberrypi.com/rpizero2/raspberry-pi-zero-2-w-product-brief.pdf (accessed on 27 March 2024).
- MG996R Datasheet (PDF)—List of Unclassifed Manufacturers. Available online: https://pdf1.alldatasheet.com/datasheet-pdf/view/1131873/ETC2/MG996R.html (accessed on 27 March 2024).
- NPN Silicon Epitaxial Planar Transistor. 2016. Available online: https://datasheetspdf.com/pdf-down/2/N/2/2N2222-SEMTECH.pdf (accessed on 27 March 2024).
- L-7113GD-5V. 2015. Available online: https://www.kingbright.com/attachments/file/psearch/000/00/20160808bak/L-7113GD-5V(Ver.9B).pdf (accessed on 24 April 2024).
- Fritzing. Available online: https://fritzing.org/ (accessed on 24 April 2024).
Field | Value |
---|---|
Layer height | 0.2 mm |
Layers on contour | 2.5 |
Filling density | 5% |
The type of filling | Straight |
Printing plate temperature | 60 °C |
Printing head temperature | 210 °C |
Description | Value |
---|---|
3D printing time | 8 h, 15 min, and 50 s |
Amount of material used [g] | 81.9 |
Estimated price [USD] | 19.49 |
Description | Value |
---|---|
3D printing time | 2 h, 52 min, and 42 s |
Amount of material used [g] | 25.9 |
Estimated price [USD] | 6.16 |
Description | Value |
---|---|
3D printing time | 5 h, and 29 min |
Amount of material used [g] | 52.6 |
Estimated price [USD] | 12.53 |
Description | Value |
---|---|
3D printing time | 16 h, 37 min, and 32 s |
Amount of material used [g] | 175.8 |
Estimated price [USD] | 41.84 |
Participants | Light On/Off | Door Locked/Unlocked | AVG | STDEV |
---|---|---|---|---|
1 | 11 | 12 | 11.5 | 0.7071 |
2 | 9 | 10 | 9.5 | 0.7071 |
3 | 8 | 7 | 7.5 | 0.7071 |
4 | 12 | 10 | 11 | 1.4142 |
5 | 9 | 10 | 9.5 | 0.7071 |
6 | 11 | 9 | 10 | 1.4142 |
7 | 10 | 11 | 10.5 | 0.7071 |
8 | 12 | 13 | 12.5 | 0.7071 |
9 | 13 | 14 | 13.5 | 0.7071 |
10 | 11 | 10 | 10.5 | 0.7071 |
11 | 12 | 12 | 12 | 0 |
12 | 7 | 8 | 7.5 | 0.7071 |
13 | 9 | 7 | 8 | 1.4142 |
14 | 11 | 9 | 10 | 1.4142 |
15 | 9 | 10 | 9.5 | 0.7071 |
16 | 13 | 12 | 12.5 | 0.7071 |
17 | 11 | 10 | 10.5 | 0.7071 |
18 | 10 | 11 | 10.5 | 0.7071 |
19 | 12 | 11 | 11.5 | 0.7071 |
20 | 12 | 13 | 12.5 | 0.7071 |
Total AVG | 10.525 | |||
STDEV AVG | 0.8131 |
Related Work | Advantages | Disadvantages | How This System Is Different |
---|---|---|---|
Controlling of Smart Home System Based on Brain–Computer Interface | - Accurate, EMOTIV Epoc has been used - Four commands are available | - Expensive - Complex | - Affordable - Door control - Real-time notifications |
Brain–Computer Interface-Based Arduino Home Automation System for Physically Challenged | - Affordable | - Not accurate - Only one command (when blinking) | - Two commands - More accurate - Door control - Real-time notifications |
Electroencephalography (EEG)-Based Home Automation for Physically Challenged People using Brain–Computer Interface (BCI) | - Affordable | - Not accurate - Only one command (when blinking) | - Two commands - More accurate - Door control - Real-time notifications |
IoBCT: A Brain–Computer Interface using EEG Signals for Controlling IoT Devices | - Accurate - Affordable | - Only one command | - Two commands - Door control - Real-time notifications |
Enhancing Home Automation through Brain–Computer Interface Technology | - Accurate, EMOTIV Epoc has been used | - Expensive | - Door control - Real-time notifications |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Drăgoi, M.-V.; Nisipeanu, I.; Frimu, A.; Tălîngă, A.-M.; Hadăr, A.; Dobrescu, T.G.; Suciu, C.P.; Manea, A.R. Real-Time Home Automation System Using BCI Technology. Biomimetics 2024, 9, 594. https://doi.org/10.3390/biomimetics9100594
Drăgoi M-V, Nisipeanu I, Frimu A, Tălîngă A-M, Hadăr A, Dobrescu TG, Suciu CP, Manea AR. Real-Time Home Automation System Using BCI Technology. Biomimetics. 2024; 9(10):594. https://doi.org/10.3390/biomimetics9100594
Chicago/Turabian StyleDrăgoi, Marius-Valentin, Ionuț Nisipeanu, Aurel Frimu, Ana-Maria Tălîngă, Anton Hadăr, Tiberiu Gabriel Dobrescu, Cosmin Petru Suciu, and Andrei Rareș Manea. 2024. "Real-Time Home Automation System Using BCI Technology" Biomimetics 9, no. 10: 594. https://doi.org/10.3390/biomimetics9100594
APA StyleDrăgoi, M. -V., Nisipeanu, I., Frimu, A., Tălîngă, A. -M., Hadăr, A., Dobrescu, T. G., Suciu, C. P., & Manea, A. R. (2024). Real-Time Home Automation System Using BCI Technology. Biomimetics, 9(10), 594. https://doi.org/10.3390/biomimetics9100594