Enhancing Communication: Real-Time Sign Language Detection with Deep Learning and Python
Abstract
Deaf and hard-of-hearing people use sign language detection to communicate. Sign language is crucial for deaf and hard of hearing people to communicate. Recent advances in computer vision and machine learning have enabled sign language gesture recognition and decipherment. Abstract: Deep learning and computer vision techniques for sign language identification systems are studied and developed. Insufficient datasets and regional sign language gesture variations are discussed in this research. The suggested methods improve sign language recognition system precision and responsiveness, improving deaf community accessibility and inclusivity. Deploying the model on powerful hardware and using TensorFlow's GPU support allows low-latency sign identification for real-world applications. Our experiments show that the system can recognize sign motions in real time with high accuracy and minimal latency. Deaf and hard-of-hearing people can communicate and live better with this technology, making sign language more inclusive. Deep Learning, TensorFlow, CNN, Real-Time, Gesture Recognition, Video Processing, Machine Learning, Low Latency, Human-Computer Interaction, Diverse Sign Language Dataset, RNN are used in Sign Language Detection (SLD).
References
J. Huang, W. Zhou, H. Li, and W. Li, “Sign Language Recognition using 3D convolutional neural networks,” in 2015 IEEE International Conference on Multimedia and Expo (ICME), 2015.
B. Kang, S. Tripathi, and T. Q. Nguyen, “Real-time sign language fingerspelling recognition using convolutional neural networks from depth map,” in 2015 3rd IAPR Asian Conference on Pattern Recognition (ACPR), 2015.
I. A. Adeyanju, O. O. Bello, and M. A. Adegboye, “Machine learning methods for sign language recognition: A critical review and analysis,” Intelligent Systems with Applications, vol. 12, no. 200056, p. 200056, 2021.
T. Petkar, T. Patil, A. Wadhankar, V. Chandore, V. Umate, and D. Hingnekar, “Real time sign language recognition system for hearing and speech impaired people,” Int. J. Res. Appl. Sci. Eng. Technol., vol. 10, no. 4, pp. 2261–2267, 2022.
B. S. Parton, “Sign language recognition and translation: a multidisciplined approach from the field of artificial intelligence,” J. Deaf Stud. Deaf Educ., vol. 11, no. 1, pp. 94–101, Winter 2006.
R. Boina, “Assessing the Increasing Rate of Parkinson’s Disease in the US and its Prevention Techniques”,” International Journal of Biotechnology Research and Development, vol. 3, no. 1, pp. 1–18, 2022.
S. Patil, S. Chintamani, J. Grisham, R. Kumar, and B. H. Dennis, “Inverse determination of temperature distribution in partially cooled heat generating cylinder,” in Volume 8B: Heat Transfer and Thermal Engineering, 2015.
O. Fabela, S. Patil, S. Chintamani, and B. H. Dennis, “Estimation of effective thermal conductivity of porous media utilizing inverse heat transfer analysis on cylindrical configuration,” in Volume 8: Heat Transfer and Thermal Engineering, 2017.
S. Patil, S. Chintamani, B. H. Dennis, and R. Kumar, “Real time prediction of internal temperature of heat generating bodies using neural network,” Therm. Sci. Eng. Prog., vol. 23, no. 100910, p. 100910, 2021.
I. Khalifa, H. Abd Al-glil, and M. M. Abbassy, “Mobile hospitalization,” International Journal of Computer Applications, vol. 80, no. 13, pp. 18–23, 2013.
I. Khalifa, H. Abd Al-glil, and M. M. Abbassy, “Mobile hospitalization for Kidney Transplantation,” International Journal of Computer Applications, vol. 92, no. 6, pp. 25–29, 2014.
M. M. Abbassy and A. Abo-Alnadr, “Rule-based emotion AI in Arabic Customer Review,” International Journal of Advanced Computer Science and Applications, vol. 10, no. 9, 2019.
M. M. Abbassy and W. M. Ead, “Intelligent Greenhouse Management System,” 2020 6th International Conference on Advanced Computing and Communication Systems (ICACCS), 2020.
M. M. Abbassy, “Opinion mining for Arabic customer feedback using machine learning,” Journal of Advanced Research in Dynamical and Control Systems, vol. 12, no. SP3, pp. 209–217, 2020.
M. M. Abbassy, “The human brain signal detection of Health Information System IN EDSAC: A novel cipher text attribute based encryption with EDSAC distributed storage access control,” Journal of Advanced Research in Dynamical and Control Systems, vol. 12, no. SP7, pp. 858–868, 2020.
M. M. and S. Mesbah, “Effective e-government and citizens adoption in Egypt,” International Journal of Computer Applications, vol. 133, no. 7, pp. 7–13, 2016.
M.M.Abbassy, A.A. Mohamed “Mobile Expert System to Detect Liver Disease Kind”, International Journal of Computer Applications, vol. 14, no. 5, pp. 320–324, 2016.
R. A. Sadek, D. M. Abd-alazeem, and M. M. Abbassy, “A new energy-efficient multi-hop routing protocol for heterogeneous wireless sensor networks,” International Journal of Advanced Computer Science and Applications, vol. 12, no. 11, 2021.
S. Derindere Köseoğlu, W. M. Ead, and M. M. Abbassy, “Basics of Financial Data Analytics,” Financial Data Analytics, pp. 23–57, 2022.
W. Ead and M. Abbassy, “Intelligent Systems of Machine Learning Approaches for developing E-services portals,” EAI Endorsed Transactions on Energy Web, p. 167292, 2018.
W. M. Ead and M. M. Abbassy, “A general cyber hygiene approach for financial analytical environment,” Financial Data Analytics, pp. 369–384, 2022.
W. M. Ead and M. M. Abbassy, “IoT based on plant diseases detection and classification,” 2021 7th International Conference on Advanced Computing and Communication Systems (ICACCS), 2021.
W. M. Ead, M. M. Abbassy, and E. El-Abd, “A general framework information loss of utility-based anonymization in Data Publishing,” Turkish Journal of Computer and Mathematics Education, vol. 12, no. 5, pp. 1450–1456, 2021.
A. M. El-Kady, M. M. Abbassy, H. H. Ali, and M. F. Ali, “Advancing Diabetic Foot Ulcer Detection Based On Resnet And Gan Integration,” Journal of Theoretical and Applied Information Technology, vol. 102, no. 6, pp. 2258–2268, 2024.
M. M. Abbassy and W. M. Ead, “Fog computing-based public e-service application in service-oriented architecture,” International Journal of Cloud Computing, vol. 12, no. 2–4, pp. 163–177, 2023.
R. Oak, M. Du, D. Yan, H. Takawale, and I. Amit, “Malware detection on highly imbalanced data through sequence modeling,” in Proceedings of the 12th ACM Workshop on Artificial Intelligence and Security - AISec’19, 2019.
C. L. Albarracín, S. Venkatesan, A. Y. Torres, P. Yánez-Moretta, and J. C. J. Vargas, “Exploration on cloud computing techniques and its energy concern,” MSEA, vol. 72, no. 1, pp. 749–758, Feb. 2023.
G. Haro-Sosa and S. Venkatesan, “Personified health care transitions with automated doctor appointment system: Logistics,” Journal of Pharmaceutical Negative Results, pp. 2832–2839, 2023.
S. Venkatesan and Z. Rehman, “The power of 5G networks and emerging technology and innovation: Overcoming ongoing century challenges,” Ion Exchange and Adsorption, vol. 23, no. 1, 2023.
J. Cruz Ángeles, The legal-community obligations of the large digital service provider platforms in the metaverse era, Cuad. transnational law , vol. 14, no. 2, p. 294-318, 2022.
J. Cruz Ángeles, The guardians of access to the metaverse. (Re)thinking the Competition Law of the European Union, Cuad. transnational law , vol. 15, no. 1, p. 275-296, 2023.
S. Venkatesan, “Challenges of datafication: Theoretical, training, and communication aspects of artificial intelligence,” Ion Exchange and Adsorption, vol. 23, no. 1, 2023.
S. Venkatesan, “Design an intrusion detection system based on feature selection using ML algorithms,” MSEA, vol. 72, no. 1, pp. 702–710, Feb. 2023.
S. Venkatesan, “Identification protocol heterogeneous systems in cloud computing,” MSEA, vol. 72, no. 1, pp. 615–621, Feb. 2023.
E. Vashishtha and H. Kapoor, "Enhancing patient experience by automating and transforming free text into actionable consumer insights: a natural language processing (NLP) approach," International Journal of Health Sciences and Research, vol. 13, no. 10, pp. 275-288, Oct. 2023.
K. Shukla, E. Vashishtha, M. Sandhu, and R. Choubey, "Natural Language Processing: Unlocking the Power of Text and Speech Data," Xoffencer International Book Publication House, 2023, p. 251.
S. Venkatesan, “Perspectives and challenges of artificial intelligence techniques in commercial social networks,” vol. 21, no. 5, 2023.
S. Venkatesan, “Utilization of media skills and technology use among students and educators in the state of New York,” NeuroQuantology, vol. 21, no. 5, pp. 111–124, 2023.
S. Venkatesan, S. Bhatnagar, and J. L. T. León, “A recommender system based on matrix factorization techniques using collaborative filtering algorithm,” NeuroQuantology, vol. 21, no. 5, pp. 864–872, 2023,
S. Venkatesan, S. Bhatnagar, I. M. Hidalgo Cajo, and X. L. G. Cervantes, “Efficient public key cryptosystem for wireless network,” NeuroQuantology, vol. 21, no. 5, pp. 600–606, 2023.
M. Awais, A. Bhuva, D. Bhuva, S. Fatima, and T. Sadiq, “Optimized DEC: An effective cough detection framework using optimal weighted Features-aided deep Ensemble classifier for COVID-19,” Biomed. Signal Process. Control, p. 105026, 2023.
D. R. Bhuva and S. Kumar, “A novel continuous authentication method using biometrics for IOT devices,” Internet of Things, vol. 24, no. 100927, p. 100927, 2023.
D. Bhuva and S. Kumar, “Securing space cognitive communication with blockchain,” in 2023 IEEE Cognitive Communications for Aerospace Applications Workshop (CCAAW), 2023.
Veena, A., Gowrishankar, S. An automated pre-term prediction system using EHG signal with the aid of deep learning technique. Multimed Tools Appl (2023).
A. Veena and S. Gowrishankar, "Context based healthcare informatics system to detect gallstones using deep learning methods," International Journal of Advanced Technology and Engineering Exploration, vol. 9, (96), pp. 1661-1677, 2022.
Veena, A., Gowrishankar, S. (2021). Healthcare Analytics: Overcoming the Barriers to Health Information Using Machine Learning Algorithms. In: Chen, J.IZ.,
Tavares, J.M.R.S., Shakya, S., Iliyasu, A.M. (eds) Image Processing and Capsule Networks. ICIPCN 2020. Advances in Intelligent Systems and Computing, vol 1200. Springer, Cham.
A. Veena and S. Gowrishankar, "Processing of Healthcare Data to Investigate the Correlations and the Anomalies," 2020 Fourth International Conference on I-SMAC (IoT in Social, Mobile, Analytics and Cloud) (I-SMAC), Palladam, India, 2020, pp. 611-617,
A. Veena and S. Gowrishankar, "Applications, Opportunities, and Current Challenges in the Healthcare Industry", 2022 Healthcare 4.0: Health Informatics and Precision Data Management, 2022, pp. 27–50.
B. Naeem, B. Senapati, M. S. Islam Sudman, K. Bashir, and A. E. M. Ahmed, "Intelligent road management system for autonomous, non-autonomous, and VIP vehicles," World Electric Veh. J., vol. 14, no. 9, 2023.
M. Soomro et al., "Constructor development: Predicting object communication errors," in 2023 IEEE International Conference on Emerging Trends in Engineering, Sciences and Technology (ICES&T), 2023.
M. Soomro et al., "In MANET: An improved hybrid routing approach for disaster management," in 2023 IEEE International Conference on Emerging Trends in Engineering, Sciences and Technology (ICES&T), 2023.
Senapati and B. S. Rawal, "Adopting a deep learning split-protocol based predictive maintenance management system for industrial manufacturing operations," in Lecture Notes in Computer Science, Singapore: Springer Nature Singapore, 2023, pp. 22–39.
Senapati and B. S. Rawal, "Adopting a deep learning split-protocol based predictive maintenance management system for industrial manufacturing operations," in Big Data Intelligence and Computing. DataCom 2022, Lecture Notes in Computer Science, vol. 13864, C. H. Hsu, M. Xu, H. Cao, H. Baghban, and A. B. M. Shawkat Ali, Eds., Singapore: Springer, 2023, pp. 22–39.
M. Sabugaa, B. Senapati, Y. Kupriyanov, Y. Danilova, S. Irgasheva, and E. Potekhina, "Evaluation of the prognostic significance and accuracy of screening tests for alcohol dependence based on the results of building a multilayer perceptron," in Artificial Intelligence Application in Networks and Systems. CSOC 2023, Lecture Notes in Networks and Systems, vol. 724, R. Silhavy and P. Silhavy, Eds., Cham: Springer, 2023, pp. 373–384.
Senapati and B. S. Rawal, "Quantum communication with RLP quantum resistant cryptography in industrial manufacturing," Cyber Security and Applications, vol. 100019, 2023.
M. Akbar, I. Ahmad, M. Mirza, M. Ali, and P. Barmavatu, “Enhanced authentication for de-duplication of big data on cloud storage system using machine learning approach,” Cluster Comput., vol. 27, no. 3, pp. 3683–3702, 2024.
M. Akbar, M. M. Waseem, S. H. Mehanoor, and P. Barmavatu, “Blockchain-based cyber-security trust model with multi-risk protection scheme for secure data transmission in cloud computing,” Cluster Comput., 2024.
J. I. D. Raj, R. B. Durairaj, S. V. Ananth, and P. Barmavatu, “Experimental investigation of the effect of e‐waste fillers on the mechanical properties of Kenaf woven fiber composites,” Environ. Qual. Manage., vol. 34, no. 1, 2024.
P. Rex, M. K. Rahiman, P. Barmavatu, S. B. Aryasomayajula Venkata Satya Lakshmi, and N. Meenakshisundaram, “Catalytic pyrolysis of polypropylene and polyethylene terephthalate waste using graphene oxide‐sulfonated zirconia (GO‐Szr) and analysis of its oil properties for Bharat Stage VI fuel production,” Environ. Qual. Manage., vol. 33, no. 4, pp. 501–511, 2024.
J. Immanuel Durai Raj, R. I. B. Durairaj, A. John Rajan, and P. Barmavatu, “Effect of e-waste nanofillers on the mechanical, thermal, and wear properties of epoxy-blend sisal woven fiber-reinforced composites,” Green Process. Synth., vol. 12, no. 1, 2023.
S. Ohol, V. K. Mathew, V. Bhojwani, N. G. Patil, and P. Barmavatu, “Effect of PCM-filled hallow fin heat sink for cooling of electronic components — a numerical approach for thermal management perspective,” Int. J. Mod. Phys. C., 2024.
K. Subramanian, N. Meenakshisundaram, and P. Barmavatu, “Experimental and theoretical investigation to optimize the performance of solar still,” Desalination Water Treat., vol. 318, no. 100343, p. 100343, 2024.
K. Subramanian, N. Meenakshisundaram, P. Barmavatu, and B. Govindarajan, “Experimental investigation on the effect of nano-enhanced phase change materials on the thermal performance of single slope solar still,” Desalination Water Treat., vol. 319, no. 100416, p. 100416, 2024.
P. Rex, N. Meenakshisundaram, and P. Barmavatu, “Sustainable valorisation of kitchen waste through greenhouse solar drying and microwave pyrolysis– technology readiness level for the production of biochar,” J. Environ. Health Sci. Eng., 2024.
T. Prasad, B. Praveen, Y. A. Kumar, and K. Krishna, “Development of carbon and glass fiber-reinforced composites with the addition of nano-egg-shell powder,” in Lecture Notes in Mechanical Engineering, Singapore: Springer Nature Singapore, 2022, pp. 569–577.
B. Praveen, M. Mohan Reddy Nune, Y. Akshay Kumar, and R. Subash, “Investigating the effect of minimum quantity lubrication on surface finish of EN 47 steel material,” Mater. Today, vol. 38, pp. 3253–3257, 2021.
S. Das, R. K. Ghadai, G. Sapkota, S. Guha, P. Barmavatu, and K. R. Kumar, “Optimization of CNC turning parameters of copper–nickel (Cu–Ni) alloy using VIKOR, MOORA and GRA techniques,” Int. J. Interact. Des. Manuf. (IJIDeM), 2024.
U. B. Vishwanatha, Y. D. Reddy, P. Barmavatu, and B. S. Goud, “Insights into stretching ratio and velocity slip on MHD rotating flow of Maxwell nanofluid over a stretching sheet: Semi-analytical technique OHAM,” J. Indian Chem. Soc., vol. 100, no. 3, p. 100937, 2023.
S. Sultana, P. Pandian, J. K. Gupta, B. Rajkamal, P. Barmavatu, and D. Mohanty, "Hybrids of imidazole with indoline derivatives: Microwave assisted synthesis, molecular docking studies, possible biological activities," Journal of the Indian Chemical Society, vol. 101, no. 4, pp. 101143-10, 2024.
P. P. Anand, U. K. Kanike, P. Paramasivan, S. S. Rajest, R. Regin, and S. S. Priscila, “Embracing Industry 5.0: Pioneering Next-Generation Technology for a Flourishing Human Experience and Societal Advancement,” FMDB Transactions on Sustainable Social Sciences Letters, vol.1, no. 1, pp. 43–55, 2023.
G. Gnanaguru, S. S. Priscila, M. Sakthivanitha, S. Radhakrishnan, S. S. Rajest, and S. Singh, “Thorough analysis of deep learning methods for diagnosis of COVID-19 CT images,” in Advances in Medical Technologies and Clinical Practice, IGI Global, pp. 46–65, 2024.
G. Gowthami and S. S. Priscila, “Tuna swarm optimisation-based feature selection and deep multimodal-sequential-hierarchical progressive network for network intrusion detection approach,” Int. J. Crit. Comput.-based Syst., vol. 10, no. 4, pp. 355–374, 2023.
A. J. Obaid, S. Suman Rajest, S. Silvia Priscila, T. Shynu, and S. A. Ettyem, “Dense convolution neural network for lung cancer classification and staging of the diseases using NSCLC images,” in Proceedings of Data Analytics and Management, Singapore; Singapore: Springer Nature, pp. 361–372, 2023.
S. S. Priscila and A. Jayanthiladevi, “A study on different hybrid deep learning approaches to forecast air pollution concentration of particulate matter,” in 2023 9th International Conference on Advanced Computing and Communication Systems (ICACCS), Coimbatore, India, 2023.
S. S. Priscila, S. S. Rajest, R. Regin, and T. Shynu, “Classification of Satellite Photographs Utilizing the K-Nearest Neighbor Algorithm,” Central Asian Journal of Mathematical Theory and Computer Sciences, vol. 4, no. 6, pp. 53–71, 2023.
S. S. Priscila and S. S. Rajest, “An Improvised Virtual Queue Algorithm to Manipulate the Congestion in High-Speed Network”,” Central Asian Journal of Medical and Natural Science, vol. 3, no. 6, pp. 343–360, 2022.
S. S. Priscila, S. S. Rajest, S. N. Tadiboina, R. Regin, and S. András, “Analysis of Machine Learning and Deep Learning Methods for Superstore Sales Prediction,” FMDB Transactions on Sustainable Computer Letters, vol. 1, no. 1, pp. 1–11, 2023.
R. Regin, Shynu, S. R. George, M. Bhattacharya, D. Datta, and S. S. Priscila, “Development of predictive model of diabetic using supervised machine learning classification algorithm of ensemble voting,” Int. J. Bioinform. Res. Appl., vol. 19, no. 3, 2023.
S. Silvia Priscila, S. Rajest, R. Regin, T. Shynu, and R. Steffi, “Classification of Satellite Photographs Utilizing the K-Nearest Neighbor Algorithm,” Central Asian Journal of Mathematical Theory and Computer Sciences, vol. 4, no. 6, pp. 53–71, 2023.
A. G. Usman et al., “Environmental modelling of CO concentration using AI-based approach supported with filters feature extraction: A direct and inverse chemometrics-based simulation,” Sustain. Chem. Environ., vol. 2, p. 100011, 2023.
M. A. Yassin et al., “Advancing SDGs : Predicting Future Shifts in Saudi Arabia ’ s Terrestrial Water Storage Using Multi-Step-Ahead Machine Learning Based on GRACE Data,” 2024.
M. A. Yassin, A. G. Usman, S. I. Abba, D. U. Ozsahin, and I. H. Aljundi, “Intelligent learning algorithms integrated with feature engineering for sustainable groundwater salinization modelling: Eastern Province of Saudi Arabia,” Results Eng., vol. 20, p. 101434, 2023.
S. I. Abba, A. G. Usman, and S. IŞIK, “Simulation for response surface in the HPLC optimization method development using artificial intelligence models: A data-driven approach,” Chemom. Intell. Lab. Syst., vol. 201, no. April, 2020.
S. R. S. Steffi, R. Rajest, T. Shynu, and S. S. Priscila, “Analysis of an Interview Based on Emotion Detection Using Convolutional Neural Networks,” Central Asian Journal of Theoretical and Applied Science, vol. 4, no. 6, pp. 78–102, 2023.
S. S. Priscila, D. Celin Pappa, M. S. Banu, E. S. Soji, A. T. A. Christus, and V. S. Kumar, “Technological frontier on hybrid deep learning paradigm for global air quality intelligence,” in Cross-Industry AI Applications, IGI Global, pp. 144–162, 2024.
S. S. Priscila, E. S. Soji, N. Hossó, P. Paramasivan, and S. Suman Rajest, “Digital Realms and Mental Health: Examining the Influence of Online Learning Systems on Students,” FMDB Transactions on Sustainable Techno Learning, vol. 1, no. 3, pp. 156–164, 2023.
S. S. Rajest, S. Silvia Priscila, R. Regin, T. Shynu, and R. Steffi, “Application of Machine Learning to the Process of Crop Selection Based on Land Dataset,” International Journal on Orange Technologies, vol. 5, no. 6, pp. 91–112, 2023.
T. Shynu, A. J. Singh, B. Rajest, S. S. Regin, and R. Priscila, “Sustainable intelligent outbreak with self-directed learning system and feature extraction approach in technology,” International Journal of Intelligent Engineering Informatics, vol. 10, no. 6, pp.484-503, 2022.