ChapterPDF Available

Augmented Intelligence: Deep Learning Models for Healthcare

April 2022

April 2022

DOI:10.1007/978-981-19-1076-0_10

In book: Augmented Intelligence in Healthcare: A Pragmatic and Integrated Analysis (pp.167-187)

Authors:

Paranthaman Mohanraj

Kongunadu College of Engineering and Technology

Dr-Palanivel Rajan S

Sri Eshwar College of Engineering

Actionable insights and learning from a highly complex biomedical dataset is a key challenge in smart healthcare. Traditional data processing algorithms fails to provide the better results with complex data. Recent advancements in artificial intelligence methods introduced an end to end complex learning models called deep neural networks often referred as deep learning models. In this chapter, we reviewed recent advancements of deep learning models and its applications related to healthcare. We also discussed the challenges and opportunities faced by the deep learning models.

Content uploaded by Dr-Palanivel Rajan S

Content may be subject to copyright.

Studies in Computational Intelligence 1024

SushrutaMishra

HrudayaKumarTripathy

PradeepMallick

KhaledShaalanEditors

Augmented

Intelligence

in Healthcare:

APragmatic and

Integrated Analysis

Studies in Computational Intelligence

Volume 1024

Series Editor

Janusz Kacprzyk, Polish Academy of Sciences, Warsaw, Poland

The series “Studies in Computational Intelligence” (SCI) publishes new develop-

ments and advances in the various areas of computational intelligence—quickly and

with a high quality. The intent is to cover the theory, applications, and design methods

of computational intelligence, as embedded in the ﬁelds of engineering, computer

science, physics and life sciences, as well as the methodologies behind them. The

series contains monographs, lecture notes and edited volumes in computational

intelligence spanning the areas of neural networks, connectionist systems, genetic

algorithms, evolutionary computation, artiﬁcial intelligence, cellular automata, self-

organizing systems, soft computing, fuzzy systems, and hybrid intelligent systems.

Of particular value to both the contributors and the readership are the short publica-

tion timeframe and the world-wide distribution, which enable both wide and rapid

dissemination of research output.

Indexed by SCOPUS, DBLP, WTI Frankfurt eG, zbMATH, SCImago.

All books published in the series are submitted for consideration in Web of Science.

More information about this series at https://link.springer.com/bookseries/7092

Sushruta Mishra ·Hrudaya Kumar Tripathy ·

Pradeep Mallick ·Khaled Shaalan

Editors

Augmented Intelligence

in Healthcare: A Pragmatic

and Integrated Analysis

Editors

Sushruta Mishra

Kalinga Institute of Industrial Technology

Bhubaneswar, Odisha, India

Pradeep Mallick

School of Computer Engineering

Kalinga Institute of Industrial Technology

Bhubaneswar, Odisha, India

Hrudaya Kumar Tripathy

Kalinga Institute of Industrial Technology

Bhubaneswar, Odisha, India

Khaled Shaalan

Dubai International Academic City

The British University in Dubai

Dubai, United Arab Emirates

ISSN 1860-949X ISSN 1860-9503 (electronic)

Studies in Computational Intelligence

ISBN 978-981-19-1075-3 ISBN 978-981-19-1076-0 (eBook)

https://doi.org/10.1007/978-981-19-1076-0

Singapore Pte Ltd. 2022

This work is subject to copyright. All rights are solely and exclusively licensed by the Publisher, whether

the whole or part of the material is concerned, speciﬁcally the rights of translation, reprinting, reuse

of illustrations, recitation, broadcasting, reproduction on microﬁlms or in any other physical way, and

transmission or information storage and retrieval, electronic adaptation, computer software, or by similar

or dissimilar methodology now known or hereafter developed.

The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication

does not imply, even in the absence of a speciﬁc statement, that such names are exempt from the relevant

protective laws and regulations and therefore free for general use.

The publisher, the authors and the editors are safe to assume that the advice and information in this book

are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or

the editors give a warranty, expressed or implied, with respect to the material contained herein or for any

errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional

claims in published maps and institutional afﬁliations.

This Springer imprint is published by the registered company Springer Nature Singapore Pte Ltd.

The registered company address is: 152 Beach Road, #21-01/04 Gateway East, Singapore 189721,

Singapore

Preface

When machine intelligence assists human intelligence, it leads to augmented intel-

ligence. While artiﬁcial intelligence promises to replicate all of the human intelli-

gence, augmented intelligence claims to enhance and scale it. Augmented intelligence

can increase the efﬁciency of diagnosis in healthcare organizations as it reﬂects the

enhanced capabilities of human decision-making in clinical settings when augmented

with computation systems and methods. The goal of this edited book is to publish

original manuscripts that address broad challenges on both theoretical and applica-

tion aspects of augmented intelligence in e-Health, biomedical, health informatics,

medical data analysis, and other aspects of health care.

This book accommodates twenty-ﬁve chapters. The initial three chapters deal

with succinct analysis of machine intelligent techniques in healthcare domain and

disease assessment like breast cancer and health metrics monitoring. Chapters four

and ﬁve highlight the role of machine learning in Alzheimer disease detection

and medical imaging. Subsequent chapters from six to ten discuss the impact of

various deep learning models on augmented intelligence health care for different

health risks diagnosis like lung carcinoma, brain tumor, personalized health care,

etc. Further, different aspects of sentiment analysis and emotion recognition are

explored in context to augmented intelligence in chapters eleven to thirteen. Chap-

ters fourteen and ﬁfteen detail the relevance of cognitive intelligence in healthcare

setting. Distinct scenarios related to blockchain concepts in health care are detailed in

chapters sixteen and seventeen. Next two chapters present the signiﬁcance of security

and privacy using augmented models in healthcare zone. Further chapters twenty and

twenty-one throw light on the application of augmented intelligent frameworks in

handling COVID-19 pandemic. Finally, the last four chapters present some advanced

approaches in healthcare domain. Thus, the handbook intends to play a signiﬁcant

vi Preface

role in assessment and acquiring knowledge in context to augmented intelligence in

healthcare domain.

Bhubaneswar, India

Dubai, United Arab Emirates

Sushruta Mishra

Hrudaya Kumar Tripathy

Pradeep Mallick

Khaled Shaalan

Contents

A Bibliometric Analysis on the Role of Artiﬁcial Intelligence

in Healthcare ..................................................... 1

Faisal Suhail, Mouhand Adel, Mostafa Al-Emran, and Khaled Shaalan

Supervised Intelligent Clinical Approach for Breast Cancer Tumor

Categorization .................................................... 15

Lambodar Jena, Lara Ammoun, and Premkumar Chithaluru

Health Monitoring and Integrated Wearables ........................ 41

S. Sivaranjani, P. Vinoth Kumar, and S. Palanivel Rajan

A Comprehensive Review Analysis of Alzheimer’s Disorder Using

Machine Learning Approach ....................................... 63

Priyom Dutta and Sushruta Mishra

Machine Learning Techniques in Medical Image: A Short Review ...... 77

Ashwini Kumar Pradhan, Kaberi Das, and Debahuti Mishra

Analysis of Diabetic Retinopathy Detection Techniques Using

CNN Models ...................................................... 87

P. Prabhavathy, B. K. Tripathy, and M. Venkatesan

Experimental Evaluation of Brain Tumor Image Segmentation

and Detection Using CNN Model .................................... 103

Debjit Koner and Soumya Sahoo

Effective Deep Learning Algorithms for Personalized Healthcare

Services .......................................................... 121

Anjana Mishra, Siddha Sachida Mohapatra, and Sukant Kishoro Bisoy

Automatic Lung Carcinoma Identiﬁcation and Classiﬁcation

in CT Images Using CNN Deep Learning Model ...................... 143

Ritu Tandon, Shweta Agrawal, Rachana Raghuwanshi,

Narendra Pal Singh Rathore, Lalji Prasad, and Vishal Jain

vii

viii Contents

Augmented Intelligence: Deep Learning Models for Healthcare ........ 167

M. Paranthaman and S. Palanivel Rajan

Sentiment Analysis and Emotion Detection with Healthcare

Perspective ....................................................... 189

Sathish Kumar, Rama Prabha, and Selvakumar Samuel

Augmented Intelligence in Mental Health Care: Sentiment

Analysis and Emotion Detection with Health Care Perspective ......... 205

Asmita De and Sushruta Mishra

NLP Applications for Big Data Analytics Within Healthcare ........... 237

Aadarsh Choudhary, Anurag Choudhary, and Shubham Suman

Cognitive Computing Driven Healthcare: A Precise Study ............. 259

Rohan Sharma and Uday Bhanu Ghosh

Cognitive Techniques for Brain Disorder Management: A Future

Trend ............................................................ 281

Mihir Narayan Mohanty

Relevance of Blockchain in Revolutionizing Health Records ........... 301

Amlan Mishra, Kashif Moin, Mayank Shrivastava,

and Hrudaya Kumar Tripathy

A Systematic Review on Blockchain Technology: Concepts,

Applications, and Prospects in Healthcare ........................... 315

Adarsh Tikmani, Saurabh Bilgaiyan, Bhabani Shankar Prasad Mishra,

and Santwana Sagnika

Integrated Machine Learning Models for Enhanced Security

of Healthcare Data ................................................ 355

Shasank Periwal, Tridiv Swain, and Sushruta Mishra

Symptoms-Based Biometric Pattern Detection and Recognition ........ 371

Uday Bhanu Ghosh, Rohan Sharma, and Abhishek Kesharwani

Time Series Analysis of COVID-19 Waves in India for Social Good ..... 401

Lakshmi Swarna Durga Nallam, Sindhu Sankati,

Hiren Kumar Thakkar, and Priyanka Singh

Detection of COVID-19 Using a Multi-scale Deep Learning

Network: Covid-MSNet ............................................ 417

S. V. Aruna Kumar, S. Nagashree, and B. S. Mahanand

Immersive Technologies in the Healthcare Space ..................... 433

Selvakumar Samuel

Artiﬁcial Intelligence in Telemedicine: A Brief Survey ................ 453

Sibanjan Debeeprasad Das and Pradip Kumar Bala

Contents ix

Infectious Diseases Reporting System Using Naïve Bayes

Classiﬁcation Algorithm ........................................... 463

Ishola D. Muraina and Abdullahi Umar Farouk

A Comprehensive Study of Explainable Artiﬁcial Intelligence

in Healthcare ..................................................... 475

Aryan Mohanty and Sushruta Mishra

Editors and Contributors

About the Editors

Dr. Sushruta Mishra is working as Assistant Professor in the School of Computer

Engineering, KIIT University, Bhubaneswar, Odisha, India. He pursued his M.Tech.

from IIIT Bhubaneswar in 2012 and has completed his Ph.D. in Computer Science

from KIIT University, Bhubaneswar, Odisha, India, in 2017. He has more than 8

years of teaching experience in various educational institutions. He has handled

many subjects such as computer networks, data mining, software engineering, and

machine learning during his academic experience. His research interest includes

image processing, machine learning, Internet of things, and cognitive computing.

He has published several research articles in reputed SCIE journals, Scopus indexed

international journals, edited books, and conferences.

Dr. Hrudaya Kumar Tripathy is presently working as Associate Professor and

Program Head of Master Program at the School of Computer Engineering, KIIT

(Deemed to be University), Bhubaneswar, India. He has completed Ph.D. in

Computer Science (Berhampur University) and an M.Tech. in Computer Science

and Engineering from the Indian Institute of Technology Guwahati. He had been

Visiting Faculty at Asia Paciﬁc University, Kuala Lumpur, Malaysia, and Univer-

siti Utara Malaysia, Sintok, Malaysia. He has received the research Post Doctoral

research fellowship from the Ministry of Higher Education Malaysia. He has 20 years

of teaching experience with post-doctorate research experience in the ﬁeld of artiﬁ-

cial intelligence, machine learning, mobile robotics, and data analysis. He received

many certiﬁcates of merit and highly applauded in a presentation of research papers

at international conferences. He has published many research papers in reputed inter-

national and national refereed journals and conferences. The Computer Society of

India (CSI) has awarded the young IT professional award 2013 to him. He is Senior

Member of IEEE, Life Member of CSI, and having membership in other professional

bodies such as IET, IACSIT, and IAENG.

xii Editors and Contributors

Dr. Pradeep Mallick is currently working as Associate Professor in the School of

Computer Engineering, Kalinga Institute of Industrial Technology (KIIT) Deemed to

be University, Odisha, India. He has also served as Professor and Head Department

of Computer Science and Engineering, Vignana Bharathi Institute of Technology,

Hyderabad. He has completed his Post Doctoral Fellow (PDF) in Kongju National

University South Korea, Ph.D. from Siksha ‘O’ Anusandhan University, M.Tech.

(CSE) from Biju Patnaik University of Technology (BPUT), and MCA from Fakir

Mohan University, Balasore, India. Besides academics, he is also involved in various

administrative activities, Member of Board of Studies, Member of Doctoral Research

Evaluation Committee, Admission Committee, etc. His area of research includes

algorithm design and analysis, data mining, image processing, soft computing, and

machine learning. Now, he is Editorial Member of the Korean Convergence Society

for SMB. He has published 9 books and more than 70 research papers in national

and international journals and conference proceedings to his credit.

Dr. Khaled Shaalan is Full Professor and Programme Head of Computer Science

at the British University in Dubai (BUiD), UAE. He is ranked among the top 2% of

scientists in 2019 according to a study led by Dr. Ioannidis and his research team at

Stanford University. He is Honorary Fellow at the School of Informatics, University

of Edinburgh (UoE), UK. Over the last two decades, he has been contributing to a

wide range of research topics in AI, Arabic NLP, knowledge management, health

informatics, and educational technology. He has published 240+ refereed publi-

cations. His research work is cited extensively worldwide, and the impact of his

research using Google Scholar’s H-index metric is 40+. He has been actively and

extensively supporting the local and international academic community. He acts as

the chair of international conferences, journals and books editor, keynote speaker, an

external member of promotions committees, among others. He is Associate Editor

on ACM Transactions of Asian and Low Resource Language Information Processing

(TALLIP) editorial board, published by the Association for Computing Machinery

(ACM), USA. He is also Member of the editorial board of the AKCE International

Journal of Graphs and Combinatorics (AKCE), Taylor and Francis.

Contributors

Mouhand Adel Faculty of Engineering and IT, The British University in Dubai,

Dubai, UAE

Shweta Agrawal SAGE University, Indore, M.P., India

Mostafa Al-Emran Faculty of Engineering and IT, The British University in Dubai,

Dubai, UAE

Lara Ammoun Department of Computer Science and Engineering, Siksha ‘O’

Anusandhan Deemed to be University, Bhubaneswar, Odisha, India

Editors and Contributors xiii

S. V. Aruna Kumar Department of Computer Science and Engineering, Malnad

College of Engineering, Hassan, Karnataka, India

Asmita De School of Computer Engineering, Kalinga Institute of Industrial Tech-

nology, Deemed to be University, Bhubaneswar, India

Pradip Kumar Bala Indian Institute of Management, Ranchi, India

Saurabh Bilgaiyan School of Computer Engineering, Kalinga Institute of Indus-

trial Technology, Deemed to be University, Bhubaneswar, Odisha, India

Sukant Kishoro Bisoy Department of Computer Science and Information Tech-

nology, C.V. Raman Global University, Bhubaneswar, Odisha, India

Premkumar Chithaluru Department of Computer Science and Engineering,

Koneru Lakshmaiah Education Foundation (KLEF), Vaddeswaram, Vijayawada,

Andhra Pradesh, India

Aadarsh Choudhary Xformics Inc., Bangalore, India

Anurag Choudhary Xformics Inc., Bangalore, India

Kaberi Das Department of Computer Science and Engineering, Siksha ‘O’

Anusandhan (Deemed To Be University), Bhubaneswar, Odisha, India

Sibanjan Debeeprasad Das Indian Institute of Management, Ranchi, India

Priyom Dutta School of Computer Engineering, KIIT Deemed To Be University,

Bhubaneswar, Odisha, India

Abdullahi Umar Farouk Computer Science Department, Faculty of Science,

Yusuf Maitama Sule University (Formerly, Northwest University), Kano, Nigeria

Uday Bhanu Ghosh HighRadius Corporation, Bhubaneswar, India

Vishal Jain Sharda University, Noida, India

Lambodar Jena Department of Computer Science and Engineering, Koneru

Lakshmaiah Education Foundation (KLEF), Vaddeswaram, Vijayawada, Andhra

Pradesh, India

Abhishek Kesharwani Vellore Institute of Technology, Vellore, India

Debjit Koner Accenture Solutions Pvt Ltd, Bangalore, Karnataka, India

Sathish Kumar Asia Paciﬁc University of Technology and Innovation, Kuala

Lumpur, Malaysia

B. S. Mahanand Department of Information Science and Engineering, Sri

Jayachamarajendra College of Engineering, JSS Science and Technology University,

Mysuru, Karnataka, India

Amlan Mishra Kalinga Institute of Industrial Technology, Deemed to be Univer-

sity, Bhubaneswar, India

xiv Editors and Contributors

Anjana Mishra Department of Computer Science and Information Technology,

C.V. Raman Global University, Bhubaneswar, Odisha, India

Bhabani Shankar Prasad Mishra School of Computer Engineering, Kalinga

Institute of Industrial Technology, Deemed to be University, Bhubaneswar, Odisha,

India

Debahuti Mishra Department of Computer Science and Engineering, Siksha ‘O’

Anusandhan (Deemed To Be University), Bhubaneswar, Odisha, India

Sushruta Mishra School of Computer Engineering, Kalinga Institute of Industrial

Technology, Deemed to be University, Bhubaneswar, Odisha, India

Aryan Mohanty Kalinga Institute of Industrial Technology, Bhubaneswar, India

Mihir Narayan Mohanty ITER, Siksha ‘O’ Anusandhan Deemed to be University,

Bhubaneswar, Odisha, India

Siddha Sachida Mohapatra Department of Computer Science and Information

Technology, C.V. Raman Global University, Bhubaneswar, Odisha, India

Kashif Moin Kalinga Institute of Industrial Technology, Deemed to be University,

Bhubaneswar, India

Ishola D. Muraina Computer Science Department, Faculty of Science, Yusuf

Maitama Sule University (Formerly, Northwest University), Kano, Nigeria

S. Nagashree Department of Information Science and Engineering, JSS ATE,

Bangalore, Karnataka, India

Lakshmi Swarna Durga Nallam Department of Computer Science and Engi-

neering, SRM University, Andhra Pradesh, Amaravati, India

S. Palanivel Rajan Department of Electronics and Communication Engineering,

M. Kumarasamy College of Engineering, Karur, Tamilnadu, India

M. Paranthaman Department of Electronics and Communication Engineering, M.

Kumarasamy College of Engineering, Karur, Tamilnadu, India

Shasank Periwal Kalinga Institute of Industrial Technology, Deemed to be Univer-

sity, Bhubaneswar, Odisha, India

Rama Prabha Nehru Arts and Science College, Coimbatore, Tamil Nadu, India

P. Prabhavathy School of Information Technology and Engineering, VIT, Vellore,

Tamil Nadu, India

Ashwini Kumar Pradhan Department of Computer Science and Engineering,

Koneru Lakshmaiah Education Foundation, Vaddeswaram, Andhra Pradesh, India;

Department of Computer Science and Engineering, Siksha ‘O’ Anusandhan (Deemed

To Be University), Bhubaneswar, Odisha, India

Lalji Prasad SAGE University, Indore, M.P., India

Editors and Contributors xv

Rachana Raghuwanshi SAGE University, Indore, M.P., India

Narendra Pal Singh Rathore SAGE University, Indore, M.P., India

Santwana Sagnika School of Computer Engineering, Kalinga Institute of Indus-

trial Technology, Deemed to be University, Bhubaneswar, Odisha, India

Soumya Sahoo C.V. Raman Global University, Bhubaneswar, Odisha, India

Selvakumar Samuel Asia Paciﬁc University of Technology and Innovation, Kuala

Lumpur, Malaysia

Sindhu Sankati Department of Computer Science and Engineering, SRM Univer-

sity, Andhra Pradesh, Amaravati, India

Khaled Shaalan Faculty of Engineering and IT, The British University in Dubai,

Dubai, UAE

Rohan Sharma HighRadius Corporation, Bhubaneswar, India;

Vellore Institute of Technology, Vellore, India

Mayank Shrivastava Kalinga Institute of Industrial Technology, Deemed to be

University, Bhubaneswar, India

Priyanka Singh Department of Computer Science and Engineering, SRM Univer-

sity, Andhra Pradesh, Amaravati, India

S. Sivaranjani ECE, M. Kumarasamy College of Engineering, Karur, India

Faisal Suhail Faculty of Engineering and IT, The British University in Dubai,

Dubai, UAE

Shubham Suman Indian Institute of Technology (ISM), Dhanbad, India

Tridiv Swain Kalinga Institute of Industrial Technology, Deemed to be University,

Bhubaneswar, Odisha, India

Ritu Tandon SAGE University, Indore, M.P., India

Hiren Kumar Thakkar Department of Computer Engineering, Marwadi Univer-

sity, Rajkot, Gujarat, India

Adarsh Tikmani School of Computer Engineering, Kalinga Institute of Industrial

Technology, Deemed to be University, Bhubaneswar, Odisha, India

B. K. Tripathy School of Information Technology and Engineering, VIT, Vellore,

Tamil Nadu, India

Hrudaya Kumar Tripathy Kalinga Institute of Industrial Technology, Deemed to

be University, Bhubaneswar, India

M. Venkatesan Department of Computer Science Engineering, NIT Puducherry,

Puducherry, India

P. Vinoth Kumar ECE, Nandha College of Technology, Erode, India

A Bibliometric Analysis on the Role

of Artiﬁcial Intelligence in Healthcare

Faisal Suhail, Mouhand Adel, Mostafa Al-Emran, and Khaled Shaalan

Abstract The rapid growth of artiﬁcial intelligence (AI) has reached unprecedented

levels across different ﬁelds. In this bibliometric analysis, we reviewed 1999 studies

published between 2011 and 2021 on the role of AI applications in facilitating health-

care services. This review aims to shed light on the scientiﬁc achievements of AI in

healthcare through examining the research focus of existing studies, major diseases,

major AI tasks and applications, most productive authors and countries, and most

common journals in the domain. The results showed that the extant literature has

focused on four distinct clusters, including the theory and process behind machine

learning, deep learning algorithms, experiments and results, and COVID-19 related

issues. The results indicated that COVID-19, pneumonia, different cancer types,

neurodegenerative diseases, and diabetes are the major diseases that received careful

attention from AI applications. The results also indicated that image processing

and diagnostic imaging were the most common tasks, while deep learning tech-

niques were the most common applications of AI in healthcare. The taxonomy of

the analyzed literature would be helpful for practitioners, researchers, and decision-

makers working in healthcare sectors to advance the wheel of medical informatics.

It can be argued that the door is still open for improving the role of AI in healthcare,

whether in its theoretical (e.g., models and algorithms) or physical (e.g., surgical

robots) form.

Keywords Bibliometric analysis ·Artiﬁcial intelligence ·Applications ·Tas ks ·

Diseases ·Healthcare

F. Suhail ·M. Adel ·M. Al-Emran (B)·K. Shaalan

Faculty of Engineering and IT, The British University in Dubai, Dubai, UAE

e-mail: mustafa.n.alemran@gmail.com

F. Suhail

e-mail: 20181838@ug.buid.ac.ae

M. Adel

e-mail: 20180854@ug.buid.ac.ae

K. Shaalan

e-mail: khaled.shaalan@buid.ac.ae

S. Mishra et al. (eds.), Augmented Intelligence in Healthcare: A Pragmatic and Integrated

Analysis, Studies in Computational Intelligence 1024,

https://doi.org/10.1007/978-981-19- 1076-0_1

2F. Suhail et al.

1 Introduction

With the appearance of cutting-edge technologies [1–3], speciﬁcally artiﬁcial intelli-

gence (AI), individuals, organizations, and society have experienced unprecedented

changes in performing their daily tasks. AI refers to the intelligence of machines,

contrary to human intelligence [4]. AI brought several beneﬁts to humanity, specif-

ically in situations where human capabilities are limited and require intelligence

beyond their abilities [5]. With its unlimited capabilities, AI can carry out tasks that

take a lot of time and cost in a shorter time frame and with fewer costs [6]. These

tasks have an interdisciplinary impact and can be employed across different domains,

such as marketing [7], banking [8], education [9], healthcare [10], etc.

In healthcare, AI applications have played a vital role in various medical tasks,

including surgical, clinical, rehabilitative, diagnostic, and predictive procedures.

AI also signiﬁcantly contributed to disease diagnosis and clinical decision making

[11]. These tasks have several implications speciﬁcally for healthcare providers and

patients [12]. At the same time, whether in its theoretical employment in the form

of models and algorithms or its physical use (e.g., surgical robots), AI can have

potentially unintended consequences for clinical outcomes and patients [13].

The employment of AI in healthcare has become an emergent domain both in

medicine and informatics, speciﬁcally during the last decade. The importance of the

topic motivates several scholars to conduct reviews (e.g., [14–16]) of the previous

studies to provide more insights into the role of AI and its applications in facilitating

healthcare services. However, these reviews were limited to speciﬁc diseases (e.g.,

COVID-19), tasks, or applications. Therefore, the purpose of this bibliometric anal-

ysis is to enrich the existing reviews with cluster-based content to show the research

trajectory of AI in healthcare. To do so, this study aims to achieve the following

research objectives:

•To determine the annual publication progress in the domain.

•To determine the research focus of the existing studies of AI in healthcare.

•To identify the major diseases that received signiﬁcant attention from AI scholars.

•To determine the major AI tasks and theoretical applications in healthcare.

•To identify the most productive authors in the domain.

•To identify the most productive countries.

•To shed light on the most frequent journals that published studies on AI in

healthcare.

2 Materials and Methods

This research employs the bibliometric analysis technique to analyze the qualitative

and quantitative issues related to the applications of AI in healthcare. This technique

has been primarily used to assess the content of existing research articles [17] and

examine the characteristics and patterns of a speciﬁc topic [18]. We have selected

A Bibliometric Analysis on the Role of Artiﬁcial Intelligence … 3

the VOSviewer tool for visualizing the content of the collected studies and their

bibliometric mapping [19]. The search strategy for the collected articles chosen for

ﬁnal analysis is depicted in Fig. 1. The ﬂow diagram for the search process is adapted

from Zakaria et al. [20]. Figure 1shows the topic, scope and coverage, keywords

and search strings, date extracted, records identiﬁed and screened, records removed,

and records included for bibliometric analysis. The search covers the journal articles

published between 2011 and 2021 on the topic of AI in healthcare, which are indexed

in the Scopus database. It is essential to mention that we were restricted to download

only 2000 records when collecting the articles from Scopus.

Fig. 1 Flow diagram of the search strategy

4F. Suhail et al.

3 Results and Discussion

3.1 Annual Publication Trends

Figure 2shows the annual publication trends on AI in healthcare from 2011 to 2021.

We can observe that studies on AI in healthcare ﬂuctuated between 2011 and 2014.

Since 2014, we have seen a steady rise until 2018. During these four years, the

number of publications has increased from 137 in 2014 to 322 in 2018. The number

is increased by 20–30 articles between 2014 and 2016, then increased by 80–90

articles between 2016 and 2018. However, there is a sharp drop in publications from

297 in 2019 to 18 in 2021. The reasons are twofold. First, since we conducted this

study in October 2021, a lot of recent papers have not yet been released or are still

under review. The second reason refers to the restrictions during the period of data

collection in which only the ﬁrst 2000 articles can be downloaded. It is imperative

to mention that the sorting of the articles was based on the most cited papers. This

means that the articles published in 2021 did not have sufﬁcient time to be cited to

appear ﬁrst in the search. Therefore, the role of AI in healthcare is on the rise, and we

can expect more publications in the future. This observation is supported by a recent

study [21], which indicated that applications like telemedicine, wearable devices,

mobile applications, and robotics, dominate the healthcare sector.

Fig. 2 Annual publication trends

A Bibliometric Analysis on the Role of Artiﬁcial Intelligence … 5

3.2 Research Focus

Figure 3shows the research focus of AI in healthcare. This is generated based on the

most frequent keywords. It is important to mention that the analysis does not include

keywords that appeared less than 12 times. It can be seen that the topics are divided

into four different clusters (red, green, blue, and yellow). The red cluster mainly

focuses on the theory and process behind machine learning. The blue cluster, on the

other hand, focuses on the implementation of machine learning as it covers a few

machine learning algorithms like neural networks, deep learning, and convolutional

neural networks, which indicates that these machine learning algorithms are the

most commonly utilized in healthcare. The green cluster is related to experiments

and results, as it covers words like outcome assessment, major clinical study, cross-

sectional studies, etc. The yellow cluster refers to the studies conducted on COVID-

19, as all the keywords refer to COVID and its consequences. While there was a

Fig. 3 Research focus of AI in healthcare

6F. Suhail et al.

shortage of studies on the role of AI in combating the COVID-19 at the beginning

of the pandemic [22], the results of the yellow cluster show how much research has

been carried out on this virus in less than two years.

3.3 Major Diseases

Table 1shows the most common diseases that were examined through AI applica-

tions. To unify the statistics, we have combined the counts for the keywords that carry

out the same meaning. For example, keywords like “covid-19” and “coronavirus-

19” were combined together. Incredibly, the COVID-19 is the most studied disease,

although it was discovered recently. This is because AI applications can quickly deter-

mine COVID-19 cases [23,24]. Apart from that, these applications have also been

used to diagnose and monitor COVID-19 patients [25]. The second most studied

disease, pneumonia, is also a side effect of COVID-19 while also being its own

disease at the same time. This shows how crucial the role of AI has been in combating

COVID-19 during the past two years.

Cancer is also one of the most studied diseases. Several types of cancer have

been studied, such as breast cancer, lung cancer, brain tumors, and skin tumors.

Thus, it can be argued that AI can treat and diagnose active and inactive cancers.

The role of AI has not been denied for treating and diagnosing neurodegenerative

diseases, which are characterized by the progressive deterioration of nerve cells in

the nervous system, including diseases such as Alzheimer’s, ataxia, and Parkinson’s

[26]. Diabetes is among the few studied diseases, and one of its side effects is diabetic

retinopathy, which leads to the loss of vision over time for people with diabetes [27].

Tabl e 1 Top 10 diseases Ranking Diseases Count

1Covid-19 455

2Pneumonia 190

3Breast cancer 165

4Lung cancer 93

5Neurodegenerative diseases 61

6Diabetes mellitus 36

7Brain tumor 34

8Heart disease 34

9 Skin tumor 27

10 Diabetic retinopathy 26

A Bibliometric Analysis on the Role of Artiﬁcial Intelligence … 7

Tabl e 2 Top10AItasksin

healthcare Ranking Tasks Count

1Image processing 1129

2 Diagnostic imaging 1093

3 Computer-assisted diagnosis 643

4Classiﬁcation 609

5Computerized tomography 442

6Information processing 122

7 Risk assessment 79

8Thorax radiography 59

9Neuroimaging 52

10 Computer simulation 47

3.4 Major AI Tasks in Healthcare

Table 2shows the major AI tasks in healthcare. As in the previous section, we

combined the counts for keywords with the same meaning. The most common AI

tasks in healthcare are image processing and diagnostic imaging. Diagnostic imaging

is a variety of non-invasive ways that doctors can see inside the human body, as

opposed to image processing, which is the task of performing operations on an

image. The third common task is computer-assisted diagnosis. This includes the

systems that help doctors to understand medical images, which are not common as

image processing and diagnostic imaging.

Classiﬁcation dominates the fourth rank in the list of tasks. There are many types

of classiﬁcation. This could include the classiﬁcation of images, text, or other special-

ized purposes in healthcare. Computerized tomography (CT), which comes in the ﬁfth

category, involves taking MRI images and other X-ray images. This category has been

extensively studied during the COVID-19 pandemic to assess the severity of infec-

tion [28]. This has been examined by applying deep learning techniques on CT scan

images to predict the infection of COVID-19 [29]. Information processing dominates

the sixth category; it involves storing and processing data. Risk assessment appears

the seventh on the list, which helps doctors to assess the risks involved in the steps

they are taking. On the other hand, tasks like computer simulation, neuroimaging,

and thorax radiography have received less attention from AI scholars.

3.5 Most Studied Theoretical Applications of AI

in Healthcare

Table 3lists the most studied theoretical AI applications in healthcare. As in the

previous sections, we have combined the keywords that share the same meaning.

It is essential to mention that applications in this scenario refer to AI theories or

8F. Suhail et al.

Tabl e 3 Top 9 AI

applications in healthcare Ranking Applications Count

1Machine learning 1318

2 Artiﬁcial neural network 1059

3 Deep learning 625

4Convolutional neural network 560

5Support vector machine 327

6Decision making 250

7 Data mining 150

8 Decision trees 122

9Natural language processing 119

algorithms. It can be seen that machine learning is the most prevalent AI application.

A better example would be the artiﬁcial neural network, the most commonly used

machine learning algorithm in healthcare, followed by deep learning, and convolu-

tional neural networks. These results provide insight into how machine learning

algorithms are important to the healthcare sector. This does not ignore the role

of other algorithms, such as support vector machines and decision trees. While

machine learning algorithms are currently the most common in healthcare, research

is still being conducted on employing rule-based techniques (e.g., [30,31]). Natural

language processing (NLP), which received less attention compared to other appli-

cations, can be used to facilitate the logistical side of healthcare and help doctors and

nurses with their daily tasks.

3.6 Most Productive Authors

To determine the most productive authors who studied the role of AI in healthcare,

we have generated a list of all authors along with their publications and citations. The

list was obtained through the co-authorship feature in VOSviewer tool. A minimum

of ﬁve documents were required for an author to be added to the list. Concerning the

minimum number of citations, we set it to zero, meaning that we did not place any

restrictions on citations. Table 4lists the top 10 authors. The authors are sorted based

on the number of published papers and citations. As per Fig. 4, the co-authorship

network shows how many publications the authors have co-authored. The size of the

circle indicates how many publications the author has published. For instance, if we

look at “Wang I.”, we can observe that he has the largest circle, which means the

largest number of publications.

A Bibliometric Analysis on the Role of Artiﬁcial Intelligence … 9

Tabl e 4 Top 10 authors based on number of published papers

Authors Number of published papers Citations

Wan g I. 28 2998

Zhang Y. 26 3199

Chen Y. 25 2369

Wan g G. 24 4049

Wan g Y. 24 2319

Li Y. 22 1999

Zhang J. 21 1820

Li Q. 19 1990

Zhang I. 19 1825

Liu Y. 18 2150

Fig. 4 Co-authorship network

10 F. Suhail et al.

Tabl e 5 Top 10 productive

countries Country Publications Citations

United states 811 102,867

China 389 40,591

United Kingdom 244 30,026

India 164 17,103

Germany 132 14,607

Italy 103 9429

Australia 97 8859

Canada 93 11,318

South Korea 87 8110

Netherlands 74 10,648

3.7 Most Productive Countries

We used the bibliographic coupling feature in VOSviewer to examine the most

productive countries in the domain of AI in healthcare. The threshold for the

minimum number of documents per country was set to 5, and the threshold for

the minimum number of citations per country was set to 0. Table 5lists the top 10

countries in terms of the number of publications and citations. We can clearly observe

that the USA (N=811) has the most publications, followed by China (N=389),

UK (N=244), India (N=164), Germany (N=132), Italy (N=103), Australia (N

=97), Canada (N=93), South Korea (N=87), and Netherlands (N=74).

On the other hand, slight changes can be noticed if we look at the list from the

perspective of citations. In that, the USA (N=102,867) still dominates the list,

followed by China (N=40,591), UK (N=30,026), India (N=17,103), Germany

(N=14,607), Canada (N=11,318), Netherlands (N=10,648), Italy (N=9429),

Australia (N=8859), and South Korea (N=8110). These results are almost similar

to the conclusions observed in earlier research [32]. The bibliographic links between

the countries are shown in Fig. 5. The link represents that the two linked countries cite

the same paper. It is evident that the USA has the largest circle, indicating the largest

number of publications. Although the USA has close ties with many countries, its

strongest ties are with China, the UK, Canada, Germany, and South Korea.

3.8 Most Frequent Journals

The bibliographic coupling feature in VOSviewer is used to analyze the most frequent

journals. The minimum number of publications per journal was set to 5, and no

restrictions were imposed on the number of citations. Table 6demonstrates the top

10 journals based on the number of publications and citations. It can be seen that

the IEEE Transactions on Medical Imaging (N=81) has the most publications,

A Bibliometric Analysis on the Role of Artiﬁcial Intelligence … 11

Fig. 5 Bibliographic coupling network for most productive countries based on publications

Tabl e 6 Top 10 journals based on publications count

Journals Publications Citations

IEEE Transactions on Medical Imaging 81 15,788

Medical Image Analysis 58 8630

Expert Systems with Applications 51 4472

PLOS One 51 4098

Journal of Biomedical Informatics 49 3681

IEEE Access 44 4329

Journal of the American Medical Informatics Association 40 3760

Scientiﬁc Reports 38 4264

Computer Methods and Programs in Biomedicine 35 4278

IEEE Journal of Biomedical and Health Informatics 33 4398

followed by Medical Image Analysis (N=58), Expert Systems with Applications

(N=51), PLOS One (N=51), Journal of Biomedical Informatics (N=49), IEEE

Access (N=44), Journal of the American Medical Informatics Association (N=

40), Scientiﬁc Reports (N=38), Computer Methods and Programs in Biomedicine

(N=35), and IEEE Journal of Biomedical and Health Informatics (N=33).

12 F. Suhail et al.

4 Conclusion

Research on AI and healthcare has a considerable overlap, with applications

supporting various health issues [33]. Medical professionals found it attractive to

automate tasks that boost their efﬁciency and assist them in medical diagnosis. Stem-

ming from that, we conducted a bibliometric analysis on the contribution of AI in

healthcare. The data were collected from the Scopus database and were analyzed

using the VOSviewer tool. The analysis provided insights into annual publication

trends of AI in healthcare, research focus, signiﬁcant diseases, major AI tasks, most

studied theoretical applications of AI, most productive authors, most productive

countries, and most frequent journals.

The results showed that publications on AI in healthcare are on the rise, indicating

the importance of the relationship between informatics and healthcare. The results

also showed that the extant literature has focused on the theory and process behind

machine learning, deep learning algorithms, experiments and results, and COVID-19

related issues. For the diseases that received much attention from AI applications,

COVID-19 has dominated the list in a short period. This is followed by pneumonia,

different cancer types, neurodegenerative diseases, and diabetes. The results indi-

cated that image processing and diagnostic imaging were the most common AI tasks

in healthcare. It has also been found that deep learning techniques were the most

common applications of AI in healthcare. For the most productive countries in the

domain, the USA, China, and the UK have dominated the list in terms of publica-

tions and citations count. IEEE Transactions on Medical Imaging and Medical Image

Analysis were the top two journals that published research on AI in healthcare.

It is believed that these ﬁndings would add signiﬁcant value to the emerging

literature on AI in healthcare. The taxonomy of the analyzed literature would be

helpful for practitioners, researchers, and decision-makers working in healthcare

sectors to advance the wheel of medical informatics and develop more advanced

approaches toward a digitized sector.

Acknowledgements This work is a part of students’ project submitted to The British University

in Dubai.

References

1. Al-Emran, M. (2015). Hierarchical reinforcement learning: A survey. International Journal of

Computing and Digital Systems, 4(2), 137–143.

2. Al-Emran, M., Zaza, S., & Shaalan, K. (2015). Parsing modern standard Arabic using Treebank

resources. https://doi.org/10.1109/ICTRC.2015.7156426

3. Al-Emran, M., Mezhuyev, V., Kamaludin, A., & AlSinani, M. (2018). Development of M-

learning application based on knowledge management processes. In 2018 7th International

Conference on Software and Computer Applications (ICSCA 2018) (pp. 248–253).

A Bibliometric Analysis on the Role of Artiﬁcial Intelligence … 13

4. Weng, J., McClelland, J., Pentland, A., Sporns, O., Stockman, I., Sur, M., & Thelen, E. (2001).

Autonomous mental development by robots and animals. Science (80-), 291(5504), 599–600.

https://doi.org/10.1126/SCIENCE.291.5504.599

5. Shuaib, A., Arian, H., & Shuaib, A. (2020). The increasing role of artiﬁcial intelligence in health

care: Will robots replace doctors in the future? International Journal of General Medicine, 13,

891–896. https://doi.org/10.2147/IJGM.S268093

6. Saheb, T., Saheb, T., & Carpenter, D. O. (2021). Mapping research strands of ethics of artiﬁcial

intelligence in healthcare: A bibliometric and content analysis. Computers in Biology and

Medicine, 135.https://doi.org/10.1016/J.COMPBIOMED.2021.104660

7. De Bruyn, A., Viswanathan, V., Beh, Y. S., Brock, J. K. U., & von Wangenheim, F. (2020). Arti-

ﬁcial intelligence and marketing: Pitfalls and opportunities. Journal of Interactive Marketing,

51, 91–105. https://doi.org/10.1016/J.INTMAR.2020.04.007

8. Zaza, S., & Al-Emran, M. (2015). Mining and exploration of credit cards data in UAE. In

Proceedings—2015 5th International Conference on e-Learning, ECONF 2015 (pp. 275–279).

https://doi.org/10.1109/ECONF.2015.57

9. Al-Emran, M., Al-Maroof, R., Al-Sharaﬁ, M. A., & Arpaci, I. (2020) What impacts learning

with wearables? An integrated theoretical model. Interactive Learning Environment.

10. Al-Emran, M., & Ehrenfeld, J. M. (2021). Breaking out of the box: Wearable technology

applications for detecting the spread of COVID-19. Journal of Medical Systems, 45, 1–2.

https://doi.org/10.1007/S10916-020-01697-1

11. Secinaro, S., Calandra, D., Secinaro, A., Muthurangu, V., & Biancone, P. (2021). The role of

artiﬁcial intelligence in healthcare: A structured literature review. BMC Medical Informatics

and Decision Making, 21, 1–23. https://doi.org/10.1186/S12911-021-01488-9/FIGURES/12

12. Reddy, S. (2018). Use of artiﬁcial intelligence in healthcare delivery. IntechOpen.

13. Quinn, T. P., Senadeera, M., Jacobs, S., Coghlan, S., & Le, V. (2021). Trust and medical AI:

The challenges we face and the expertise needed to overcome them. Journal of the American

Medical Informatics Association, 28(4), 890–894. https://doi.org/10.1093/JAMIA/OCAA268

14. Sarker, S., Jamal, L., Ahmed, S. F., & Irtisam, N. (2021). Robotics and artiﬁcial intelligence

in healthcare during COVID-19 pandemic: A systematic review. Robotics and Autonomous

Systems, 146.https://doi.org/10.1016/J.ROBOT.2021.103902

15. Albahri, A. S., Hamid, R. A., Alwan, J. K., Al-Qays, Z. T., Zaidan, A. A., Zaidan, B. B.,

Albahri, A. O. S., AlAmoodi, A. H., Khlaf, J. M., Almahdi, E. M., Thabet, E., Hadi, S. M.,

Mohammed, K. I., Alsalem, M. A., Al-Obaidi, J. R., & Madhloom, H. T. (2020). Role of

biological data mining and machine learning techniques in detecting and diagnosing the novel

coronavirus (COVID-19): A systematic review. Journal of Medical Systems, 44(7). https://doi.

org/10.1007/s10916-020-01582-x

16. Rong, G., Mendez, A., Bou Assi, E., Zhao, B., & Sawan, M. (2020). Artiﬁcial intelligence in

healthcare: Review and prediction case studies. Engineering, 6(3), 291–301. https://doi.org/

10.1016/J.ENG.2019.08.015

17. Zyoud, S. H., Al-Jabi, S. W., Sweileh, W. M., & Awang, R. (2014). A bibliometric analysis

of research productivity of Malaysian publications in leading toxicology journals during a 10-

year period (2003–2012). Human and Experimental Toxicology, 33(12), 1284–1293. https://

doi.org/10.1177/0960327113514101

18. Mao, G., Zou, H., Chen, G., Du, H., & Zuo, J. (2015). Past, current and future of biomass energy

research: A bibliometric analysis. Renewable and Sustainable Energy Reviews, 52, 1823–1833.

https://doi.org/10.1016/j.rser.2015.07.141

19. Van Eck, N. J., & Waltman, L. (2010). Software survey: VOSviewer, a computer program for

bibliometric mapping. Scientometrics, 84(2), 523–538. https://doi.org/10.1007/s11192-009-

0146-3

20. Zakaria, R., Ahmi, A., Ahmad, A. H., & Othman, Z. (2021). Worldwide melatonin research:

A bibliometric analysis of the published literature between 2015 and 2019. Chronobiology

International, 38(1), 27–37. https://doi.org/10.1080/07420528.2020.1838534

21. Alqudah, A. A., Al-Emran, M., & Shaalan, K. (2021). Technology acceptance in healthcare:

A systematic review. Applied Sciences, 11(22). https://doi.org/10.3390/APP112210537

14 F. Suhail et al.

22. Al-Emran, M., & Arpaci, I. (2021). Intelligent systems and novel coronavirus (COVID-19):

A bibliometric analysis. In Emerging technologies during the era of COVID-19 pandemic

(pp. 59–67). Springer.

23. Rao, A. S. R. S., & Vazquez, J. A. (2020). Identiﬁcation of COVID-19 can be quicker through

artiﬁcial intelligence framework using a mobile phone-based survey in the populations when

cities/towns are under quarantine. Infection Control & Hospital Epidemiology.https://doi.org/

10.1017/ice.2020.61

24. Arpaci, I., Huang, S., Al-Emran, M., Al-Kabi, M. N., & Peng, M. (2021). Predicting the COVID-

19 infection with fourteen clinical features using machine learning classiﬁcation algorithms.

Multimedia Tools and Applications, 80, 11943–11957. https://doi.org/10.1007/s11042-020-

10340-7

25. Vaishya, R., Javaid, M., Khan, I. H., & Haleem, A. (2020). Artiﬁcial intelligence (AI) appli-

cations for COVID-19 pandemic. Diabetes and Metabolic Syndrome: Clinical Research &

Reviews, 14(4), 337–339. https://doi.org/10.1016/J.DSX.2020.04.012

26. Amor, S., Puentes, F., Baker, D., & Van Der Valk, P. (2010). Inﬂammation in neurodegenerative

diseases. Immunology, 129(2), 154–169. https://doi.org/10.1111/J.1365-2567.2009.03225.X

27. Porta, M., & Bandello, F. (2002). Diabetic retinopathy. Diabetologia, 45(12), 1617–1634.

https://doi.org/10.1007/S00125-002-0990-7

28. Tang, Z., Zhao, W., Xie, X., Zhong, Z., Shi, F., Liu, J., & Shen, D. (2020). Severity assessment

of coronavirus disease 2019 (COVID-19) using quantitative features from chest CT images.

29. Al-Emran, M., Al-Kabi, M. N., & Marques, G. (2021). A survey of using machine learning algo-

rithms during the COVID-19 pandemic. In Emerging technologies during the era of COVID-19

pandemic (pp. 1–8). Springer Science and Business Media Deutschland GmbH.

30. Asadi, S., Nilashi, M., Abumalloh, R. A., Samad, S., Ahani, A., Ghabban, F., Yusuf, S. Y.

M., & Supriyanto, E. (2021). Evaluation of factors to respond to the COVID-19 pandemic

using DEMATEL and fuzzy rule-based techniques. International Journal of Fuzzy Systems,

1–17. https://doi.org/10.1007/S40815-021-01119-5/FIGURES/7

31. Zamri, N., Abdullah, L., & Ibrahim, M. A. H. (2021). Electronic healthcare system: Mental

disorder assessment and intervention with self-treatment using rule-based techniques. In

Signals and communication technology (pp. 453–479). Springer.

32. Fosso Wamba, S., & Queiroz, M. M. (2021). Responsible artiﬁcial intelligence as a secret ingre-

dient for digital health: Bibliometric analysis, insights, and research directions. Information

Systems Frontiers, 1–16. https://doi.org/10.1007/S10796-021-10142-8/TABLES/7

33. Penteado, B. E., Fornazin, M., & Castro, L. (2021). The evolution of artiﬁcial intelligence in

medical informatics: A bibliometric analysis. In EPIA Conference on Artiﬁcial Intelligence,

September 2021. LNAI (Vol. 12981, pp. 121–133). https://doi.org/10.1007/978-3-030-86230-

5_10

Supervised Intelligent Clinical Approach

for Breast Cancer Tumor Categorization

Lambodar Jena, Lara Ammoun, and Premkumar Chithaluru

Abstract Cancer is treated to be the most dangerous and life killing disease in the

world, especially lung cancer and breast cancer. The breast cancer is one of the most

common diseases present among women that cause an increase in the death rate

for them. Studies have revealed the importance of early diagnosis of breast cancer

as it increases the likelihood of cure and reduces mortality. Due to the difﬁculty of

manually diagnosing the disease in addition to the lack of efﬁciency of automated

systems that diagnose the disease, therefore there is a need to develop an automated

system. This will facilitate the diagnostic process with high efﬁciency through the

use of machine learning techniques that use the previous data to train the network.

Thus it can predict the new input data intelligently through the use of classiﬁcation

algorithms under the supervision in order to classify benign and malignant tumors.

Here the results of multiple machine learning algorithms will be compared to obtain

high classiﬁcation accuracy.

Keywords Breast cancer ·Supervised learning ·Intelligent system ·Clinical

approach

1 Introduction

As per the global statistics, cancer is presumed to be the most complex disease that

cause the most death in women (the second disease after lung cancer) [1]. It arises

from abnormal growth of cells and may spread rapidly throughout the body and then

it is di cult to control it so the best way to avoid the many deaths resulting from

this disease, is early detection, as early detection leads to speeding up treatment,

dispensing with many difﬁcult surgeries and increasing the survival rate of patients

L. Jena (B)·P. Chithaluru

Department of Computer Science and Engineering, Koneru Lakshmaiah Education Foundation

(KLEF), Vaddeswaram, Vijayawada, Andhra Pradesh, India

e-mail: jlambodar@gmail.com

L. Ammoun

Department of Computer Science and Engineering, Siksha ‘O’ Anusandhan Deemed to be

University, Bhubaneswar, Odisha, India

S. Mishra et al. (eds.), Augmented Intelligence in Healthcare: A Pragmatic and Integrated

Analysis, Studies in Computational Intelligence 1024,

https://doi.org/10.1007/978-981-19- 1076-0_2

16 L. Jena et al.

[2]. While searching for the causes of breast cancer, we ﬁnd that there is no clear

evidence of this disease, so we can say that the true cause of breast cancer is unknown,

but there are many dangerous factors that may be the cause of its existence, such

as genetic factors, exposure to chemical rays, dietary fats, exposure to pesticides

and solvents, as well as pressure, Psychological and stress, we also note that the

presence of a lump in the breast may be a sign of breast cancer, but not all lumps

are evidence of the presence of this disease, and there are some possibilities that

may be evidence of the possibility of developing this disease, such as Lymph node

changes, Nipple secretions, Nipple retraction or inverted nipple, Redness, Swelling,

Skin texture changes, noting that the main cause of breast lumps are cysts and acute

breast cysts. To support the idea of the necessity of early detection of this disease,

many researches have been done to be able to detect breast cancer automatically.

Predicting the breast cancer presence and classifying tumors into benign and malig-

nant tumors greatly help in early detection of this complex disease. Thus, it can

reduce mortality rate of women, increase access to treatment, get proper health care,

avoid unnecessary surgeries, and save the huge ﬁnancial expenses [3]. In this work,

a comparison has been made between the different techniques in machine learning

that aims at expanding the knowledge regarding classiﬁcation of breast cancer, selec-

tion of traits by looking at different previous studies, and, use of different data sets,

different algorithms, and different tools.

2 Motivation

There is a need for early detection of the presence of cancer to increase the likeli-

hood of obtaining an optimal treatment and thus achieving a cure through patient

management according to classiﬁcation cases.

3 Objective

The prime objectives of this work are to implement machine learning classiﬁers for

(1) Early prediction of the tumor for optimal treatment and (2) Accurate detection of

the disease to avoid the suffering of the patient resulting from unnecessary surgical

procedures.

4 Problem Statement

Breast cancer is a complex and dangerous disease. But an accurate diagnosis and

classiﬁcation of patients into malignant and benign groups prior to surgery for a

Supervised Intelligent Clinical Approach for Breast Cancer … 17

breast lesion helps reduce risks and speed up recovery from this disease and for

planning the ideal treatment.

To do this classiﬁcation, we use supervised machine learning techniques that have

been identiﬁed as a good prediction method, and using these techniques can help

reduce mortality. Many research papers were highlighted that used many algorithms

and techniques to diagnose breast cancer, but as a result of the urgent need, it proposes

to study several algorithms on a dataset to study the effectiveness of these algorithms

in classifying breast cancer and obtaining high accuracy in diagnosing this disease.

5 Literature Review

A deep study of numerous research papers related to the classiﬁcation of cancer

patients using machine learning techniques was covered, where the objectives, advan-

tages and limitations of several research papers were compared (shown in Table 1),

as well as a comparison has been done between the tools, the data set, the techniques

used, and the accuracy of the results of many research papers (shown in Table 2).

Table 2shows research output of various authors who worked on different datasets

related to accuracy and techniques used.

6 Experimental Setup

In experiment various classiﬁcation algorithms like SVM, NB, and Decision Tree

are used and the outputs with respect to different performance indicators are closely

observed in order to obtain the best classiﬁcation [31–35]. The search provides

a comparison of several supervised machine learning algorithms to obtain high

accuracy in the tumor classiﬁcation of breast cancer patients.

The Wisconsin dataset is used for experiment with 11 columns [“ID”, “Clump”,

“UnifSize”, “UnifShape”, “MargAdh”, “SingEpiSize”, “BareNuc”, “BlandChrom”,

“NormNucl”, “Mit”, “Class”] and 699 rows representing patient data.

7 Result Analysis

We observe by applying all the features to train the classiﬁer to classify the class

[benign or malignant] of each patient, including the ID to each patient [the ID feature

has no disease association]. So when it is included it will affect the training of the

classiﬁer. Thus the results of all classiﬁers are analyzed for two scenarios when all

features are used with ID of patients and when all features are used without ID of

patients.

18 L. Jena et al.

Tabl e 1 Comparison of research outcomes related to their objective, advantage, and limitation

Objectives Advantages Limitations

This paper aims to predict the

presence of cancer by using

the CSSFFS algorithm to

achieve an optimal feature set

with the lowest error rate to

improve classiﬁcation

accuracy [1]

Determine which feature set

has the most impact on

classiﬁcation and remove

irrelevant and useless features

The researchers suggest trying

the CSSFFS algorithm in other

areas to conﬁrm its usefulness

in improving classiﬁcation

Study the ability to advance

cancer prediction with high

accuracy for big data [3]

This research analyzed a set of

big data GE and DM and a

combined dataset that contains

both GE and DM using the

Spark and Weka platforms,

and get the lowest error rate

and highest accuracy using the

SVM algorithm on the Spark

platform

The paper proposes to improve

prediction accuracy through

the use of feature selection

techniques and the application

of proposed classiﬁcation

algorithms using a balanced

data set

The research provides a

classiﬁcation of several areas,

ﬁrst to predict the presence of

cancer before diagnosis, the

other ﬁeld to predict the

diagnosis and treatment, and

the last ﬁeld to predict the

outcome [4]

The research studies the

possibility of predicting with

effective accuracy using

different techniques as per

requirements

The paper proposes to reduce

error rates while achieving

high accuracy by choosing an

in-depth study of the features

and minimizing the dimensions

Increase the accuracy of

prediction of breast cancer by

a modiﬁed method called

ANOVA-BOOTSTRAP-SVM

[5]

The possibility of using the

proposed algorithm to conduct

research and to with more

accuracy

The sensitivity of the algorithm

varies according to the

parameter Cvalue and the type

of kernel used

Develop a system for early

detection of cancer using some

techniques of data mining [6]

The techniques provide good

accuracy and the paper claims

that SVM achieves the best

classiﬁer among the proposed

technologies

Difﬁculty choosing the best

dimensions and features to

improve performance

The ability to classify breast

cancer with high accuracy

using two algorithms, Support

Vector Machine and

K-Nearest Neighbors [7]

The proposed algorithms are a

useful classiﬁer both in the

diagnostic and in the medical

ﬁeld

The sensitivity of the algorithm

is affected by the kernel used

Building an automated model

that uses routine blood

analysis data to determine the

presence of breast cancer in

patients [8]

Features were determined

based on correlation and study

their impact on different

algorithms

The paper suggests using a

larger database and thus

potentially higher classiﬁcation

accuracy

(continued)

Supervised Intelligent Clinical Approach for Breast Cancer … 19

Tabl e 1 (continued)

Objectives Advantages Limitations

This research provides a new

methodology for training the

neural network to obtain a

model that efﬁciently predicts

breast cancer [9]

It has been observed that, in

most cases, setting limits on

neural network weights results

in increased classiﬁcation

accuracy

Determining the optimal limits

of weights is a difﬁcult process

and requires further study,

research and experimentation

on a different set of data

Building an automated system

that helps diagnose breast

cancer based on the values of

some features using a

Decision Tree [10]

A study is presented to

determine the range of trait

values associated with cancer

The paper proposes to use

more data and study the limits

of other features to produce

reliable results for

higher-precision classiﬁcation

Predict the recurrence of

breast cancer for three years

by building a model that uses

a Decision Tree based

learning algorithm [11]

The model can be considered

a tool with good accuracy to

measure the likelihood of

cancer recurrence

The paper proposes an

in-depth study of the patient

database that is increasing

every year so that the useful

patterns of information present

in it can be revealed, thus

improving the model

Study the effectiveness of

routine analysis of blood in

the detection of cancer using

several algorithms [12]

Parameters that help achieve

effective accuracy were

researched using four machine

learning techniques

The accuracy rate was not very

high but the usefulness of this

type of data in diagnosing

breast cancer using ML

methods have been studied

Study feature selection

techniques and compare their

effect on the accuracy of some

classiﬁcation algorithms [13]

The results showed that the

random-forest algorithm gives

the highest accuracy with

feature selection, in addition,

the f-test gives better results

for the smaller data set, while

the serial forward selection

gives better results for the

larger dataset

It was suggested that an

in-depth study of the

importance of features and

their ability to categorize by

testing them on another dataset

and using other algorithms

Diagnosing breast cancer by

building a Bayesian network

and identifying the most

inﬂuential features for greater

accuracy in knowing a

person’s likelihood of

developing breast cancer [14]

Developing a tool to give an

accurate assessment of the

overall situation when

diagnosing breast cancer and

to support decision-making

for the most difﬁcult cases

The paper proposes applying

the algorithm to a larger

database to obtain more

consistent and

Comparison of several

algorithms to classify cancer

with high accuracy [15]

High resolution and low error

was obtained using the SVM

algorithm

The paper proposes to study

the effectiveness of algorithms

over other data sets to conﬁrm

their robustness and measure

their sensitivity across the data

(continued)

20 L. Jena et al.

Tabl e 1 (continued)

Objectives Advantages Limitations

Study the effect of human

parameters in routine blood

analysis on the classiﬁcation

accuracy of cancer patients

[16]

Increased ease of diagnosis of

cancer

Difﬁculty deciding on which

characteristics achieve better

classiﬁcation accuracy and

lower sensitivity

Improve the classiﬁcation

accuracy of a breast cancer

prediction system using a

hybrid classiﬁer called

WPSO-SSVM [17]

The proposed algorithm

outperformed many

algorithms and received high

classiﬁcation accuracy

The paper proposes to delve

into the robustness of the

algorithm by testing it against

another standard dataset

High accuracy of breast

cancer classiﬁcation was

achieved using a hybrid

classiﬁer called RFSVM [18]

The limitations of the need to

adjust parameters that we face

when using SFM has been

overcome in addition to the

limitations related to the

problem of over-allocation of

random forests

The paper suggests searching

for the best way to handle data

and explore the best weight

features so as to develop the

system through depth study of

thedataset

The k-Nearest Neighbor

algorithm was analyzed for

various rules and distance to

predict the cancer [19]

Effective results using both

the Euclidean and Manhattan

distance types

The use of both types of

Euclidean distance and

Manhattan leads effective

results in the expectation but

consumes a lot of time

7.1 The Results When All Features Are Used with ID Feature

of Patients

When the patient’s ID feature is included it will affect the training of the classiﬁer and

the accuracy will be lower. It is observed that the accuracy is 0.66, i.e., 66% (Table 3)

for the RBF SVM, 0.69, i.e., 69% (Table 4) for the Linear SVM, and 0.80, i.e., 80%

(Table 5) for Naïve Bayes. But according to the fact that Python programs contain a

special function of the Decision Tree algorithm that chooses the best features to train

the classiﬁer depending on the tree depth, here the maximum tree depth is 3. So we

note that the accuracy of the Decision Tree algorithm is high which is 0.95 i.e. 95%

(Table 6).

The confusion matrix generated for all the above four classiﬁers is shown in

Figs. 1,2,3, and 4.

There are two target classes for risk prediction of breast cancer namely “benign”

and “malignant” with the performance indicators like precision, recall, and F1-score

shown in Tables 7and 8.

For both the classes “benign” and “malignant” the precision is represented in

Figs. 5and 6, recall is represented in Figs. 7and 8, and F1-score is represented in

Figs. 9and 10.

Supervised Intelligent Clinical Approach for Breast Cancer … 21

Tabl e 2 Research outcomes on different datasets related to tool, technique used and accuracy

Dataset used Tool used Technique used Accuracy

WDBC dataset from

UCI machine learning

depository [1]

Wek a Simple CART, RBF

network, NB and J48

Simple CART

(93.6731), RBF

network (95.0791), NB

(94.3761) and J48

(94.0246)

Data gathered from

clinical experts in

cancer [2]

Wek a NB, K-NN and J84 NB (98.2%), K-NN

(98.8%) and J84

(98.5%)

University of

California Irvine

(WDBC) [3]

Weka, Spark SVM, DT and RF SVM for Weka

(98.03%), SVM for

Spark (99.68%), DT

for Weka (95.09%), DT

for Spark (98.80%), RF

for Weka (96.07%) and

RF for Spark (98.09%)

UCI machine learning

repository USA [20]

Wek a NB, RF, LR, MP,

K-NN

NB (71.6%), RF

(69.5%), LR (68.8%),

MP (64.6%), K-NN

(72.37%)

Wisconsin hospital

[21]

Wek a ANN and SVM ANN (95.4%) and

SVM (96.9%)

Wisconsin breast

cancer data set [22]

Wek a J84, NB, MLP, LR,

SVM, K-NN

J84 (95.59%), NB

(96.79%), MLP

(94.78%), LR

(96.79%), SVM

(97.59%), K-NN

(95.19%)

Using different

datasets [4]

JUPYT ER platform SVM, C4.5, Naïve

Bayes and k-NN

SVM (97.13%)

outperforms C4.5,

Naïve Bayes and k-NN

in terms of accuracy

(varies between 95.12

and 95.28%)

UCI repository [6]Spyder SVM, K-NN, Logistic

Regression

SVM (92.78), K-NN

(92.23), Logistic

Regression (92.10)

The breast cancer

termed as Wisconsin

breast cancer diagnosis

data set is taken from

UCI machine learning

repository [7]

Spyder SVM, K-NN SVM (98.57), K-NN

(97.14)

(continued)

22 L. Jena et al.

Tabl e 2 (continued)

Dataset used Tool used Technique used Accuracy

Breast cancer

Wisconsin (original)

and breast cancer

Wisconsin (diagnostic)

datasets [10]

WEKA Decision Tree

algorithm

90.52%

The records of

Mizoram Cancer

Institute, Aizawl [11]

WEKA J48 84.2105%

UCI library [12]Matlab Artiﬁcial Neural

Network (ANN),

Standard Extreme

Learning Machine

(ELM), Support Vector

Machine (SVM) and

K-Nearest Neighbor

(k-NN)

ANN (79.4304%),

ELM (80%), k-NN

(77.5%), SVM (73.5%)

Wisconsin breast

cancer (original) data

set [15]

Wek a SVM, C4.5, NB,

K-NN

SVM (97.13%), C4.5

(95.13%), NB

(95.99%), K-NN

(95.27%)

Liver disorder, WBCD,

and mammographic

mass data from UCI

machine learning

repository [17]

Wek a WPSO-SSVM 83.76% for liver

disorder, 98.42% for

WBCD, 95.21% for

mammographic mass

data

WBCD [18]Python RFSVM 99.714

(WBCD) taken from

UCI machine learning

repository [23]

WEKA DT—SVM DT—SVM (91%)

Mammogram images

[24]

WEKA J48, AD-tree, and

CART

J48 (98.1%), AD-tree

(97.7%), and CART

(98.5%)

Thedatafromthe

Iranian Center for

breast cancer dataset

[25]

Wek a Decision Tree (C4.5),

Support Vector

Machine (SVM), and

Artiﬁcial Neural

Network (ANN)

DT (0.936%), ANN

(0.947%) and SVM

(0.957%)

Wisconsin diagnosis

breast cancer data [26]

Matlab AdaBoost SVM

classiﬁcation

algorithm, combined

with k-means

98.85%

WDBC [27]Wek a KNN, SVM, RF and

KNN (96.1%), SVM

(97.9%), RF (96%) and

NB (92.6%)

WDBC [28]Wek a NB, LR, simple CART

and J48

NB (95.26%), LR

(65.42%), simple

CART (98.13%) and

J48 (97.27%)

(continued)

Supervised Intelligent Clinical Approach for Breast Cancer … 23

Tabl e 2 (continued)

Dataset used Tool used Technique used Accuracy

WDBC [29]Wek a NB, regression, SVM,

J48 and bagging

algorithm

NB (72.7%), bagging

algorithm (65.3%),

regression algorithm

(60.03%), SVM

(82.53%) and J48

(79.8%)

WDBC [30]Wek a Linear_kernel-SVM,

RBF_kernel-SVM,

Polynomial

kernel-SVM,

Sigmoid_kernel-SVM

Result of SVM for

various kernel are

(99%), (98%), (87%),

and (94%),

respectively

Wisconsin breast

cancer (original)

dataset [21]

Wek a Artiﬁcial Neural

Network, Support

Vector Machine

ANN (95.4%), SVM

(96.9%)

Tabl e 3 Results for RBF SVM classiﬁer

Rate of positive prediction Sensitivity F1score Support

2 (“benign”) 0.66 1.00 0.79 90

4 (“malignant”) 0.00 0.00 0.00 47

Accuracy rate 0.66 137

Macro-avg 0.33 0.50 0.40 137

Weighted-avg 0.43 0.66 0.52 137

Tabl e 4 Results for linear SVM classiﬁer

Rate of positive prediction Sensitivity F1score Support

2 (“benign”) 0.69 0.98 0.81 90

4 (“malignant”) 0.78 0.15 0.25 47

Accuracy rate 0.69 137

Macro-avg 0.73 0.56 0.53 137

Weighted-avg 0.72 0.69 0.62 137

Tabl e 5 Results for Naïve Bayes classiﬁer

Rate of positive prediction Sensitivity F1score Support

2 (“benign”) 0.77 1.00 0.87 90

4 (“malignant”) 1.00 0.43 0.60 47

Accuracy rate 0.80 137

Macro-avg 0.88 0.71 0.73 137

Weighted-avg 0.85 0.80 0.78 137

24 L. Jena et al.

Tabl e 6 Results for Decision Tree classiﬁer

Rate of positive prediction Sensitivity F1score Support

2 (“benign”) 0.99 0.93 0.96 90

4 (“malignant”) 0.88 0.98 0.93 47

Accuracy rate 0.95 137

Macro-avg 0.94 0.96 0.94 137

Weighted-avg 0.95 0.95 0.95 137

Fig. 1 RBF SVM

It is seen from Figs. 5and 6that in case of benign class Decision Tree classiﬁer

performs better with precision value of 0.99 as compared to other three classiﬁers.

In case of malignant class, the Naïve Bayes classiﬁer performs better with precision

value of 1.00 as compared to other three classiﬁers.

Figure 11 shows the classiﬁcation accuracy level obtained, 0.66, i.e., 66% by the

RBF SVM classiﬁer, 0.69, i.e., 69% by the Linear SVM classiﬁer, 0.80, i.e., 80% by

Naïve Bayes classiﬁer and 0.95 i.e. 95% by Decision Tree classiﬁer.

Supervised Intelligent Clinical Approach for Breast Cancer … 25

Fig. 2 Linear SVM

Fig. 3 Naïve Bayes

26 L. Jena et al.

Fig. 4 Decision Tree

Tabl e 7 Benign class Rate of positive

prediction

Sensitivity F1score

RBF SVM 0.66 1.00 0.79

Linear SVM 0.69 0.98 0.81

Naïve Bayes 0.77 1.00 0.87

Decision Tree 0.99 0.93 0.96

Tabl e 8 Malignant class Rate of positive

prediction

Sensitivity F1score

RBF SVM 0.00 0.00 0.00

Linear SVM 0.78 0.15 0.25

Naïve Bayes 1.00 0.43 0.60

Decision Tree 0.88 0.98 0.93

7.2 The Results When All Features Are Used Without ID

of Patients

When all features except ID of patients are used the accuracy becomes high for all

the algorithms used. The accuracy levels obtained are 0.96, i.e., 96% (Table 9)for

Supervised Intelligent Clinical Approach for Breast Cancer … 27

Precision

0.8

0.6

0.4

0.2

Precision

RBF SVM Linear SVM Naïve Bayes

Decision

Tree

Fig. 5 Precision of benign class

Precision

0.8

0.6

0.4

0.2

Precision

RBF SVM Linear SVM Naïve Bayes

Decision

Tree

Fig. 6 Precision of malignant class

SVM RBF, 0.96 i.e. 96% (Table 10) linear SVM, 0.95, i.e., 95% (Table 11)forNaïve

Bayes, and 0.95, i.e., 95% (Table 12) for Decision Tree.

Figure 12 shows the classiﬁcation accuracy level obtained by the classiﬁers. It is

observed that the classiﬁcation accuracy level suddenly increased in comparison to

the previous experiment shown in Fig. 11. The accuracy is increased to 96% from

0.66, i.e., 66% for the RBF SVM classiﬁer, 96% from 0.69, i.e., 69% for the Linear

SVM classiﬁer, 95% from 0.80, i.e., 80% for Naïve Bayes classiﬁer and it remains

same 0.95 i.e. 95% for Decision Tree classiﬁer.

The confusion matrix generated for all the above four classiﬁers without using

patient’s ID feature is shown in Figs. 13,14,15, and 16.

28 L. Jena et al.

10.98

0.98

0.96

0.94

0.93

0.92

0.9

0.88

RBF SVM Linear SVM Naïve Bayes Decision Tree

Recall

Fig. 7 Recall of benign class

0.98

0.8

0.6

0.43

0.4

0.15

0.2

RBF SVM Linear SVM Naï ve Bayes Decision Tr ee

Recall

Fig. 8 Recall of malignant class

Tables 13 and 14 depicts the positive predicted value (precision), Sensitivity

(recall), and F1-score values of all four classiﬁers used above for the two target

classes “benign” and “malignant”.

For both the classes “benign” and “malignant” the precision is represented in

Figs. 17 and 18. It is seen that the precision of benign class dominates the precision

of malignant class in all these four classiﬁers RBF SVM, Linear SVM, Naive Bayes,

and Decision Tree shown in Fig. 19. The recall is represented in Figs. 20 and 21, and

the F1-score is represented in Figs. 22 and 23.

Supervised Intelligent Clinical Approach for Breast Cancer … 29

F1-score

0.8

0.6

0.4

0.2

RBF SVM

Linear SVM

Naïve Bayes

Decision

Tree

F1-score 0.79 0.81 0.87 0.96

Fig. 9 F1-score of benign class

F1-score

0.8

0.6

0.4

0.2

RBF SVM

Linear SVM

Naïve Bayes

Decision

Tree

F1-score 0 0.25 0.6 0.93

Fig. 10 F1-score of malignant class

Figures 24 and 25 show the Recall analysis and F1-Score analysis of the machine

learning classiﬁers RBF SVM, Linear SVM, Naive Bayes, and Decision Tree for

both the classes Benign and Malignant. It is clearly seen that the Recall value of all

classiﬁers remains low for Benign class in comparison to Malignant class. However,

the F1-Score value of all classiﬁers remains high for benign class in comparison to

malignant class.

30 L. Jena et al.

100 80

80 66

RBF SVM Linear SVM Naïve Bayes Decision Tree

Acuracy in %

Fig. 11 Accuracy of the classiﬁcation algorithms with ID attribute

Tabl e 9 Results for RBF SVM classiﬁer without using patient ID feature

Rate of positive prediction Sensitivity F1score Support

2 (“benign”) 1.00 0.94 0.97 90

4 (“malignant”) 0.90 1.00 0.95 47

Accuracy rate 0.96 137

Macro-avg 0.95 0.97 0.96 137

Weighted-avg 0.97 0.96 0.96 137

Tabl e 10 Results for linear SVM classiﬁer without using patient ID feature

Rate of positive prediction Sensitivity F1score Support

2 (“benign”) 1.00 0.94 0.97 90

4 (“malignant”) 0.90 1.00 0.95 47

Accuracy rate 0.96 137

Macro-avg 0.95 0.97 0.96 137

Weighted-avg 0.97 0.96 0.96 137

Tabl e 11 Results for Naïve Bayes classiﬁer without using patient ID feature

Rate of positive prediction Sensitivity F1score Support

2 (“benign”) 1.00 0.92 0.96 90

4 (“malignant”) 0.87 1.00 0.93 47

Accuracy rate 0.95 137

Macro-avg 0.94 0.96 0.95 137

Weighted-avg 0.96 0.95 0.95 137

Supervised Intelligent Clinical Approach for Breast Cancer … 31

Tabl e 12 Results for Decision Tree classiﬁer without using patient ID feature

Rate of positive prediction Sensitivity F1score Support

2 (“benign”) 0.99 0.93 0.96 90

4 (“malignant”) 0.88 0.98 0.93 47

Accuracy rate 0.95 137

Macro-avg 0.94 0.96 0.94 137

Weighted-avg 0.95 0.95 0.95 137

96 96

95.8

95.6

95.4

95.2

94.8

94.6

94.4

RBF SVM Linear SVM Naïve Bayes Decision Tree

Accuracy in %

Fig. 12 Accuracy of the classiﬁcation algorithms without ID attribute

8 Conclusion

Several researchers in their research papers have highlighted the use of many algo-

rithms and methods to detect and predict breast cancer. But as a result of the urgent

need for early diagnosis of breast cancer several supervised learning algorithms have

been considered for the model to work. In this work, four machine learning clas-

siﬁers have been used for classiﬁcation of breast cancer tumors into benign class

and malignant class. From the experiment outcome, it is seen that Decision Tree

algorithm results better in comparison to other three classiﬁers with classiﬁcation

accuracy of 95% when all the features are used and SVM (both RBF SVM and

Linear SVM) performs better in accuracy in comparison to other three classiﬁers

with classiﬁcation accuracy of 96% when the features are used without patient ID.

The positive predicted value (precision), sensitivity (recall), and F1-score for both

the cases are compared with one another. The observation and outcomes of machine

learning methods implemented in the work have signiﬁcant role in risk prediction of

breast cancer tumors in early stage.

32 L. Jena et al.

Fig. 13 Confusion-matrix of RBF SVM without patient ID feature

Fig. 14 Confusion-matrix of linear SVM without patient ID feature

Supervised Intelligent Clinical Approach for Breast Cancer … 33

Fig. 15 Confusion-matrix of Naïve Bayes without patient ID feature

Fig. 16 Confusion-matrix of Decision Tree without patient ID feature

34 L. Jena et al.

Tabl e 13 Benign class

without ID feature Rate of positive

prediction

Sensitivity F1score

RBF SVM 1.00 0.94 0.97

Linear SVM 1.00 0.94 0.97

Naïve Bayes 1.00 0.92 0.96

Decision Tree 0.99 0.93 0.96

Tabl e 14 Malignant class

without ID feature Rate of positive

prediction

Sensitivity F1score

RBF SVM 0.90 1.00 0.95

Linear SVM 0.90 1.00 0.95

Naïve Bayes 0.87 1.00 0.93

Decision Tree 0.88 0.98 0.93

1.002 11 1

0.998

0.996

0.994

0.992 0.99

0.99

0.988

0.986

0.984

RBF SVM Linear SVM Naïve Bayes Decision Tree

precision

Fig. 17 Precision of benign class without patient ID feature

Supervised Intelligent Clinical Approach for Breast Cancer … 35

0.905 0.9 0.9

0.9

0.895

0.89

0.885 0.88

0.88

0.875 0.87

0.87

0.865

0.86

0.855

RBF SVM Linear SVM Naïve Bayes Decision Tree

precision

Fig. 18 Precision of malignant class without patient ID feature

1.05

0.95

precision Benign

0.9

precision Malignant

0.85

0.8

RBF SVM Linear SVM Naïve Bayes

Decision

Tree

Fig. 19 Comparisons of precision value in benign and malignant classes

36 L. Jena et al.

0.945

0.94 0.94

0.94

0.935

0.93

0.925

0.92

0.915

0.91

RBF SVM Linear SVM Naïve Bayes Decision Tree

Recall

Fig. 20 Recall of benign class without patient ID feature

0.995

0.99

0.985

0.98

0.975

0.97

RBF SVM Linear SVM Naïve Bayes Decision Tree

Recall

Fig. 21 Recall of malignant class without patient ID feature

Supervised Intelligent Clinical Approach for Breast Cancer … 37

0.972

0.97 0.97

0.97

0.968

0.966

0.964

0.962

0.96 0.96

0.96

0.958

0.956

0.954

RBF SVM Linear SVM Naïve Bayes Decision Tree

F1-score

Fig. 22 F1-score of benign class without patient ID feature

0.955 0.95 0.95

0.95

0.945

0.94

0.935 0.93 0.93

0.93

0.925

0.92

RBF SVM

Linear SVM

Naïve Bayes Decision Tree

F1-score

Fig. 23 F1-score of malignant class without patient ID feature

38 L. Jena et al.

Recall

Recall Benign class Recall Malignant class

0.98

0.94 0.94

0.93

0.92

RBF SVM Linear SVM Naïve Bayes

Decision Tree

Fig. 24 Recall analysis of both benign class and malignant class

F1-Score

F1-Score Benign class F1-Score Malignant class

0.97

0.96 0.96

0.95 0.95

0.93 0.93

RBF SVM Linear SVM Naïve Bayes Decision Tree

Fig. 25 F1-score analysis of both benign class and malignant class

References

1. Aruna, S., & Dr, S. (2011). A novel SVM based CSSFFS feature selection algorithm for

detecting breast cancer. International Journal of Computer Applications, 31, 14–20.

2. Maliha, S. K., Ema, R. R., Ghosh, S. K., Ahmed, H., Mollick, M. R. J., & Islam, T.

(2019). Cancer disease prediction using naive Bayes, k-nearest neighbor and J48 algorithm. In

10th International Conference on Computing, Communication and Networking Technologies

(ICCCNT) (pp. 1–7). IEEE.

3. Alghunaim, S., & Al-Baity, H. H. (2019). On the scalability of machine-learning algorithms

for breast cancer prediction in big data context. IEEE Access, 7, 91535–91546.

4. Rawal, R. (2020). Breast cancer prediction using machine learning. Journal of Emerging

Technologies and Innovative Research, 7(5), 13–24.

Supervised Intelligent Clinical Approach for Breast Cancer … 39

5. Petrova, V. B. (2020). Detection of malignant and benign breast cancer using the ANOVA-

BOOTSTRAP-SVM. Journal of Data and Information Science, 5(2), 62–75.

6. Shravya, C., Pravalika, K., & Shaik, S. (2019). Prediction of breast cancer using supervised

machine learning techniques. International Journal of Innovative Technology and Exploring

Engineering (IJITEE), 8(6), 1106–1110.

7. Islam, M. M., Iqbal, H., Haque, M. R., & Hasan, M. K. (2017). Prediction of breast cancer using

support vector machine and k-nearest neighbors. In IEEE Region 10 Humanitarian Technology

Conference (R10-HTC) (pp 226–229). IEEE.

8. Salod, Z., & Singh, Y. (2019). Comparison of the performance of machine learning algorithms

in breast cancer screening and detection: A protocol. Journal of Public Health Research, 8(3).

9. Livieris, I. E. (2019). Improving the classiﬁcation efﬁciency of an ANN utilizing a new training

methodology. Informatics, 6, 1. Multidisciplinary Digital Publishing Institute.

10. Akben, S. B. (2019). Determination of the blood, hormone and obesity value ranges that indicate

the breast cancer, using data mining based expert system. IRBM, 40(6), 355–360.

11. Dawngliani, M. S., Lalmuanawma, S., & Chandrasekaran, N. (2019). Development of a model

to predict breast cancer recurrence using decision tree based learning algorithms. Think India

Journal, 22(10), 4008–4013. ISSN: 0971-1260.

12. Fatih, A. M., Yunus, C., Kadir, S., & Akif, D. (2018). Breast cancer diagnosis by different

machine learning methods using blood analysis data. International Journal of Intelligent

Systems and Applications in Engineering, 6(4), 289–293.

13. Dhanya, R., Paul, I. R., Akula, S. S., Sivakumar, M., & Nair, J. J. (2019). A comparative study

for breast cancer prediction using machine earning and feature selection. In International

Conference on Intelligent Computing and Control Systems (ICCS) (pp. 1049–1055). IEEE.

14. Carvalho, D., Pinheiro, P. R., & Pinheiro, M. C. D. (2016). A hybrid model to support the early

diagnosis of breast cancer. Procedia Computer Science, 91, 927–934.

15. Asri, H., Mousannif, H., Al Moatassime, H., & Noel, T. (2016). Using machine learning

algorithms for breast cancer risk prediction and diagnosis. Procedia Computer Science, 83,

1064–1069.

16. Patrício, M., Pereira, J., Crisóstomo, J., Matafome, P., Gomes, M., Seiça, R., & Caramelo, F.

(2018). Using Resistin, glucose, age and BMI to predict the presence of breast cancer. BMC

Cancer, 18, 29. https://doi.org/10.1186/s12885-017-3877-1

17. Latchoumi, T. P., & Parthiban, L. (2017). Abnormality detection using weighed particle swarm

optimization and smooth support vector machine. Biomedical Research, 28(11).

18. Soni, B., Bora, A., Ghosh, A., & Reddy, A. (2019). RFSVM: A novel classiﬁcation technique

for breast cancer diagnosis. International Journal of Innovative Technology and Exploring

Engineering (IJITEE), 8(12). ISSN: 2278-3075.

19. Medjahed, S. A., Saadi, T. A., & Benyettou, A. (2013). Breast cancer diagnosis by using

k-nearest neighbor with different distances and classiﬁcation rules. International Journal of

Computer Applications, 62(1).

20. Bharati, S., Rahman, M. A., & Podder, P. (2018). Breast cancer prediction applying different

classiﬁcation algorithm with comparative analysis using Weka. In 4th International Conference

on Electrical Engineering and Information & Communication Technology (iCEEiCT) (pp. 581–

584). IEEE.

21. Bayrak, E. A., Kırcı, P.,& Ensari, T. (2019). Comparison of machine learning methods for breast

cancer diagnosis. In Scientiﬁc Meeting on Electrical-Electronics & Biomedical Engineering

and Computer Science (EBBT) (pp. 1–3). IEEE.

22. Kumar, G. R., Ramachandra, G. A., & Nagamani, K. (2013). An efﬁcient prediction of breast

cancer data using data mining techniques. International Journal of Innovations in Engineering

and Technology (IJIET), 2(4), 139.

23. Sivakami, K. (2015). Mining big data: Breast cancer prediction using DT-SVM hybrid model.

International Journal of Scientiﬁc Engineering and Applied Science (IJSEAS), 1(5), 418–429.

24. Padmapriya, B., & Velmurugan, T. (2016). Classiﬁcation algorithm based analysis of breast

cancer data. International Journal of Data Mining Techniques and Applications, 6(1), 43–49.

40 L. Jena et al.

25. Ahmad, L. G., Eshlaghy, A. T., Poorebrahimi, A., Ebrahimi, M., & Razavi, A. R. (2013). Using

three machine learning techniques for predicting breast cancer recurrence. Journal of Health

and Medical Informatics, 4(124), 3.

26. Senkamalavalli, R., & Bhuvaneswari, T. (2017). Improved classiﬁcation of breast cancer data

using hybrid techniques. International Journal of Advanced Engineering Researchand Science,

5(5), 237467.

27. Khourdiﬁ, Y., & Bahaj, M. (2018). Applying best machine learning algorithms for breast cancer

prediction and classiﬁcation. In International Conference on Electronics,Control, Optimization

and Computer Science (ICECOCS) (pp. 1–5). IEEE.

28. Singh, S. N., & Thakral, S. (2018). Using data mining tools for breast cancer prediction and anal-

ysis. In 4th International Conference on Computing Communication and Automation (ICCCA)

(pp. 1–4). IEEE.

29. Zheng, B., Yoon, S. W., & Lam, S. S. (2014). Breast cancer diagnosis based on feature extrac-

tion using a hybrid of k-means and support vector machine algorithms. Expert Systems with

Applications, 41(4), 1476–1482.

30. Memon, M. H., Li, J. P., Ul Haq, A., Memon, M. H., & Zhou, W. (2019). Breast cancer

detection in the IOT health environment using modiﬁed recursive feature selection. Wireless

Communications and Mobile Computing.

31. Tripathy, H. K., Mishra, S., Thakkar, H. K., & Rai, D. (2021). CARE: A collision-aware

mobile robot navigation in grid environment using improved breadth ﬁrst search. Computers &

Electrical Engineering, 94, 107327.

32. Mishra, S., Thakkar, H., Mallick, P. K., Tiwari, P., & Alamri, A. (2021). A sustainable IoHT

based computationally intelligent healthcare monitoring system for lung cancer risk detection.

Sustainable Cities and Society, 103079.

33. Mishra, S., Panda, A., & Tripathy, K. H. (2018). Implementation of re-sampling technique

to handle skewed data in tumor prediction. Journal of Advanced Research in Dynamical and

Control Systems, 10, 526–530.

34. Tripathy, H. K., Mallick, P. K., & Mishra, S. (2021). Application and evaluation of classiﬁcation

model to detect autistic spectrum disorders in children. International Journal of Computer

Applications in Technology, 65(4), 368–377.

35. Mishra, S., Tripathy, H. K., Mallick, P. K., Bhoi, A. K., & Barsocchi, P. (2020). EAGA-MLP—

An enhanced and adaptive hybrid classiﬁcation model for diabetes diagnosis. Sensors, 20(14),

4036.

Health Monitoring and Integrated

Wearables

S. Sivaranjani, P. Vinoth Kumar, and S. Palanivel Rajan

Abstract In today’s highly competitive world, balancing work and ﬁtness has

become a major worry for the majority of individuals. Waiting long times in hospi-

tals and mobility surveillance are well-known problems. The problem necessitates a

health monitoring system that can smoothly monitor normal schedule health metrics

and heart rate monitoring and communicate the results to the appropriate person using

a GSM module. With the advancement of technology, numerous monitoring systems

have emerged, providing convenience to individuals. The current state of health

research and development is depicted in this chapter. Different wearable sensors

systems have been explained such as biosensors, Implantable sensors, Thermoelec-

tric Sensors, Power generation sensors and reviewed in order to determine which type

of sensors need to use and what may be done to improve throughput over present

scenario systems.

Keywords Wearable sensors ·Implantable sensors ·Thermoelectric sensors ·

Power generation sensors

1 Introduction

In this COVID context, health monitoring and integrated wearable’s systems are

playing an increasingly signiﬁcant role in preventing disease and lowering medical

costs. People in today’s fast-paced society are rushing through their stressful jobs,

paying little attention to food preparation, and trying hard to maintain a sophisti-

cated lifestyle. People are not ready to take care about their blood pressure, Heart

rate, and Lungs functioning etc. Wearable technology supports the fast growing

people to monitor the basic parameter such as blood pressure, Heart rate, and Lungs

functioning [1]. These wearable devices can implant in the user body or skin or

S. Sivaranjani (B)·S. Palanivel Rajan

ECE, M. Kumarasamy College of Engineering, Karur, India

e-mail: swame12@gmail.com

P. Vinoth Kumar

ECE, Nandha College of Technology, Erode, India

S. Mishra et al. (eds.), Augmented Intelligence in Healthcare: A Pragmatic and Integrated

Analysis, Studies in Computational Intelligence 1024,

https://doi.org/10.1007/978-981-19-1076-0_3

42 S. Sivaranjani et al.

embedded in the cloth to monitor their health status. Wearable sensors can be used

for both diagnostic and monitoring purposes. Sensing of physiological and metabolic

variables, as well as detection of movement, are among their existing capabilities.

It’s hard to overstate the magnitude of the problems that these techniques could actu-

ally ﬁx [2]. Physiological monitoring could aid in the detection and treatment of a

wide range of neurological, cardiovascular, and pulmonary illnesses such as seizures,

hypertension, dysrhythmias, and asthma in a large number of people. Home-based

motion monitoring could help people live more independently and participate in their

communities by minimizing the risks [3].

In Fig. 1, Wearable devices capture physiological and movement data, enabling

for continuously monitoring patients. Depending on the clinical purpose, sensors

are employed in a variety of ways. Sensors that capture movement data could be

utilized in various applications such as evaluating the effectiveness of residence

rehabilitation programmes in stroke survivors or the use of mobility aid devices in

the elderly. For example, Patients’ data is transmitted to a mobile phone or an access

point via wireless transmission, which is then relayed to a remote center over the

Internet. Data processing applied throughout the system detects emergency situations

(such as falls), and an alarm message is delivered to an emergency service center

to provide rapid assistance to patients. Family members and caregivers are notiﬁed

in the event of an emergency, but they may also be notiﬁed in other circumstances,

such as when the patient requires assistance with taking his or her medications [4].

Clinical staff can remotely monitor a patient’s condition and be notiﬁed if a medical

decision needs to be taken.

Fig. 1 Wearable sensor-based wellness surveillance system [1]

Health Monitoring and Integrated Wearables 43

2 Wearable Sensors

Wearable refers to items such as sweaters, caps, pants, spectacles, bras, socks,

watches, patches, or belt-mounted devices that do not obstruct daily activities or

impede mobility. Wearability is particularly important in domains such as health-

care monitoring, well-being, and ﬁtness/sport. There is an overwhelming amount of

fashionable wearable devices on the market today, but not all of them are respon-

sible for accurately detecting or reporting our health condition. On the other hand,

there are numerous sensors that monitor physiological characteristics but are not

accessible. Wearable biomedical sensors are a subset of equipment that can detect

physiological parameters while also being worn. Wearable technology is frequently

built on traditional electronics, either stiff or bendable, and powered by traditional

batteries. Mobile device peripherals (devices, interfaces, or sensors attached to the

mobile) are included. In other circumstances, wearable devices are much more “dis-

ruptive,” such as clothes and materials with distributed functionalities that are tightly

integrated with electronics. Because gadgets must be washable, elastic, and fold-

able, as well as occasionally printable or ﬂexible. Figure 2shows that few Wearable

Biomedical Sensors.

Biological indicators of importance in rehabilitative include pulse rate, breathing

rate, hypertension, blood oxygen levels, and limb movements. The variables

produced out of these observations have a diagnostic accuracy signiﬁcance and it

Fig. 2 Wearable biomedical sensors [5]

44 S. Sivaranjani et al.

can be employed to provide general health markers. Traditionally, only a health-

care organization provide some close monitoring of physiological indicators [5–8].

Precise, constant, real-time tracking of physiological signs is now feasible thanks to

advancements in ﬁtness trackers.

3 Biosensors

Biosensors have been extensively studied and developed as a tool in the ﬁelds of

medicine, the environment, food, and pharmaceuticals. In the presence of a variety

of interfering species, the biosensors are designed to create a digital electrical signal

proportional to the concentration of a given biochemical or a group of biochemicals.

These are referred to as “biosensors” because they make use of biofunctionalities

such as recognition and catalysis [6,7]. Biosensors are often built with a combination

of biological components and transducers. Biosensors are analytical devices that

are self-contained and consist of a biological sensing element, or bio-recognition

element, that is responsible for speciﬁcity, and a physical transducer that turns the

recognition phenomena into a measured signal.

3.1 Evolution of Biosensors

Lee, Mutharasan et al., Wearable sensors were divided into three periods depending

on the extent of implementation of the following elements is, the process of binding

its biorecognition or range of structures molecules to a basic output waveform [8]

(Fig. 3). The ﬁrst iteration is going through basic sensors behind the selective barrier,

such as a dialysis bag. Covalent bonding at a well ﬁtted transducer interface or inte-

gration into a polymer matrix at the transduction surface are used in multiple gener-

ations to freeze the transducers. Different pieces of the descendants are considerably

different (for example, supervisory hardware), whereas a biosensor protein is an

essential part of the base individuals are prone [9].

Fig. 3 Basic diagram of transferring biosensor signal [9]

Health Monitoring and Integrated Wearables 45

3.2 Biodegradable Sensors

The Biosensing technology is rapidly required to determine several critical physi-

ological processes, such as circulation of blood, hypertension, intravascular stress,

bone cracks, and so on, because of the ever-increasing diseases and a plethora of prob-

able surgery complications. A secondary surgery to remove the device, on the other

hand, can be difﬁcult, painful, and costly for patients. Furthermore, people’s percep-

tions of traditional electronics are harmful and non-degradable, making the devel-

opment of biodegradable sensors an unrealistic aim [10–12]. Furthermore, advances

in material engineering and innovation have opened up a new path for the develop-

ment of biodegradable sensors. Rogers’ group was the ﬁrst to report an implanted

sensor for intracranial pressure monitoring built completely of biodegradable mate-

rials in 2016. This sensor could be able to function normally in certain situations. In

this world circumstance, biodegradable systems have piqued interest since, due to

its composition and use of compostable metals, they can disappear in their respec-

tive surroundings with controlled deterioration. Some examples of biomaterials are

biodegradable conductive materials, paper substrates, and alloys composite poly-

mers. In this regard, compostable sensors have been employed to get parameters

such as temperature, strain, blood pressure, oxygen content, and levels of pollution

through impermanent embed in the body or even further surface. Those implants

must be made from harmless and biocompatible materials. Biodegradability, which

can be classiﬁed as partially or entirely compostable, is another desirable and critical

feature for medical equipment [13]. The primary version consists of a biodegradable

antenna for information and/or power transmission outside the body, as well as non-

biodegradable sensor and electronics blocks. These sensors can lower the danger of

elimination and chronic inﬂammation in implantable devices, as well as the need to

change sensors for durable devices and health costs.

4 Implantable Sensors

Wearable and implantable technology is helping to alter healthcare and health

outcomes in the mobile health era, as well as line with the expected tips on better

health monitoring and management [14]. Biomedical applications of wireless smart

medical gadgets, ranging from disease detection to health prevention, and also the

components used in their production and requirements for wireless medical implants

and apps. Finally, we will go through several of the technological obstacles with

embedded and wearable technology, as well as potential solutions for overcoming

the problem. In Fig. 4, Wearable and implantable devices or technologies had already

quickly dominated the era of digital health in a variety of biomedical applica-

tions, including monitoring, tracking, and recording people’s vital signs in order to

improve their own and their family’s health [15]. Accessories such as smart watches,

armbands, and eyewear are incorporating some of these technologies into our daily

46 S. Sivaranjani et al.

Fig. 4 Wearable and implantable device for cardiovascular health management systems [19]

life. One of the ﬁrst embedded health devices was a completely transplanted cardiac

pacemaker for patients in the 1960s. Improved pacemakers, implanted cardioverter

deﬁbrillators (ICDs), and implant deep brain stimulators were already created and

utilized to treat millions of patients. These implantable devices are rechargeable and

made out of biocompatible electrodes and cables as well as programming circuitry

[16–18].

The assessment of activities in the human body is one of the most important

applications of molecular sensors. Implantable sensors, among those technologies

for biomedical wearable electronics, provide distinct challenges in terms of device

design and implementation that meet the requirements for usage in the human body.

Implantable sensors have a number of beneﬁts over many other analysis techniques.

Implantable systems support self-monitoring and detecting changes in patient status

consciously or unintentionally.

4.1 Implantable Devices for Cardiovascular Healthcare

Cardiovascular disease is the most common reason for death, and its prevalence has

risen rapidly in recent years. Constant cardiac supervise is critical for initial detection

and prevention, and ﬂexible and Smart wearable devices have evolved into helpful

tools. These sections will provide you a quick summary of how wearable/implantable

sensors can help you monitor your heart rate [19]. The heart is a critical organ that

carries oxygen via vascular system to deliver oxygen (O2) as well as nutrients to all

parts of the body while also removing carbon dioxide (CO2) and metabolic waste.

According to the World Health Organization, cardiovascular diseases affect about

one-third of the worldwide people. As cardiovascular disorders become consider-

ably easier to control when discovered early, continuous cardiovascular monitoring

is becoming increasingly crucial. Various cardiovascular diseases, for example, can

be detected by evaluating EKG patterns (ECGs). Meanwhile, hypertension and blood

Health Monitoring and Integrated Wearables 47

oxygen saturation (BOS) are routinely monitored. This is because high blood pres-

sure is one of the primary causes of heart sickness, and a poor oxygen saturation

baseline can aid in the diagnosis of acute coronary syndrome. Treatment should be

started very away when abnormal heartbeats are discovered. Implantable sensors in

the CRT device provide a unique opportunity for continuous monitoring of a detailed

medical Heart Failure state [19] by measuring heart rate, intracoronary hypertension,

cardiac problems, including physical ﬁtness, as well as diagnosing signiﬁcant hard-

ware malfunctioning. Early detection of a deteriorating clinical situation allows for

proactive medical action to improve HF management.

4.2 Implantable Pressure Sensors

Hypertension is a crucial indicator that may be mechanically assessed from the

patient’s body. In the year of 1950s, implantable pressure sensors have become a hot

topic. In 1969, a detector coupled with a nylon catheter was used in the ﬁrst time to

measure heart rate in the afferent arteriole, when Scheinman, Abbot, and Rapaport

employed it. Swan and Ganz produced a catheterization with a circulation ballooning

tip in the following year. Because of its ﬂexibility its use and adaptability that will

be used without ﬂuoroscopy, their catheter quickly became the industry standard for

coronary care. An inﬂatable balloon stuck in a minor pulmonary conduit gives an

approximate measure of mitral valve pressure with pulmonary artery catheters [20].

Advanced innovations and revisions to the device have also been changed over the

decades for a number of purposes, including the thermodilution method for heart rate

unit of measure, the inclusion of multiple lumens to permit drug injection through

the catheter, and the use of ﬁber optic Venous oxygen saturation probes to obtain

instant oxygen saturation measured data in the vessel, among others [21].

New protocols and techniques are continually being created for genuine, in-vivo

monitoring of blood. Providing genuine, non-invasive measurements are among the

new applications for capillary pulse oximetry. ICU operations and disaster care were

the primary drivers driving novel capillary heart rate measuring technologies. The

work into investigating potential bounding boxes and holes, regulating device bubble

pressure, and navigating the device through the microvasculature is still underway

[22].

For genuine, in-vivo regulation of blood oxygen, new procedures and technologies

are continuously being updated. Intensive care unit surgery and emergency medicine

inspired the development of new uses for intravascular hypertension measures.

The research into ﬁnding potential bounding boxes and perforations, regulating

device inﬂatable pressure, and guided guidance through all the vascularization is

still ongoing.

In terms of innovation, design, performance, usefulness, and cost, pressure sensors

are vastly different. Three primary technologies based upon piezo resistive, inductive

capacitive, and optical transduction have developed mostly on marketplace to be used

as blood pressure sensors as a result of the expansion of tubes with several lumens.

48 S. Sivaranjani et al.

In the mid-year of 1950s piezo resistivity was discovered in silicon, Copper was

used to develop a sensor module, which is still widely utilized [23]. Fabrication

technique has beneﬁted handsomely from electronic circuit sector breakthroughs

and techniques using materials, procedures, and expertise toolkits. Microelectrome-

chanical systems (MEMS)-based gadgets have exploded as a result, and are now

positioned to drive the growth of new markets.

All pressure sensors have one thing in common: They create pressure to a physical

element to cause it to deﬂect. Several devices take utilization of membrane distortion.

Gauges were frequently used in tympanic membrane systems. The bending moment

is equal to the required pressure created so over most of a given cluster. This direct

ratio does not remain valid in the context of a membrane with such a lot of produced

force or a lot more deformations. Because these methods are easy to test and analyze,

a linear response inside this deﬂection measurement is frequently desired.

4.3 Implantable Piezo Resistive Pressure Sensors

Smith described the properties of piezo resistive in Si and Ge in 1954, indicating

that the material’s resistance changes with applied stress. This breakthrough paved

the way for the development of semiconductor-based piezo resistive sensors. In vivo

blood pressure monitoring is frequently done with piezo resistive which is based

silicon cell pressure sensors attached to the point of an arteries tube. In a human

body, catheter tube is placed in a blood vessel. In addition to many functionalities

that can be installed on it, catheters allow drainage, injection of ﬂuids, and access

by surgical equipment [24]. Catheterization is the process of inserting a catheter.

Piezo resistors are positioned on or in a diaphragm in piezo resistive-based pressure

sensors. The resistance changes linearly with the pressure for thin diaphragms and

minor deﬂections.

Figure 5depicts the development of piezo resistive pressure sensor technology.

Initially, metal diaphragms with metal strain gauges were employed. Metal cavity

was promptly replaced with distributed piezo resistors and Si wafer diaphragms, in

Fig. 5, removing the hysteresis and creep difﬁculties that metal diaphragms had.

Silicon is fully elastic at normal temperature and will not distort plastically. Up to

1% strain, silicon obeys Hooke’s law, a tenfold improvement above standard metallic

materials. As a result, silicon-based diaphragms became an instant favorite over metal

diaphragms.

The invention of oxide layer bonding and the device’s capacity to endure 500–

1500 V and 400–600 C made semiconductor reed valves bound to parfait crystal

support feasible. The use of anodic bonding in the sensor fabrication process resulted

in a signiﬁcant cost saving. However, piezo resistive sensors could not achieve the

needed downsizing as a result of this for biological applications.

In the 1980s, the introduction of silicon on insulator (SOI) technology provided

a numerous advantage to Micro Electro-Mechanical System-based sensor systems,

chieﬂy due to the hidden insulation that may serve as an etched stop, permitting

Health Monitoring and Integrated Wearables 49

Fig. 5 Diaphragm pressure sensor [24]

accurate control of the membrane width [25]. Ever since, various tiny pressure sensors

with silicon nitride membranes have been described.

When Macmillan Equipment developed a tiny catheterization tip pressure trans-

ducer in 1973, that comprised of polysilicon system that is able in a semi Wheatstone

bridge, silicon piezo resistor-based monitoring has indeed been regarded typical for

cannula multiple sensors. Millar Instruments currently have a commercially avail-

able single-use pressure catheter, the Mikro-Cath, that has been approved by the

FDA in Fig. 6. It’s designed to be used as a minimally invasive gadget with only a

few hours of body contact. The sensor is mounted on a 3.5 French catheter with a

diameter of 1.1 mm and can detect the pressure ranging from 200 to 1200 mmHg.

Although the Millar Instrumentation detector is structurally stable, its stiffness, high

power consumption, and temperature dependence limit it. Several ﬁrms, including

50 S. Sivaranjani et al.

Fig. 6 Pressure catheter (Mikro-Cath) [26]

sentron and SciSense, sell a comparable sort of catheter tip pressure sensor on the

market [26].

The research of permanently embedded heart rate monitors is still ongoing,

although there’ve been a few promising efforts. Mills et al. created an injectable

system for long-term peristaltic hypertensive surveillance in awake individuals,

continuously movement experimental mice. Invasive blood pressure is transferred

to a silicon-based force measure sensor 1.35–1.6 mm via a 5-cm long ﬂuid-ﬁlled

catheter. Hypertension and pulse rate were measured in 14 chronically implanted

animals (30–150 days).

5 Self-charging Wearable

Wearable electronics have made remarkable breakthroughs in health monitoring,

but they have also consumed more energy, demanding better battery life, and more

periodic recharging. Batteries, on the other hand, must be replaced or recharged.

In health monitoring, this causes unwelcome downtime. Despite the promise of

power generation by transforming body temperature, portable piezoelectric materials

have not been able to produce power large or consistent enough for the continuous

running of private healthcare surveillance equipment. The simultaneous merging

of a wearable thermoelectric generator with a developing lithium silicon battery

has produced power reliably, continuously, removing the most signiﬁcant barrier to

implementing thermoelectrics in wearable electronics. The fundamental problem of

low thermoelectric output power for rechargeable devices has also been signiﬁcantly

alleviated only with increased Lithium Silicon battery, which has a voltage level half

Health Monitoring and Integrated Wearables 51

that of Lithium ion batteries. The WTEG continuously generates up to 378 W of

power, which is used to operate a conventional glucose detector and stored in Li–S

batteries to ensure a steady voltage of 2 V even when electricity supply and demand

change dramatically [27]. This work, to our knowledge, is the ﬁrst to demonstrate the

viability of operating a commercial glucose sensor solely using body heat, allowing

wearables to operate without interruption and without the need for time-consuming

battery recharge or replacement.

For health and medical applications, reliable and continuous electricity is crucial.

Wearable electronics have made great gains in functionality for a variety of appli-

cations ranging from everyday actions to health supervise, but they have yet to be

offered in a continuous mode. Primary or secondary batteries are often utilized,

but replacing or charging them on a regular basis is not only inconvenient, but

also disturbs their operation. In the late 1960s, for example, pacemaker batteries

were generally replaced every 13–18 months via surgery. For more than 31 years,

implanted thermoelectric gadgets combining radioactive isotopes for use as a heating

element for artiﬁcial hearts (later known as lifetime pacemakers) have been demon-

strated to be durable and secure. Human body conversion efﬁciency is being investi-

gated as just a feasible solution for supplying remote physical condition supervises

equipment. These devices categorized as active or passive systems. Active systems

are electromagnetic, piezoelectric, electrostatic, and turboelectric. Passive systems

are wireless power transmission and thermoelectric. Because the system stops gener-

ating electricity when motion stops. It is challenging to convert an ability to present

into a stable and accurate source of electricity for sensors. Inductive charging distribu-

tion has restricted use owing to the relatively limited working transmission distance

and the receiver, regardless of the fact that active approach provides a consistent

and stable electricity supply. Body temperature is a constant form of energy which

can supply energy production 24/7 a day because the body temperature is regu-

lated. Rechargeable batteries are required for consistent operation because the energy

provided by superconductors varies due to variations in internal and outside heat.

Condition of the patient could be supplied by a generator with a recharging battery. A

great deal of energy and a battery were being used to demonstrate a tactile sense with

relative humidity detectors and an actual screen. Despite the fact that capacitors are

popular due to their relatively simple device structure, rechargeable batteries outper-

form leaky capacitors, especially when the system is not in use for long periods of

time. More crucially, unlike capacitors, which have quickly decaying discharge volt-

ages, batteries may produce consistent and continuous discharge voltages. Wearable

electronics require a steady voltage source to function properly [28]. Nonetheless,

because of lower voltages generated by thermoelectric and lower operation volt-

ages (2 V) for low-power wearable electronics, the high charging voltage of Li-ion

batteries (4.5 V) has been one of the primary hurdles in adopting batteries for storing

energy.

A seamless integration hydroelectric cell system produces reliable voltage without

disruption, removing the limitations of traditional radiative devices and enabling

unprecedented recharging-free and stick up functionality. A game’s characteristic

means it works reliably once it has been turned on. This seems to be terribly beneﬁcial

52 S. Sivaranjani et al.

Fig. 7 Finger stick versus continues monitoring glucose level [28]

in biomedical activities, where monitoring system versus non-linear process provides

a huge edge, or where cell recharging/replacement has undesired consequences.

Traditional glucose strip tests have been performed multiple times per day, but

this small number of tests frequently misses the largest or lowest rises in glucose

levels in Fig. 7. Continuous monitoring could reduce the number of false negatives

and perceive near the beginning signs of diabetic shock or coma. The integrated

system has been shown to be capable of powering a commercial glucose monitoring

sensor with the use of a near ﬁeld communication (NFC) reader without the use of

extra batteries.

6 Wearable Thermoelectric Generator Devices

For some power generation applications, thermoelectric generator gadgets have come

forward as a feasible option. TEGs are one of the allow expertise being investigated to

help the center achieve its goal of developing own-powered, wearable physical condi-

tion, and environmental supervise systems. In applications with a heat source, TEGs

can minimize the requirement of battery. Because batteries have a ﬁnite lifespan

before they need to be renewed or reﬁlled, TEG devices allow for continuous power

generation by charging or reﬁlling batteries. TEG devices’ recent use in the auto-

mobile industry represents a signiﬁcant advancement in their development. Despite

recent advances in numerous thermoelectric device applications, the advancement of

body heat harvesting via TEG devices has received relatively little attention. TEGs

have only lately gained popularity as a result of their wear ability and versatility in

various applications. Body heat harvesting, in particular, has gotten a lot of interest

as a way to power wearable sensors and devices. Wearable gadgets will be extra

durable due to their capacity to strap up steady and continuous energy from body

heat without the necessity of a battery [29]. Finally, the TEG devices being tested

on various sections of the body to see which was the most efﬁcient in terms of both

skin temperature and atmospheric pressure air cooling to charge the gadget.

Health Monitoring and Integrated Wearables 53

6.1 Upper Arm Energy Harvesting

Excess heat collecting was recorded from the inside of the right forearm. For a

multitude of reasons, this person’s body was selected. Since it is so right at the

heart, the electrocardiogram (EKG or ECG) signals are ideally measured on the

right forearm. A portable EKG monitor can be driven by a Thermoelectric generator

placed on the forearm, providing for cardiac monitoring surveillance. The forearm

has a ﬁner and wider region than that of the elbow, allowing greater heat exchange

of the TEG with both the ﬂesh. It also allows for effective air movement due to

the obvious organic swing phase that occurs when running. The bicep muscle was

examined to use the same apparatus that was being used to evaluate the wrists.

To generate power, the thermoelectric generator gadget was indeed linked to a 1.8

load resistance. The gadget was bonded to the right forearm, and the adhesives also

acted as insulator again for cables that linked the spewers, keeping the gadget from

short-circuiting out. The hand was waved in time with such a tempo set at different

velocities, demonstrating the phone’s functioning at different strolling velocities.

6.2 T-Shirt Energy Harvesting

The TEG sensor was embedded on the inside of a T-shirt to conduct the human heat

energy gathering monitoring [30]. By chopping through the T-shirt, the very same

non-PDMS device used again for hand was incorporated. The gadget was inserted

with the topmost copper wires near the surface and the bottom metal layer resting

inside the traditional male T-shirt next to the chest and the women’s corset. Between

the top and bottom copper spreaders, the cloth would operate as a layer in Fig. 8.

In terms of function, the cloth between the two metal applicators would be compa-

rable to the PDMS layer. Running in a single direction while carrying an air movement

analyzer upright has been used to perform the Sample t-test. During the trip, a volt-

meter was used to test the voltage regulation resistance connected to the TEG. The

inlet air sensor and the voltmeter readings were also collected.

6.3 Energy Collection on the Chest

The TEG gadget was attached ﬁrmly to the chest and over heart in the last body

heat harvesting trial in Fig. 9. The device utilized in the T-shirt was the same gadget

that was tested on the chest. Power generation was enabled by connecting the TEG

gadget to a 1.8 load resistor. The tests were carried out using a fan with changing

airﬂow. The investigation was carried out at temperatures of 18.3 °C in a room.

At a gait pace of around 1.1 m/s, 10.2 W/cm2of energy was calculated. The T-shirt

induces the expression a lesser outcome than the forearm test, but the ﬁgurine test

54 S. Sivaranjani et al.

Fig. 8 T-shirts with embedded TEG devices [30]

Fig. 9 Over the chest area, a TEG device is insulated directly to the skin [31]

generated a higher result. It’s worthwhile noting that wrist’s power is almost identical

to the power produced by the hidden breast under the cloak [31]. The TEG just on

wrist provides air but not much more energy than that of the TEG just on chest.

Figure 10 shows that the upper arm model provided the largest amount of power,

around 20 µW/cm2, when air speed was at its quickest, around 1.6 m/s. When

connecting air velocity to power, the upper arm testing has the strongest linear rise,

ensuing in the ﬁnest concert of all the examination. The T-shirt proved to be the

least efﬁcient in testing, consistently producing the least power but still producing

an acceptable power in the range of 3–9 W/cm2at like air velocity.

Health Monitoring and Integrated Wearables 55

Fig. 10 Comparison of TEG power on the T-shirt, chest, upper arm [31]

7 Power Generation in Wearables

The batteries were the traditional source of energy for wearables up to this moment,

but can no longer require that growing demand for upcoming wearable tech that

requires a light, transportable source of power. A uniform and continuous power

solution exists. The existing battery, in example, is inﬂexible and hefty, making

it unsuitable for outdoor activities, demanding industrial circumstances, or military

uses. The current battery also contributes to pollution in the environment. It’s exciting

that clean and renewable power technologies have developed signiﬁcantly in the last

decade, allowing for potentially superior wearable powering options [32].

Figure 11 illustrates the different type of power generation systems. The different

type of power sources is Electromagnetic ﬁeld power, PV power translation for

wearable systems, TEG for wearable systems, Mechanical energy harvesting for

wearable systems. A wearable device’s power generating component can also be

employed as a signal assessment channel to know about the wearer’s physiological

data or pathological status. The vibration-based energy production method (elec-

trostatic, electromagnetic, or piezoelectric) can, for example, collect energy gener-

ated and identify the human body’s mobility posture, workout intensity, movement

frequency, and other data using its voltage waveform. The friction force is used in the

TENG power generating method [33,34]. When human movement, it can feel the

dynamic pressure to keep on the ground or on an object. The transducer converts can

gather rhythmic power from of the patient’s psyche even while evaluating the clin-

ical parameters of the person to use the reference voltage. Thermoelectric electricity

production depending on the temperature variation can both create power through

changes in temperature and analyze the data, like the user’s core temperature, using

electronic pulses including such voltage and current [35]. Worn devices’ sources of

56 S. Sivaranjani et al.

Fig. 11 Different type of power generation systems [32]

power would be more than merely a power distribution device inside the near; they

will also have detecting and sensing abilities, allowing identity wearing devices.

Health Monitoring and Integrated Wearables 57

8 Applications

8.1 Textile ECG

Everyone covers up and puts on clothes every day to hide and protect their body.

Clothing’s original function is to keep a person warm and sheltered from the elements

[36]. Minimizations of the Wearable electronics are now conceivable thanks to

advances in electronics. But, for example, could electronics be reliably integrated

directly into clothing, such as a shirt. If intellect could be worn on a daily basis, the

wearer could keep track of her or his own physiology during a sporting event. A shirt

could be used in medical applications used as a monitoring device in the garment

industry. Electrodes may possibly be used to measure biosignals. A measurement

event’s signal could be saved in measurement equipment and later relayed to the

hospital. The ECG is a well-known biosignal that depicts a heart’s electrical activity.

The QRS-complex, P-wave, and T-wave are the three primary components of elec-

trocardiogram. Those signals generated in a range of areas of the heart, structure, and

timing are controlled by a number of parameters, including the heart’s stress level

at the time of measurement [37–39]. The electrode placements and interval between

each control the magnitude of the pulses.

Textile electrodes are electrodes that are made of textile. Textile materials are

normally insulators, but conductive thread is added to the material through the

production method of textile electrodes. These electrodes don’t require any gel to

connect to the skin. To connect to the epidermis, these electrodes don’t require any

gel. Textile electrodes are made by incorporating conductive yarn into a structure by

spinning, weaving, or needlework it. Silver coated thread or metal ﬁlaments braided

into yarn can both be used to make conductive yarn. Four distinct textile electrode

materials were evaluated, each of which had a different structural design in Fig. 12.

The conductive threads on the fabric of textile electrode 1 were knitted at an

angle on the fabric. The other textile electrode materials were hard, whereas this

material was highly ﬂexible. The conductive strands were woven into the weft of

the textile electrode 2. The textile electrode 3 was also woven, but this time the

conductive strands followed the warp. The electrodes 1–3 were manufactured in

a factory, however the electrode 4 was made by hand using a sewing machine to

embroider conductive yarn (Fineness, silver ﬁbre) in an unsystematic prototype on a

material. The electrodes that were tested were all of the same size [38]. For each of

the aforesaid skin properties: desiccated, clammy, and equipped, results are obtained

with the electrode 1, 2, 3, 4. For all four electrodes, the averages of points from

various measurements are QRS complex, P-wave, T-wave, noise, baseline, R wave

Peak amplitudes calculated. Electrode 4 provided the best evaluation result. Textile

electrodes are ideal for ECG monitoring. The electrode 4 produced excellent results

in this study (embroidered electrode). The beneﬁts, particularly for dry skin, were

outstanding.

58 S. Sivaranjani et al.

Fig. 12 Textile electrodes E1, E2, E3, E4 [38]

8.2 Textile EMG

One of its most extensively utilized biosignals in wellness technologies, physical

training, and recovery devices is surface electromyography (SEMG). In comparison

to traditional electrode design, an alternative is to use Smart Fabric and Interactive

Textile technology for EMG recording [40]. They enable non-interfering, well-ﬁtting

clothing with a scattered number of sensors to be designed and manufactured. The

ability to position electrodes in the desired location, as well as the option to employ

a duplicated number of electrodes, can be beneﬁcial not just for identifying the

appropriate combination for standard bipolar tracking, but also for large surface area

EMG recording. Electromyography control of muscle functions and activity tracking

can be achieved using e-textiles as recoding systems [41]. Textile electrodes stitched

onto a garment served as capacitive transducers that measured through with a textile

layer. Skin irritation concerns are reduced when direct skin contact is avoided.

Health Monitoring and Integrated Wearables 59

9 Conclusion

In this chapter discussed about the various type of sensors such as biosensors,

biodegradable sensors, implantable sensors, Wearable Thermoelectric Generator

devices, and Power generation sensors. The area of implantable sensors isn’t really

fresh, there are many other fresh possibilities worth exploring. Implantable sensors

can be used in a wider range of applications; for example, health tracking could be

considerably enhanced with effective implantable systems. Patients will be much

more ready to accept to monitoring if the technology shows promise, especially

if needles or other painful treatment procedures are avoided. Implantable sensors,

whether mechanical, electrical, chemical, or electronic, have the potential to revo-

lutionize the medical industry. The WTEG-battery system allows for continuous

lengthy monitoring systems. We used exact solutions that were veriﬁed by a numer-

ical analysis to carefully developed and optimize the shape of the WTEG. Even

with the voltage-boosting loss, the WTEG generated more energy than the conven-

tional glucose sensor needed. Our method clearly illustrates that no-charge batteries

believe that combining a WTEG with batteries will provide a future platform for

dependable and continuous health monitoring and medicinal applications. Wearable

body temperature gathering thermoelectric generators have emerged as a result of

recent major advances in the development of thermal systems for energy produc-

tion (TEGs). Such ﬁtness trackers will be much more dependable, long-lasting, and

enable for battery-free operation. After a series of studies, it was discovered that the

upper arm generates the most power, while the wrist and chest produce the least.

The TEG draped on the T-shirt generated the least amount of electricity. The regions

of the upper arm and chest are appropriate for electrocardiogram device implanta-

tion. The developed system of self-powered adaptable electrical gadgets, as well as

potential energy, communication, and pressure ﬂow routes, were also detailed in this

work. Those print-ready portable electronics, textiles based on ﬁber electrical compo-

nents, self-powered self-awareness wearable systems were discussed. This chapter

will assist researchers in determining the wearable sensors researchers should use to

measure the applications.

References

1. Greenspoon, A. J., Patel, J. D., Lau, E., Ochoa, J. A., Frisch, D. R., Ho, R. T., Pavri, B.

B., & Kurtz, S. M. (2012). Trends in permanent pacemaker implantation in the United States

from 1993 to 2009 increasing complexity of patients and procedures. Journal of the American

College of Cardiology, 5, 60–154.

2. Lau, C. P., Tai, Y. T., Fong, P. C., Li, J. P. S., Leung, S. K., Chung, F. L. W., & Song, S. (1992).

Clinical experience with an activity sensing DDDR pacemaker using an accelerometer sensor.

Pacing and Clinical Electrophysiology, 15, 33–43.

3. Mishra, S., Tripathy, H. K., Mallick, P. K., Bhoi, A. K., & Barsocchi, P. (2020). EAGA-MLP—

An enhanced and adaptive hybrid classiﬁcation model for diabetes diagnosis. Sensors, 20(14),

4036.

60 S. Sivaranjani et al.

4. Tripathy, H. K., Mishra, S., Thakkar, H. K., & Rai, D. (2021). CARE: A collision-aware

mobile robot navigation in grid environment using improved breadth ﬁrst search. Computers &

Electrical Engineering, 94, 107327.

5. Coats, A., Adamopoulos, S., Radaelli, A., McCance, A., Meyer, T., Bernardi, L., Solda, P. L.,

Davey, P., Ormerod, O., & Forfar, C. O. N. W. A. Y. (1992). Controlled trial of physical training

in chronic heart failure. Exercise performance, hemodynamics, ventilation, and autonomic

function. Circulation, 3, 85–119.

6. Leung, S. K., Lau, C. P., Tang, M. O., Leung, Z., & Yakimow, K. (1998). An integrated

dual sensor system automatically optimized by target rate histogram. Pacing and Clinical

Electrophysiology, 5, 21–155.

7. Bax,J.J.,Marwick,T.H.,Molhoek,S.G.,Bleeker,G.B.,VanErven,L.,Boersma,E.,Steendijk,

P., van der Wall, E. E., & Schalij, M. J. (2003). Left ventricular dyssynchrony predicts beneﬁt

of cardiac resynchronization therapy in patients with end-stage heart failure before pacemaker

implantation. American Journal of Cardiology, 6, 92–123.

8. Tripathy, H. K., Mallick, P. K., & Mishra, S. (2021). Application and evaluation of classiﬁcation

model to detect autistic spectrum disorders in children. International Journal of Computer

Applications in Technology, 65(4), 368–377.

9. Rod Gimbel, J., Bello, D., Schmitt, M., Merkely, B., Schwitter, J., Hayes, D. L., Sommer, T.,

Schloss, E. J., Chang, Y., Willey, S., & Kanal, E. (2013). Randomized trial of pacemaker and

lead system for safe scanning at 1.5 Tesla. Heart Rhythm, 5, 59–69.

10. Karami, M. A., & Inman, D. J. (2012). Powering pacemakers from heartbeat vibrations using

linear and nonlinear energy harvesters. Applied Physics Letters, 4, 42–90.

11. Tashiro, R., Kabei, N., Katayama, K., Tsuboi, E., & Tsuchiya, K. (2002). Development of

an electrostatic generator for a cardiac pacemaker that harnesses the ventricular wall motion.

Journal of Artiﬁcial Organs, 3, 05–23.

12. Navarro, X., Krueger, T. B., Lago, N., Micera, S., Stieglitz, T., & Dario, P. (2005). A critical

review of interfaces with the peripheral nervous system for the control of neuroprostheses and

hybrid bionic systems. Journal of the Peripheral Nervous System, 4, 10–22.

13. Mishra, S., Thakkar, H., Mallick, P. K., Tiwari, P., & Alamri, A. (2021). A sustainable IoHT

based computationally intelligent healthcare monitoring system for lung cancer risk detection.

Sustainable Cities and Society, 103079.

14. Wise, K. D., Angell, J. B., & Starr, A. (1970). An integrated-circuit approach to extracellular

microelectrodes. IEEE Transactions on Biomedical Engineering, 5, 215–238.

15. Najaﬁ, K., Wise, K., & Mochizuki, T. (1985). A high-yield IC-compatible multichannel

recording array. IEEE Transactions on Electron Devices, 32, 206–211.

16. Mishra, S., Panda, A., & Tripathy, K. H. (2018). Implementation of re-sampling technique

to handle skewed data in tumor prediction. Journal of Advanced Research in Dynamical and

Control Systems, 10, 526–530.

17. Stieglitz, T. (2010). Manufacturing, assembling and packaging of miniaturized neural implants.

Microsystem Technologies, 16, 723–734.

18. Shuai, C. J., Li, S., Peng, S. P., Feng, P., Lai, Y. X., & Gao, C. D. (2019). Biodegradable metallic

bone implants. Materials Chemistry Frontiers, 3, 544–562.

19. Hermawan, H. (2018). Updates on the research and development of absorbable metals for

biomedical applications. Progress in Biomaterials, 7, 93–110.

20. Keane, T. J., Swinehart, I. T., & Badylak, S. F. (2015). Methods of tissue decellularization used

for preparation of biologic scaffolds and in vivo relevance. Methods, 8, 25–34.

21. Holland, C., Numata, K., Rnjak-Kovacina, J., & Seib, F. P. (2019). The biomedical use of silk:

Past, present, future. Advanced Healthcare Materials, 8, e1800465.

22. Janani, G., Kumar, M., Chouhan, D., Moses, J. C., Gangrade, A., Bhattacharjee, S., & Mandal,

B. B. (2019). Insight into silk-based biomaterials: From physicochemical attributes to recent

biomedical applications. ACS Applied Bio Materials, 2, 5460−5491.

23. Li, L., Fu, X., Chen, S., Uzun, S., Levitt, A. S., Shuck, C. E., Han, W., & Gogotsi. Y.

(2020). Hydrophobic and stable MXene-polymer pressure sensors for wearable electronics.

ACS Applied Materials & Interfaces, 12, 15362−15369.

Health Monitoring and Integrated Wearables 61

24. Dahiya, A. S., Thireau, J., Boudaden, J., Lal, S., Gulzar, U., Zhang, Y., Gil, T., Azemard, N.,

Ramm, P., Kiessling, T., O’Murchu, C., Sebelius, F., Tilly, J., Glynn, C., Geary, S., O’Dwyer,

C., Razeeb, K. M., Lacampagne, A., Charlot, B., & Todri-Sanial, A. (2020). Review-energy

autonomous wearable sensors for smart healthcare: A review. Journal of the Electrochemical

Society, 3, 167–170.

25. Phan, H. P., Dao, D. V., Nakamura, K., Dimitrijev, S., & Nguyen, N. T. (2015). The

piezoresistive effect of SiC for MEMS sensors at high temperatures: A review. Journal of

Microelectromechanical Systems, 6, 1663−1677.

26. Barlian, A. A., Park, W. T., Mallon, J. R., Rastegar, A. J., & Pruitt, B. L. (2009). Review:

Semiconductor piezoresistance for microsystems. Proceedings of the IEEE, 3, 513−552.

27. Kirstein, T., Cottet, D., Grzyb, J., & Tröster, G. (2002). Textiles for signal transmission in

wearables. Workshop on Modelling, Analysis and Middleware Support for Electronic Textiles,

San Jose, CA.

28. Rantanen, J., Vuorela, T., Kukkonen, K., & Ryynänen, O. (2001). Improving human thermal

comfort with smart clothing. In Proceedings of the 2001 IEEE Systems, Man and Cybernetics

Conference.

29. Norstebo, C. A. (2003). Intelligent textiles, soft products. Journal of Future Materials.

30. Mishra, S., Mishra, B. K., & Tripathy,H. K. (2015, December). A neuro-genetic model to predict

hepatitis disease risk. In 2015 IEEE International Conference on Computational Intelligence

and Computing Research (ICCIC) (pp. 1–3). IEEE.

31. Karilainen, S., Hansen, S., & Muller, J. (2005). Dry and capacitive electrodes for long-term

ECG-monitoring. 8th Annual Workshop on Semiconductor Advances (Vol. 3, pp. 156–159).

32. Catrysse, M., Puers, R., Hertleer, C., van Langenhove, L., van Egmond, H., Matthys, D., &

Leuven, K. U. (2003). Fabric sensors for measurement of physiological parameters (Vol. 5,

pp. 1758–1761). Ghent University, Department of Textiles, Department of Paediatrics and

Genetics, Belgium. IEEE.

33. Paradiso, R., & De Rossi, D. (2006). Advances in textile technologies for unobtrusive moni-

toring of vital parameters and movements. In Proceedings of the 28th IEEE EMBS Annual

International Conference (Vol. 3, pp. 165–185), New York City.

34. Paradiso, R., Loriga, G., & Taccini, N. (2005). A wearable health care system based on knitted

integrated sensors. IEEE Transactions on Information Technology in Biomedicine, 9, 85–90.

35. Wong, A. S. W., Li, Y., & Zhang, X. (2004). Inﬂuence of fabric mechanical property on clothing

dynamic pressure distribution and pressure comfort on tight-ﬁt sportswear. Sen’i Gakkaishi, 6,

293–299.

36. Kim, N. Y., & Lee, H. (2019). Inﬂuence of clothing pressure on blood ﬂow and subjective

sensibility of commercial sports compression wear. Fashion & Textile Research Journal, 21,

459–467.

37. Ray, C., Tripathy, H. K., & Mishra, S. (2019). A review on facial expression based behavioral

analysis using computational technique for autistic disorder patients. In M. Singh, P. Gupta,

V. Tyagi, J. Flusser, T. Ören, & R. Kashyap (Eds.), Advances in computing and data sciences.

ICACDS 2019. Communications in computer and information science (Vol. 1046). Springer.

https://doi.org/10.1007/978-981-13-9942-8_43

38. De Silva, J. C. L., Tarassova, O., Ekbolom, M. M., Andersson, E., Rönquist, G., & Arndt,

A. (2016). Quadriceps and hamstring muscle activity during cycling as measured with

intramuscular electromyography. European Journal of Applied Physiology, 11, 1807–1817.

39. Pani, D., Achilli, A., Spanu, A., Bonﬁglio, A., Gazzoni, M., & Botter, A. (2019). Validation of

polymer-based screen-printed textile electrodes for surface EMG detection. IEEE Transactions

on Neural Systems and Rehabilitation Engineering, 27, 1370–1377.

40. Cömert, A., Honkala, M., & Hyttinen, J. (2013). Effect of pressure and padding on motion

artifact of textile electrodes. Biomedical Engineering Online, 12, 1–19.

41. Lu, F., Wang, C., Zhao, R., Du, L., Fang, Z., Guo, X., & Zhao, Z. (2018). Review of stratum

corneum impedance measurement in non-invasive penetration application. Biosensors, 8, 8–31.

A Comprehensive Review Analysis

of Alzheimer’s Disorder Using Machine

Learning Approach

Priyom Dutta and Sushruta Mishra

Abstract Alzheimer’s disease is a neural disorder which the cause for progressive

irreversible neurologic disorder. It is difﬁcult to detect in primary stages and yet the

modalities for the progression which are complex which are not completely known.

The way of detection of Alzheimer’s disease is through neuro imaging technique

by the diffusion of the resonance caused by magnetism represented as MR. In order

to understand the real and root causes there is a need of studying huge amount of

MR images. So as to resolve the issues of time constraint and faster access to the

features of MR image, machine learning techniques are needed, in order to process

large quantities of medical data. In machine learning set of rules determine the

output based on the goal. The study of the mild cognitive impairment according to

the classiﬁcation problem is done as well as the diffusion of the data along with

the other sources. The systematic review of several predictive learning methods is

presented with reference to their work and the score of their performance. From the

research work, machine learning proves to be an efﬁcient way for categorizing and

classiﬁcation of the Alzheimer’s disease with the high accuracy.

Keywords Alzheimer’s disease ·Magnetic resonance imaging (MRI) ·Machine

learning ·Support vector machine ·Cognitive impairment ·Multimodal analysis

1 Introduction

The disease named after Dr. Alois Alzheimer as Alzheimer which cannot be cured

and is neuro-degenerative disease which affects primarily the aged and the elderly

population, hence it is a very common factor of the dementia of the today’s world

[1]. The report of the world which reveals that around 50 million peoples have been

affected by the disease. Just because the symptoms of the Alzheimer’s disease are

visible after 60 years of age, it becomes very hard to detect. Study suggests some

forms may develop at the age of 30–50 years also for some individual where there

P. Dut t a ·S. Mishra (B)

School of Computer Engineering, KIIT Deemed To Be University, Bhubaneswar, Odisha, India

e-mail: mishra.sushruta@gmail.com

S. Mishra et al. (eds.), Augmented Intelligence in Healthcare: A Pragmatic and Integrated

Analysis, Studies in Computational Intelligence 1024,

https://doi.org/10.1007/978-981-19- 1076-0_4

64 P. Dutta and S. Mishra

is gene mutation [2]. The disease can result in structural and functional changes in

the brain. Patients are ﬁrst diagnosed by cognition-based strategies that eventually

proceed towards Alzheimer’s risks [3]. As per the U.S. health service data, about twice

women are affected with this risk compared to men and it gets worse in them very

fast. Shrinking of brain tends to get severe in women with this risk when compared to

men. Some researchers suggest that changes in women’s brain may be due to some

other factors. This disease is presently in the top ten ranking of reason for death in

U.S. and in recent times, it is noted that it is amongst top ﬁve leading cause of death

lagging just behind cancer and heart disorders especially amongst senior citizens [4].

The disease which involves the loss of cognitive functioning i.e. human beings’

thinking and the process of remembering and reasoning and the behaviour abilities

of daily life. The severe stage to the mild stage is the range where the person depends

on other person for daily activities. Hence the detection of the disease should be done

at early stages. Graña et al. [5] and Patil et al. [6] used the various computationally

intelligent methods for the early detection of this risk.

The detection of the disease can be done at the age of 50’s where the chances of

the detection can be highest. Now a days, Machine learning algorithms have high

training accuracy for the detection of the disease and can result in early detection, the

algorithms include various supervised as well as unsupervised learning models which

are used for training the model. The research work done by Patil and Ramakrishnan

[7] and Schouten et al. [8] includes such learning methods which include learning,

training and testing and validation methods. The predictive methods from a part

of machine intelligence has now proved to be an efﬁcient technique for detection

of disease accurately. The process involves learning from a data, for data we use

datasets and the data is cleaned and noise removal process is done in the data. From

the datasets feature selection is done where important features are selected and the

data is split into two divisions one for train set and other as test set. The supervised

and unsupervised learning methods are performed and the testing on the trained

model is done and the accuracy calculation is done. The diagram shown Fig. 1is the

process of disease identiﬁcation of Alzheimer’s disease using machine learning as

shown, here in the training of the model is done using the machine learning based

on linear as well as logistic regression.

As per the Alzheimer’s Association which published an annual report which

includes several parameters for the detection of Alzheimer’s disease. The Population

of half the adults over the age of 85 will have the Alzheimer’s disease where 1 in

10 persons who is having the age of 65 approx. 10% are having the disease. And

the good thing is that half the population in America approx. 5.4 Million person

know that they have Alzheimer’s and based on gender women tend to have more

disease than men. The symptoms develop at the age of 30. So the research work like

Mesrob et al. [9] which used neuro imaging technique for the identiﬁcation of the

Alzheimer’s risk.

From the various research work suggested in the process of training a prototype

to identify Alzheimer’s risk, the research work suggests classiﬁcation based on the

learning types which majorly focus on the logistics as well as linear regression

[10,11]. From a record of 15 records the screening process discards the combined

A Comprehensive Review Analysis of Alzheimer’s Disorder … 65

Fig. 1 Alzheimer’s disease prediction process through machine learning algorithms

results and ﬁlters the accurate resembling records which constitute the several records

meant for the identiﬁcation and are included in the ﬁnal research. Figure 2shows

the records from several research is taken into consideration identiﬁed from same

source as well as different source which involves screening of various records and

ﬁnding the eligible records and considering for the inclusion.

Figure 2shows the study of various records from the dataset and previous

research work and classiﬁcation based on the identiﬁcation, eligibility, screening

and inclusion.

2 Literature Survey

Dyrba et al. [12] used support vector machine (SVM) MK-SVM as classiﬁer and

the neuroimaging technique opted as Diffusion Tensor Imaging (DTI), functional

magnetic Resonance Imaging (fMRI), considering the measures they used FA, Mean

diffusivity (MD), Mode of anisotropy (MO), Voxel-Based Morphometry (VMB),

local cluster metric, shortest path length and the feature set was DTI measures and

Support Vector Machine (SVM), Resting-state functional (Rs-fMRI) metrics (SM),

Grey matter volume GMV (SVM) Rs-fMRI +DTI +GMV (SVM), DTI Chen et al.

[13] used SVM as classiﬁer and the neuro-imaging technique opted as (Diffusion

kurtosis imaging), considering the measures they used some methods and the feature

set was Automated Anatomical Labelling atlas Combination of kurtosis and diffusion

indices from diffusion metrics and accuracy 96.2, 92.5, 81.1, 86.8 respectively. Cai

66 P. Dutta and S. Mishra

Fig. 2 Records samples included in research based on the identiﬁcation, eligibility, screening and

inclusion

et al. [14] used LDA (Linear Discriminant Analysis) as classiﬁer and the neuro-

imaging technique generated feature set to identify alzheimer’s disease in elderly

people. Tang et al. [15] used Linear Discriminant Analysis, Support Vector Machine

as classiﬁer and the neuro-imaging technique opted as Diffusion Tensor Imaging,

Functional Magnetic Resonance Imaging (fMRI), considering the measures they

used Volume, deformation, used SVM as well as K-Nearest Neighbour KNN [6]

for training the model for detection for Alzheimer’s disease and used Fractional

Anistrophy (FA). Demirhan et al. [16] in their research used SVM to train their

machine learning model for prediction of alzheimer’s disease used SVM for training

the model and received as 84.9% and Mild cognitive impairment as 79%. Li et al.

[9] they combined Diffusion Tensor Imaging as well as fMRI indices to access their

result for their Alzheimer’s disease classiﬁcation and their results obtained with Trace

based on voxel-based FA. The effectiveness of the methods were discussed in [17]

using brain shape which is the classiﬁcation of Alzheimer’s disease, the method used

A Comprehensive Review Analysis of Alzheimer’s Disorder … 67

is used for existing changes in morphology related to Alzheimer’s for evaluation. The

detection of Alzheimer’s is done by [18] using Alzheimer’s disease neuro imaging

data and considering all the data all at a once, which required the diagnosis of the

patient and used in various settings for clinical. The study for the existing techniques

and issues was addressed by [19] so authors suggested the use of sparse inverse

covariance analysis to examine and region of the brain and the evaluation of the

brain was done in terms of the connectivity patterns. Using Economic growth centre

database ﬁltration and the noise reduction is suggested by the authors [20] in their

research and the process of wavelet transformation of feature extraction and the

classiﬁcation by the use of Support Vector Machine, provides the patients to travel

anywhere. Several other relevant works are being conducted in context to disease

diagnosis using machine learning approach. Table 1depicts the accuracy analysis of

different works done in the domain.

3 Generalized Working Model

The generalized working module works on the following steps which include the

data extracted from the datasets and the important features are extracted, the feature

extraction process includes extraction of several features and the model is trained in

the process using 80:20 ratio dataset split which will be the train and test set [18–20,

23]. After the model is trained under the supervised learning process, the prediction

process is then validated by the test set. The generalized process diagram is shown

in Fig. 3.

The above method is used by almost in the research process which includes

several datasets of the Alzheimer’s disease and the dataset is ﬁrst processed and

the data cleaning and noise reduction process is made by various data cleaning

mechanisms and data visualization is performed. After visualizing the dataset, the

feature extraction process is done, since here the dataset can be linearly or non-

linearly dependent, here the data can be trained using the SVM which can be linear

SVM or non-linear SVM. Machine learning models are used here especially to train

and test the data from the datasets. Researchers also used here ad boost and Marko’s

models to determine the classiﬁcation of disease from the dataset [24–26]. After the

classiﬁcation, the model is tested using the testing parameters which is described

as below and the accuracy of the model opted can be found. The deployment of the

model can be done either online as well as ofﬂine. Here the computer aided diagnosis

opted consists of the model which has been deployed either online or ofﬂine.

4 Testing and Validation Parameters

Accuracy denotes the proportion of accurately categorized samples with the

cumulative number of samples [27] and is determined as,

68 P. Dutta and S. Mishra

Tabl e 1 Comparative work done by researchers with their accuracy

Research Measures Classiﬁer Technique Feature set ACC%

Mishra et al. [4] Mean

diffusivity

SVM Diffusion

tensor

imaging

Mean diffusivity

100.0

−99.0

Patil et al. [5]Functional

anisotropy

AdaBoost Diffusion

tensor

imaging

FA [10 features] 84.5

Patil and

Ramakrishnan[5]

Mean

diffusivity

SVM and

decision

stumps

Diffusion

tensor

imaging

Logistic

regression

89.7

91.9

Schouten et al. [6] Mean

diffusivity

Logistic

elastic net

regression

Diffusion

tensor

imaging

FA-TBSS

Mean diffusivity

Alzheimer’s

dataset

Functional

anisotropy

82.6

80.8

81.8

84.8

Mesrob et al. [7] Mean

diffusivity,

Grey matter

volume GMV

Non-linear

SVM

Diffusion

tensor

imaging,

functional

magnetic

resonance

imaging

(fMRI)

Mean diffusivity

SVM

LDA

Non-linear SVM

72.1

72.4

65.2

76.5

Dyrba et al. [12]Grey matter

volume GMV,

diffusion

indices

Multivariate

SVM NB

Diffusion

Tensor

Imaging,

functional

magnetic

Resonance

Imaging

(fMRI)I

Functional

Anisotropy (FA)

Mean diffusivity

(SVM)

89.3

80.3

83.3

82.7

Li et al. [9]SVM SVM Diffusion

tensor

imaging,

functional

magnetic

resonance

imaging

(fMRI)I

Tract-Based

Functional

Anisotropy

94.3

Dyrba et al.[12]FA, M D, M O

sMRI: GMV

Rs-fMRI: local

clustering

coefﬁcient

SVM Diffusion

tensor

imaging,

functional

magnetic

resonance

imaging

(fMRI)I

DTI measures

(SVM)

Rs-fMRI

measures

(continued)

A Comprehensive Review Analysis of Alzheimer’s Disorder … 69

Tabl e 1 (continued)

Research Measures Classiﬁer Technique Feature set ACC%

Chen et al. [13]Resting-state

functional

(Rs-fMRI),

Voxel-based

morphometry

(VMB)

SVM Diffusion

tensor

imaging

(DTI),

diffusion

tensor

imaging

(DTI)

ALL-Diffusion

kurtosis imaging

Diff-Diffusion

kurtosis imaging

Diff-Diffusion

Tensor Imaging

Diffusion kurtosis

imaging

96.2

92.5

81.1

86.8

Caietal.[14] Connection

strength

Linear

discriminant

analysis

Diffusion

tensor

imaging

(DTI)

NeuroImaging

technique 1

NeuroImaging

technique 2

NeuroImaging

technique 3

84.6

73.0

79.8

Tang et al. [15] Volume,

deformation,

Linear

discriminant

analysis,

SVM

Diffusion

tensor

imaging

(DTI)

Results for right

hippocampus

with SVM volume

Shape original

78.4

70.3

86.5

83.8

Shao et al. [21] Mean

diffusivity

SVM, k-NN,

linear

discriminant

analysis

Diffusion

tensor

imaging

(DTI)

Computed

tomography

support vector

machine

Alzheimer’s

disease

support vector

machine

100

Nir et al. [22] Mean

diffusivity

SVM Diffusion

Tensor

Imaging

(DTI)

Positron emission

Tomography

Demirhan et al.,

2015 [16]

Mean

diffusivity

SVM Diffusion

tensor

imaging

(DTI)

Alzheimer’s

disease

84.9

Li et al. [9]Functional

anisotropy

Logistic

regression

Diffusion

tensor

imaging

(DTI)

Alzheimer’s

disease feature

selection

Fuse et al. [17]Functional

anisotropy

SVM Diffusion

Tensor

Imaging

(DTI)

Alzheimer’s

disease

Whole-brain

Hippocampal

Cingulum

Parahippocampal

Gyrus

MCI/HC

(continued)

70 P. Dutta and S. Mishra

Tabl e 1 (continued)

Research Measures Classiﬁer Technique Feature set ACC%

Dou et al. [18] Mean

diffusivity

SVM, linear

discriminant

analysis,

XGB

Diffusion

tensor

imaging

(DTI)

support vector

machine

82.5

Fig. 3 Generalized process diagram

Accuracy =TP +TN

TP +TN +FP +FN

where TP =True Positive, i.e. samples which are accurately categorized as positive

by the classiﬁer.

TN =True Negative, i.e. samples which are accurately categorized as negative

by the classiﬁer.

FP =False Positive, i.e. samples which are negative but categorized as positive

by the classiﬁer.

FN =False Negative, i.e. samples which are positive but categorized as negative

by the classiﬁer.

A Comprehensive Review Analysis of Alzheimer’s Disorder … 71

Tabl e 2 Comparative

analysis of the machine

learning models [5]

Methods Used Accuracy

SVM 100

Ad boost 84.5

SVM 89.7

Logistic regression 82.6

Non-linear SVM 76.5

Multivariate SVM 89.3

SVM 96.2

LDA, SVM 89.2

SVM, k-NN, NB 100

5 Comparison Analysis

Table 2shows the comparative analysis of the accuracy obtained by the methods

opted by various research work. The analysis shows that most of the researchers

used regression and SVM for training the machine learning model which resulted in

good accuracy which can be seen from the table and even 100% accuracy is achieved

in some of the research work.

6 Result Analysis

Patil and Ramakrishnan [5] in their research for the detection of Alzheimer’s disease

used three classiﬁer for training their machine learning models such as SVM, Deci-

sion Stumps and Logistic Regression and received a percentage of 89.7%, 91.9%

and 93.4% shown in accuracy graph in Fig. 4.

Schouten et al. [6] in their research work for detecting the Alzheimer’s disease

trained the machine learning model based on several parameters they are as follows

FA-TBSS, MD-TBSS, DA-TBSS DR-TBSS, FA-ICA, MD-ICA, DA-ICA, DR-ICA

Connectivity graph Degree, Strength, Clustering Centrality, Path length, Transitivity

and Sparse Group Lasso and the accuracy percent of the graph is shown below in the

graph. The accuracy percentage is shown in Fig. 5. The maximum accuracy received

is for FA-ICA where the accuracy is 85.1%.

Dyrba et al. [12], 2013 in their research work used various SVM techniques for

the classiﬁcation of Alzheimer’s disease. The graph below shows the WMD (SVM)

accuracy comes out to be 4.5. The accuracy percentage is shown in the graph of

Fig. 6.

Chen et al. [13] in their research work used several classiﬁcation technique for the

detection of Alzheimer’s disease. They used ALL-DKI, Diff-DKI, Diff-DTI, Kur-

DKI for training their machine learning model for detection of Alzheimer’s disease.

72 P. Dutta and S. Mishra

Fig. 4 Result matrix by Patil and Ramakrishnan [3]

Fig. 5 Result matrix by Schouten et al. [4]

The highest accuracy received is 96.2 as ALL-DKI. The accuracy percentage is

shown in graph of Fig. 7.

Cai et al. [14] used BC-AAI, CN-AAI, BC-CN (AAI), Hippocampal volume-

AAI, MMSE-AAI, Hippocampal volume-MMSE-AAI, BC-HOA, CN-HOA, BC-

CN-HOA, Hippocampal volume-HOA, MMSE-HOA, Hippocampal volume with

MMSE -HOA for training the model for prediction of Alzheimer’s disease. The

diagram shown in Fig. 8.

A Comprehensive Review Analysis of Alzheimer’s Disorder … 73

Fig. 6 Result metrics achieved by Dyrba et al. [8]

Fig. 7 Result metrics by Chen et al. [9]

7 Conclusion

The results obtained from this review depicts that machine learning algorithms may

be applicable to several image-based instances to understand the functional mecha-

nisms of Alzheimer’s risks and minor cognition issues which are primarily impor-

tant for the future research work in the ﬁeld of Alzheimer’s disease [28]. Research

works which are existing show the categorization between the Alzheimer’s disease

and the Health control patient. From the above results in the research work the

support vector machine proves to be better and dominating than other classiﬁers.

Fractional anisotropy FC overpowers the categorization of the Alzheimer’s disease

74 P. Dutta and S. Mishra

Fig. 8 Result metrics by Cai et al. [10]

in the Health control unit over the integration of the mild cognitive impairment also

the multi-modal analysis proves to be the detecting the patterns of the several neuro-

degeneration across several platforms which is gaining more attention and getting

reliable for better classiﬁcation in the categorization of Alzheimer’s disorder with

minor cognitive concern. These imaging technique is useful in comparing the several

approaches related to the mild cognitive impairment bio makers including several

types of classiﬁcations such as deep learning and feature selections and several other

imaging techniques are focussed in order to take several samples for further high

precision sampling techniques.

References

1. Tutica, L., Vineel, K. S. K., Mishra, S., Mishra, M. K., & Suman, S. (2021). Invoice deduction

classiﬁcation using LGBM prediction model. In Advances in electronics, communication and

computing (pp. 127–137). Springer: Singapore.

2. Roy, S. N., Mishra, S., & Yusof, S. M. (2021). Emergence of drug discovery in machine

learning. In Technical advancements of machine learning in healthcare (p. 119).

3. Chowdhury, S. R., Mishra, S., Miranda, A. O., & Mallick, P. K. (2020, March). Energy

consumption prediction using light gradient boosting machine model. In International

Conference on Emerging Trends and Advances in Electrical Engineering and Renewable

Energy (pp. 413–422). Springer: Singapore.

4. Mishra, S., Tripathy, H. K., & Panda, A. R. (2018). An IMPRoved and adaptive attribute

selection technique to optimize dengue fever prediction. International Journal of Engineering

and Technology, 7, 480–486.

5. Patil, R. B., & Ramakrishnan, S. (2014). Analysis of sub-anatomic diffusion tensor imaging

indices in white matter regions of Alzheimer with MMSE score. Computer Methods and

Programs in Biomedicine, 117, 13–19.

6. Schouten, T. M., Koini, M., De Vos, F., Seiler, S., De Rooij, M., Lechner, A., Schmidt, R.,

Heuvel, M. V. D., Van Der Grond, J., & Rombouts, S. A. (2017). Individual classiﬁcation of

Alzheimer’s disease with diffusion magnetic resonance imaging. NeuroImage, 152, 476–481.

A Comprehensive Review Analysis of Alzheimer’s Disorder … 75

7. Mesrob, L., Sarazin, M., Hahn-Barma, V., De, S. L. C., Dubois, B., Gallinari, P., Kinkingnéhun,

S., Mesrob, L., Marie, S., & Valerie, H.-B., et al. (2012). DTI and structural MRI classiﬁcation

in Alzheimer’s disease. Advances in Molecular Imaging, 2, 12–20.

8. Dyrba, M., Ewers, M., Wegrzyn, M., Kilimann, I., Plant, C., Oswald, A., Meindl, T., Pievani, M.,

Bokde, A. L. W., & Fellgiebel, A., et al. (2013). Robust automated detection of microstructural

white matter degeneration in Alzheimer’s disease using machine learning classiﬁcation of

multicenter DTI data. PLoS ONE, 8, e64925.

9. Li, M., Qin, Y., Gao, F., Zhu, W., & He, X. (2014). Discriminative analysis of multivariate

features from structural MRI and diffusion tensor images. Magnetic Resonance Imaging, 32,

1043–1051.

10. Mishra, S., Raj, A., Kayal, A., Choudhary, V., Verma, P., & Biswal, L. (2012). Study of cluster

based routing protocols in wireless sensor networks. International Journal of Scientiﬁc and

Engineering Research, 3(7).

11. Jena, L., Patra, B., Nayak, S., Mishra, S., & Tripathy, S. (2019). Risk prediction of kidney

disease using machine learning strategies. In Intelligent and cloud computing (pp. 485–494).

Springer: Singapore.

12. Dyrba, M., Grothe, M. J., Kirste, T., & Teipel, S. J. (2015). Multimodal analysis of functional

and structural disconnection in Alzheimer’s disease using multiple kernel SVM. Human Brain

Mapping, 36, 2118–2131.

13. Chen, Y., Sha, M., Zhao, X., Ma, J., Ni, H., Gao, W., & Ming, N. (2017). Automated detection

of pathologic white matter alterations in Alzheimer’s disease using combined diffusivity and

kurtosis method. Psychiatry Research: Neuroimaging, 264, 35–45.

14. Cai, S., Huang, K., Kang, Y., Jiang, Y., Von Deneen, K. M., & Huang, L. (2019). Potential

biomarkers for distinguishing people with Alzheimer’s disease from cognitively intact elderly

based on the rich-club hierarchical structure of white matter networks. Neuroscience Research,

144, 56–66.

15. Tang, X., Qin, Y., Wu, J., Zhang, M., Zhu, W., & Miller, M. I. (2016). Shape and diffusion

tensor imaging based integrative analysis of the hippocampus and the amygdala in Alzheimer’s

disease. Magnetic Resonance Imaging, 34, 1087–1099.

16. Demirhan, A., Nir, T. M., Zavaliangos-Petropulu, A., Jack, C. R., Weiner, M. W., Bernstein,

M. A., Thompson, P., & Jahanshad, N. (2015). Feature selection improves the accuracy of

classifying Alzheimer disease using diffusion tensor images. In Proceedings of the 2015 IEEE

12th International Symposium on Biomedical Imaging (ISBI) (pp. 126–130). Brooklyn, NY,

USA, 16–19 April 2015.

17. Fuse, H., Oishi, K., & Maikusa, N. (2018). Detection of Alzheimer’s disease with shape analysis

of MRI images. In 2018 Joint 10th International Conference on Soft Computing and Intelligent

Systems (SCIS), IEEE

18. Escudero, J., Ifeachor, E., & Zajicek, J. P. (2013). Machine learning-based method for person-

alized and cost-effective detection of Alzheimer’s disease. IEEE Transactions on Biomedical

Engineering, 60(1).

19. Patel, R., Liu, J., Chen, K., & Reiman, E. (2009). Sparse inverse covariance analysis of human

brain for Alzheimer’s disease study. In 2009 ICME International Conference on Complex

Medical Engineering, IEEE.

20. Thakare, P., & Pawar, V. R. (2016). Alzheimer disease detection and tracking of Alzheimer

patient. In 2016 International Conference on Inventive Computation Technologies (ICICT),

IEEE

21. Shao, J., Myers, N., Yang, Q., Feng, J., Plant, C., Böhm, C., Förstl, H., Kurz, A., Zimmer,

C., & Meng, C., et al. (2012). Prediction of Alzheimer’s disease using individual structural

connectivity networks. Neurobiology of Aging, 33, 2756–2765.

22. Nir, T. M., Villalon-Reina, J. E., Prasad, G., Jahanshad, N., Joshi, S. H., Toga, A. W., Bernstein,

M. A., Jack, C. R., Weiner, M. W., & Thompson, P., et al. (2015). Diffusion weighted imaging-

based maximum density path analysis and classiﬁcation of Alzheimer’s disease. Neurobiology

of Aging, 36, S132–S140.

76 P. Dutta and S. Mishra

23. Ray, C., Tripathy, H. K., & Mishra, S. (2019). Assessment of autistic disorder using machine

learning approach. In Proceedings of the International Conference on Intelligent Computing

and Communication (pp. 209–219). Hyderabad, India, 9–11 January 2019.

24. Mishra, S., Mishra, B. K., Tripathy, H. K., & Dutta, A. (2020). Analysis of the role and scope

of big data analytics with IoT in health care domain. In Handbook of data science approaches

for biomedical engineering (pp. 1–23). Academic Press.

25. Mishra, S., Dash, A., & Jena, L. (2021). Use of deep learning for disease detection and diagnosis.

In Bio-inspired neurocomputing (pp. 181–201). Springer: Singapore.

26. Mishra, S., Thakkar, H., Mallick, P. K., Tiwari, P., & Alamri, A. (2021). A sustainable IoHT

based computationally intelligent healthcare monitoring system for lung cancer risk detection.

Sustainable Cities and Society, 103079.

27. Sahoo, S., Das, M., Mishra, S., & Suman, S. (2021). A hybrid DTNB model for heart disorders

prediction. In Advances in electronics, communication and computing (pp. 155–163). Springer:

Singapore.

28. Rath, M., & Mishra, S. (2019). Advanced-level security in network and real-time applications

using machine learning approaches. In Machine learning and cognitive science applications

in cyber security (pp. 84–104). IGI Global.

Machine Learning Techniques in Medical

Image: A Short Review

Ashwini Kumar Pradhan, Kaberi Das, and Debahuti Mishra

Abstract At present, the analysis and diagnosis of a particular disease is a big chal-

lenge for doctors. So, to get the prior information regarding the internal anatomical

structure of human organs or tissue, different imaging modalities techniques are used

to capture the medical image which is represented pixel by pixel. Due to the large

volume of data in the image dataset, it is more difﬁcult to analyze. In this study,

a different sequence of operations on the medical image such as pre-processing,

feature extraction, feature selection, existing classiﬁcation techniques with pros and

cons are studied and compared. Finally, an improvement of classiﬁcation techniques

in terms of efﬁciency, accuracy is summarized which will be helpful for a researcher

working in this ﬁeld.

Keywords Medical image ·Image classiﬁcation ·Feature extraction ·Disease

diagnosis ·Artiﬁcial intelligence

1 Introduction

Recently there has been a big interest in the medical image which plays an important

role for diagnosis and research purposes. The data behind the medical image is formed

with the help of different imaging modalities. Classiﬁcation analysis is one of the

important techniques out of many data mining techniques that extract the pattern

from the dataset. Since a large number of images is captured by different types of

equipment in various hospitals, an efﬁcient and effective technique is required for

A. K. Pradhan (B)

Department of Computer Science and Engineering, Koneru Lakshmaiah Education Foundation,

Green Fields, Vaddeswaram, Andhra Pradesh, India

e-mail: ashwini.apply@gmail.com

A. K. Pradhan ·K. Das ·D. Mishra

Department of Computer Science and Engineering, Siksha ‘O’ Anusandhan (Deemed To Be

University), Bhubaneswar, Odisha, India

e-mail: kaberidas@soa.ac.in

D. Mishra

e-mail: debahutimishra@soa.ac.in

S. Mishra et al. (eds.), Augmented Intelligence in Healthcare: A Pragmatic and Integrated

Analysis, Studies in Computational Intelligence 1024,

https://doi.org/10.1007/978-981-19- 1076-0_5

78 A. K. Pradhan et al.

classiﬁcation. There are two phases in medical image classiﬁcation (a) Training (b)

Testing. In the training phase, the extracted features from the images are given to a

classiﬁer with output labels which will develop a model. The performance evaluation

of the model is carried out by the testing phase for unknown sat of image dataset

where labels are not supplied to a classiﬁer. So, for better classiﬁcation accuracy, we

should focus on the right representation of the image.

In this work, we were reviewed and compared different techniques required for

image classiﬁcation such as imaging modalities, feature extraction and selection

along with its pros and cons.

2 Imaging Modalities Techniques

To know whether a disease is aggressive or more aggressive, the diagnosis and

analysis are required by doctors and researchers. There are mainly four different types

of imaging modalities such as; Computed Tomography (CT), Magnetic Resonance

Imaging (MRI), Ultrasound, and X-rays are required for generating medical images

[1]. The merits and demerits of different imaging modalities techniques are described

in Table 1. Different imaging modalities of chest are also represented in Fig. 1.Due

to high resolution of Magnetic Resonance Imaging (MRI) modality, it is chosen as

best source of information for brain and chest study.

Tabl e 1 Various imaging modalities techniques

Modalities Developed by and year Pros and cons

Computed tomography (CT) British engineer Godfrey

Hounsﬁeld in 1972

Improves the contrast level

Takes a maximum of 30 min. for

a total body scan

DNA may damage by radiation

Electromagnetic radiation may

lead to cancer

Magnetic resonance imaging

(MRI)

Peter Mansﬁeld in 1977 Radiation is not used by MRI

Provides detail and a clear

image of soft tissue

All cancers are not found by

MRI

It is too expensive

Ultrasound imaging Ian Donald in 1970 Ultrasound machines are very

simple and less size

Less expensive

Low resolution

X-rays imaging Wilhelm Roentgen in 1895 Image quality is medium

Dangerous for bone cancer

Machine Learning Techniques in Medical Image: A Short Review 79

(a) CT scan of chest (b) MRI scan of chest

Fig. 1 Different types of imaging modalities for chest

3 Image Pre-processing Techniques

To enhance the quality of the image, pre-processing is required. Different techniques

are described below:

3.1 Enhancement Phase

Here the image attributes are changed. It consists of a grayscale technique, binary

conversion technique, and contrast stretching technique [2].

3.1.1 Grayscale Technique

To reduce the complexity of the RGB (Red, Green, and Blue) image, it changes to

a grayscale image. When the computer adds red, green, and blue colors in different

proportions, a color image is created which is stored in an array. But in the case of

grayscale images, the machine stores only intensity values of black and white color

which is stored on a single array. For image pre-processing, it is not mandatory to

add this technique because maximum data may be lost.

There are different methods used for converting an RGB image to gray image.

•Luminosity method: The formula for calculating Grayscale value is as follows

80 A. K. Pradhan et al.

Grayscale =0.299R +0.587G +0.114B (1)

•Average Method: It will take the average value of R, G, and B to compute the

Grayscale value. The formula for average method is as follows

Grayscale =(R+G+B)/3(2)

3.1.2 Binary Conversion Technique

Local binary pattern (LBP): Here the local spatial structure in an image is described

which is used for texture classiﬁcation. It is the comparison between center pixel

intensity with its surrounding pixel intensity values. Due to its low computational

cost, it is very popular in pattern reorganization in the medical ﬁeld [3]. Histograms

(H) of an image are created by collecting the LBP codes from images. It is calculated

using the below formula

Hi=

x,y

I{fl(x,y)=i},i=0,1,...n−1,(3)

where (x,y) is LBP neighborhood operator and different labels of LBP operator

represented by n. The function I{X}=1, if Xis true else 0.

3.1.3 Contrast Stretching Technique

It is used for normalization of the image where the range of grayscale increases means

intensity range spread over all the image size. For brightening the intensity, Gamma

transformation or logarithm transformation are used. It transforms high contrast

images to low contrast images because there is a very small difference between dark

intensity and light intensity values [4].

3.2 Noise Removal Phase

In this phase, the unwanted information can be removed with the help of ﬁlters such

as Robert and Prewitt ﬁlters, Median ﬁlters, and Laplacian ﬁlters. Weighted Median

Filter (WMF) is used to reduce noise, blur when it passes over the neural network

[5]. It provides more weightage to middle position of each window. In this technique,

various coefﬁcients are multiplied with intensity values and replace the value of the

pixel with the median of its neighboring pixel value. It is so effective due to its

excellent impulse noise reduction capacity and less blurring of an image.

Machine Learning Techniques in Medical Image: A Short Review 81

In another communication, the author [6] uses an Adoptive weighted median ﬁlter

(AWMF) to remove speckles. Here the weight coefﬁcient is adjusted instead of ﬁxed

weight where the window size is large.

4 Feature Extraction Techniques

The characteristics of the image or object are called a feature. This technique extracts

the most important feature from the image and assigns them to a classiﬁer to distin-

guish a different pattern. In this section, we compared and reviewed texture and shape

feature extraction techniques.

4.1 Texture Feature Extraction Technique

It is a low-label feature in an image that consists of smoothness, coarseness, and

regularity. In the case of the medical image, it refers to the appearance and structure

of objects in an image for clinical practice. Though most medical images are 2D

digital images, this technique tries to evaluate the position and intensity of gray level

intensity. There are two types of methods i.e. Gray Level Co-occurrence Matrix

(GLCM), and Multilevel Discrete Wavelet Transform (DWT) are mainly used in the

medical image [7].

4.1.1 Gray Level Co-occurrence Matrix (GLCM)

This technique allows to extract the information regarding a pair of pixels such as

the distance between them with the horizontal and vertical direction. Contrast and

entropy parameters are calculated by this technique. Using GLCM, different statistics

are calculated such as; Contrast, Correlation, Energy, and Homogeneity.

4.1.2 Multilevel Discrete Wavelet Transform (DWT)

This technique analyzes the frequency and time domain of an image with multiple

scales. Lots of signals are found through this method which helps the ﬁlters to extract

more components.

82 A. K. Pradhan et al.

Tabl e 2 Classiﬁcation

results by different authors

used different feature

extraction techniques

Techniques Authors Classiﬁcation accuracy

(%)

ED Xiaogian Xu, et al.

2008

Ramamurthy, 2011

ZM Calebi, 2016

Fesharaki, 2017

FD Rajaei et al., 2015

Oberoi and Singha,

2017

LBP Ko, kim and Nam,

2011

Reza Zara, 2013

GLCM Raju and Modi, 2011

Alvarenga et al.,

2007

DWT Guler et al.

Weng et al.

73.40

97.15

4.2 Shape Feature Extraction Technique

This technique is used to extract the circular shape, triangular shape, or diameter

of objects in the image. It is up to two types 1. Boundary-based feature extraction

two. Region-based feature extraction. It includes a Contour-based method such as

Fourier Descriptors (FD), Edge Detection (ED), and Zernike Moments (ZM). Table 2

describes and summarized the different extraction techniques with their classiﬁcation

accuracy proposed by different authors [8].

5 Feature Reduction Techniques

It comes after feature extraction where the dataset size has been minimized. It includes

techniques such as Linear Discriminate Analysis (LDA), Principal Component Anal-

ysis (PCA), and GA (Genetic Algorithms). LDA reduces the feature by preserving

maximum data (“without losing data”) which will increase the ratio between-class

variance and within-class variance. It will work better for larger datasets and multi-

class classiﬁcation. PCA will work better when the number of samples per dataset

is low. Table 3describes and summarized the different feature reduction techniques

with their description proposed by different authors [9–15].

Machine Learning Techniques in Medical Image: A Short Review 83

Tabl e 3 Different feature reduction techniques with their description

Feature reduction techniques Description

PCA +KSVM [9]PCA reduced 65,536 to 1024 feature vectors.

DWT +PCA +KSVM with GRB kernel

achieved best accurate classiﬁcation result of

99.38% as compared with IPOL and HPOL

kernel

GLCM, PCA, and SVM using RBF kernel

function [10]

Feature extracted by using GLCM and

classiﬁed with RBF kernel provides 100%

classiﬁcation accuracy than PCA

DWT, PCA, k-mean clustering, and k-nearest

neighbor classiﬁer [11]

Seven statistical measures including skewness,

kurtosis, speciﬁcity, etc. are measured

GLCM (gray level co-occurrence matrix) and

SVM [12]

Texture-based feature selection using GLCM

and SVM combination has been provided

Wavelet-based PCA with fuzzy C-means

clustering [13]

PCA based fuzzy C-means clustering system

yield more and accurate information about

abnormal tissues than conventional PCA

LDA, PCA, and SVM [14]LDA selects vital features than PCA and

achieved accuracy of 98.87%

GLCM, KNN, ANN, PCA +LDA [15]GLCM, PCA +LDA combination is superior

to others in terms of reducing the dimension of

features

6 Image Classiﬁcation Techniques

In this section, we discuss four different types of classiﬁcation techniques with their

pros and cons (Table 4).

7 Conclusion and Future Scope

In this paper, different medical image classiﬁcation techniques such as image modal-

ities, pre-processing, feature extraction and reduction, and different classiﬁer with

their combination has been analyzed by various authors. This work will also assist in

selecting the best technique for classiﬁcation purposes. Out of all feature extraction

techniques, it can be concluded that the edge detection technique provides the lowest

classiﬁcation accuracy than the texture extraction method. From the literature work,

we found that PCA is a widely used method for feature reduction. In the future, we

planned to survey existing nature-inspired optimization algorithms that will optimize

the parameters of classiﬁer architecture to provide better accuracy.

84 A. K. Pradhan et al.

Tabl e 4 Different image classiﬁcation techniques with pros and cons

Different classiﬁers

and developer

Illustration Pros Cons

Bayes—classiﬁcation

(1763, Thomas Bayes)

Based on probability

theory

Very less data is

required for training

Variable selection is

difﬁculty task

Neural network

classiﬁers (1944

by Warren

McCullough and

Walter Pitts)

Inspired by human

nervous system.

Example: ANN,

RBFN, RNN, etc.

Gives better results in

complex domains

Training slow,

computational

complexity is more

Decision tree (1959,

British

researcher, William

Belson)

It is a node-like

structure. Inner node

represents a test on

attributes and leaf

node denotes class

labels

Example: ID3, C4.5

No need for domain

knowledge. Easy to

learn and procedure is

so simple

Not suitable for

practical

implementation

Support vector

machines (1963,

Vladimir N. Vapnik

and Alexey

Ya. Chervonenkis)

It is a binary classiﬁer

that uses kernel

function to ﬁnd out a

best boundary

For non-linear data,

provides better results

More memory is

needed

References

1. Othman, M. F. B., Abdullah, N., & Rusli, N. A. B. A. (2010). An overview of MRI brain

classiﬁcation using FPGA implementation. In IEEE Symposium on Industrial Electronics and

Applications (ISIEA) (pp. 623–628).

2. Miranda, E., Aryuni, M., & Irwansyah, E. (2016, November). A survey of medical image classi-

ﬁcation techniques. In International Conference on Information Management and Technology

(ICIMTech) (pp. 56–61).

3. Mishra, S., Thakkar, H., Mallick, P. K., Tiwari, P., & Alamri, A. (2021). A sustainable

IoHT based computationally intelligent healthcare monitoring system for lung cancer risk

detection. Sustainable Cities and Society, 103079.

4. Anitha, S., & Radha, V. (2010). Comparison of image preprocessing techniques for textile

texture images. International Journal of Engineering Science and Technology, 2, 7619–7625.

5. Bauer, S., Nolte, L. P., & Reyes, M. (2011, September). Fully automatic segmentation of

brain tumor images using support vector machine classiﬁcation in combination with hierar-

chical conditional random ﬁeld regularization. In International Conference on Medical Image

Computing and Computer-Assisted Intervention (pp. 354–361). Springer, Berlin, Heidelberg.

6. Tripathy, H. K., Mishra, S., Thakkar, H. K., & Rai, D. (2021). CARE: A collision-aware

mobile robot navigation in grid environment using improved breadth ﬁrst search. Computers

and Electrical Engineering, 94, 107327.

7. Pradhan, A., Mishra, D., Das, K., Panda, G., Kumar, S., & Zymbler, M. (2021). On the

classiﬁcation of MR images using “ELM-SSA” coated hybrid model. Mathematics, 9.

8. Ping Tian, D. (2013). A review on image feature extraction and representation techniques.

International Journal of Multimedia and Ubiquitous Engineering, 8, 385–396.

9. Zhang, Y., & Wu, L. (2012). An MR brain images classiﬁer via principal component analysis

and kernel support vector machine. Progress in Electromagnetic Research, 130, 369–388.

Machine Learning Techniques in Medical Image: A Short Review 85

10. Singh, D., & Kaur, K. (2012). Classiﬁcation of abnormalities in brain MRI images using GLCM,

PCA, and SVM. International Journal of Engineering and Advanced Technology (IJEAT), 1.

11. Takate, V. S., & Vikhe, P. S. (2012). Classiﬁcation of MRI brain images using K-NN and

k-means. International Journal on Advanced Computer Theory and Engineering (IJACTE), 1.

12. Tripathy, H. K., Mallick, P. K., & Mishra, S. (2021). Application and evaluation of classiﬁcation

model to detect autistic spectrum disorders in children. International Journal of Computer

Applications in Technology, 65(4), 368–377.

13. Mishra, S., Tripathy, H. K., Mallick, P. K., Bhoi, A. K., & Barsocchi, P. (2020). EAGA-MLP—

an enhanced and adaptive hybrid classiﬁcation model for diabetes diagnosis. Sensors, 20(14),

4036.

14. Bhaiya, L. P., & Verma, V. K. (2012). Classiﬁcation of MRI brain images using neural network.

International Journal of Engineering Research and Applications (IJERA).

15. Jafarpour, S., Sedghi, Z., & Amirani, M. C. (2012). A robust brain MRI classiﬁcation with

GLCM features. International Journal of Computer Applications.

Analysis of Diabetic Retinopathy

Detection Techniques Using CNN Models

P. Prabhavathy, B. K. Tripathy, and M. Venkatesan

Abstract Diabetic Retinopathy (DR) is an eye disease and is caused by changes

in retinal blood vessels. It is common in diabetes patients. Severity level of DR

classiﬁed based on changes in the DR-lesions in the retina. Microaneurysms (MAs)

is inﬂammation of walls of blood vessels inside the retina, and is an early indi-

cator of Diabetic Retinopathy. In the early stages, detecting Diabetic retinopathy

is difﬁcult, even for an experienced expert diagnostic procedure can be hard. So, a

Computer-Aided Diagnosis (CAD) method should be proposed to automate the diag-

nosis of Diabetic Retinopathy by classifying fundus images into ﬁve severity classes

to provide appropriate suggestions to Diabetic Retinopathy patients. We have taken

the APTOS and Kaggle Diabetic Retinopathy dataset to train the models and to detect

the severity levels of Diabetic Retinopathy. In this chapter, DenseNet, EfﬁcientNet

and Convolution Neural Network models are used to detect Diabetic Retinopathy.

Keywords Diabetic Retinopathy ·Fundus images ·CNN ·DenseNet ·

EfﬁcientNet ·Accuracy ·Precision ·Recall ·F1-score

1 Introduction

Diabetic Retinopathy (DR) is an eye disease and is caused by changes in retinal

blood vessels. It is common in diabetes patients. According to the World Health

Organization (WHO), at least 2.2 billion people have vision loss globally, of whom

at least 1 billion have a vision impairment that could have been prevented or addressed

[16]. This 1 billion people includes those with moderate or severe distance vision loss

or vision impairment due to unaddressed presbyopia (826 million), refractive error

P. Prabhavathy ·B. K. Tripathy (B)

School of Information Technology and Engineering, VIT, Vellore, Tamil Nadu 632014, India

e-mail: tripathybk@vit.ac.in

P. Prabhavathy

e-mail: pprabhavathy@vit.ac.in

M. Venkatesan

Department of Computer Science Engineering, NIT Puducherry, Puducherry, India

S. Mishra et al. (eds.), Augmented Intelligence in Healthcare: A Pragmatic and Integrated

Analysis, Studies in Computational Intelligence 1024,

https://doi.org/10.1007/978-981-19- 1076-0_6

88 P. Prabhavathy et al.

(123.7 million), cataract (65.2 million), glaucoma (6.9 million), corneal opacities

(4.2 million), diabetic retinopathy (3 million) and blinding trachoma (2 million)

[16].

Severity level of DR classiﬁed based on changes in the DR-lesions in the retina

[18]. The common lesions that indicate DR include microaneurysms (is inﬂamma-

tion of walls of blood vessels inside the retina), hard exudates (formed when microa-

neurysms or walls of capillaries become fragile and burst), hemorrhages (protein

leakage from the blood vessel) and soft exudates or cotton wool spots (leakage of

blood vessels block the vessels) [9,14,17,24,25]. DR is divided into two classes

namely non-proliferative DR (NPDR) and proliferative DR (PDR).

NPDR is caused when inside the retina blood vessels are damaged resulting in

the leakage of blood or ﬂuid and PDR is caused when inside the retina blood vessels

are blocked [14]. NPDR is further divided into three sub-stages: mild, moderate and

severe NPDR. Together, these ﬁve stages make up the widely used ‘International

Clinical Diabetic Retinopathy Disease Severity Scale’ [11]. Microaneurysms appear

in mild NPDR stage. Ruptured microaneurysms cause the appearance of hemor-

rhage and exudates are present in moderate NPDR. Number of hemorrhages and

exudates gets bigger in the severe NPDR and circulation of blood vessels experiences

a lack of oxygen and causes new blood vessels in proliferative diabetic retinopathy

(PDR) [13,20]. Fundus images are the most commonly used approach to diagnose

disease. Manual screening of fundus images takes time to detect and evaluate DR

and sometimes it is error prone.

In recent decades, Neural networks (NNs) [22] are giving good performance

results in areas like image classiﬁcation, image segmentation, speech processing,

natural language processing, etc. The deep architecture is framed to learn features in

a hierarchical manner with feature at a level being formed by composition of features

at lower levels [5]. It is an auto learning procedure which allows learning complex

functions as outputs from input data. The number of levels where compositions are

performed using many non-linear operations determine the depth. This is in line with

the brain of mammals which follow a deep architecture. This inspired the scientists

from the ﬁeld of neural networks to develop deep neural networks (DNNs). Variants

of DNNs have inﬂuenced research in AI [3]. There are several applications of DNNs

to Audio Signal Classiﬁcation [6], in health care [12], Age and gender Estimation

[7], in the ﬁeld of Sentiment Analysis [23] and in image processing [1]. A survey of

many such applications can be found in [2].

Convolutional neural Networks (CNN) are DNNs which are structured upon

the human visual system [15]. These models use a mathematical operator called

convolution. CNNs have been applied for handwritten character recognition Pattern

recognition systems based upon CNN are the best possible systems developed so far.

In this chapter, we have taken APTOS and Kaggle Diabetic Retinopathy

Dataset and DenseNet, EfﬁcientNet and CNN models are used to classify Diabetic

Retinopathy.

Analysis of Diabetic Retinopathy Detection Techniques Using CNN … 89

2 Related Work

Diabetic retinopathy is getting difﬁcult to detect in the early stages, because mild

NPDR shows minute changes in the retina. Detecting microaneurysms is difﬁcult in

some cases by the bare eye. In recent years, deep learning models have been success-

fully used to automate the diagnosis of a disease. Qummar et al. [19] proposed an

approach which ensembles ﬁve deep CNN models, they are Inceptionv3, Resnet50,

Xception, DenseNet—121 and DenseNet—169 to classify ﬁve stages of DR (No

DR, mild NPDR, moderate NPDR and severe NPDR and PDR). They used Kaggle

Diabetic Retinopathy Dataset and Resized, cropped, mean normalized and rotated

images to pre-process data. Data is a perfectly imbalanced dataset, did upsampling

and downsampling on the dataset. The accuracy achieved by imbalanced, upsam-

pling and downsampling dataset by using SGD optimizer is 70%, 60% and 51%

respectively and by using Adam optimizer is 55%, 57% and 51% respectively.

Gao et al. [8] they are grading images based on its abnormalities in the retina

and treatment required to diagnose disease. They collected 4476 images from three

clinical departments in Sichuan Provincial People’s Hospital. They labeled fundus

images with 4 classes: Normal, mild or moderate NPDR, severe NPDR or mild PDR

and PDR. Trained on Inception-v3 model with 299 ×299 pixels. When images are

resized to small size, small lesions in the image get reduced. It makes it hard to

detect lesions. So, they proposed a new model, resized fundus images to 600 ×600

pixels and then they cut each image into four 300 ×300 pieces and feed each piece

to four different Inception-V3 models. The accuracy achieved by Inceptionv3 and

Inception@4 are 88.35% and 88.72% respectively.

Arora and Pandey [4] proposed a CNN model is applied on Kaggle Diabetic

Retinopathy Dataset. They experimented on 1000 images and considered ﬁve classes.

They achieved 74.58% training and 74.4% testing accuracy on the proposed model.

Wu and Hu [26] they used pre-trained models such as VGG19, ResNet50 and

Inception-V3. They used Kaggle Diabetic Retinopathy Dataset and oversampled

data to balance the dataset. The performance of VGG19, ResNet50 and Inception-V3

models are 51%, 49% and 61% respectively.

Mishra et al. [17] proposed a deep supervision of the Inception-Residual network

to classify DR images. They took Kaggle Diabetic Retinopathy dataset and applied

preprocessing. They achieved 80.6% accuracy on the proposed model. Zeng et al.

[27], in this paper, instead of giving a single eye as input, they are feeding both eyes

as inputs and giving output as classiﬁcation of each eye. They used Kaggle Diabetic

Retinopathy dataset. These images have been labeled by 5 classes. In this paper they

considered 2 classes. Fundus images with labels of 0 and 1 are classiﬁed as without

RDR (Referable DR) and 2, 3 and 4 are classiﬁed as with RDR. They got 82.2%

sensitivity and 70.7% speciﬁcity.

Some of the above classiﬁcation works are combining severity levels of one or

more classes into one class. First indicator of DR is Microaneurysms which can be

found in the mild NPDR stage. Combining mild and moderate NPDR makes it unable

to detect the DR at early stages. Combining later NPDR classes with PDR class makes

90 P. Prabhavathy et al.

it unable to detect DR at later stages. We balanced the dataset to improve the classiﬁ-

cation accuracy because the imbalanced dataset has bias toward the majority classes.

In this paper, considered ﬁve stages of DR and used a DenseNet and EfﬁcientNet

Convolution Neural Network models to better classify the Diabetic Retinopathy.

3 Methodology

Convolutional Neural Network (CNN) is an artiﬁcial neural network used to analyze

images and text data. CNNs are made up of neurons with learn-able biases and

weights. Every neuron in the network receives nnumber of inputs, then calculates a

weighted sum over them and then passes it through activation function to get output.

CNNs contain the layers, an input, dense and a hidden layer that includes multiple

convolution layers, pooling layers and batch normalization layers.

3.1 Convolution Layer

Convolution layers make use of ﬁlters (also known as kernels) to detect features in

an image. A ﬁlter is a matrix of weights that moves on parts of the image to detect

speciﬁc features. Convolution operation provides a value which is how conﬁdent that

a speciﬁc feature is to present. Convolution operation is an element-wise product and

sum of the two matrices. The convolution operation calculated using following Eq. 1

nout =(nin −f)

x+1(1)

To introduce non-linearity, Activation function used is Rectiﬁed Linear Unit

(ReLU). Mathematical Eq. 2for ReLU

f(y)=0,y<0;

y,y≥0.(2)

3.2 Batch Normalization Layer

Batch normalization is a method used to normalize the inputs of each layer, in

order to resolve the internal Covariate shift problem. Covariate shift problem occurs

when distribution of input variables is different in datasets of training and testing.

Mathematically, covariate shift problem occurs if

Analysis of Diabetic Retinopathy Detection Techniques Using CNN … 91

ptrain(x)= ptest(x)(3)

and

(y/x)= ptest (x)(4)

where xis a feature.

3.3 Pooling Layer

Pooling operation is a vector to scalar transformation that operates on each local part

of an image, they do not have ﬁlters and do not compute dot products with the local

part, instead they compute the average of the pixels in the region (Average Pooling),

pick the pixel with the highest intensity value and discard the rest (Max Pooling) and

pick the pixel with the lowest intensity and discard the rest (Min Pooling).

3.4 Dense Layer

The dense layer of a CNN produces probability for each class. To obtain these

probabilities, initialize the ﬁnal Dense layer to the same number of neurons as the

number of classes we need. The dense layer outputs are passed through the Softmax

activation function, which maps outputs of dense layer to a vector where sum of the

elements are sum up to one. Equation 5is the mathematical formula for SoftMax

σ(x)j=exj

K

k=1exk

(5)

where xis a vector of the inputs to the output layer and jindices of the output. Variant

CNN models have been proposed. We used EfﬁcientNet and DenseNet models.

3.5 EfﬁcientNet

Scaling up Convolutional neural networks are a widely used approach for better

accuracy. Common ways to scale up a network by depth (d)orwidth(w)or

by image resolution. The EfﬁcientNet model, balances all dimensions of network

depth/width/image resolution with constant ratio to improve accuracy [21]. Depth

(d), deeper CNNs can capture richer and more complex features, and generalize well

92 P. Prabhavathy et al.

on new problems. Width (w), wider networks are able to capture more ﬁne-grained

features and easier to train. Image resolution (r), high resolution images, CNNS

potentially capture more ﬁne-grained patterns.

For higher resolution images, increasing network depth can help to capture

features like bigger size images. And increasing width helps to capture more

ﬁne-grained patterns.

Compound coefﬁcient Φto uniformly scale network width, depth, and resolution

in a balanced way

depth;d=αφ

width;w=βφ

resolution;r=γφ

s.t,α·β2·γ2≈2

α≥1,β≥1,γ ≥1

(6)

Here α, β and γare constants. By choosing different φvalues there are 7 variants

in EfﬁcientNet B0–B7. The model used is EfﬁcientNet-B0 where α,β and γare 1.

The architecture of the EfﬁcientNet-B0 is shown in Table 1.

DenseNet improves information ﬂow between layers. The key characteristic of

DenseNet model is direct connection from preceding layers to subsequent layers [10].

Each layer in the dense block, receives feature maps from all of its previous layers as

inputs and its feature maps passed as inputs to the subsequent layers. DenseNets are

divided into Dense blocks, within a block dimension of feature maps are constant.

Mathematical equation of DenseNet connectivity Eq. 7.

x=Hx0,x1, ... x−1(7)

Tabl e 1 EfﬁcientNet-b0

architecture Operator Channels Layers

Conv3 ×332 1

MBConv1, k3 ×316 1

MBConv6, k3 ×324 2

MBConv6, k5 ×540 2

MBConv6, k3 ×380 3

MBConv6, k5 ×5112 3

MBConv6, k5 ×5192 4

MBConv6, k3 ×3256 1

Conv1 ×1, Pool, FC 512 1

Analysis of Diabetic Retinopathy Detection Techniques Using CNN … 93

Tabl e 2 DenseNet-121

architecture Layers DenseNet-121 (k=32)

Conv 7×7conv, stride =2

Pool 3×3max pool, stride =2

Dense block 1 [1 ×1conv,3 ×3conv] * 6

Transition 1 1×1conv, 2 ×2avg pool, stride =2

Dense block 2 [1 ×1conv,3 ×3conv] * 12

Transition 2 1×1conv, 2 ×2avg pool, stride =2

Dense block 3 [1 ×1conv,3 ×3conv] * 24

Transition 3 1×1conv, 2 ×2avg pool, stride =2

Dense block 4 [1 ×1conv,3 ×3conv] * 16

Classiﬁcation layer 7×7global avg pool, 512FC, Softmax

where x0,x1, ... x−1are concatenation of the feature maps produced in layers. H(.)

is made up of three operations: Batch Normalization (BN), ReLU and Convolution

layer.

Between the dense blocks transition layer introduced to down sample the layer that

reduces the size of feature maps. Transition layer consists of convolution followed

by an average pooling layer. Concatenating feature maps increase channel dimension

at every layer. To produce kfeature maps in each layer a hyper parameter is used

that is growth rate (k). Before each 3 ×3 convolution, to reduce the number of input

feature-maps to improve computational efﬁciency 1 ×1 convolution is introduced

as a bottleneck layer. To further improve compactness of model, number of feature-

maps at transition layers are reduced by using compression (φ) factor, 0 ≤φ≤1

[10]. The architecture of the DenseNet-121 is shown in Table 2.

4 Experimentation Results

4.1 Data Set

The dataset used in this paper is taken from Kaggle Diabetic Retinopathy compe-

tition website provided by Asia Paciﬁc Tele-Ophthalmology Society (APTOS)

contains 3662 high resolution fundus images and some images from Kaggle Diabetic

Retinopathy Detection dataset. Based on severity of DR-lesions present in the retina,

images are labeled with a scale of 0–4. According to the scale No DR—0, mild

NPDR—1, moderate NPDR—2, severe NPDR—3 and PDR—4. The distribution of

data is shown in Table 3

94 P. Prabhavathy et al.

Tabl e 3 Distribution of

dataset information Dataset APTOS APTOS +Kaggle Balanced APTOS +

Kaggle

class 0 1805 1805 950

class 1 370 1020 950

class 2 999 1020 950

class 3 193 1034 950

class 4 295 967 950

4.2 Pre-processing

In Data Pre-processing, APTOS and Kaggle dataset are having each image of huge

size. Resizing an image into smaller size, Pixel normalization, pixel values of images

are converted from [0, 255] to [0, 1], Crop to square of desired image, images are

having black part in each image. So, removing black parts from an image and cropping

it to the desired image which contains information makes training faster and applied

Gaussian blur on images which suppresses the high frequency information (noise,

edges), while preserving the low-frequency information of the image. The original

image is shown in Fig. 1and processed image is shown in Fig. 2.

Fig. 1 Original image

Fig. 2 Processed image

Analysis of Diabetic Retinopathy Detection Techniques Using CNN … 95

5 Results

Accuracy, Precision, Recall, F1-score, Speciﬁcity, ROC and AUC are used as

performance metrics to evaluate performance of the model.

Mathematical formula for Accuracy

Accuracy =tp +tn

tp +tn +fp +fn (8)

Mathematical formula for Precision

Precision =tp

tp +fp (9)

Mathematical formula for Recall/Sensitivity

Recall =tp

tp +fn (10)

Mathematical formula for Speciﬁcity

Speciﬁcity =tn

tn +fp (11)

F1-score is the weighted harmonic mean of precision and recall

F1−score =2∗precision ∗recall

precision +recall (12)

where tp (true positive) is the number of samples correctly classiﬁed under the class,

tn (true negative) is the number of samples correctly classiﬁed in rest of the classes,

fp (false positive) is the number of samples incorrectly classiﬁed in rest of the classes

and fn (false negative) is the number of samples incorrectly classiﬁed under the class.

The Receiver Operating Characteristic (ROC) curve is a graphical plot, plotted

with True Positive Rate (TPR) on y-axis and False Positive Rate (FPR) on the x-axis.

Area Under the Curve (AUC) provides the performance measure of classiﬁcation

of different classes.

Loss function used in this experiment is categorical cross entropy.Cross

entropy loss measures the performance of the classiﬁcation model whose output

is a probability between 0 and 1. Mathematical formula,

−M

c=1yi,clog Pi,c(13)

96 P. Prabhavathy et al.

Tabl e 4 Distribution of test

dataset of EfﬁcientNet Dataset APTOS +Kaggle Balanced APTOS +Kaggle

class 0 351 185

class 1 205 199

class 2 206 192

class 3 210 197

class 4 194 177

where Mdenotes classes, yi,cis an indicator of binary, Pis predicted probability and

cis true class.

The dataset we have used is APTOS and Kaggle Diabetic Retinopathy dataset. We

have taken the whole dataset of APTOS. From Kaggle dataset we took all images of

class 3 and class 4 and added a few numbers of images from the remaining classes in

APTOS +Kaggle dataset balanced the number of images in each class in Balanced

APTOS +Kaggle dataset.

The distribution of Test dataset of APTOS +Kaggle and Balanced APTOS +

Kaggle of EfﬁcientNet is shown in Table 4. The Confusion matrix of APTOS +

Kaggle dataset on EfﬁcientNet is shown in Table 5and accuracy is 89.28%, recall,

precision, speciﬁcity and f1-score of all classes of DR are shown in Table 6.Efﬁ-

cientNet model classiﬁed all ﬁve stages of Diabetic retinopathy with good values.

And also calculated the ROC on EfﬁcientNet which is shown in Fig. 3. ROC plot

shows performance of classiﬁcation of classes. AUC 0.99 for class 0 is the highest,

which means model predicts well for class 0. It also had above 0.90 on class 1, 2 and

4 and class 3 has AUC value of 0.88.

Tabl e 5 Confusion matrix of APTOS +Kaggle dataset on EfﬁcientNet

Label class 0 class 1 class 2 class 3 class 4

class 0 347 2 1 0 1

class 1 2181 315 4

class 2 115 174 11 5

class 3 016 1169 24

class 4 0 3 10 11 170

Tabl e 6 Performance measure of each class of APTOS +Kaggle dataset on EfﬁcientNet

Label Recall Precision Speciﬁcity F1-score

class 0 0.99 0.99 0.99 0.99

class 1 0.88 0.83 0.96 0.86

class 2 0.84 0.92 0.98 0.88

class 3 0.80 0.82 0.96 0.81

class 4 0.88 0.83 0.97 0.85

Analysis of Diabetic Retinopathy Detection Techniques Using CNN … 97

Fig. 3 Roc curve of APTOS

+Kaggle dataset on

EfﬁcientNet

We balanced the dataset because the network is biased toward high-occurrence

classes than the low-occurrence classes. The Confusion matrix of Balanced APTOS

+Kaggle dataset on EfﬁcientNet is shown in Table 7and accuracy is 89%, recall,

precision, speciﬁcity and f1-score of all classes are shown in Table 8. On a balanced

dataset, model classiﬁed all ﬁve stages of DR with good values than APTOS +

Kaggle dataset. ROC curve on EfﬁcientNet is shown in Fig. 4.

The highest AUC is 0.99 for class 0, which means model predicts well for class

0. It also has above 0.91 in class 1, 2, 3 and 4.

Tabl e 7 Confusion matrix of balanced APTOS +Kaggle dataset on EfﬁcientNet

Label class 0 class 1 class 2 class 3 class 4

class 0 173 9 3 0 0

class 1 2175 515 2

class 2 114 168 2 7

class 3 015 6167 9

class 4 1 7 5 8 156

Tabl e 8 Performance measure of each class of balanced APTOS +kaggle dataset on EfﬁcientNet

Label Recall Precision Speciﬁcity F1-score

class 0 0.94 0.98 0.99 0.96

class 1 0.88 0.80 0.94 0.84

class 2 0.88 0.90 0.97 0.89

class 3 0.85 0.87 0.98 0.86

class 4 0.88 0.90 0.98 0.89

98 P. Prabhavathy et al.

Fig. 4 ROC curve of

balanced APTOS +

KAGGLE dataset on

EfﬁcientNet

The distribution of Test dataset of APTOS +Kaggle and Balanced APTOS +

Kaggle of DenseNet is shown in Table 9. The Confusion matrix of APTOS +Kaggle

dataset on DenseNet is shown in Table 10 and accuracy is 72%, recall, precision,

speciﬁcity and f1-score of all classes are shown in Table 11. And also calculated

the ROC on DenseNet which is shown in Fig. 5. Roc plot shows performance of

classiﬁcation of classes. AUC 0.97 for class 0 is the highest, which means model

predicts well for class 0. It also had above 0.80 on class 1 and 2, 0.73 on class 3 and

0.65 on class 4.

Tabl e 9 Distribution of test

dataset of DenseNet Dataset APTOS +Kaggle Balanced APTOS +Kaggle

class 0 366 189

class 1 230 176

class 2 215 200

class 3 189 190

class 4 166 195

Tabl e 10 The confusion matrix of APTOS +Kaggle dataset on DenseNet

Label class 0 class 1 class 2 class 3 class 4

class 0 360 5 0 1 0

class 1 17 162 25 23 3

class 2 927 163 11 5

class 3 340 26 104 16

class 4 524 30 54 53

Analysis of Diabetic Retinopathy Detection Techniques Using CNN … 99

Tabl e 11 Performance measure of each Class of Aptos +Kaggle Dataset on DenseNet

Label Recall Precision Speciﬁcity F1-score

class 0 0.98 0.91 0.96 0.95

class 1 0.70 0.63 0.90 0.66

class 2 0.76 0.67 0.91 0.71

class 3 0.55 0.54 0.91 0.54

class 4 0.32 0.69 0.98 0.44

Fig. 5 ROC curve of

APTOS +Kaggle dataset on

DenseNet

The Confusion matrix of Balanced APTOS +Kaggle dataset on DenseNet is

shown in Table 12 and accuracy is 69%, recall, precision, speciﬁcity and f1-score

of all classes are shown in Table 13. ROC plot on DenseNet is shown in Fig. 6.The

highest

AUC is 0.96 for class 0, which means model predicts well for class 0. It also had

above 0.73 on class 1, 2 and class 3 and 0.64 on class 4.

In the above experimentation results EfﬁcientNet model has better performance

than the DenseNet model. Compared to DenseNet, EfﬁcientNet has lesser parameters

Tabl e 12 Confusion matrix of balanced APTOS +Kaggle dataset on DenseNet

Label class 0 class 1 class 2 class 3 class 4

class 0 179 6 2 1 1

class 1 7132 19 16 2

class 2 733 153 3 4

class 3 250 20 106 12

class 4 848 30 47 62

100 P. Prabhavathy et al.

Tabl e 13 Performance measure of each class of balanced APTOS +Kaggle dataset on DenseNet

Label Recall Precision Speciﬁcity F1-score

class 0 0.95 0.88 0.97 0.91

class 1 0.75 0.49 0.82 0.59

class 2 0.77 0.68 0.91 0.72

class 3 0.56 0.61 0.91 0.58

class 4 0.32 0.77 0.97 0.45

Fig. 6 ROC curve of

balanced APTOS +

KAGGLE dataset on

DenseNet

and runs faster. DenseNet was over ﬁtting after a certain number of epochs when

we were training model. We have applied data augmentation, l1l2 regularization,

drop out and even tried changing batch size values to avoid overﬁtting. Even reduced

complexity of model to see the performance of model but it has no improvement on

the performance. But EfﬁcientNet has shown better performance than the DenseNet

model on both the datasets.

6 Conclusions

Diabetic Retinopathy damages the blood vessels of the retina; it is the most common

disease for people who are having diabetes. Lack of detection of Diabetic Retinopathy

in the early stages may lead to loss of vision. Therefore, early detection of Diabetic

Retinopathy is required to save the vision. We considered ﬁve DR stages, those

are NO DR, mild NPDR, moderate NPDR, severe NPDR and PDR. Classiﬁca-

tion of each stage of Diabetic Retinopathy with good performance is important

Analysis of Diabetic Retinopathy Detection Techniques Using CNN … 101

to protect vision. The deep learning model gives better performance on classiﬁ-

cation. Classifying retinal blood vessels to automatically detect stages of Diabetic

Retinopathy and providing appropriate treatment to Diabetic Retinopathy patients.

We used Kaggle Diabetic Retinopathy and Asia Paciﬁc Tele-Ophthalmology Society

(APTOS) dataset. We have used EfﬁcientNet and Dense Neural Networks to clas-

sify stages of Diabetic Retinopathy. EfﬁcientNet has better classiﬁcation results

than DenseNet. DenseNet classiﬁed well on 3 classes only but EfﬁcientNet Model

classiﬁed very well on all the stages of Diabetic Retinopathy for all metrics.

References

1. Adate, A., & Tripathy, B. K. (2018). Deep learning techniques for image processing. In S.

Bhattacharyya, H. Bhaumik, A. Mukherjee, & S. De (Eds.), Machine learning for big data

analysis (pp. 69–90). De Gruyter.

2. Adate, A., & Tripathy, B. K. (2021). A survey on deep learning methodologies of recent

applications. In D. P. Acharjya, A. Mitra, & N. Zaman (Eds.), Deep learning in data analytics:

Recent techniques, practices and applications (pp. 145–170). Springer Publications.

3. Adate, A., Tripathy, B. K., Arya, D., & Shaha, A. (2020). Impact of deep neural learning on

artiﬁcial intelligence research. In S. Bhattacharyya, A. E. Hassanian, S. Saha and B. K. Tripathy

(Eds.), Deep learning research and applications (pp. 69–84). De Gruyter Publications.

4. Arora, M., & Pandey, M. (2019). Deep neural network for diabetic retinopathy detection. In:

International Conference on Machine Learning, Big Data, Cloud and Parallel Computing

(COMITCon) (pp. 189–193). Faridabad, India. https://doi.org/10.1109/COMITCon.2019.886

2217.

5. Bhattacharyya, S., Snasel, V., Hassanian, A. E., Saha, S., & Tripathy, B. K. (2020). Deep

learning research with engineering applications, De Gruyter Publications.

6. Bose, A., & Tripathy, B. K. (2020). Deep learning for audio signal classiﬁcation. In S. Bhat-

tacharyya, A. E. Hassanian, S. Saha and B. K. Tripathy (Eds.), Deep learning research and

applications (pp. 105–136). De Gruyter Publications.

7. Debgupta, R., Chaudhuri, B. B., & Tripathy, B. K. (2020). A wide ResNet-based approach

for age and gender estimation in face images. In A. Khanna, D. Gupta, S. Bhattacharyya, V.

Snasel, J. Platos, & Hassanien A. (Eds.), International Conference on Innovative Computing

and Communications, Advances in Intelligent Systems and Computing (vol. 1087, pp. 517–530).

https://doi.org/10.1007/978-981-15-1286-5_44.

8. Gao, Z., Li, J., Guo, J., Chen, Y., Yi, Z., & Zhong, J. (2019). Diagnosis of diabetic retinopathy

using deep neural networks. IEEE Access, 7, 3360–3370. https://doi.org/10.1109/ACCESS.

2018.2888639

9. Harun, N. H., Yusof, Y., Hassan, F., & Embong, Z. (2019). Classiﬁcation of fundus images for

diabetic retinopathy using artiﬁcial neural network. In 2019 IEEE Jordan International Joint

Conference on Electrical Engineering and Information Technology (JEEIT) (pp. 498–501).

Amman, Jordan. https://doi.org/10.1109/JEEIT.2019.8717479.

10. Huang, G., Liu, Z., Maaten, L. V. D., & Weinberger, K. Q. (2017). Densely connected convo-

lutional networks. In 2017 IEEE Conference on Computer Vision and Pattern Recognition

(CVPR) (pp. 2261–2269). Honolulu, HI. https://doi.org/10.1109/CVPR.2017.243.

11. International clinical diabetic retinopathy disease severity scale detailed table, American

Academy of Ophthalmology, San Francisco, CA, USA, Tech. Rep. (2002).

12. Kaul, D., Raju, H., & Tripathy, B. K. (2021). Deep learning in healthcare. In D. P. Acharjya,

A. Mitra, & N. Zaman (Eds.), Deep learning in data analytics—Recent techniques, practices

and applications (pp. 97–115). Springer Publications.

102 P. Prabhavathy et al.

13. Khan, S. H., Abbas, Z., & Rizvi, S. D. (2019). Classiﬁcation of diabetic retinopathy images

based on customized CNN architecture. In The Proceedings of Amity International Conference

on Artiﬁcial Intelligence (AICAI) (pp. 244–248). Dubai, United Arab Emirates. https://doi.org/

10.1109/AICAI.2019.8701231.

14. Leeza, M., & Farooq, H. (2019). Detection of severity level of diabetic retinopathy using bag

of features model. IET Computer Vision, 13(5), 523–530. https://doi.org/10.1049/iet-cvi.2018.

5263

15. Maheswari, K., Shaha, A., Arya, D., Tripathy, B. K., & Rajkumar, R. (2020). Convolutional

neural networks: A bottom-up approach. In S. Bhattacharyya, A. E. Hassanian, S. Saha and

B. K. Tripathy (Eds.), Deep learning research with engineering applications (pp. 21–50). De

Gruyter Publications.

16. Mishra, S., Dash, A., & Jena, L. (2021). Use of deep learning for disease detection and diagnosis.

In Bio-inspired neurocomputing (pp. 181–201). Springer, Singapore.

17. Mishra, S., Tadesse, Y., Dash, A., Jena, L., & Ranjan, P. (2021). Thyroid disorder analysis

using random forest classiﬁer. In Intelligent and cloud computing (pp. 385–390). Springer,

Singapore.

18. Omar, Z. A., Hanaﬁ, M., Mashohor, S., Mahfudz, N. F. M., & Muna’im, M. (2017). Auto-

matic diabetic retinopathy detection and classiﬁcation system. In 2017 7th IEEE International

Conference on System Engineering and Technology (ICSET) (pp. 162–166). Shah Alam. https://

doi.org/10.1109/ICSEngT.2017.8123439.

19. Qummar, S., et al. (2019). A deep learning ensemble approach for diabetic retinopathy

detection. IEEE Access, 7, 150530–150539. https://doi.org/10.1109/ACCESS.2019.2947484

20. Ray, C., Tripathy, H. K., & Mishra, S. (2019, June). Assessment of autistic disorder

using machine learning approach. In International Conference on Intelligent Computing and

Communication (pp. 209–219). Springer, Singapore.

21. Tan, M., & Le, Q. (2019). EfﬁcientNet: Rethinking model scaling for convolutional neural

networks. In The Proceedings International Conference on Machine Learning (ICML 2019)

(pp. 6105–6114). PMLR.

22. Tripathy, B. K., & Anuradha, J. (2015). Soft computing—Advances and applications.Cengage

Learning publishers.

23. Tripathy, B.K., & Baktha, K. (2017). Investigation of recurrent neural networks in the ﬁeld of

sentiment analysis. In The proceedings of IEEE International Conference on Communication

and Signal Processing (ICCSP17) (pp. 2047–2050).

24. Tripathy, H. K., Mishra, S., Thakkar, H. K., & Rai, D. (2021). CARE: A collision-aware

mobile robot navigation in grid environment using improved breadth ﬁrst search. Computers

and Electrical Engineering, 94, 107327.

25. Wang, X., Lu, Y., Wang, Y., & Chen, W. (2018). Diabetic retinopathy stage classiﬁcation using

convolutional neural networks. In IEEE International Conference on Information Reuse and

Integration (IRI) (pp. 465–471). Salt Lake City, UT. https://doi.org/10.1109/IRI.2018.00074.

26. Wu, Y., & Hu, Z. (2019). Recognition of diabetic retinopathy based on transfer learning. In 2019

IEEE 4th International Conference on Cloud Computing and Big Data Analysis (ICCCBDA)

(pp. 398–401). Chengdu, China. https://doi.org/10.1109/ICCCBDA.2019.8725801.

27. Zeng, X., Chen, H., Luo, Y., & Ye, W. (2019). Automated detection of diabetic retinopathy

using a binocular Siamese-like convolutional network. In 2019 IEEE International Symposium

on Circuits and Systems (ISCAS) (pp. 1–5). Sapporo, Japan. https://doi.org/10.1109/ISCAS.

2019.8702328.

Experimental Evaluation of Brain Tumor

Image Segmentation and Detection Using

CNN Model

Debjit Koner and Soumya Sahoo

Abstract In this study, the brain tumor detection technique using the digital image is

discussed. There are two ways to perform the detection: one is segmentation and the

other is classiﬁcation. The segmentation process used the adaptive threshold by which

the areas of interest are ﬁrst extracted from the digital image, and also reliable segmen-

tation is engineered by a model-based approach, i.e., modiﬁed Markov random ﬁeld

(MRF). The classiﬁcation process used the MRF segmented areas that are sorted

into regular and suspicious and used a method called Convolution Neural network

(CNN). In this chapter, the dataset is divided into two parts and it consists of a set of

screen/ﬁlms of the brain with abnormal (105) brain images and normal (167) brain

images that have been tested. Proven that there are biopsies of 48, with the malignant

mass of different types, and subtlety are contained later. A free-response receiver

characteristic operating curve is used to detect the algorithm accuracy which demon-

strates the association between the true positive massed detection and the variety of

false-positive alarms per image. The outcome indicated that a sensitivity of 90% can

be carried out in the prediction of various types of masses at the expense of false

detection of two signals per image. The algorithm becomes notably successful in the

prediction of the nominal cancers demonstrated by SPL les/masses/size of 10 mm.

In the dataset of 16 cases, a sensitivity of 94% was observed with 1.5 false alarms

per image. An extensive analysis of the consequences of the algorithm’s parame-

ters on its sensitivity and speciﬁcity became additionally carried out to optimize the

technique for a medical, observer overall performance analysis.

Keywords A brain tumor ·Segmentation ·Convolutional neural network (CNN) ·

Epochs ·Healthcare ·Image augmentation

D. Koner (B)

Accenture Solutions Pvt Ltd, Bangalore, Karnataka, India

e-mail: debjitkoner@gmail.com

S. Sahoo

C.V. Raman Global University, Bhubaneswar, Odisha, India

e-mail: soumya.sahoo@cvrce.edu.in

S. Mishra et al. (eds.), Augmented Intelligence in Healthcare: A Pragmatic and Integrated

Analysis, Studies in Computational Intelligence 1024,

https://doi.org/10.1007/978-981-19-1076-0_7

103

104 D. Koner and S. Sahoo

1 Introduction

Nowadays in the Medical ﬁeld, Magnetic Resonance Images (MRI) are very useful

for medical image processing. As brain tumor deﬁnes the uncommon increment of

tissues and augments of the unrestricted cells, because of this, the herbal pattern of

the growth of cell and demise failed. There are two stages of a brain tumor which

include the primary stage and secondary stage.

Brain tumor refers to when the tumor is unfolded in any part of the brain. Brain

tumors can be identiﬁed using certain symptoms like mood changes, difﬁculty in

hearing, vision, seizures, difﬁculty in walking, muscular moments, and so on. Clas-

siﬁcation of brain tumors is CNS lymphoma, ependymomas, oligodendroglioma,

Gliomas, and medulloblastoma.

The tumor can be unfolded in the primary stage but the tumor spreads in the

secondary stage, due to which the left out part of the seldom regrows again in the

second stage of the tumor which creates a big problem.

Why does this complication happen? The tumor occurs in an inaccurate area.

There are some techniques by the brain tumor can be detected, by using any of these

techniques brain tumors can be recognized and can identify the stages of the tumor

it can be achieved via—Magnetic resonant image is the MRI scanning Computed

tomography is the CT scanning Ultrasound, etc.

A brain tumor can be detected in numerous ways, by using the tumor method

we can easily detect and identify the tumor. Spectrometry of some mass is a CNN

algorithm for tumor prediction, using the irregular shape of the brain the abnor-

mality is predicted. The evaluation of the MRI images of the brain is considered

assertive because brain tumor is very deadly disease and many people lose their

lives due to brain tumor in developed countries like the USA, Germany, and so on;

according to the report of NBTF in the USA each year 29,000 people are diagnosed

with a brain tumor in which 13,000 of patients die each year. Several superior MRS

(MR Spectroscopy), MRI (Magnetic Resonance Imaging) methods that include DTI

(Diffusion Tensor Imaging) and for the evaluation of brain tumors, MRI perfusion

of MR are used. There are two main types of brain tumor that we can classify

broadly (i) benign tumor which is a noncancerous tumor, (ii) malignant tumor which

is a cancerous tumor. By WHO (World Health Organization) the malignant tumor

is further classiﬁed into four types. Types of malignant tumors are (i) Glioblas-

toma is called type IV tumor, (ii) Anaplastic Astrocytoma is called type III tumor,

(iii) Low-Grade Astrocytoma is called type II tumor, (iv) Pilocytic Astrocytoma is

called type I tumor. Semi malignant tumors are (i) Low-grade Astrocytoma and (ii)

Pilocytic Astrocytoma, which are two malignant tumors that are less aggressive.

The other two malignant tumors Glioblastoma and Anaplastic Astrocytoma, are the

most dangerous malignant tumor which causes the death of a tumor patient. For

the treatment and diagnosis of a brain tumor, various image processing methods

are used. In image processing methods segmentation is the fundamental step that is

used to abstract the abnormal section tissues from the brain MRI images. The role

of segmentation is crucial in the cancer region for treatment, diagnosis of cancer,

Experimental Evaluation of Brain Tumor Image Segmentation … 105

and it also helps to estimate the outcome of the treatment. For the subdivision of the

tumor, a vast number of auto-segmentation and semi-auto segmentation techniques

are used. MRI consists of strategies with more than one sequence that consist of

T1-weighted (TI) contrast-stronger (T1c) and T1-weighted, T2-weighted, Fluid, and

Techniques for brain tumor segmentation that use T2-weighted Attenuated Inversion

Recovery (ﬂair). There are numerous features in MRI images that are used to study

the segmentation of brain tumors that in-clues textures of the images, structure tensor

eigenvalues, and histograms of the local. To classify the pattern in the segment of the

tumor, the most common machine learning algorithm that is used namely are random

forest and support vector machine. For the study of the segmentation of brain tumors,

the deep learning method becomes popular due to its higher analysis like detection of

an object, semantic segmentation, and image classiﬁcation. For automatic segmenta-

tion of brain tumors using multi-model MRIs, deep learning algorithms have reached

state-of-the-art performance. However, a convolution neural network is generally

used for the segmentation of brain tumors, image classiﬁcation, and prediction of

the lifeline of the patients. Techniques used by deep learning for the segmentation

of the tumor, image classiﬁcation, and detection which includes mainly Convolu-

tional Restricted Boltzmann Machine, and Stacked De-Noising Autoencoders. For

classiﬁcation, segmentation of the image, and prediction CNN gives much higher

accuracy than all others in deep learning methods. For Segmenting, classifying, and

predicting brain tumors methods both the 3D and 2D Convolution neural network are

used. In distinct classes, the segmentation is classiﬁed as healthy tissue strengthening

the core, necrosis, non-strengthening the core, and edema.

For segmentation, speciﬁc tumor cells display morphological facts. and distinct

phenotypic prediction, and classiﬁcation, along with motility, gene expression, prolif-

eration, cell morphology, and metabolism metastatic capability. This paper offers an

evaluation of CNN, frameworks, architectures, techniques, algorithms, and essen-

tially examine by using deep learning to know the classiﬁcation, prediction of survival

time, and segmentation. The assessment plays an evaluation of the dataset applied,

and libraries that might be used for assessment measures, implementation, and

recognition.

2 Background Study

Several signiﬁcant works based on brain tumor classiﬁcation and analysis are under-

taken by different authors. Some of those relevant studies are presented here. Non-

invasive magnetic resonance techniques were proposed by Dong et al. [1], which

were used to identify the brain tumor without using ionizing radiation. The 3D MRI

volumes of the manual segmentation need a longer time, and its main performance

was based on the operator’s experience. Hence the author encouraged a primary

unit-based neural network which was a deep convolution network. On the datasets

of 2015 BRATS, the author implements this segmentation, There were 220 cases

of high-grade glioma and 54 cases of low-grade glioma in this study. The overall

106 D. Koner and S. Sahoo

performance of the proposed approach became a comparison to the manual delin-

eated ﬂoor truth U-net-based network which is a deep neural network that offers an

advanced result for the core tumor regions.

To glioblastomas (both high and low grades) MRI image, Haveri et al. [2] proposed

a segment of brain tumor using a deep neural network. A brain tumor of this kind can

manifest itself in any part of the brain. And it may have different contrast, sheep, and

size. As a machine learning algorithm, the article utilizes the convolutional neural

network. The segmentation of the tumor has exploited both the global feature and

the local feature. For the research work, the BRATS dataset was used by the author.

The brain tumor segment proposed by Isin et al. [3], was one of the toughest tasks

in the ﬁeld of medical science. With the help of the early brain tumor diagnosis, it

shows some major improvement in the lifespan of the patient. For larger data, manual

segmentation took a lot of time in the computation. Automatic segmentation is needed

for this reason. In modern medical science, there was a use of deep learning for

automatic segmentation. For a larger MRI dataset, it gives effective segmentation. The

methods of deep learning states were reviewed by this article. A prior challenge was

there for the conventional auto segment method, maps probability, or feature selection

for the classiﬁcation of the high representative, which is a tough assignment. With

the help of multimode of the brain MRI images, the convolutional neural network

technique learns the complicated feature automatically for both the tissues, i.e., tumor

and healthy. Gliomas, one of the most clusters of a brain tumor which was proposed

by Pereira et al. [4], which was the results for the short lifetime, and their grades

are high. As a large amount of data was being produced by the MRI method and

segmentation which is done manually needs a longer time. Due to the structural, large,

and spatial variability among the brain, automatic brain tumor detection becomes the

most difﬁcult task. The author suggests a method, i.e., convolution neural network

with a kernel of small size, i.e., 3 ×3 for the new segmentation. In the neural

network by assigning a few numbers of weights the kernel with a smaller size in the

architecture helps to avoid overﬁtting. To provide effective segmentation intensity

normalization was used for pre-processing along with a convolutionalneural network.

The rest of the discussion was implemented on the 2013 BRATS database. Brain

tumor based on gliomas a segmented algorithm was proposed by Hussian et al. [5].

To ﬁgure out the tumor, it uses a neural network method, i.e., a deep convolutional

neural network. Due to the brain tumor accuracy segmentation, the lifespan of the

patient increases. By using the drop-out and max-out layer in the patch process,

helps the model to eliminate the overﬁtting process. The pre-processing technique

was used to propose this algorithm to eliminate the unwanted noise and the small

false positive was eliminated by post-processing using morphological operators. For

research work for the segmentation, the 2013 BRATS dataset was used by the author.

A convolutional neural network provides better performance for automatic

segmentation of medical images which was proposed by Wang et al. [6]. But a

robust result was not provided for clinical use. This model’s weakness was its

inability to generalize previously unknown classes objects. For the speciﬁc image

test, the discussed technique, CNN becomes adaptive, which may be supervised or

unsupervised.

Experimental Evaluation of Brain Tumor Image Segmentation … 107

The segmentation process which was estimated by Devkota et al. [7] is based on

the Operations of Morphological Mathematics and the processing time was improved

by the FCM algorithm, the model which the author proposed, for the testing stages

of evaluation gives an accuracy of 92% and 86.6 were the classiﬁcation accuracy.

Histogram-based segmentation method was proposed by Yantao et al. [8]. There

were three different classes of segmentation of tumor (edema, normal tissue, and

tumor include tumor and necrosis), T1 and FLAIR are two different types of a classi-

ﬁcation model. In the abnormal regions, the tissue of the tumor and the edema were

distinguished which enhance the T1 modality contrast using the k-means method

and a Dice coefﬁcient to accomplish with a sensitive accuracy of 73.6 and 90.3,

respectively.

Badran et al. [9] adopted the canny edge detection model which is inspired by the

edge detection approach and it is accumulating for the extraction of ROI for adaption

thresholding.

A method was proposed by Pei et al. [10] to determine the pattern of tumor growth

which is a novel feature for the improvement of the segmentation of the texture-based

tumor of MRI longitudinal. To ﬁgure out the modeling of tumor growth label maps

were used and to predict the density of cells after extracting intensity and texture

features.

A technique proposed by Dina et al. [11] on the Probabilistic Neural Network

model was related to Vector Quantization learning. On 64 MRI images, the model

tested practically in which the test case contains 18 MRI images, and the remaining

46 MRI images for the training set, for the smoothness Gaussian ﬁlter used. By

modifying the PNN method, 79% of the processing time was reduced.

A method being proposed by Othman et al. was a probabilistic Neural Network.

To extract the feature PCA (Principle Component Analysis) was used and it also

reduces the dimension of the larger data [12]. Based on the spread value the range

of the accuracy is between 73 and 100%.

Rajendran et al. [13] proposed a model, that was region-based fuzzy clustering. It

has an accuracy of 93.3 and 82.1% of ASM and enhancement of Probabilistic Fuzzy

C-Means model based on Jaccard index with morphological operations.

Tables 1and 2highlights relevant research works conducted in brain tumor

diagnosis.

3 Discussed Methodology

By using deep learning techniques apart from traditional machine learning techniques

a lot of work is saved by the people in the extraction of feature and selection [31–

34]. The traditional method is compared with the proposed MRI images of the brain

tumor prediction (dataset contains with Tumor or without Tumor). CNN model is

used in this paper for the prediction of the image which is mentioned in Fig. 1, which

has two main processes and they are pre-processing of 2D image and Training.

108 D. Koner and S. Sahoo

Tabl e 1 Overview of deep learning research works on brain tumor prediction

Study Method Discussed

approach

Implemented

software/tools

Evaluation

(accuracy %)

Sun et al. [14]3D convolutional

neural network

Multi-view deep

learning framework

Toolbox of

PyTorch

88%

Arietal.[15]Convolutional

neural network

Extreme learning

machine with local

receptive ﬁelds

2015 MATLAB 97.18%

Aminrt et al. [16]Random forest Gabor wavelet

features,

histograms of

oriented gradient

and segmentation

based texture

analysis

FLAIR Dice scores

Complete: 91

Non-enhancing:

Enhancing: 90

Nie et al. [17]Convolutional

neural network

Multi-channel

convolutional

neural networks

and SVM

Python 90.66%

Virupakshappa

and Amarapur

[18]

Adaptive

artiﬁcial neural

network

Approach to

modiﬁed level sets

MATLAB 98%

Suter et al. [19]3D convolutional

neural network

Multi-view deep

learning framework

(MvNet) and

SPNet

cikit-learn3

version 0.19.1

72.2%

Chato and Latiﬁ

[20]

Linear

discriminant,

convolutional

neural network

SVM, KNN,

ensemble learner

and logistic

regression

Python 68.8%

While training a Convolution Neural Network (CNN) model the image normaliza-

tion creates a big problem and the workﬂow of the pre-processing part is mentioned

in Fig. 2. All the images present in the dataset for testing purposes are not in the same

aspect ratio or size which creates a problem. Due to the above reasons, the image is

being pre-proceeded for training and testing purposes, the actual size of the image

is 256 ×256 which is converted into 64 ×64 size.

Training-set

Augmentation of image: To increase the accuracy and performance of a model there

is a technique in deep learning which is called image augmentation is used here in

this paper. The reasons behind the overﬁtted model are too much data in the training

set. Augmentation of the image is used to overcome such a situation. In diverse

ways such as processing or combination of multiple processing, i.e., random rotate,

random ﬂip it creates images artiﬁcially.

Experimental Evaluation of Brain Tumor Image Segmentation … 109

Tabl e 2 Overview of classiﬁcation of brain tumor prediction research works using deep learning

Study Method Discussed

approach

Software tool for

implementation

Evaluation

(accuracy %)

Nandi [21]Convolutional

neural networks

Local receptive

ﬁelds on an

extreme learning

machine

Matlab 97.18%

Abiwinanda et al.

[22]

Convolutional

neural networks

AlexNet, VGG16,

ResNet

Matlab 84.19%

Banerjee et al. [23]Using

multi-sequence

MR images, deep

convolutional

neural networks

(ConvNets) were

created

Nil Python and

tensor ﬂow

97%

Zhou et al. [24]Convolutional

neural networks

DenseNet-RNN,

DenseNet-LSTM

Tensor ﬂow,

Nvidia Titan Xp

GPU

92.13%

Paula et al. [25]CNN, random

forests, fully

connected neural

network

Different machine

learning models

R language 91.43%

Alberts et al. [26]SVM, RF, KNN,

LOG, MLP and

PCA

LBP, BRIEF, and

HOG

Not mention 83%

Mohsen et al. [27]Deep

convolutional

neural networks

Discrete wavelet

transform (DWT),

principal

components

analysis (PCA)

2015 Matlab R

and 3.9 Weka

96.87%

Jena et al. [28]Convolutional

neural networks +

PSO

Capsule networks

(CapsNets)

2.7 Python and

Keras library

86.56%

Xu et al. [29]Activation

characteristics of

deep convolutional

neural networks

ImageNet

information was

used to train deep

convolutional

activation features

Not mention 97.5%

Ishikawa et al. [30] Deep CNN BING objectness Not mention 98.5

Rescaling of an image: It is a maneuver that moves statistics from one numerical

variety to another via simple division using a predeﬁned consistent. Due to the

possibility of optimization, overﬁtting, and stability issue in a deep neural network,

the input is restricted between 0 and 1.

110 D. Koner and S. Sahoo

Fig. 1 Workﬂow of the model

Fig. 2 Workﬂow of image pre-processing part

Shearing of an image: Shear displaces or mapping each factor within the vertical

direction by an amount proportional to its distance from an image edge. Though, the

direction does not have to be vertical but can be random.

Zoom in the image: It arbitrarily zooms in and out into the image. The factor zooming

controlled the zoom_range. For example, if 0.2 is the parameter of the zoom_range

that means the range is [0.2, 1.2].

Experimental Evaluation of Brain Tumor Image Segmentation … 111

Image ﬂip: The image is being turned over vertically and horizontally. In the model,

we have used horizontal ﬂip. The concerning image is ﬂipped in the vertical axis.

The parameter horizontal_ﬂip is used to make it active and inactive.

Test-set

The images used here are predominantly stowed in Red, Green, and Blue, i.e., RGB

format. In this format, the representation of the image is in the format of a 3 channel

or 3D array. One dimension is for channels (red, green, and blue colors) and the

spatial dimension is the two other dimensions. Thus, though three numbers every

pixel is encoded. An 8-bit unsigned integer is usually used to store each number

(0–255). To overcome the issue like overﬁtting, stability, and optimization rescaling

is used. The images with and without tumors are depicted in Fig. 3.

To predict images directly in the deep neural network we use a CNN model and

the workﬂow of the architecture of the Convolution Neural Network model is shown

in Fig. 4.

Convolution layer: From the given two functions it is derived by integration which

demonstrates how one shape is modiﬁed by another. To extract vital information

from the given MRI image we use a feature detector of 32, 64 and 128 which is

stored in a 3 ×3 dimensions matrix image. The feature detector is also mentioned

as a “Filter” or “Kernal”. In the mentioned Fig. 3, the convolution layer is being

used to reduce the loss of important information from the image and it also helps

to boost the efﬁciency and accuracy of the prototype [35]. The Convolution Layer

will accomplice their outputs to the local 2D area of their inputs, each computing a

Convolution operation with a 2D ﬁlter of size 32 ×3×3, 64 ×3×3, 128 ×3×

3. These outputs are in the 2D matrix as it is a black and white or MRI image and

in the max-pooling, the size of the matrix will get reduced to extract the important

information only.

Input shape: Here the image input matrix size each of 64 ×64 consists of two

channels as it is an MRI or black and white image.

Fig. 3 a Without tumor and bwith tumor

112 D. Koner and S. Sahoo

Fig. 4 Workﬂow of our model

Activation layer: This function is responsible to introduce qualities that are not’

linear to the network by input mapping to the response variables. The transformation

function we have used here is Rectiﬁed Linear Unit, i.e., ReLU as an activation layer,

it is used here as its job is to omit the values of the pixel which are negative in the

feature map and the Sigmoid activation function is used to categorized binary output

[36–38].

Max-pooling: It is in a simple matrix format but this layer shrinks the matrix using

the feature detector. It carries only the vital information from the previous layer and

leaves out the rest. In this paper, 2 ×2 window size matrix is used with a pooling

stride of 2.

Flattering: It converts all the weights of the neurons into a 1D matrix or a simple

vector. Its miles are used to ﬂatten the pooled function map into a column. It converts

the shrink input image into a vector which is used for the classiﬁcation.

Full connection: In this layer, every neuron receives input from each node of the

previous layers. Only from a restricted subarea of previous layers of neurons, do

the convolution neuron layers receive input. Here 128 nodes are used for a fully

connected layer as this number is not too large nor not too small for the system

which helps the model to run smoothly [39].

Loss function: It plays a very signiﬁcant role in neural networks. The value of the

loss function is non-negative, in which the model increases its robustness along

Experimental Evaluation of Brain Tumor Image Segmentation … 113

with the decrement of the loss function value. Here, to calculate the cross-entropy

binary-cross-entropy is used.

Optimizer: Optimizer helps to update the neural network weights. In the model,

Adam optimizer is used for weight updating.

CNN compiling: In this phase, all the layers mentioned previously are compiled into

one.

Prediction: Finally, the prediction of whether or not a tumor may be seen in an MRI

scan is shown using binary encoding which helps classiﬁed from which class the

given image belongs, i.e., with has a brain tumor or without a brain tumor [40–43].

4 Result Discussion

In this analysis, the dataset is divided into three sub-section, i.e., 60, 50, 40% testing

set, with 15 epochs and 30 epochs. Table 3represents the total accuracy that comes in

each part. To get the different results in three different parts the training and test set

are divided and two different epochs, respectively (i.e., 40, 50, 60% and 30, 15 into

a test set and training set) the same architecture is followed. In Table 3we show the

accuracy we get from each part from two epochs, to represent the model differently

we took a different test set and training set size with diverse epoch sizes, the test set

size increases slowly to collate the model accurateness with more training size.

In Fig. 5we get an accuracy of about 64.99, Fig. 6we get an accuracy of about

67.04, Fig. 7we get an accuracy of about 69.71, Fig. 8we get an accuracy of about

61.44, Fig. 9we get an accuracy of about 64.14, Fig. 10 we get accuracy about 66.48,

here a line graph represented where the loss is compared which ﬁgure out the error

in the model (i.e., it gives us an overview whether the model is overﬁtted, underﬁt

or it is a good model), precision helps to test the precision of the model, the blue

line represents the accuracy which is mention in the graph, in all the graph given

shows the diverse accurateness to get so, the model is being trained three times with

different partition and different epoch of the same dataset.

In Fig. 5, a graph is plotted against loss versus accuracy on 30 epochs, and we get

an accuracy of 64.99%, in which the dataset is spilled, where 40% of data from the

dataset is for training purposes and remaining for testing purpose.

Tabl e 3 Accuracy analysis

with different size of epochs Test set Accuracy

15 epochs 30 epochs

40% 61.44 64.99

50% 64.14 67.04

60% 66.48 69.71

114 D. Koner and S. Sahoo

Fig. 5 Accuracy with loss graph for test set 40% (epoch of 30)

Fig. 6 Accuracy with loss graph for test set 50% (epoch of 30)

In Fig. 6, a graph is plotted against loss versus accuracy on 30 epochs, and we get

an accuracy of 67.04, in which the dataset is spilled, where 50% of the data from the

dataset is for training purposes and remaining for testing purpose.

In Fig. 7, a graph is plotted against loss versus accuracy on 30 epochs, and we

get an accuracy of 69.71, in which the dataset is spilled, where 60% of data from the

dataset is for training purposes and remaining for testing purposes.

Experimental Evaluation of Brain Tumor Image Segmentation … 115

Fig. 7 Accuracy with loss graph for test set 60% (30 epoch)

Fig. 8 Accuracy versus loss for test set 40% (15 epoch)

In Fig. 8, a graph is plotted against loss versus accuracy on 15 epochs, and we

get an accuracy of 61.44, in which the dataset is spilled, where 40% of data from the

dataset is for training purposes and remaining for testing purposes.

In Fig. 9, a graph is plotted against loss versus accuracy on 15 epochs, and we

get an accuracy of 64.14, in which the dataset is spilled, where 50% of data from the

dataset is for training purposes and remaining for testing purposes.

116 D. Koner and S. Sahoo

Fig. 9 Accuracy versus loss for test set 50% (15 epoch)

Fig. 10 Accuracy versus loss for test set 60% (15 epoch)

In Fig. 10, a graph is plotted against loss versus accuracy on 15 epochs, and we

get an accuracy of 66.48, in which the dataset is spilled, where 60% of data from the

dataset is for training purposes and remaining for testing purposes.

Experimental Evaluation of Brain Tumor Image Segmentation … 117

5 Conclusion

We have presented a novel technique in this chapter to detect and segment Brain

tumors from the MRI brain images. To extract the important information from the

segment of the brain tumor we used techniques, i.e., Convolution Neural Network.

We have split the dataset into three subsections to compare the accuracy and test the

performance of the different splits that we had made while testing. We had split the

training and test set into three different segments, i.e., giving more data in the training

set for training, and testing less data for testing and vice versa, and these changes

reﬂect a humongous change in the accuracy or performance. This technique can be

used in a machine that supports artiﬁcial intelligence which the process hassle-free

and fast compare to the normal traditional way of testing. In some studies or the

laboratory, it is seen that detection of Brain tumors from MRI images is done by

various methods and it achieve more accuracy and more efﬁciency than the normal

way of testing.

References

1. Dong, H., Liu, F., Mo, Y., Guo, Y., & Yang, G. (2017). Automatic brain tumor detection and

segmentation using U-net-based fully convolutional networks. MIUA, 3, 1–12.

2. Havaei, M., Davy, A., & Warde Farley, D., et al. (2017). Brain tumor segmentation with deep

neural networks. Medical Image Analysis, 35, 18–31.

3. Ism, A., Direkoglu, C., & Sah, M. (2016). Review of MRI-based brain tumor image segmen-

tation using deep learning methods. In Proceedings of 12th International Conference on

Application of Fuzzy Systems and Soft Computing, Vienna, Austria.

4. Alves, V., Carlos, A. S., Pereira, S., & Pinto, A. (2016). Brain tumor segmentation using

convolutional neural networks in MRI images. IEEE Transactions on Medical Imaging, 35,

1240–1251.

5. Hussain, S., Anwar, S. M., & Majidmil, M. (2017). Brain tumor segmentation using cascaded

deep convolutional neural network. In Proceedings of 39th International Conference of the

IEEE Engineering in Medicine and Biology Society, Jeju, South Korea.

6. Wang, G., Zuluaga, M. A., Pratt, R., Aertsen, M., Doel, T., Klusmann, M., David, A. L., Deprest,

J., Vercauteren, T., & Ourselin, S. (2016). Slic-Seg: A minimally interactive segmentation of

the placenta from sparse and motion-corrupted fetal MRI in multiple views. Medical Image

Analysis, 34, 137–147.

7. Alsadoon, A., Singh, A. K., Elchouemi, A., Devkota, B., & Prasad, P. W. C. (2017, December 7–

8). Image segmentation for early stage brain tumor detection using mathematical morphological

reconstruction. In 6th International Conference on Smart Computing and Communications,

ICSCC 2017, Kurukshetra.

8. Song, Y., Ji, Z., & Sun, Q., et al. (2017). A novel brain tumor segmentation from multi-modality

MRI via a level-set-based model. Journal of Signal Processing Systems, 87, 249–257. https://

doi.org/10.1007/s11265-016-1188-4.

9. Badran, E. F., Mahmoud, E. G., & Hamdy, N. (2017). An algorithm for detecting brain

tumors in MRI images. In 7th International Conference on Cloud Computing, Data Science &

Engineering—Conﬂuence.

10. Davatzikos, C., Iftekharuddin, K. M., Li, W., Pei, L., & Reza, S. M. S. (2017). Improved brain

tumor segmentation by utilizing tumor growth model in longitudinal brain MRI. Proceedings

of SPIE—The International Society for Optical Engineering.

118 D. Koner and S. Sahoo

11. Dahab, D. M., Ghoniemy, S., & Selim, G. E. (2012). Automated brain tumor detection and

identiﬁcation using image processing and probabilistic neural network techniques. IJIPVC,

1(2), 1–8.

12. Ariffanan, M., Basri, M., & Othman, M. F. (2011). Probabilistic neural network for brain

tumor classiﬁcation. In 2nd International Conference on Intelligent Systems, Modelling and

Simulation.

13. Rajendran, A., & Dhanasekaran, R. (2011). Fuzzy clustering and deformable model for tumor

segmentation on MRI brain image: A combined approach. In International Conference on

Communication Technology and System Design.

14. Sun, L., Zhang, S., Chen, H., & Luo, L. (2019). Brain tumor segmentation and survival predic-

tion using multimodal MRI scans with deep learning. Frontiers in Neuroscience, 13, 810.

https://doi.org/10.3389/fnins.2019.00810.

15. Ari, A., & Hanbay, D. (2018). Deep learning based brain tumor classiﬁcation and detection

system. Turkish Journal of Electrical Engineering & Computer Sciences, 26(5), 2275–2286.

https://doi.org/10.3906/elk-1801-8.

16. Amin, J., Sharif, M., Raza, M., & Yasmin, M. (2018). Detection of brain tumor based on features

fusion and machine learning. Journal of Ambient Intelligence and Humanized Computing,1–17.

17. Nie, Z., et al. (2016). Integrated time-resolved ﬂuorescence and diffuse reﬂectance spectroscopy

instrument for intraoperative detection of brain tumor margin. IEEE Journal of Selected Topics

in Quantum Electronics, 22(3), 49–57.

18. Amarapur, B. (2017, September). An automated approach for brain tumor identiﬁcation using

ANN classiﬁer. In 2017 International Conference on Current Trends in Computer, Electrical,

Electronics and Communication (CTCEEC) (pp. 1011–1016). IEEE.

19. Suter, R. K., Rodriguez-Blanco, J., & Ayad, N. G. (2020). Epigenetic pathways and plasticity

in brain tumors. Neurobiology of Disease, 145, 105060. https://doi.org/10.1016/j.nbd.2020.

105060. PMID: 32877743.

20. Chato, L., & Latiﬁ, S. (2021). Machine learning and radiomic features to predict overall survival

time for glioblastoma patients. Journal of Personalized Medicine, 11(12), 1336. https://doi.org/

10.3390/jpm11121336. PMID: 34945808; PMCID: PMC8705288.

21. Nandi, A. (2015, November). Detection of human brain tumour using MRI image segmentation

and morphological operators. In 2015 IEEE International Conference on Computer Graphics,

Vision and Information Security (CGVIS) (pp. 55–60), Bhubaneswar.

22. Abiwinanda, N., Hanif, M., Hesaputra, S. T., Handayani, A., & Mengko, T. R. (2019).

Brain tumor classiﬁcation using convolutional neural network. In Lhotska, L., Sukupova, L.,

Lackovi´c, I., Ibbott, G. (eds) World Congress on Medical Physics and Biomedical Engineering

2018, IFMBE Proceedings,vol68(1), (pp. 183–189). Springer, Singapore. https://doi.org/10.

1007/978-981-10-9035-6_33

23. Banerjee, S., Khajanchi, S., & Chaudhuri, S. (2015). A mathematical model to elucidate brain

tumor abrogation by immunotherapy with T11 target structure. PLoS One, 10(5), e0123611.

https://doi.org/10.1371/journal.pone.0123611.

24. Zhou, M., Scott, J., Chaudhury, B., Hall, L., Goldgof, D., Yeom, K. W.,Iv, M., Ou, Y., Kalpathy-

Cramer, J., Napel, S., Gillies, R., Gevaert, O., & Gatenby, R. (2018). Radiomics in brain

tumor: image assessment, quantitative feature descriptors, and machine-learning approaches.

American Journal of Neuroradiology, 39(2), 208–216. https://doi.org/10.3174/ajnr.A5391.

25. de Robles, P., Fiest, K. M., Frolkis, A. D., Pringsheim, T., Atta, C., St. Germaine-Smith, C.,

Day, L., Lam, D., & Jette, N. (2015). The worldwide incidence and prevalence of primary brain

tumors: a systematic review and meta-analysis. Neuro-Oncology, 17(6), 776–783. https://doi.

org/10.1093/neuonc/nou283

26. Alberts, E., Tetteh, G., Trebeschi, S., Bieth, M., Valentinitsch, A., Wiestler, B., & Menze, B. H.

(2017). Multi-modal image classiﬁcation using low-dimensional texture features for genomic

brain tumor recognition. In Graphs in Biomedical Image Analysis, Computational Anatomy

and Imaging Genetics (pp. 201–209). Springer, Berlin, Germany.

27. Mohsen, H., El-Dahshan, E. S. A., El-Horbaty, E. S. M., & Salem, A. B. M. (2018). Classiﬁca-

tion using deep learning neural networks for brain tumors. Future Computing and Informatics

Journal, 3(1), 68–71.

Experimental Evaluation of Brain Tumor Image Segmentation … 119

28. Jena, L., Mishra, S., Nayak, S., Ranjan, P., & Mishra, M. K. (2021). Variable optimiza-

tion in cervical cancer data using particle swarm optimization. In Advances in Electronics,

Communication and Computing (pp. 147–153). Springer Singapore.

29. Xu, L., Lin, Y., Han, J. C., Xi, Z. N., Shen, H., & Gao, P. Y. (2007). Magnetic resonance

elastography of brain tumors: Preliminary results. Acta Radiologica, 48, 327–330.

30. Ishikawa, E., Yamamoto, T., Matsuda, M., Akutsu, H., Zaboronok, A., Kohzuki, H., Miki, S.,

Takano, S., & Matsumura, A. (2015). Intraparenchymal brain lesion biopsy guided by a rigid

endoscope and navigation system. Surgical Neurology International, 6, 149. https://doi.org/

10.4103/2152-7806.165764

31. Mishra, S., Dash, A., Ranjan, P.,& Jena, A. K. (2021). Enhancing heart disorders prediction with

attribute optimization. In Advances in Electronics, Communication and Computing (pp. 139–

145). Springer Singapore.

32. Ray, C., Tripathy, H. K., & Mishra S. (2019). A review on facial expression based behavioral

analysis using computational technique for autistic disorder patients. In M. Singh, P. Gupta, V.

Tyagi, J. Flusser, T. Ören, & R. Kashyap (Eds.), Advances in Computing and Data Sciences.

ICACDS 2019. Communications in Computer and Information Science (Vol. 1046). Springer

Singapore. https://doi.org/10.1007/978-981-13-9942-8_43

33. Mukherjee, D., Tripathy, H. K., & Mishra, S. (2021). Scope of medical bots in clinical domain.

In Technical Advancements of Machine Learning in Healthcare (Vol. 936, p. 339).

34. Mishra, S., Thakkar, H. K., Mallick, P. K., Tiwari, P., & Alamri, A. (2021). A sustainable IoHT

based computationally intelligent healthcare monitoring system for lung cancer risk detection.

Sustainable Cities and Society, 103079.

35. Tripathy, H. K., Mishra, S., Thakkar, H. K., & Rai, D. (2021). CARE: A collision-aware

mobile robot navigation in grid environment using improved breadth ﬁrst search. Computers &

Electrical Engineering, 94, 107327.

36. Chattopadhyay, A., Mishra, S., & González-Briones, A. (2021). Integration of machine learning

and IoT in healthcare domain. In Hybrid Artiﬁcial Intelligence and IoT in Healthcare (pp. 223–

244). Springer Singapore.

37. Sahoo, S., Das, M., Mishra, S., & Suman, S. (2021). A hybrid DTNB model for heart disor-

ders prediction. In Advances in Electronics, Communication and Computing (pp. 155–163).

Springer Singapore.

38. Mishra, S., Tripathy, H. K., & Acharya, B. (2021). A precise analysis of deep learning for

medical image processing. In Bio-Inspired Neurocomputing (pp. 25–41). Springer Singapore.

39. Mishra, S., Tadesse, Y., Dash, A., Jena, L., & Ranjan, P. (2021). Thyroid disorder analysis

using random forest classiﬁer. In Intelligent and Cloud Computing (pp. 385–390). Springer

Singapore.

40. Mishra, S., Mishra, B. K., & Tripathy,H. K. (2015, December). A neuro-genetic model to predict

hepatitis disease risk. In 2015 IEEE International Conference on Computational Intelligence

and Computing Research (ICCIC) (pp. 1–3). IEEE.

41. Mishra, S., Dash, A., & Mishra, B. K. (2020). An insight of Internet of Things applications in

pharmaceutical domain. In Emergence of Pharmaceutical Industry Growth with Industrial IoT

Approach (pp. 245–273). Academic Press.

42. Mishra, S., Sahoo, S., & Mishra, B. K. (2019). Addressing security issues and standards in

Internet of things. In Emerging Trends and Applications in Cognitive Computing (pp. 224–257).

IGI Global.

43. Mishra, S., Mahanty, C., Dash, S., & Mishra, B. K. (2019). Implementation of BFS-NB hybrid

model in intrusion detection system. In Recent Developments in Machine Learning and Data

Analytics (pp. 167–175). Springer Singapore.

Effective Deep Learning Algorithms

for Personalized Healthcare Services

Anjana Mishra, Siddha Sachida Mohapatra, and Sukant Kishoro Bisoy

Abstract The main motive to present this chapter is to provide effective deep

learning algorithms for personalized healthcare services and to make the reader

understand the importance of deep learning in the ﬁeld of healthcare. We realize

that acquiring ability and signiﬁcant viewpoints from nuanced, high-dimensional,

and heterogeneous biomedical information keeps on being a signiﬁcant test in

medical care change. Electronic wellbeing reports, imaging, omics, sensor infor-

mation, and text are instances of nuanced, heterogeneous, gravely explained, and

generally unstructured information that have arisen in contemporary biomedical

science. As feature engineering is needed in traditional data mining approach to

extract efﬁcient and more scalable features from data, and after that predicting and

clustering the models make it more challenging in these steps, especially in case of

complicated data, the latest advancements in deep learning in the ﬁeld of healthcare

give effective calculations to get start to ﬁnish models from complex information. In

the coming days, it is believed that deep learning will have a major role in converting

big and complex biomedical data into improved human health. However, we also

notice limitations and the need for improved method creation and implementations,

especially in terms of domain experts’ and citizen scientists’ comprehension. To

close the gap between deep learning models and human interpretability, we propose

designing holistic and practical interpretable algorithms.

Keywords Deep learning ·Healthcare ·Neural networks ·Biomedical data ·

Feature engineering

1 Introduction

The turn of events and use of mechanical gadgets have brought about consistent

advancement in medication, genomics, drugs, and medical care following. This has

made data collection, interpretation, and processing much easier. Deep Learning is

A. Mishra (B)·S. S. Mohapatra ·S. K. Bisoy

Department of Computer Science and Information Technology, C.V. Raman Global University,

Bhubaneswar, Odisha, India

e-mail: anjanamishra2184@gmail.com

S. Mishra et al. (eds.), Augmented Intelligence in Healthcare: A Pragmatic and Integrated

Analysis, Studies in Computational Intelligence 1024,

https://doi.org/10.1007/978-981-19- 1076-0_8

121

122 A. Mishra et al.

right now having a critical impact in different areas of medical services. The striking

consequences of medical services have been made conceivable by the extending

accessibility of medical services information and the quick development of varieties

in deep learning strategies. Deep learning approaches can reveal clinically signiﬁcant

data covered in immense measures of medical care information, which would then be

able to be utilized for medical care dynamic, therapy, control, and avoidance. Deep

learning has numerous applications like in wellbeing conduct reaction, electronic

wellbeing record preparing and recuperating experimentally solid therapies from

text, walk investigation and mechanical helped recuperation, hearing issue therapy,

malignant growth treatment, heart determination, and mental action examination, to

give some examples. This simpliﬁes treatment for medical services suppliers and is

more helpful for patients, just as taking into account more proﬁcient and successful

checking. Because of advances in profound learning, essential clinical hardware,

for example, a stethoscope and thermometer have developed into registered tomog-

raphy (CT), radio atomic imaging, ultrasound analytic gadgets, radiation treatment,

ventilators, dialysis, and other clinical gadgets.

Medical care therapy and ofﬁces will without a doubt work on it, long time in

various manners, making them more powerful and conveying greater administrations.

There is no question that, in the upcoming time [1], medical care therapy and ofﬁces

will work in an assortment of ways, making it more proﬁcient and offering more

excellent types of assistance.

Deep learning has unmatched depth of learning and feature extraction when

compared to classical machine learning algorithms. Through nonlinear change in

the secret layers, the profound organization design can inexact complex capacities.

From low to an undeniable level, the portrayal of highlights turns out to be progres-

sively conceptual, permitting the ﬁrst information to be all the more accurately distin-

guished. Deep learning models and approaches have been used in a vast number of

experimental works, and there are many deep learning models. The purpose of this

paper is to emphasize the key concepts and applications of deep learning in the health

care and medical ﬁelds, rather than to demonstrate all of the methods and models. The

autoencoder is a plain feedforward DNN design that comprises encoder and decoder

capacities for the information and yield layers, separately. Convolutional ﬁlters are

introduced through interleaved feedforward layers, which are accompanied by recti-

ﬁcation, reduction, or pooling layers. The CNN generates high-level abstract features

for each layer [2]. The Recurrent Neural Network (RNN), which is a successive infor-

mation NN with an inbuilt memory that changes the condition of every neuron with

earlier info, is another deep learning rendition. The Deep Belief Network (DBN)

model has only a couple layers of covered-up units, and each layer’s units are asso-

ciated with the following layer’s units. It needs a lot of computing power for deep

learning. The advancement of graphics processing units (GPUs), which play a major

role in accelerating the computing requirements of deep learning, can be attributed

to the success and duplication documented in deep learning [3,4].

The portions of this chapter that follow are arranged as follows. In the next section,

which discusses deep learning methods, we’ll go over ﬁve common deep learning

algorithms and their fundamental principles of operation. In the part survey of deep

Effective Deep Learning Algorithms for Personalized Healthcare … 123

Fig. 1 Deep learning-based

published papers over the

years

learning execution in medical care, an audit of writing in the medical care and biomed-

ical spaces that have utilized deep learning was researched and tended to. The factual

information of deep learning-based distributed papers throughout the years is plotted

as a reference chart in Fig. 1given.

2 Literature Survey

Deep learning affects the medical care ﬁeld. Google has contributed a great deal of

exertion seeing how profound learning models might be utilized to create forecasts

about hospitalized patients, helping doctors with clinical information and results in

the executives.

The blog article, named “Deep Learning for Electronic Health Records,”

proceeded to clarify how deep learning might be used to limit managerial weight

while further developing experiences into patient consideration and necessities. This

is the best utilization of deep learning in medical services since it diminishes manage-

rial overhead while permitting clinical specialists to zero in on the thing they are doing

best: wellbeing.

The NHS in the United Kingdom has made a commitment to using deep learning,

AI, and machine learning to become a leader in healthcare. This technology’s worth

has been acknowledged by the healthcare provider. The likelihood to upgrade patient

consideration by means of the utilization of knowledge investigation and profound

learning tool stash is absolutely interesting to the NHS, which is assailed by cost

decreases, Brexit, and chronic talent shortages [5].

Investing in deep learning technologies might help the business avoid some of the

historical issues that have hampered efﬁciency while also simplifying patient care.

It may also give much-needed assistance to healthcare practitioners.

124 A. Mishra et al.

Deep learning implementations include diagnostic imaging solutions, automa-

tion that can recognize patterns in clinical conditions, deep learning algorithms that

can help identify forms of cancer, and imaging solutions that use deep learning to

recognize rare diseases or particular kinds of pathology.

Deep learning has shown to be critical in providing medical professionals with

knowledge that allows them to recognize issues early on, allowing them to give much

more personalized and relevant care for patients.

3 Deep Learning Algorithms

The principles of operation of ﬁve deep learning algorithms are described in this

section. Each algorithm has a number of variations. The basic idea is to estimate

a function that produces the expected output given a set of inputs. Various models

are better suited to manage different problems, as well as different types of data and

planned tasks. Image classiﬁcation requires a different model than speech classiﬁca-

tion or time series classiﬁcation. To minimize the dimensionality of the data, certain

models may be used as a pre-processing step. The hidden layer, which interfaces the

contribution to the yield, is basically the design of the model. Therefore, through

the weighted association from neurons seeing the climate, a succession of actuation

is created, which is alluded to as feedforward [6]. The utilization of more secret

layers in deep learning contrasted with NN, which just has 1 or 2 hidden layers, is

one of the contrasts between profound learning and NN. NN must be prepared for

administered learning assignments, while deep learning can be prepared for both

unaided and managed to learn errands. The yield unit’s result is contrasted with the

anticipated worth toward the ﬁnish of the feedforward interaction. This evaluation

would yield a mistake esteem, which will be utilized to change connected loads in

reverse from the yield layer to the secret layer and afterward to the information layer

until the yield is near the normal outcome [7,8].

3.1 Autoencoders

AE is a feature extraction tool that uses data-driven learning. It is unsupervised and

it is taught to reconstruct the input vector rather than assigning a class mark. As

showninFig.2, the info and yield layers of AE are typically planned and organized

to give an identical number of neurons, with complete associations between neurons

in each layer. The hidden layer has less neurons than the info and yield layers [9].

The point of this design is to scramble information in a low-dimensional space and

concentrate highlights from it. The main goal is to achieve a deep AE architecture,

where various AE can be stacked together in order to obtain high dimensionality of

data. Over the past decade, several variations of AE have been proposed to manage

various data patterns and execute basic functions [10].

Effective Deep Learning Algorithms for Personalized Healthcare … 125

Fig. 2 Structure of simple

autoencoder Input layer(I)

Hidden layer(h)

Output layer(o)

II I

ooo

There are several types of autoencoders. Some of them are denoizing and sparse

autoencoders. Let’s brieﬂy discuss about them.

3.1.1 Denoizing Autoencoder

Denoizing autoencoders introduce noise into the signal, resulting in a corrupted copy.

This prohibits autoencoders from simply copying data features from the input to the

output. When training, these autoencoders use a partly corrupted input to reconstruct

the original undistorted input [11]. This basic approach of the model is to employ a

vector ﬁeld for mapping the input data to a lower-dimensional domain that describes

the natural data to smooth out the extra noise.

3.1.2 Sparse Autoencoder

Hidden nodes in sparse autoencoders are larger than input nodes. They can still

extract useful information from the results. The degree of activation is represented

by the obscurity of a node in a generic sparse autoencoder. On the hidden layer,

a sparsity constraint is applied. This is to avoid the output layer from copying the

data from the input layer. Sparsity can be obtained mostly during training phase by

applying additional terms to the loss function, by physically zeroing all but the largest

hidden unit activations, or by matching the frequency distribution of the hidden unit

activations with a small desired value.

3.2 Recurrent Neural Network

The hidden layer of this type of DL has guided graph sequence relationships between

neurons. Because of this component, it has a transient powerful condition. This is

signiﬁcant in applications including text analysis, sound analysis, DNA sequence

analysis, and continuous electric signal analysis from the body, where the output is

based on previous computations. To keep track of what happened in the previous

interval, RNNs are trained with information that has interdependencies.

126 A. Mishra et al.

Fig. 3 Feedforward recurrent neural network

By giving the same weights and preferences to all the layers, RNN transforms

independent activations into dependent activations, minimizing the difﬁculty of

increasing parameters and memorizing each previous output by feeding each output

into the next hidden layer [12].

As a result, all three layers can be combined into a single recurring layer with the

same weights and bias as the hidden layers [13]. The feedforward implementation

of recurrent neural networks has been shown in Fig. 3.

3.3 Convolutional Neural Network

CNN was motivated by biochemical instruments in the human cerebrum, where

the example of network between neurons matches the human visual cortex [14,

15]. An information layer, a few secret layers, and a yield layer make up a stan-

dard CNN. Convolutional, pooling, completely associated (FC), and standardization

layers are the most widely recognized segments of a CNN’s secret layers. A deep

learning calculation can take in a picture as info, assign importance (learnable loads

and predispositions) to different angles/objects in the picture, and recognize them

[16]. When contrasted with other characterization calculations, the measure of pre-

handling required by a CNN is fundamentally less. Albeit crude strategies require

Effective Deep Learning Algorithms for Personalized Healthcare … 127

hand-designing of channels, CNN can gain proﬁciency with these channels/attributes

with sufﬁcient experience.

By changing over the information picture into a solitary vector for arrangement,

FC layers interface the neurons in the past layer to the neurons in the last layer.

This layer contains the channel that is utilized to assess the info picture’s class. The

classmark is applied to the yield with the most noteworthy worth [17]. The greatest

beneﬁt of a CNN is that it needs to change a ton of boundaries in the channel during

back proliferation utilizing procedures like angle plummet, which extraordinarily

lessens the quantity of associations in the CNN design.

3.4 Deep Boltzmann Machine

A Boltzmann framework (BM) is an evenly coupled organization of noticeable and

inconspicuous stochastic units. The BM worldview is appropriate for displaying

and eliminating idle semantic portrayals from immense assortments of unstructured

writings [18]. To decide the information ward and information-free assumptions

in a bound pair of double factors, the ﬁrst BM calculation includes haphazardly

instated Markov chains to accomplish balance circulations. Truth be told, utilizing

this strategy, the learning method is amazingly sluggish. The Restricted Boltzmann

Machine (RBM), which has no relations between covered-up modules, was made to

accomplish productive learning.

Given the noticeable units, the restrictive circulation over the secret units factor-

izes, which is a helpful capacity of RBM. Since the RBM work portrayal is taken

to be an assortment of peripheral back appropriations acquired by unequivocally

upgrading the likelihood, deductions are manageable [18].

DBM and DBN are two signiﬁcant DL structures in this gathering that have been

presented in literary works. DBM NN has a comparative design to RBM, yet with

more obscure factors and layers. The two layers of the DBM engineering have totally

undirected collaborations between neurons [19]. To amplify the lower bound of the

probability, a stochastic most extreme likelihood-based calculation is regularly used

to prepare a DBM.

Due to the connections between the hidden neurons, assessing the conveyance over

the back secret neurons gave the noticeable neurons is impossible by unequivocally

enhancing the likelihood. DBM’s execution is imperative since it can possibly adapt

progressively complex interior portrayals, which is seen as a productive technique for

tackling acknowledgment issues. Besides, in semi-administered learning, undeniable

level portrayals can be built from a modest quantity of marked information, and

countless unlabeled data sources can be utilized to tweak the model for a speciﬁc

assignment. Notwithstanding a base up pass, DBM might coordinate hierarchical

criticism to help it spread intricacy and along these lines adapt all the more heartily

to hazy data sources [2].

128 A. Mishra et al.

3.5 Deep Belief Network

DBN is an RBM variation wherein a few secret layers can learn by utilizing the secret

yield of one RBM as information for preparing the following RBM layer [19,20]. It

has undirected associations between the main two layers and directed associations

between the layers after that. The eager layer astute preparing procedure is utilized

when the DBN is prepared to utilize solo learning and its boundaries are changed

relying upon the anticipated outcomes. The backpropagation calculation is utilized

to prepare the DBN structure, which comprises numerous secret layers of neurons

[21,22].

4 Implementation of Deep Learning in Health Care

This portion looks at a few wellbeings and biomedical ﬁelds where DL techniques

have been effectively used to assemble a model to address a speciﬁc errand. Here the

execution and use of DL is primarily examined in four regions: clinical picture, natural

framework, EHR, and report the executives and physiological signs and sensors. The

point is to show a portion of the executions of profound learning approaches that

have been created to address biomedical-related assignments that have had frustrating

results with different systems, for example, handmade, or that appear to be unsolvable

because of the errand’s intricacy.

Figure 2portrays a rundown of space ﬁelds and DL moves toward that have been

applied. The consider is isolated along with two sections, one for every application

classiﬁcation and the other for every application representation. The communication

between the substance in the application class and the substance in the application

model makes the association between the two squares.

4.1 Biological System

Natural records like DNA, RNA, genomes, sickle cell action, bacterial and viral

replication, and change have attributes that empower DL calculations to develop

prescient models that beat human specialists. Discriminative quality acknowledg-

ment, orderly evidence in protein–protein communications, picture results from

organic cell propensities, drug synthesis response proﬁling, and DNA–protein or

protein–arrangement restricting are a portion of the forecasts that have been made

[23]. The DL methodology to be used is determined by the data structure and planned

targets. CNN, AE, DBN, and RNN algorithms have been frequently used in a variety

of applications [24].

Effective Deep Learning Algorithms for Personalized Healthcare … 129

Biological

System

Physiological

Signals and

Sensors

Medical

Image

EHR and

Report

Management

1 Mitosis Detecon using breast

histopathology (CNN)

2 Prostate cancer diﬀerenaon (RNN)

1 Seizure detecon (DBN)

2 Capture the sleep informaon (CNN)

1 Cancer diagnosis (AE)

2 Alzheimer disease idenﬁcaon (DBM)

1 Disease code classiﬁcaon using free-

text (CNN)

2 Inferring phenotypic paern (AE)

Fig. 4 Application of deep learning algorithms in different medical areas

4.2 Health Record and Report Management

The electronic health record (EHR) records patient data in order to improve health

services and provide customized service as well as a chronological history [25]. Radi-

ological photographs, identiﬁcation, temporal event detection, doctor health analysis,

condition classiﬁcation, medical data, medication, laboratory testing, and outcomes

are all examples of this material. These records have grown in proportion over time,

making it impossible and daunting for health workers and medical practitioners to

keep track of them. Until the last decade, the majority of methods relied on math-

ematical tools, with just a few attempts to use machine learning [26]. Because of

the vast and growing volume of data, such approaches have now become infeasible

(Fig. 4).

4.3 Medical Image

As of late, the ubiquity of profound learning calculations for picture division, conﬁne-

ment, naming, and acknowledgment assignments has agreed with a huge expansion

in clinical picture proof. Picture information is simpler for clinicians to compre-

hend and are moderately requested and numbered, so examining them has become

130 A. Mishra et al.

a famous ﬁeld. A few distributions additionally recognized exactness in distin-

guishing an assortment of irregularities, including harmful tumors, bosom mass limi-

tation, pathology (organ parts) recognition, irresistible illnesses, and coronary vein

stenosis assignment. CNN and AE have been generally used to address troublesome

symptomatic picture issues [27].

4.4 Physiological Signals and Sensors

Sensing technology has advanced to the point that it is now possible to receive and

interpret signals from patients in order to track their emotional health, heart condition,

and illness diagnosis [28]. Feature extraction and feature selection are critical steps in

achieving an effective and high-performing model. When compared to conventional

ML techniques, DL algorithms have been effective in modeling physiological signals

and sensor data with higher precision.

The fact that this type of medical data is linear and time-dependent is one of its

most distinguishing features [29]. Accordingly, a model-building procedure ought to

think about not simply the type of the information (spatial provisions), yet in addition

the time factor (ﬂeeting elements).

5 Challenges Faced in Personalized Health Care for Deep

Learning Applications

DL is as yet in its outset in biomedical and bioengineering applications, notwith-

standing the entirety of its wonderful advances and capacities referenced in the past

area. For DL to have the option to address the basic clinical and medical care issues,

signiﬁcant difﬁculties should be survived. This section talks about a portion of the

hardships that accompany carrying out DL measures.

5.1 Medical Data Representation and Transformation

DL calculations can mention the most dependable observable facts and gauges when

given the right kind and amount of information. Unstructured clinical information is

normally contained in arrangements (DNA, time series, video signs, and sound so

on), trees (parse trees, XML reports, RNA, etc.), text information (clinical records,

manifestations portrayals, tumor depictions), or a blend of these sorts [30].

Sadly, the DL strategy’s middle can possibly deal with twofold info informa-

tion when it is progressively separated into arrangements of zeros and ones for the

handling machine to measure. Any subjective information is hard to convert into an

Effective Deep Learning Algorithms for Personalized Healthcare … 131

open conﬁguration, and handling can be tedious [31]. People can without much of

a stretch cycle and decipher these measurements, and where there is a synchronous

progress, for example, in force and amount, it very well may be effectively deci-

phered and changes made because of the changes; temperature and light are two

models.

Related cycles and conditions in DL require a lot of encoding and cautious

numerical articulations in DR and change.

How about we consider utilizing a numerous DR DL models to follow a child’s

wellbeing. At the point when the model detects a strange conduct in the newborn

child, it examinations the child’s temperature and physiological manifestations, and

it delivers a reaction for the child’s wellbeing status dependent on contribution from

the light power and the temperature of the climate and the child.

5.2 Controlling Biomedical Data Stream

Managing quick and streaming information is another DL deterrent [32]. The medical

care area is going through sensational change, with a monstrous measure of medical

services information being created at a fast speed. The beneﬁt is that clinical experts

will utilize these related to the DL model to recognize and furnish medical care

therapy with an assortment of neurotic conditions. This information can be utilized

continuously biomedical signs from an assortment of sources, for example, blood

glucose control, mind work, pulse, and oxygen immersion level, biomedical imaging

from ultrasound, electrography, and MRI, and a great many terabytes of information

for understanding into ailments.

ECG and EEG physiological detecting information are fundamental signs

acquired from different pieces of the body. DL should have the option to sort out

immense amounts of constantly changing information while as yet considering when

earlier information becomes excess. Albeit a few DL design variations have endeav-

ored to give procedures for managing this case, there are as yet inexplicable issues as

far as memory utilization, work assortment, missing information, and computational

intricacy with regards to fruitful investigations of quick, enormous scope streaming

information.

5.3 Analyzing Medical Big Data

The amazingly dependable discoveries recorded in DL are because of a lot of

information.

Elements in the information are utilized to develop or create boundaries in the

neurons during the learning interaction to accomplish forecast. It is important that

the information be huge.

132 A. Mishra et al.

Besides, it is important that the information give basic and essential usefulness to

the preparation. Any clinical areas that wish to utilize DL are restricted because of the

troubles in delivering or gathering information, and stamping information frequently

requires space specialists that are not generally qualiﬁed.

The inquiry is how much detail can be found in immense measures of information.

The decision of DL design and approval computation techniques are utilized to tune

the boundaries of neurons [33]. Around here, DL offers the apparatus to make astute

and solid information examinations, helping specialists in settling on more intelligent

choices, revealing patient wellbeing status, and building a successful AI. In any case,

there are a few deterrents that DL should defeat in this space. The fundamental

concern is getting (Medical Big Data) MBD because of the danger of information

control and absence of information sharing affectability, which could undermine

patient wellbeing, administrative issues, costly instruments, and the contribution of

clinical specialists. Another issue is the information assortment measure, which is

performed through application structures and conventions, which can be tedious. The

insights are likewise little in contrast with information from different settings (e.g.,

web-based media), and they are gotten from nonreplicable conditions or conditions

that are not broadly experienced. Other fundamental difﬁculties in MBD can be

confronted.

Notwithstanding lost records, there are blunders in encoding clinical record

information during capacity, just as contrasts in estimation hardware and size. On

account of an absence of comprehension of the worth of enormous information and

information assortment, information is not open or is deﬁcient in speciﬁc cases [34].

Manufactured information is frequently made and joined with genuine informa-

tion to arrive at an expansive information measure and hold an equilibrium of factors,

yet what amount of certainty can engineered information be given?

Various kinds of patient attributes can bring about inconsistencies in clinical

signs, for example, weight and care time, which can be an extra measurement in the

examination. These issues should be addressed to make an interconnected medical

care foundation that works on understanding quality while lessening dependence on

specialists. For great execution, DL techniques require exact and wide information.

5.4 Hardware Requirements for Medical Big Data

To work properly, a deep learning approach necessitates a signiﬁcant amount of

training data.

Health evidence in the real world is typically very broad and growing all the time.

The computer system must have adequate computational capacity to carry out

tasks and generate models.

Since a solitary focal handling unit is lacking to oversee huge scope DL activities

[35], information researchers and designers fabricated multicore superior GPUs and

related preparing units. These GPUs are expensive, devour a ton of power, and are

not broadly utilized or found in clinical establishments and clinics where information

Effective Deep Learning Algorithms for Personalized Healthcare … 133

is gathered and produced. The more perplexing the DL design, the really preparing

power is expected to ﬁnish guidance. Sending a DL arrangement in the actual world in

this manner turns into a tedious and costly undertaking. Without ﬁtting DL insightful

ways to deal with recover helpful data and mystery designs from information is futile.

6 Future Scope of Deep Learning

The recent progress in deep learning would open up new research areas and allow for

changes to existing models. This section discusses future research and development

directions, with an emphasis on healthcare and physiological signal applications.

6.1 Computational Complexity

DL is turning out to be all the more normally utilized in the clinical ﬁeld, particularly

in the space of physiological signals like ECG, electromyography (EMG), and EEG.

The bioelectrical activity of the heart is monitored by ECG, the bioelectrical activity

of the brain is monitored by EEG, and the operation of the body’s muscles and

nerves is monitored by EMG. As a result of this ﬁeld’s success, further application

and conﬁguration variance techniques will arise.

The objective of robotized signal investigation is to incorporate it into medical

services hardware as a helpful clinical symptomatic instrument that will work on

understanding quality and consider persistent wellbeing observing. To achieve this

accomplishment, future examinations should expand the intricacy of the classiﬁer

calculation to boost computational proﬁciency and intricacy [36].

6.2 Deep Learning Multitasking

Various physiological signs may now be recorded at the same time and continually

because of the advancement in the utilization of wearable gadgets lately. For different

purposes, various DL approaches might be needed for characterization and investi-

gation of these signs. Future examination should zero in on a solitary conventional

DL approach that can deal with an assortment of orders.

This strategy will save time and effort that otherwise would have been required to

develop a unique solution for each categorization. One research looked at a compli-

cated case of learning across multivariate and relational time series with missing

information, in which the relationships are represented by a graph.

They were able to ﬁll in missing data as well as forecast future values for the time

series.

134 A. Mishra et al.

Designing DL algorithms that incorporate numerous physiological data for

categorization is another potential focus.

Multiple signals from wearable devices allow for the creation of a cohesive model

by combining various signals.

Constructive use of these signals will undoubtedly improve accuracy and that

will serve numerous functions, allowing the system to work even though one of the

signals is unavailable.

6.3 Deep Learning-Based Related Work

The rise of a more astute climate is because of the Internet of Things (IoT) and huge

information.

A keen climate, as per one exploration [37], is an actual world lavishly and imper-

ceptibly joined with actuators, sensors, shows, computational parts and coordinated

immaculately in our regular things, and associated by a consistent organization and

savvy versatility. IoT gadgets are for the most part on the ascent; by 2020, it is normal

that 50 billion contraptions will be associated with the web.

This will bring about an enormous expansion in the measure of information

produced. In view of the outstanding advancement of information from connected

gadgets like remote body sensors, brilliant meters, and different devices, DL has

become the favored method for sorting out this information. On account of associ-

ation bottlenecks and generally decreases in assistance quality because of inertness

concerns, a cloud-based DL turns into a trouble. Edge registering is an idea that

proposes moving processing administrations from brought together cloud workers

to edge hubs that are nearer to end clients.

The expanding spread of cell phones and wearable gadgets has helped the

change of IoT-empowered advances from customary single-focus frameworks to

more altered medical services frameworks. mHealth interfaces patients and medical

care suppliers utilizing IoT and versatile advancements from the portable area,

permitting individuals to become advocates for their own wellbeing and advancing

correspondence among specialists and patients.

DL has been used to build a discourse pathology discovery framework using the

mHealth structure in IoT. Voices are recorded in the framework using shrewd cell

phones [38].

Prior to being shipped off a CNN, voice signals are handled. In one case, analysts

in Peru utilized DL and mHealth innovation to further develop TB discovery.

To execute DL investigation and calculation, the edge hubs devour speciﬁc infor-

mation from the cloud that is mentioned by customer gadgets. Edge ﬁguring further

develops distributed computing: it measures huge measures of information prior to

moving it to the cloud and it empowers computational capacity at the edge hub,

which upgrades cloud assets.

On an edge registering administration framework, a DL-based food distinguishing

proof framework for dietary assessment is one model. More investigation might

Effective Deep Learning Algorithms for Personalized Healthcare … 135

be done around here to build the productivity of DL execution, for example, an

appropriated, layer-focused DL design that permits cloud asset edge hub activity.

Because of limited assistance ability, network execution, and adaptability, DL

approaches to augment the quantity of occupations in a PC framework. Another

angle to consider is the exhibition evaluation and appraisal of DL tense registering.

As a result of the ascent in IoT mechanical gadgets, there would be scattered and

incorporated varieties of DL approaches for edge registering later on.

6.4 Semi-supervised Learning for Biomedical Data

Investigating the situation of both named and unlabeled information, which happens

in numerous natural areas like proteins and DNA, is another huge subject of

examination.

In such occurrences, the objective of DL is to join semi-regulated preparing

approaches to meet the necessities for effective DR learning. One technique for

future examinations to assist DL with distinguishing examples and DRs in such situ-

ations (unlabeled/solo information) is to utilize current marked/directed information

to change learned example and portrayal to get the best display for the information

[39].

Use of DL in biomedical boundaries for clinical protection observing and inves-

tigation is currently in its early stages and ordinarily experiences deﬁcient or lacking

marked information.

More exploration is required in this ﬁeld, just as the utilization of crossbreed

approaches and semi-regulated learning varieties to beat the restrictions of unlabeled

information.

6.5 Replacement of Biomedical Research Methods by Deep

Learning

There are different areas where DL might be utilized to improve wellbeing and clin-

ical beneﬁts, activities, hardware, and programming. One examination [40]intro-

duced a clinical approval strategy for publicly supporting information for diabetic

retinopathy, which is a noticeable reason for visual misfortune because of diabetes

mellitus.

A strategic relapse approach was utilized to prepare half of the dataset, which

included ordinary and obsessive order marks. A 50% dataset was utilized for testing

and approval. The affectability of the ﬁnding was 90%.

Be that as it may, the technique requires human judgment, which is inclined to bias

and incorrectness. To anticipate half of the test set, CNN might be applied to half of

136 A. Mishra et al.

the named pictures to become familiar with the components utilizing a progression

of convolution and pulling layers.

Since CNN reﬂection of elements at various levels will build the affectability

discoveries, the affectability is projected to be over 90%. Also, constant video recon-

naissance was utilized to investigate falls in dementia patients for early analysis and

avoidance [41].

The Hopkins Falls Grading Scale was utilized to direct the examination. A ﬁtting

profound learning strategy will be a mix of RNN and CNN (intermittent CNN),

which can appraise a capacity from various video outlines in a constant video casing

to deliberately survey the clinical manifestations: pre-fall, fall, and post-fall. This

can be utilized to set off a caution for the patient’s clinical consideration. To help

customers in settling on buy choices, a mainstream wearable device was offered in

wellbeing observing.

The insightful procedure utilized may become pointless as the information

increases, while a DNN will remain compelling paying little heed to the information

size and will not endure execution debasement.

Another utilization is heart auscultation, which utilizes an essential symptomatic

technique to offer data on cardiovascular hemo-elements and sicknesses.

To separate between typical and pathologic cardiovascular sounds, the LSTM

approach will proﬁciently follow the grouping of sounds from the gadget gathering

the sound.

The door and memory circuit, which is a vital part of the DL calculation, permits

LSTM to decipher the example from the sound information.

Multi-facet stack AE among patients and doctors might be utilized to accomplish

the application.

The AE design can encode and disentangle patient contribution to the planned

yield for the treatment clinician, and the other way around. The test–retest depend-

ability will be taken care of by means of the multi-facet thought. Furthermore, to

characterize the vital provisions of the considered marvel, a deduced examination

was built [42].

This involved a 6-month self-directed, cell phone-based electronic assent (eCon-

sent) methodology inside the Parkinson mPower application to recognize normal and

inventive topics connected to unequivocal assent.

The information procured during the set time frame might be used to prepare a

cloud-based DBN model, with the mPower application speaking with the model to

get the ideal yield.

The DBN layer-wise preparing approach guarantees that input information

attributes are considered while building the model.

This plan decouples handling from the portable application (accordingly making

the gadget lighter), considers model extension, permits various clients to get to the

framework, and takes into consideration focal refreshing as essential.

Meetings were ﬁnished with nine clinicians and 12 discouraged understudies who

were chosen from a guiding ofﬁce. The meeting kept going somewhere in the range

of 40 and 50 min, and it was recorded and translated. In light of the little information

size, the achievement was archived.

Effective Deep Learning Algorithms for Personalized Healthcare … 137

In any case, tremendous measures of information will make it troublesome, and

human constraints will affect execution. Subsequently, with the use of LSTM to

demonstrate the recorded sound and DNN for the organized data, the crossover DL

system will convey further developed execution.

The reception of a crossover strategy will assist with separating more signiﬁcance

from the information since the model will proceed to learn and improve as more

information opens up, wiping out human inclination and the impediments of topic

scientiﬁc models.

The utilization of a spinner for classiﬁcation of proactive tasks utilizing a mobile

phone movement locator might be perceived utilizing DL.

7 Case Studies

There are plenty of related works associated with deep learning and healthcare, some

of them are:

7.1 Clinical Imaging

Following the achievement in PC vision, the primary uses of profound ﬁguring out

how to clinical information were on picture preparing, particularly on the inves-

tigation of cerebrum Magnetic Resonance Imaging (MRI) sweeps to anticipate

Alzheimer sickness and its varieties. In other clinical areas, CNNs were utilized

to surmise a progressive portrayal of low-ﬁeld knee MRI sweeps to naturally section

ligament and foresee the danger of osteoarthritis. Regardless of utilizing 2D pictures,

this methodology got preferable outcomes over a best-in-class technique utilizing

physically chosen 3D multi-scale highlights. Profound learning was likewise applied

to portion various sclerosis sores in multi-channel 3D MRI and for the differential

ﬁnding of harmless and dangerous bosom knobs from ultrasound pictures. All the

more as of late, Gulshan utilized CNNs to distinguish diabetic retinopathy in retinal

fundus photos, acquiring high affectability and particularity over around 10,000 test

pictures concerning guaranteed ophthalmologist comments. CNNs likewise acquired

exhibitions comparable to 21 board-ensured dermatologists on ordering biopsy-

demonstrated clinical pictures of various sorts of skin malignancy over an enormous

informational collection of 130,000 pictures (1942 biopsy-named test pictures).

7.2 Electronic Health Records

All the more as of late profound learning has been applied to measure accumulated

EHRs, including both structured (e.g. conclusion, drugs, research facility tests) and

138 A. Mishra et al.

unstructured (for example free-text clinical notes) information. The best piece of

this writing handled the EHRs of a medical care framework with a profound design

for a particular, typically managed, prescient clinical errand. Speciﬁcally, a typical

methodology is to show that profound learning acquires preferable outcomes over

regular AI models regarding certain measurements, like Area Under the Receiver

Operating Characteristic Curve, exactness, and F-score. In this situation, while most

papers present start to ﬁnish regulated organizations, a few works likewise propose

unaided models to infer dormant patient portrayals, which are then assessed utilizing

shallow classiﬁers (for example irregular backwoods, strategic relapse).

A few works applied profound ﬁguring out how to anticipate sicknesses from the

patient clinical status. Liu et al. utilized a four-layer CNN to anticipate congestive

cardiovascular breakdown and ongoing obstructive aspiratory sickness and showed

critical beneﬁts over the baselines. RNNs with long transient memory (LSTM) stowed

away units, pooling and word implanting were utilized in Deep Care, a start to ﬁnish

profound unique organization that induces current disease states and predicts future

clinical results. The creators additionally proposed to direct the LSTM unit with a rot

impact to deal with sporadic coordinated occasions (which are normal in longitudinal

EHRs). In addition, they joined clinical intercessions in the model to powerfully shape

the forecasts.

8 Conclusion

ML is progressively changing how clinical treatment and checking are completed.

The entirety of this might be attributed to DL’s achievements. DL can give a reac-

tion to information examination and learning difﬁculties seen in enormous measures

of information when contrasted with customary ML and element designing.

Biomedical picture handling, wellbeing record preparing, sensors and wellbeing

record handling, human movement and feeling examination, and different spaces

have all utilized DL methods.

A decent ML part is needed for an effective AI framework; DL is rapidly turning

into the AI innovation of decision.

In this survey study, we tended to the fundamental design of DL approaches to

more readily appreciate it. The focal point of the discussion was on the standards of

activity and their application in the wellbeing and clinical ﬁelds.

The accompanying calculations were introduced: (1) AE, (2) RNN, (3) CNN,

(4) DBN, and (5) DBM. From 2012 to 2017, we investigated the direction of DL

establishment. We saw a consistent increment, with CNN having the biggest pace of

increment.

For effectiveness and versatility, framework and correspondence engineering will

dynamically modify to help large information and DL draws near. What is more,

there are some innate issues in DL that should be dealt with. In the real world, the

majority of these informations are in alternate organizations that cannot be dealt with

utilizing DL methods and requires an extra layer of encoding and portrayal.

Effective Deep Learning Algorithms for Personalized Healthcare … 139

Clinical information is expensive to get, and the dataset contains passages that are

absent or conﬂicting. As per the factual outcomes announced in this survey study,

future applications and improvements in DL will see expanding utilization of CNN

in clinical picture preparation.

Inside the overall DL draws near, there will be many sorts of DL draws near.

The utilization of physiological signs for analysis using DL procedures will turn

out to be more normal. With the appearance of IoT and edge ﬁguring advancements,

another model of DL will arise to oblige this innovation.

To examine monstrous information from wearable and cell phones, DL will be

utilized related to cloud and edge ﬁguring to foster AI for mHealth.

Accordingly, we might presume that DL gives an extraordinary calculation and

is a suitable answer for MBD’s issues.

Nonetheless, utilizing DL in each application that includes information examina-

tion ought not to be done at the expense of elective ML calculations that produce

equivalent outcomes yet require less preparation and memory. Moreover, different

AI calculations that have a solid shot at getting superior with immense information

ought to be considered to address the issue for information examination.

References

1. Muller, H., & Unay, D. (2017). Retrieval from and understanding of large-scale multi-modal

medical datasets: A review. IEEE Transactions on Multimedia, 19(9), 2093–2104.

2. Ravi, D., Wong, C., Deligianni, F., Berthelot, M., Andreu-Perez, J., Lo, B., & Yang, G.-

Z. (2017). Deep learning for health informatics. IEEE Journal of Biomedical and Health

Informatics, 21(1), 4–21.

3. Alam, M., Vidyaratne, L. S., & Iftekharuddin, K. M. (2018). Sparse simultaneous recurrent

deep learning for robust facial expression recognition. IEEE Transactions on Neural Networks

and Learning Systems, 29(10), 4905–4916.

4. Mishra, S., Panda, A., & Tripathy, K. H. (2018). Implementation of re-sampling technique

to handle skewed data in tumor prediction. Journal of Advanced Research in Dynamical and

Control Systems, 10, 526–530.

5. Tobore, I., Li, J., Yuhang, L., Al-Handarish, Y., Kandwal, A., Nie, Z., & Wang, L. (2019). Deep

learning intervention for health care challenges: Some biomedical domain considerations. JMIR

mHealth and uHealth, 7(8), e11966.

6. Sahoo, S., Das, M., Mishra, S., & Suman, S. (2021). A hybrid DTNB model for heart disorders

prediction. In Advances in electronics, communication and computing (pp. 155–163). Springer.

7. Yousoff, S. N. M., Baharin, A., & Abdullah, A. (2016). A review on optimization algorithm for

deep learning method in bioinformatics ﬁeld. In 2016 IEEE EMBS Conference on Biomedical

Engineering and Sciences (IECBES).

8. Jena, L., Mishra, S., Nayak, S., Ranjan, P., & Mishra, M. K. (2021). Variable optimiza-

tion in cervical cancer data using particle swarm optimization. In Advances in electronics,

communication and computing (pp. 147–153). Springer.

9. Vincent, P., Larochelle, H., Bengio, Y., & Manzagol, P.-A. (2008). Extracting and composing

robust features with denoising autoencoders. In Proceedings of the 25th International

Conference on Machine Learning—ICML’08.

10. Roy, S. N., Mishra, S., & Yusof, S. M. (2021). Emergence of drug discovery in machine

learning. In Technical advancements of machine learning in healthcare (Vol. 936, p. 119).

140 A. Mishra et al.

11. Masci, J., Meier, U., Cire¸san, D., & Schmidhuber, J. (2011). Stacked convolutional auto-

encoders for hierarchical feature extraction. In Lecture notes in computer science (Vol. 52,

pp. 52–59). Springer Berlin Heidelberg.

12. Alom, M. Z., Taha, T. M., Yakopcic, C., Westberg, S., Sidike, P., Nasrin, M. S., Van Esesn, B.

C., Awwal, A. A. S., & Asari, V. K. (2018). The history began from AlexNet: A comprehensive

survey on deep learning approaches. arXiv [cs.CV].

13. Bengio, Y., Simard, P., & Frasconi, P. (1994). Learning long-term dependencies with gradient

descent is difﬁcult. IEEE Transactions on Neural Networks, 5(2), 157–166.

14. Matsugu, M., Mori, K., Mitari, Y., & Kaneda, Y. (2003). Subject independent facial expression

recognition with robust face detection using a convolutional neural network. Neural Networks,

16(5–6), 555–559.

15. Hubel, D. H., & Wiesel, T. N. (1962). Receptive ﬁelds, binocular interaction and functional

architecture in the cat’s visual cortex. Journal of Physiology, 160(1), 106–154.

16. Tripathy, H. K., Mishra, S., Thakkar, H. K., & Rai, D. (2021). CARE: A collision-aware

mobile robot navigation in grid environment using improved breadth ﬁrst search. Computers &

Electrical Engineering, 94, 107327.

17. Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017). ImageNet classiﬁcation with deep

convolutional neural networks. Communications of the ACM, 60(6), 84–90.

18. Salakhutdinov, R., & Brain, H. L. (2010). Efﬁcient learning of deep Boltzmann machines.

Mlr.press. [Online]. Available: http://proceedings.mlr.press/v9/salakhutdinov10a/salakhutdino

v10a.pdf. Accessed June 28, 2021.

19. Hinton, G. E., & Salakhutdinov, R. R. (2006). Reducing the dimensionality of data with neural

networks. Science, 313(5786), 504–507.

20. Kim, J. Y., Lee, H. E., Choi, Y. H., Lee, S. J., & Jeon, J. S. (2019). CNN-based diagnosis models

for canine ulcerative keratitis. Science and Reports, 9(1), 14209.

21. Ryu, S., Noh, J., & Kim, H. (2016). Deep neural network based demand side short term load

forecasting. Energies, 10(1), 3.

22. Tripathy, H. K., Mallick, P. K., & Mishra, S. (2021). Application and evaluation of classiﬁcation

model to detect autistic spectrum disorders in children. International Journal of Computer

Applications in Technology, 65(4), 368–377.

23. Mishra, S., Mohapatra, S. K., Mishra, B. K., & Sahoo, S. (2018). Analysis of mobile cloud

computing: Architecture, applications, challenges, and future perspectives. In Applications of

security, mobile, analytic, and cloud (SMAC) technologies for effective information processing

and management (pp. 81–104). IGI Global.

24. Xu, M., Papageorgiou, D. P., Abidi, S. Z., Dao, M., Zhao, H., & Karniadakis, G. E. (2017). A

deep convolutional neural network for classiﬁcation of red blood cells in sickle cell anemia.

PLoS Computational Biology, 13(10), e1005746.

25. Nguyen, P., Tran, T., Wickramasinghe, N., & Venkatesh, S. (2017). $\mathtt Deepr$: A convo-

lutional net for medical records. IEEE Journal of Biomedical and Health Informatics, 21(1),

22–30.

26. Lin, C., Hsu, C. J., Lou, Y. S., Yeh, S. J., Lee, C. C., Su, S. L., & Chen, H. C. (2017).

Artiﬁcial intelligence learning semantics via external resources for classifying diagnosis codes

in discharge notes. Journal of Medical Internet Research, 19(11), e380.

27. Wang, S.-H., Lv, Y.-D., Sui, Y., Liu, S., Wang, S.-J., & Zhang, Y.-D. (2017). Alcoholism

detection by data augmentation and convolutional neural network with stochastic pooling.

Journal of Medical Systems, 42(1), 2.

28. Zhang, J., & Wu, Y. (2017). A new method for automatic sleep stage classiﬁcation. IEEE

Transactions on Biomedical Circuits and Systems, 11(5), 1097–1110.

29. Dominguez Veiga, J. J., O’Reilly, M., Whelan, D., Caulﬁeld, B., & Ward, T. E. (2017). Feature-

free activity classiﬁcation of inertial sensor data with machine vision techniques: Method,

development, and evaluation. JMIR mHealth and uHealth, 5(8), e115.

30. Sacha, D., Sedlmair, M., Zhang, L., Lee, J. A., Weiskopf, D., North, S., & Keim, D. (2016).

Human-centered machine learning through interactive visualization. In ESANN 2016: 24th

European Symposium on Artiﬁcial Neural Networks, Computational Intelligence and Machine

Learning, Proceedings (pp. 641–646), Bruges, April 27–29, 2016.

Effective Deep Learning Algorithms for Personalized Healthcare … 141

31. Yang, X., Zhang, T., & Xu, C. (2015). Cross-domain feature learning in multimedia. IEEE

Transactions on Multimedia, 17(1), 64–78.

32. Najafabadi, M. M., Villanustre, F., Khoshgoftaar, T. M., Seliya, N., Wald, R., & Muharemagic,

E. (2015). Deep learning applications and challenges in big data analytics. Journal of Big Data,

2(1).

33. Chattopadhyay, A., Mishra, S., & González-Briones, A. (2021). Integration of machine learning

and IoT in healthcare domain. In Hybrid artiﬁcial intelligence and IoT in healthcare (pp. 223–

244). Springer.

34. Lindsey, M. L., Kassiri, Z., Virag, J. A. I., de Castro Brás, L. E., & Scherrer-Crosbie, M. (2018).

Guidelines for measuring cardiac physiology in mice. American Journal of Physiology-Heart

and Circulatory Physiology, 314(4), H733–H752.

35. Raina, R., Madhavan, A., & Ng, A. Y. (2009). Large-scale deep unsupervised learning using

graphics processors. In Proceedings of the 26th Annual International Conference on Machine

Learning—ICML’09.

36. Page, A., Turner, J. T., Mohsenin, T., & Oates, T. (2014). Comparing raw data and feature

extraction for seizure detection with deep learning methods. Umbc.edu. [Online]. Available:

http://eehpc.csee.umbc.edu/publications/pdf/2014/Adam_FLAIRS14.pdf. Accessed June 28,

2021.

37. Ahmed, E., Yaqoob, I., Gani, A., Imran, M., & Guizani, M. (2016). Internet-of-things-based

smart environments: State of the art, taxonomy, and open research challenges. IEEE Wireless

Communications, 23(5), 10–16.

38. Liu, C., Cao, Y., Luo, Y., Chen, G., Vokkarane, V., Yunsheng, M., Chen, S., & Hou, P. (2018). A

new deep learning-based food recognition system for dietary assessment on an edge computing

service infrastructure. IEEE Transactions on Services Computing, 11(2), 249–261.

39. Mukherjee, D., Tripathy, H. K., & Mishra, S. (2021). Scope of medical bots in clinical domain.

In Technical advancements of machine learning in healthcare (Vol. 936, p. 339).

40. Brady, C. J., Mudie, L. I., Wang, X., Guallar, E., & Friedman, D. S. (2017). Improving

consensus scoring of crowdsourced data using the Rasch model: Development and reﬁnement

of a diagnostic instrument. Journal of Medical Internet Research, 19(6), e222.

41. Bayen, E., Jacquemot, J., Netscher, G., Agrawal, P., Tabb Noyce, L., & Bayen, A. (2017).

Reduction in fall rate in dementia managed care through video incident review: Pilot study.

Journal of Medical Internet Research, 19(10), e339.

42. Doerr, M., Maguire Truong, A., Bot, B. M., Wilbanks,J., Suver, C., & Mangravite, L. M. (2017).

Formative evaluation of participant experience with mobile eConsent in the app-mediated

Parkinson mPower study: A mixed methods study. JMIR mHealth and uHealth, 5(2), e14.

Automatic Lung Carcinoma

Identiﬁcation and Classiﬁcation in CT

Images Using CNN Deep Learning Model

Ritu Tandon, Shweta Agrawal, Rachana Raghuwanshi,

Narendra Pal Singh Rathore, Lalji Prasad, and Vishal Jain

Abstract Lung cancer is the most prevalent malignancy that cannot be avoided and

that causes late health care death. At now, CT scans can be used to assist physi-

cians to diagnose early stage lung cancer. In many situations, lung cancer detec-

tion depends on doctors’ experience, which might neglect some patients and create

certain issues. Deep learning in several diagnostic ﬁelds of medical imaging has

become a popular and powerful approach. The deep study models employ the Convo-

lutional Neural Network (CNN), which extracts features and classiﬁes the picture

using a fully connected network. The CNN leverages this functionality. The chapter

presented study of deep learning algorithm for lung cancer detection. The experiment

is performed with CNN by utilizing LIDC-IDRI dataset. It is referred to as the Lung

Image Database Consortium image collection and comprises of diagnosis thoracic

computed tomography (CT) scans which are labeled. It is accessible worldwide and

is updated on a regular basis. The classiﬁcation performance is measured for the

matrices F1 score, Recall, precision, support, and accuracy. The accuracy achieved

with experiment is 96.5%

Keywords CNN ·Deep learning ·CT scans ·Medical image processing ·AI

1 Introduction

The most frequent cause of death worldwide is cancer [1]. For researchers, clinicians

and patients, cancer is a challenging disorder to tackle [2]. In 2019, 96,480 deaths

are caused by skin cancer, followed by lung cancer, 142,670 deaths, 42,260 deaths

of breast cancer and 31,620 deaths of the prostate and 17,760 deaths of brain cancer

R. Tandon (B)·S. Agrawal ·R. Raghuwanshi ·N. P. S. Rathore ·L. Prasad

SAGE University, Indore, M.P., India

e-mail: ritu.tondon@sageuniversity.in

N. P. S. Rathore

e-mail: narendrasingh@acropolis.in

V. Ja i n

Sharda University, Noida, India

S. Mishra et al. (eds.), Augmented Intelligence in Healthcare: A Pragmatic and Integrated

Analysis, Studies in Computational Intelligence 1024,

https://doi.org/10.1007/978-981-19- 1076-0_9

143

144 R. Tandon et al.

Fig. 1 Deaths due to cancer disease

(American Cancer Society, new cancer release report 2). Figure 1shows the number

of deaths due to cancer [3].

Early identiﬁcation of cancer is necessary for many people’s lives to be saved.

For these types of cancer diagnoses, eye examination and manual procedures are

typically used. This method of manually interpreting medical images takes a long

time and is prone to errors.

About 334 million people have asthma, and tuberculosis kills 1.4 million people

each year, Lung cancer affects the lives of 1.6 million individuals each year and

pneumonia kills millions more [4]. The epidemic of COVID-19 inﬂuenced the entire

planet. Lung problems are undeniably one of the world’s top causes of mortality

and disability. Early identiﬁcation is key to improving long-term survival rates and

boosting the possibilities of recovery [5,6]. Lung problems can be examination using

skin tests, sputum sample tests, blood tests [7].

Deep learning is a subset of machine learning based on the structure and function

of the human brain [8]. Machine learning advancements have helped to discover,

measure, and describe patterns in medical pictures, in particular in deep learning [9].

Mechanical learning (ML) algorithms are becoming extremely prominent in the ﬁeld

of identiﬁcation of cancer cells utilizing medical imagery and feature extraction. In

medical imaging, several kinds of feature extraction methods have therefore been

examined for the detection of various forms of cancer such as brain, lung, skin, liver,

cervical, and brain [10–17].

Deep learning offers the beneﬁt of producing high-level pictures from raw photos.

Graphics processing units (GPU) are used for functional extraction and picture

recognition in accordance with a deep learning. For example, convolutional neural

networks have shown to be capable of detecting cancer with good accuracy [18–21].

We are using publicly available datasets like LIDC-IDRI dataset that can be used

to test these algorithms. In this book chapter we will focus on proposed methodology,

evaluation, and results using the LIDC-IDRI dataset [22].

Automatic Lung Carcinoma Identiﬁcation and Classiﬁcation … 145

2 Augmented Intelligence in Healthcare

A better word for smart healthcare systems is “augmented intelligence,” a concept

that more appropriately reﬂects their aim because they are designed to live with

human decision-making.

Augmented intelligence is also gaining very much popularity in medical domain.

In healthcare, augmented intelligence makes more sense than artiﬁcial intelligence.

This is because it emphasizes a human’s improved capabilities when supplemented

with the appropriate tools and technologies—something that might become a reality

in the near future. Deep learning algorithms with augmented reality can also be used

to classify medical images.

An AI system that uses machine learning employs a learner algorithm that is

programmed to learn from data referred to as “training data.” Based on the training

data, the learner algorithm will automatically change the machine learning model.

As fresh data is supplied, a “continuous learning system” updates the model without

human intervention, whereas “locked learners” do not automatically update the model

with new data. In order to assess systems for quality, safety, and bias in health care, it’s

critical to determine whether the learner algorithm is eventually locked or continues

to learn once implemented into clinical practice. Understanding the risk of applying

a health care AI system to individuals whose personal features differ signiﬁcantly

from those in the training data set requires being able to track the source of training

data.

Of course, this technology is still in its early stages, and regulations have yet to

catch up. Though adoption is modest in the industry, the technology’s potential for

wider use forces us to consider the problems it may represent. Integrating private

patient records with any type of technology, especially one as advanced as AI, carries

a number of cyber security hazards.

Another danger is the possibility of malicious image manipulation. Researchers

are concerned that hackers could tamper with digital radiological scans and “trick”

an AI into warning a patient for lung cancer, while this is an unlikely scenario.

Of course, clinicians would assess these ﬂagged individuals accordingly, referring

them to specialists and ordering follow-up testing as needed, but experts nevertheless

recommend that developers be mindful of possible hazards like these. Figure 2shows

the various applications of augmented intelligence.

3 Deep Learning

Machine Learning is an artiﬁcial intelligence ﬁeld and deep learning is the machine

learning branch [23]. AI’s a wide phrase referring to technologies that let computers

to imitate the human behavior. All of this is made possible through machine learning,

which is a set of algorithms trained on data. Figure 3shows the relationship between

machine learning and deep learning.

146 R. Tandon et al.

Fig. 2 Applications of augmentation intelligence

Fig. 3 Relationship between

deep learning and machine

learning

3.1 Artiﬁcial Intelligence

A program that can observe, compute, execute, and adapt is called as Artiﬁcial, Intel-

ligence. Artiﬁcial intelligence (AI) is human intelligence simulation of computers

designed for people to think and act like human beings [24]. Any machine with

human-like features such as problem-solving, and learning is known as a humanoid

machine.

Automatic Lung Carcinoma Identiﬁcation and Classiﬁcation … 147

3.2 Machine Learning

Algorithms that improve performance if more data is accessible over time. Machine

learning is an artiﬁcial intelligence (AI) branch focused on the design of programs

that accurately learn from data without being expressly planned.

3.3 How Deep Learning Is Used in Image Processing

and Medical Text

Deep learning is a machine learning branch that learns from a vast amount of data in

a diversiﬁed neural network. Deep learning is an Artiﬁcial Intelligence and Machine

Learning (AI) approach which replicates the gathering of human knowledge. Deep

learning is a major component of data science, which includes statistics and predictive

modeling. Data scientists who are in charge of obtaining, analyzing, and interpreting

large amounts of data would actually beneﬁt from deep learning. It enhances and

improves the procedure. In other words, accuracy. Deep learning achieves higher

recognition accuracy than ever before. This enables consumer electronics to satisfy

user expectations, which is vital for safety-sensitive applications such as self-driving

cars. For deep learning, large quantities of labeled data are necessary. For instance,

millions of photographs and hundreds of hours of videos involve the development

of self-driving automobiles.

In a machine learning system, domain knowledge and a range of methods are

required in order for raw data to be organized for learning models like classiﬁca-

tion, pattern identiﬁcation, and inference making. Input data are typically converted

in linear form in machine learning methods and their ability to analysis naturally

occurring in its primitive form is limited [25,26]. In the representation of the training

model’s input data, deep learning differs from conventional ML. In deep learning,

feature extraction from input data is not necessary. Because of their complexity, Deep

learning models are also known as deep neural networks when compared to ordi-

nary artiﬁcial neural networks (ANN’s) [27]. In most cases, have three layers (input

layer, hidden layer, and output layer). In compared to ANN, deep learning models

have more interconnections which allow them to comprehend important and helpful

ideas. The data inputs deep learning architectures include a hierarchical learning

framework, which allows them to include heterogeneous data from a variety of data

sources. Figure 4depicts a schematic diagram of the process ﬂow in deep learning.

The technique is identical to that of the machine learning technique, except that no

feature extraction block. The task is completed via the deep learning algorithm.

148 R. Tandon et al.

Fig. 4 Deep learning ﬂow diagram

3.4 Convolutional Neural Network (CNN)

A convolutional neural network (CNN) is a feed-forward neural network that

processes data in a grid-like structure and is commonly used to evaluate visual

images. A ConvNet is another name for this. To detect and identify objects in images,

a convolutional neural network is utilized [28–32].

A neural network that can distinguish between two types of ﬂowers, Marigold

and Rose, is shown in Fig. 5.

Every image in CNN is represented as a collection of pixel values. Figure 6shows

the representation of digit 7.

Any convolutional neural network starts with the convolution operation. Let’s

look at the convolution operation with two one-dimensional matrices, aand b.

Fig. 5 Identiﬁcation of ﬂowers using CNN

Automatic Lung Carcinoma Identiﬁcation and Classiﬁcation … 149

Fig. 6 Pixels representation of digit 7

a=[5,3,7,5,9,7]

b=[1,2,3]

The arrays are multiplied element-by-element in the convolution process, and the

product is summed to generate a new array that represents a*b.

The elements of matrix bare multiplied by the ﬁrst three components of matrix

a. To obtain the result, the product is added together.

150 R. Tandon et al.

This procedure is repeated until the convolution operation is ﬁnished.

Example: Image recognition assists in the development of Disease detection or

self-driving cars in healthcare is done via visual imagery. It is all possible using

convolutional neural networks work. Figure 7is an illustration of how convolutional

neural networks work:

Consider the following scenario: You have a picture of a bird and want to know if

it is a bird or anything else. The initial step is to feed the image’s pixels to the neural

network’s input layer in the form of arrays. By conducting various calculations and

manipulations, the hidden layers extract features. Multiple hidden layers, such as the

Fig. 7 Identiﬁcation of bird using CNN

Automatic Lung Carcinoma Identiﬁcation and Classiﬁcation … 151

Fig. 8 Filters in convolution layer

ReLU layer, the convolution layer, and the pooling layer, extract features from the

image. Finally, there’s a fully linked layer that recognizes the image’s object.

3.5 Convolutional Neural Network Layers

Multiple hidden layers in a convolution neural network improve in the extraction of

information from an image [33,34]. CNN’s four most signiﬁcant layers are:

1. Convolution layer

2. ReLU layer

3. Pooling layer

4. Fully connected layer.

3.5.1 Convolution Layer

This is the initiation phase in obtaining useful information from an image. The

convolution action is performed by many ﬁlters in a convolution layer. Every image

is viewed as a pixel value matrix.

Consider the 5 ×5 image below, where the pixel values are either 0 or 1. A ﬁlter

matrix with a 3 ×3 dimension is also included. Slide the ﬁlter matrix over the picture

and calculate the point’s product to get the combined feature matrix (Fig. 8).

3.5.2 ReLU Layer

The rectiﬁed linear unit is abbreviated as ReLU. After the feature maps have been

obtained, they must be moved to a ReLU layer.

ReLU performs an element-by-element process, setting all negative pixels to zero.

It causes the network to become non-linear, and the result is a rectiﬁed feature map.

152 R. Tandon et al.

3.5.3 Pooling Layer

Pooling is a down sampling method, which reduces the dimension of the charac-

teristic map. The corrected function map is now transmitted via a pooling layer to

generate a pooled function map.

The pooling layer includes a variety of ﬁlters to recognize distinct types of data.

Like Edges, corners, body, feathers, eyes, and beak are examples of picture elements.

3.5.4 Fully Connected Layer

The ﬂattened matrix is sent as an input to the fully connected layer to identify the

picture.

4 Lung Cancer Classiﬁcation Using CNN

From earlier researchers, it is apparent that CNN has the capability to perform better

to detect lung cancer using CT images as compared to the manual detection done by

the radiologists. Some studies have also demonstrated that CNN also exceeds expert

radiologists for the identiﬁcation of lung nodules [35,36].

CNN has taken two measures to identify and classify lung cancer. The ﬁrst stage

is the extraction of the functionalities by the CNN and then in the second step the

categorization of lung pictures is carried out by the artiﬁcial neural network. In CNN

end to end learning is used or we can use transfer learning by using the pre-trained

models. For achieving higher accuracy, a large dataset is required to overcome the

over-ﬁtting issues. The transfer learning approach performs better as compared to

the end-to-end learning in the case of the small dataset also. Figure 9provides the

process of classiﬁcation model to classify lung images as being normal or malignant

lung nodule.

4.1 Data Collection

In this chapter, we have used the LIDC-IDRI dataset for collecting the CT images

of the lungs. This LIDC-IDRI dataset contains 1018 low dose CT image cases of

1010 patients that are collected by the collaboration of 8 medical companies and 7

academic centers. All the 1018 cases include XML ﬁles that contain explanations

about the CT images. This dataset has been developed by 04 radiologists to produce

annotations of the CT pictures and classify these images into the three groups (nodule

<> 3 mm, nodule <> 3 mm, non-nodule >> respectively). This dataset comprises a

total of 244,527 pictures of 512 * 512 with the width varying from 65 to 764. With

Automatic Lung Carcinoma Identiﬁcation and Classiﬁcation … 153

Fig. 9 Process ﬂow of classiﬁcation model

the help of this dataset, two level diagnoses can be done ﬁrst at the patient level and

second at the nodule level. The CT imaging nodules are categorized as:

•unknown levels

•normal or initial lung cancer

•metastatic lesion

•Malignancy.

154 R. Tandon et al.

4.2 Data Preprocessing and Augmentation

In the preprocessing phase, the collected CT Dicom images are converted into the jpg

format by using the radiant Dicom viewer. These images are having various pixels

that do not contain any information about the images, so these pixels are removed

using the compression technique of artiﬁcial intelligence. Images of LIDC-IDRI are

categorized into two categories of normal images and malignant images. A deep

learning model performs with a lot of information, but it is a time-consuming and

laborious process to acquire huge numbers of data. Therefore, the artiﬁcial dataset

from the available dataset was generated using the data augmentation technique using

rotation, cutting, padding, and ﬂipping and brightness changes.

4.3 Convolution and Pooling Layer for Feature Extraction

Extracting features is a key element in CT images categorization. The automated

learning of CNN functions will so differ from the learning of manual function. The

malignant nodule has lesions at ≥3 mm in the visual depiction of lungs CT scans.

CNN model is used to process the images with the labeled class such as normal or

malignant. At the time of the training process from the automatic weight updation,

the image’s features will be extracted by CNN. CNN model has two layers in our

proposed design.

•Convolutional layer

•Pooling layer.

The number of ﬁlters used to improve the input depth will be raised after convo-

lution and a pooling layer [37] will be employed to maintain the same depth while

lowering their size.

4.4 Classiﬁcation

Flattened weighted layer feature map that is collected from the ﬁnal pooling layer

is used in the classiﬁcation process and it will be utilized as an input to the fully

connected layer, which will generate the performance metric for displaying the loss

and accuracy, and the internal node values will be automatically updated to improve

the outcome based on the weights of the metric values.

These layers are stacked when the preprocessing is completed. As is usual, the

output of the last layer is used as the ﬁnal output.

Automatic Lung Carcinoma Identiﬁcation and Classiﬁcation … 155

5 Experimental Work

5.1 Model Creation

Here we are discussing the process to detect malignant lung nodule from the CT

images. For the collection of images, we have used the LIDC-IDRI dataset that

contains 2 classes (i) normal and (ii) Malignant. The LIDI-IDRI dataset in this model

employed a total of 1000 CT scans. The data set is divided into 90% training and

10% test samples. Figure 10a, b shows the normal and malignant image of the lung

from the LIDC-IDRI dataset.

This CNN model for lung cancer detections is implemented in Python and uses

NumPy, OpenCV, and Keras as deep learning libraries. The batch size is 32 bits, and

the most essential hyper-parameter to alter in deep learning is batch size. Medium

size of data is used to train our algorithms by employing a small lung image dataset.

Some data increase is also employed to produce artiﬁcial training from the existing

images using other ways. An image generator object is used which performs random

ﬂips, crop rotations, and shifts on our lung CT images. This allows us to achieve

higher accuracy by using the smaller dataset also. In the model, Our CONV layer

contains 32 ﬁlters for 3 ×3 kernel and RELU (Rectiﬁed Linear Unit), a 3 ×3size

pooling layer and a high density for ReLu and Softmax. 25 percent dropout (0.25),

batch normalization, and max pooling are employed. The visualization is done after

each layer and Adam optimizer is employed at with an epoch of 15. Graphical

representation of architecture of model is shown in Fig. 11.

5.2 Result and Discussion

After the successful implementation of our model for classiﬁcation of lung nodule

with convolution and pooling layer sample images are taken from the training dataset

and after every layer output visualization is done and shown in Fig. 12a–f.

5.2.1 Performance Matrices

The accuracy just is not adequate in the classiﬁcation to assess the model’s entire

efﬁciency. Consequently, accuracy, recall, F1 score, and support are assessed for each

type of lung lesion illness [38]. We have created the confusion matrix to illustrate

how well our model works in all classes.

Now we will learn the calculation and meaning of those performance metrics.

In ﬁgure where TP =True Positive, FN =False Negative, FP =False Positive,

TN =True Negative, the terms for creating confusion matrix are deﬁned. This is

the genuine positive scenario if the lung nodule, for example, is marked with the

malignancy and is also predicted as malignant in the model. If the picture is labeled

156 R. Tandon et al.

Fig. 10 a Normal lung

image. bMalignant lung

image

(a)

(b)

Automatic Lung Carcinoma Identiﬁcation and Classiﬁcation … 157

Fig. 11 Model architecture

malignant yet classed as normal, the wrong negative is the case. False positive case

occurs when the categorization model indicates that the lung nodule is cancerous,

while in fact it is normal. If the classiﬁer suggests that a regular lung nodule is normal,

the negative is true.

Accuracy

Accuracy is the most important measure that is calculated by total predicted samples

out of total samples.

Accuracy =TP +TN

TP +TN +FP +FN (1)

Precision

Precision is required to determine the right percentage of positive observations?

Precision =TP

TP +FP (2)

Sensitivity/Recall

Recall is the real positive rate computation. It displays how much of the real positive

has been accurately observed?

Recall =TP

TP +FN (3)

F1 Score

F1 score is also called F, and it is calculated by taking and recalling a weighted

average accuracy.

158 R. Tandon et al.

F1 Score =2TP

2TP +FN +FP (4)

Support

To calculate support, the number of real class incidents in the data set is utilized.

(a)

(b)

Fig. 12 a Visualization of layers (conv2d_1). bVisualization of layers (max_pooling2d_1). c

Visualization of layers (conv2d_2). dVisualization of layers (max_pooling2d_2). eVisualization

of layers (conv2d_3). fVisualization of layers (max_pooling2d_3)

Automatic Lung Carcinoma Identiﬁcation and Classiﬁcation … 159

(c)

(d)

Fig. 12 (continued)

160 R. Tandon et al.

(e)

(f)

Fig. 12 (continued)

Automatic Lung Carcinoma Identiﬁcation and Classiﬁcation … 161

Tabl e 1 Hyper parameters

detail S. No. Hyper parameter Valu e

1Optimizer Adam

2Epoch 15

3Dropout 0.25

4 Learning rate 0.0001

5Batch size 32

5.3 Hyper Parameters

Table 1represents the different hyper parameters to train the CNN for lung cancer

detection.

5.4 Model Performance

This section provides the classiﬁcation of the CNN model for lung cancer detection in

terms of accuracy, loss, and other performance metrics. Figure 13a, b shows training

and validation accuracy and loss.

The classiﬁcation report of the model is depicted in Table 2and confusion matrix

is shown in Fig. 14.

5.5 Classiﬁcation Report

Validation loss: 0.0804193764925003.

Validation accuracy: 0.9650145769119263.

After the successful implementation for prediction of image to detect normal or

malignant we have tested our model by passing one image for prediction. Prediction

output is generated as normal image in Fig. 15.

162 R. Tandon et al.

(a)

(b)

Fig. 13 a Training and validation accuracy. bTraining and validation loss

Tabl e 2 Classiﬁcation report

Precision Recall F1-score Support Average accuracy

Normal 0.57 0.56 0.57 195 96.5

Cancer 0.44 0.45 0.45 148

Macro avg. 0.51 0.51 0.31 343

Weighted avg. 0.52 0.51 0.51 343

Automatic Lung Carcinoma Identiﬁcation and Classiﬁcation … 163

Fig. 14 Confusion matrix

Fig. 15 Output image

164 R. Tandon et al.

6 Conclusion

Lung cancer is a complicated problem and cause of many deaths. Deep learning has

been demonstrated to be a precision that allows for the use not only as a second

diagnostic opinion, but also as a strong tool that clinicians may consider in their job.

Many Deep Learning algorithms have been used in different applications of medical

domain. The performance presented by CNN in medical imagining is commendable

and can be utilized to classify and detect cancerous or non-cancerous images in

collaboration with medical practitioners. The chapter presented study of CNN model

for lung cancer detection. We have provided in this article the CNN model with the

common aim to alleviate radiologist’s task in the identiﬁcation of lung nodules.

The experiment is performed with LIDC-IDRI dataset. The classiﬁcation accuracy

achieved is 96.5%.

References

1. Ali, I., Wani, W. A., & Saleem, K. (2011). Cancer scenario in India with future perspectives.

Cancer Therapy, 8.

2. Siegel, R. L., Miller, K. D., & Jemal, A. (2016). Cancer statistics, 2016. CA: A Cancer Journal

for Clinicians, 66(1), 7–30.

3. Mishra, S., Chaudhury, P., Mishra, B. K., & Tripathy, H. K. (2016, March). An implemen-

tation of feature ranking using machine learning techniques for diabetes disease prediction.

In Proceedings of the Second International Conference on Information and Communication

Technology for Competitive Strategies (pp. 1–3).

4. Fass, L. (2008). Imaging and cancer: A review. Molecular Oncology, 2(2), 115–152.

5. Barentsz, J., Takahashi, S., Oyen, W., Mus, R., De Mulder, P., Reznek, R., Oudkerk, M., &

Mali, W. (2006). Commonly used imaging techniques for diagnosis and staging. Journal of

Clinical Oncology, 24(20), 3234–3244.

6. Chaudhury, P., Mishra, S., Tripathy, H. K., & Kishore, B. (2016, March). Enhancing the capabil-

ities of student result prediction system. In Proceedings of the Second International Conference

on Information and Communication Technology for Competitive Strategies (pp. 1–6).

7. Khan, M. A., Akram, T., Sharif, M., Shahzad, A., Aurangzeb, K., Alhussein, M., & Altamrah, A.

(2018). An implementation of normal distribution based segmentation and entropy controlled

features selection for skin lesion detection and classiﬁcation. BMC Cancer, 18(1), 638.

8. Nasir, M., Attique Khan, M., Sharif, M., Lali, I. U., Saba, T., & Iqbal, T. (2018). An improved

strategy for skin lesion detection and classiﬁcation using uniform segmentation and feature

selection based approach. Microscopy Research and Technique, 81(6), 528–543.

9. Esteva, A., Robicquet, A., Ramsundar, B., Kuleshov, V., DePristo, M., Chou, K., Cui, C.,

Corrado, G., Thrun, S., & Dean, J. (2019). A guide to deep learning in healthcare. Nature

Medicine, 25(1), 24–29.

10. Hawkins, S. H., Korecki, J. N., Balagurunathan, Y., Gu, Y., Kumar, V., Basu, S., Hall, L. O.,

Goldgof, D. B., Gatenby, R. A., & Gillies, R. J. (2014). Predicting outcomes of nonsmall cell

lung cancer using CT image features. IEEE Access, 2, 1418–1426.

11. Tripathy, H. K., Mishra, S., Thakkar, H. K., & Rai, D. (2021). CARE: A collision-aware

mobile robot navigation in grid environment using improved breadth ﬁrst search. Computers &

Electrical Engineering, 94, 107327

12. Balagurunathan, Y., Gu, Y., Wang, H., Kumar, V., Grove, O., Hawkins, S., Kim, J., Goldgof,

D. B., Hall, L. O., Gatenby, R. A., & Gillies, R. J. (2014). Reproducibility and prognosis of

quantitative features extracted from CT images. Translational Oncology, 7(1), 72.

Automatic Lung Carcinoma Identiﬁcation and Classiﬁcation … 165

13. Jena, L., Patra, B., Nayak, S., Mishra, S., & Tripathy, S. (2021). Risk prediction of kidney

disease using machine learning strategies. In Intelligent and Cloud Computing (pp. 485–494).

Springer Singapore.

14. Mishra, S., Raj, A., Kayal, A., Choudhary, V., Verma, P., & Biswal, L. (2012). Study of

cluster based routing protocols in wireless sensor networks. International Journal of Scientiﬁc

Engineering and Research, 3(7).

15. Afza, F., Khan, M. A., Sharif, M., & Rehman, A. (2019). Microscopic skin laceration segmen-

tation and classiﬁcation: A framework of statistical normal distribution and optimal feature

selection. Microscopy Research and Technique, 82(9), 1471–1488.

16. Khan, M. A., Javed, M. Y., Sharif, M., Saba, T., & Rehman, A. (2019, April). Multi-model deep

neural network based features extraction and optimal selection approach for skin lesion clas-

siﬁcation. In 2019 International Conference on Computer and Information Sciences (ICCIS)

(pp. 1–7). IEEE.

17. Mishra, S., Mishra, B. K., Tripathy, H. K., & Dutta, A. (2020). Analysis of the role and scope

of big data analytics with IoT in health care domain. In Handbook of data science approaches

for biomedical engineering (pp. 1–23). Academic Press.

18. Sreelatha, T., Subramanyam, M. V., & Prasad, M. G. (2019). Early detection of skin cancer

using melanoma segmentation technique. Journal of Medical Systems, 43(7), 190.

19. Rehman, A., Khan, M. A., Mehmood, Z., Saba, T., Sardaraz, M., & Rashid, M. (2020).

Microscopic melanoma detection and classiﬁcation: A framework of pixel-based fusion and

multilevel features reduction. Microscopy Research and Technique, 83(4), 410–423.

20. LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature, 521(7553), 436–444.

21. Xu, B., Wang, N., Chen, T., & Li, M. (2015). Empirical evaluation of rectiﬁed activations in

convolutional network. arXiv preprint arXiv:1505.00853

22. Armato Samuel, G., III, McLennan, G., Bidaut, L., McNitt-Gray, M. F., Meyer, C. R., Reeves,A.

P., Zhao, B., Aberle, D. R., Henschke, C. I., Hoffman, E. A., et al. (2015). Data from LIDC-IDRI.

Available online: https://wiki.cancerimagingarchive.net/display/Public/LIDC-IDRI. Accessed

January 7, 2020.

23. Kumar, P. R., & Manash, E. B. K. (2019). Deep learning: A branch of machine learning. Journal

of Physics: Conference Series, 1228(1). IOP Publishing.

24. Coccia, M. (2019). Artiﬁcial intelligence technology in oncology: A new technological

paradigm. National Research Council of Italy & Yale University.

25. Mishra, S., Dash, A., & Jena, L. (2021). Use of deep learning for disease detection and diagnosis.

In Bio-inspired neurocomputing (pp. 181–201). Springer Singapore.

26. Tandon, R., & Agrawal, S. (2020). Sequential CNN for automatic breast cancer detection using

histopathological images. Journal of Critical Reviews JCR, 7(15), 6104–6117.

27. Ray, C., Tripathy, H. K., & Mishra, S. (2019, June). Assessment of autistic disorder

using machine learning approach. In International Conference on Intelligent Computing and

Communication (pp. 209–219). Springer Singapore.

28. Saravagi, D., Agrawal, S., & Saravagi, M. (2021). Opportunities and challenges of ML model

for prediction and diagnosis of spondylolisthesis: A systematic review International Journal

of Engineering Systems Modelling and Simulation, 12(2/3), 127–138.

29. Saravagi, D., Agrawal, S., & Saravagi, M. (2021). Indian stock market analysis and predic-

tion using the LSTM model during COVID-19. International Journal of Engineering Systems

Modelling and Simulation, 12(2/3), 139–147.

30. Agrawal, S., & Kamal, R. (2015). Computational orchestrator: A super class for matrix, robotics

and control system orchestration. International Journal of Computer Applications, 12–17. ISSN

0975-8887.

31. Agrawal, S., & Jain, S. (2020). Medical text and image processing: Applications, issues and

challenges. Springer Nature. https://doi.org/10.1007/978-3-030-40850-3_11

32. Ganapathy, N., Swaminathan, R., & Deserno, T. M. (2018). Deep learning on 1-D biosignals:

A taxonomy-based survey. Yearbook of Medical Informatics, 27(01), 098–109.

33. Xu, Y., Hosny, A., Zeleznik, R., Parmar, C., Coroller, T., Franco, I., Mak, R. H., & Aerts, H.

J. W. L. (2019). Deep learning predicts lung cancer treatment response from serial medical

imaging. Clinical Cancer Research, 25(11), 3266–3275.

166 R. Tandon et al.

34. Sankar, V., Kumar, D., Clausi, D. A., Taylor, G. W., & Wong, A. (2019). Sisc: End-to-

end interpretable discovery radiomics-driven lung cancer prediction via stacked interpretable

sequencing cells. arXiv preprint arXiv:1901.04641

35. Raghu, M., Zhang, C., Kleinberg, J., & Bengio, S. (2019). Transfusion: Understanding transfer

learning for medical imaging. arXiv preprint arXiv:1902.07208

36. Shorten, C., & Khoshgoftaar, T. M. (2019). A survey on image data augmentation for deep

learning. Journal of Big Data, 6(1), 1–48.

37. Springenberg, J. T., Dosovitskiy, A., Brox, T., & Riedmiller, M. (2014). Striving for simplicity:

The all convolutional net. arXiv preprint arXiv:1412.6806

38. Raihan-Al-Masud, Md., & Mondal, M. R. H. (2020). Data-driven diagnosis of spinal

abnormalities using feature selection and machine learning algorithms. PLoS ONE, 15(2),

e0228422.

Augmented Intelligence: Deep Learning

Models for Healthcare

M. Paranthaman and S. Palanivel Rajan

Abstract Actionable insights and learning from a highly complex biomedical

dataset is a key challenge in smart healthcare. Traditional data processing algorithms

fails to provide the better results with complex data. Recent advancements in artiﬁ-

cial intelligence methods introduced an end to end complex learning models called

deep neural networks often referred as deep learning models. In this chapter, we

reviewed recent advancements of deep learning models and its applications related

to healthcare. We also discussed the challenges and opportunities faced by the deep

learning models.

Keywords Deep learning models ·Health care ·Electronic health records ·

Clinical imaging ·Genomics

1 Introduction

Technological advancement of artiﬁcial intelligence methods paved the way for

future healthcare systems. Deep learning, a part of artiﬁcial intelligence gaining a

dramatic advancement nowadays. Deep learning algorithms are proﬁcient in learning

and manipulating data including speech, language, and images. Due to the availability

of a large set of electronic health records in an unstructured manner, conventional data

managing algorithms failed to the requirements of smart health care systems. But

the deep learning algorithms have the computational power to extract the required

medical information from larger datasets [1]. Recent researches show deep learning

models have achieved an accuracy level equal to or greater than the physician. Diag-

nostic tasks like spinal analysis with MRI images [2], optical coherence tomography

[3], and diabetic retinopathy [4], referrals cardiovascular risks from fundus images

[5] and moles identiﬁcation from melanomas [6,7].

Electronic health record, a revolution in information technology ﬁeld generates

the best resources for AI algorithms. Electronic health records maintain the best

M. Paranthaman ·S. Palanivel Rajan (B)

Department of Electronics and Communication Engineering, M. Kumarasamy College of

Engineering, Karur, Tamilnadu 639113, India

e-mail: drspalanivelrajan@gmail.com

S. Mishra et al. (eds.), Augmented Intelligence in Healthcare: A Pragmatic and Integrated

Analysis, Studies in Computational Intelligence 1024,

https://doi.org/10.1007/978-981-19- 1076-0_10

167

168 M. Paranthaman and S. P. Rajan

Fig. 1 Deep learning for healthcare

treatment suggested by the best physician in the past which helps the deep learning

model to learn in-depth. The combination of deep learning models and electronic

health records will make huge development in fully automated health care systems

in the future.

A deep learning system for healthcare applications is shown in Fig. 1. This system

has three major components called digital data, AI analysis, and clinical decision.

Digital data is an important resource for any AI-based model to train and validate

the models’ decisions. A decade ago or still many hospitals do not have digitized

data about the patients’ health records. So the generation of electronic health records

also needs an AI-based model called natural language processing. So the AI and

electronic health record combination will help each other to form a smart health care

system [1,8].

AI analysis is the next in deep learning-based healthcare systems. Deep learning

framework has to learn and provide the correct decision based on what it learned

during the training phase. Multiple levels of abstraction can be used to train the

computational models to achieve high accuracy [1]. So multiple layers have to be

invoked for processing the data inside the neural network. Convolutional Neural

Network (CNN), Restricted Boltzmann Machine (RBM), Data Belief Network

(DBN), Recurrent Neural Network (RNN), and Auto Encoder (AE) are the deep

learning architectures widely used in healthcare systems. The applications of these

deep learning models are broadly classiﬁed into electronic health records, clinical

imaging, and genomics.

In the context of clinical imaging, deep learning models can be applied to brain

MRI images to diagnose Alzheimer’s disease [9]. RBM is also used for the detection

of ﬂuctuations in Alzheimer’s disease from brain MRI images [10] and segmentation

of multiple sclerosis [11]. CNN is used to predict osteoarthritis by automated knee

cartilage segmentation [12], from retinal fundus images diabetic retinopathy [13] can

be diagnosed and skin cancer classiﬁcation [14]. A stacked denoising AI model is

used for the diagnosis of breast cancer from ultrasound images. After the diagnosis

or prediction or decision on the tasks assigned to deep learning models, the clinical

decision has to be made. In the healthcare domain, deep learning models assist the

physician in many ways including the ﬁnal decision.

Augmented Intelligence: Deep Learning Models for Healthcare 169

2 Deep Learning Framework

Deep learning algorithms that solve complex industry problems gained massive

attraction in information computing. To perform speciﬁc tasks, different neural

networks are used by different deep learning algorithms. Each algorithm uses or

creates artiﬁcial neural networks which mimic the function of the human brain to do

practical computations on large datasets. A basic neural network consists of three

stacked layers and the computing element of the network is called nodes or neurons.

The stacked layers of artiﬁcial neural networks are the input layer, output layer, and

hidden layer.

Basic neural network is shown in Fig. 2. The input layer has ‘n’ number of

neurons that hold the information. This neuron is multiplied with random weights

and all values are added in the hidden layer(s). Finally, the output is determined by

the nonlinear function called, activation function.

To build a deep learning model different algorithms are used for training. Since

deep learning models are capable of self-learning due to effective training. So there is

a need for good training process and a huge amount of data to train the deep learning

model. This kind of dependence shows that no deep learning model is perfect for all

applications. We have to choose an appropriate deep learning model that depends on

the application in which the model is going to be implemented.

In this chapter, we have listed some deep learning algorithms that can be

used for healthcare applications. There are ﬁve major deep learning algorithms

used for healthcare applications. They are Convolutional Neural Networks (CNN),

Restricted Boltzmann Machine (RBM), Deep Belief Network (DBN), Recurrent

Neural Network (RNN), and Generative Adversarial Networks (GAN).

Fig. 2 Basic neural network

170 M. Paranthaman and S. P. Rajan

2.1 Convolutional Neural Networks (CNN)

The convolutional neural network is a basic deep neural network developed in the

1990s. For image recognition, this deep neural network is used. No other neural

network exactly imitates the human brain to recognize images. The working of CNN

is similar to the visual cortex of the brain. The convolutional neural network is widely

used for image classiﬁcation. For example, classiﬁcation of the image into tiger or lion

class is the same as recognizing the image of a picture as tiger or lion. The output layer

should be a multi-class classiﬁcation network. Therefore, the convolutional neural

network has an input layer, one output layer with a multi-class neural network, and

many hidden layers.

Regardless of image recognition algorithms, poor results are only obtained if

original images are used to train the neural network. So the feature of images should

be extracted. Various methods for extracting the image feature have been employed

earlier. Before the revolution of machine learning methods image, the feature extrac-

tion process is separated from the classiﬁer. We need to develop a feature extraction

model with or without considering the classiﬁcation network [15]. The conventional

feature extraction method is shown in Fig. 3.

The Independent feature extraction model has the following blocks.

Data: Training data (Input images and Correct output information).

Feature Extraction: Based on contrast, features of the images are extracted.

Classiﬁer: Based on extracted features, the classiﬁer decides to provide output.

Output: Store the output and check whether the classiﬁer output is correct or not.

If incorrect, the information will send back to the classiﬁer in some deep neural

networks.

But, this independent feature extraction model is inconsistent and also it required

additional time and cost. So the deep learning model includes a feature extraction

model inside the classiﬁer network rather than keeping it separated process. The

advantage of the CNN classiﬁer network is that the manual feature extraction model

is converted into an automated process. Using special neural networks, the weights are

determined during the training process. This model yields signiﬁcant improvement

in image classiﬁcation but it requires more cost. The combined phases of feature

extraction and classiﬁer are shown in Fig. 4.

Fig. 3 Independent feature extraction model

Augmented Intelligence: Deep Learning Models for Healthcare 171

Fig. 4 Special kind of feature extraction model

Fig. 5 Basic convolutional neural network

The increase in cost is due to the adaptation of two neural networks inside the

architecture of the convolutional neural network which is shown in Fig. 5. One neural

network is used for feature extraction and the other is used for image classiﬁcation.

The extracted features of an input image are given to the classiﬁer network. Based

on the features the classiﬁer generates the output.

The feature extraction neural network is constructed using several layers. The main

and important layers are the convolutional layer and pooling layer. In the context of

image feature extraction and classiﬁcation, the difference between convolutional

neural networks and other neural network is the processing of images in a two-

dimensional plane. First, the convolutional layer performs convolution operation

and the pooling layer merges the nearby pixels into a single pixel and this leads to

the reduction in image size.

2.2 Recurrent Neural Networks (RNN)

Generally, an artiﬁcial neural network consists of an input layer and one or more

hidden layers then one output layer. The complexity of the network increases with

an increase in the number of hidden layers. Recurrent connections are helped to

increase the depth at various levels of the neural network. So that the network is

capable enough to handle it in a nonlinear manner too [1]. Handling nonlinear units

requires additional optimization methods and there is no neural network designed

for that till 2006 [16,17]. High dimensional nonlinearity in the hidden layers of the

network provides ease in modeling data recognition and prediction. This process

is possible only through recurrent connections inside the artiﬁcial neural networks.

Generally, neural networks with recurrent connections are termed recurrent neural

172 M. Paranthaman and S. P. Rajan

networks [18,19]. The hidden layers of RNN are capable to remember and store

complex data for a long period which can be processed in the future. This could be

easy for any neural network to predict the output [20].

Recurrent Neural Network should contain a minimum of one feedback connection

in the hidden layer. So that the activation function can operate as a loop. This recur-

rent connection enables the network to learn the sequence and temporal processing.

Nonlinear processing is the backbone of the RNN architecture and that is possible if

the network hidden layer contains a loop along with Multi-Layer Perceptron (MLP).

MLP is having some memory that helps the entire network to learn the sequence and

predict the output [21].

A simple form of recurrent network (folded and unfolded RNN architecture

through time) is shown in Figs. 6and 7which has three layers. The input, hidden

layer (with recurrent connection), and output are the three layers present in the archi-

tecture of RNN. The stability of the network and overall performance is completely

depending on the hidden layer if the initialization element is non-zero [22]. Through

time, the sequence learning happens consistently. At each time, the hidden layer

learns and predicts the output and also it stores the prediction so that next times-

tamp, the information can be utilized [18]. In each timestamp, the hidden layer uses

a nonlinear function.

If the recurrent neural network is trained well, a rich nonlinear dynamic model

can be designed even with simple architecture. Since the recurrent neural network

is suitable for nonlinear operation, it should have the following parameters so that it

will be considered as a dynamic system.

Observability: Observable RNN is the determination of network state from a set of

input or output measurement.

Controllability: Within a hidden layer, RNN is controllable if any desired state could

be chosen as the initial state through time.

Fig. 6 Folded recurrent

neural network architecture

Augmented Intelligence: Deep Learning Models for Healthcare 173

Output

Layer

Hidden

Layer

Input

Layer

Output

Layer

Hidden

Layer

Input

Layer

Output

Layer

Hidden

Layer

Input

Layer

Fig. 7 Unfolded recurrent neural network architecture

Stability: A network is said to be stable if it gives a new prediction output even if

small changes occur over time.

Due to the dynamic nature of handling a nonlinear function, the training of recur-

rent neural networks maybe difﬁcult. But efﬁcient training of the RNN results

in a good deep learning model. The instability maybe caused due to hidden

states of a dynamic system. Some approaches used for training recurrent neural

networks are expectation maximization [23], global optimization, and approximated

Levenberg–Marquardt [24], Extended Kalman Filter, multigrid random search, and

time-weighted pseudo-newton optimization [25].

2.3 Restricted Boltzmann Machine (RBM)

Restricted Boltzmann Machine is a powerful deep learning model. This is a proba-

bilistic model used to distribute the variables of a visible layer. Hidden layer units

are used to increase the modeling efﬁciency [26]. Restricted Boltzmann machine is

applied to the practical problems that contain text [27] and high dimensional images

[1,28]. Generally, RBM is trained based on the unsupervised learning method. One or

more RBM can be trained by stacking layers [29]. This model can be used for solving

supervised learning problems. The pre-processing of input data can be done in two

ways: either a feed-forward network imposed or the hidden layer of the network

174 M. Paranthaman and S. P. Rajan

Fig. 8 Restricted Boltzmann machine example

itself pre-processes the input. In both cases, RBM should be added with some other

algorithm.

In classiﬁcation, Restricted Boltzmann Machine is sometimes called harmoniums

when nonbinary hidden layer units are used to extract the features [27]. But most of

the time the learning happens without label information [30,31]. A separate classiﬁer

is used to learn the missed label information. So the design of the RBM classiﬁer

should not incorporate any other classiﬁer to process and extract the features [26]. As

in the Bayes classiﬁer, the restricted Boltzmann machine can be designed for each

class and that increases the training efﬁciency. But this approach is not suitable for

the network which has more hidden layers. Globally this approach cannot be used for

modeling each class of the network. So we need to concentrate on sharing weights

across all classes to expect the RBM classiﬁer to perform well [32]. An example

network of RBM is shown in Fig. 8.

2.4 Deep Belief Networks (DBN)

Deep belief networks are having many hidden layers with causal variables. It is a

probabilistic model where each hidden layer output is conditionally given to another

hidden layer which is on the top of the stack. The basic building block of a deep belief

network is the restricted Boltzmann machine. RBM uses an unsupervised learning

method to train the network. The hidden layer output of one of the RBM can be used

for the input for another RBM while extracting the features of the input data. This

process will continue until the upper hidden layer gives output to the output layer as

showninFig.9. Since RBM follows unsupervised learning, a deep belief network

cannot work with a supervised learning model. It needs an additional classiﬁer or

network to use the RBN output. A two-step process is followed to train the deep belief

network as it uses a restricted Boltzmann machine as a basic building block [29].

Augmented Intelligence: Deep Learning Models for Healthcare 175

Fig. 9 Training process of deep belief network

The steps are layer-wise unsupervised learning and ﬁne turning. Steps in layer-wise

unsupervised learning are:

•Fix parameters for RBM and train the ﬁrst RBM with input data.

•The output of the ﬁrst RBM is given to the input of the second RBM and it

continues till the ﬁnal RBM in the hidden layer.

•Output has the extracted data from all layers.

Fine Turning: Add a suitable classiﬁer to avoid errors in the training. Classiﬁer

like backpropagation network at the end of deep belief network [33].

2.5 Generative Adversarial Networks (GAN)

Generative Adversarial Networks are a special type of artiﬁcial intelligence algo-

rithm to give a solution for generative modeling problems. The generative model

aims to learn the probability distribution from a wide variety of training samples

and to generate them. Once the model is learned, we can design a generative adver-

sarial network to generate more samples from the previous learning of probability

distribution [34]. The generative models used in GAN are based on the game theory

whereas other generative models are based on optimization techniques. Using two

machine learning models, a neural network is implemented [35,36]. The ﬁrst model

is a generator and the second model is a discriminator. A simple training model of

GAN is shown in Fig. 10.

3 Applications of Deep Learning

3.1 Clinical Imaging

Deep learning algorithm is used to predict diabetic macular edema and diabetic

retinopathy in retinal fundus images. 128,175 retinal images from EyePACS in the

U.S and three major hospitals from India, were used to train the convolutional neural

176 M. Paranthaman and S. P. Rajan

Fig. 10 Training process in general adversarial network

network. To train a deep neural network model, a known function is needed to detect

diabetic retinopathy. So during the training random values are set for the severity

parameters. After one set of training, we can further train our model by comparing

the known values from the training. As long as we train the model with different

images, the learning happened and we can check the accuracy of the trained model

[13].

Automatic prediction of Skin cancer is explained in [14]. A deep convolutional

neural network is trained and tested. The deep learning algorithm competency level

is compared with two dermatologists on classifying skin cancer with only disease

labels and pixels as inputs. 129,450 clinical images comprising of 2032 different

diseases were used to train the model. The effectiveness of CNN is validated in

two ways. First, three class partition of diseases on classifying skin lesions. The

accuracy of CNN is 72.1% whereas dermatologists achieve 65.56 and 66%. Second,

nine class partition of diseases on classifying skin lesions. The accuracy of CNN is

55.4% whereas dermatologists achieve 55 and 53.3%.

Stacked denoising autoencoder deep learning model is used to differentiate the

types of lesions. The CADx framework paradigm is changed by the deep learning

model. Due to the deep architecture, this model is capable of uncovering the training

data directly, simpliﬁed feature selection, and optimized feature extraction and clas-

siﬁcation. So the stacked denoising autoencoder-based CADx model is used for the

diagnosis of pulmonary nodules in CT images and breast lesions in ultrasound images

[37].

Augmented Intelligence: Deep Learning Models for Healthcare 177

Based on Region-based CNN, construction, and diagnosis of the onychomycosis

dataset were explained in [38]. From existing photographs, the R-CNN-based deep

learning model generates a large number of datasets to train the AI algorithm. There

are six datasets created with reference to the existing images. This deep learning

model generates 49,597 images of training datasets. The results were compared

with 42 dermatologists and the developed model outperformed all of them. The

basic difference between R-CNN and CNN is that CNN checks the entire image for

matching whereas R-CNN locates the desired object in the image.

Image segmentation assisted by a deep learning system is demonstrated in [39].

Pixel-level deep segmentation is carried on CT images for morphometric analysis.

A convolutional neural network is used here for image segmentation. Digital CT

images are converted into grayscale images. The reason to give grayscales images as

input to the deep learning model is its capability to identify the shades of grayscale

images. A normal human can identify only around 700 shades whereas the fully

convolutional network can identify up to 4096 shades in a digital CT image.

Fully automated deep learning system for the segmentation of fat and abdominal

muscle areas on CT images was developed and validated. From 467 subjects, 883 CT

images were used to train the fully convolutional neural network. A well-structured

patient selection process is carried out to get good-quality CT images. The supervised

learning model is used to develop the FCN and the data augmentation method is

used to generate the training images from 883 CT images. Totally 11,167 data were

generated for training the deep learning model. The accuracy of the model is validated

with external and internal data validation [40]. Novel FCN is proposed to automate

cartilage segmentation to improve the assessment of knee osteoarthritis. This 3D-

FCN model is trained by SKI10 data with no pre-processing or up sampling [41]

An automated fetal detection using recurrent neural networks is explained in [42].

A hybrid deep learning model consists of knowledge transferred RNN and CNN is

proposed. The basic classiﬁer is trained along with CNN to locate the fetal plane.

Based on the learning the temporal features are extracted via the LSTM model. Three

sets of training samples were generated from 300 videos and the total images of the

datasets are 12,343, 11,942, and 13,091. To evaluate the performance of the hybrid

deep learning model, another three datasets were generated as follows: 2252 images

from 60 videos, 2278 images from 52 videos, and 8718 images from 219 videos.

Segmentation of 3D images using a hybrid deep learning architecture is proposed

in [43]. Combination FCN and RNN are responsible for 3D image segmentation.

RNN is used to exploit inter-slice contexts whereas FCN is used to exploit intra-

slice contexts. FCN aims to separate object relevant such as shapes, texture, and

object irrelevant information such as image contrast, uneven illumination. So that

RNN can focus only on object-relevant information in inter-slice context. In clinical

applications, medical image fusion of multi-modal images is important in measuring

similarity. Denoising autoencoder-based deep learning architecture is proposed in

[44]. MRI and CT images are used to train the deep neural network model.

Convolutional neural network is widely used to detect breast cancer. A large

quantity of data is required to train the CNN model and the data must be labeled.

The creation of new mammogram images that can be used for training the model is

178 M. Paranthaman and S. P. Rajan

proposed as a solution in [45]. A generative adversarial network is used to generate

mammogram images. From the synthetic digital mammogram images, the GAN

network is trained with a speciﬁc region of interest.

Diabetic Retinopathy grading using deep belief networks is explained in [46].

Early diagnosis of diabetic retinopathy helps the patient as well as physician to take

necessary steps. For that diabetic progression, a degree has to be identiﬁed as early

as possible. Restricted Boltzmann machine-based deep belied network is applied to

identify the progression degree. First, the images are converted into HSV and then a

3D image. The proposed network converts these 3D images into a two-dimensional

weighted mean. With raw data of an image, the deep neural network grades the degree

of diabetic retinopathy.

3.2 Electronic Health Record

Prediction of nonmelanoma skin cancer using convolutional neural network in Elec-

tronic Health Record data of Taiwan Insurance is explained in [47]. The Electronic

health record consists of 9494 patients. With patient medical history and clinical

decisions, the CNN model predicted skin cancer. Prediction of opioid drug overdose

based on past electronic health records is explained in [48]. The sequential recurrent

neural network model is used here to improve the prediction. This deep learning

model is based on the temporal processing of old opioid prescriptions.

Identiﬁcation of high-risk patients from past electronic health records using the

RNN LSTM model is proposed in [49]. To reduce the cost of care and readmissions,

it is essential to ﬁnd out high-risk patients especially patients with Congestive Heart

Failure. Prediction of sepsis early can save many lives as the history of medical

records shows it is a deadly condition with high mortality rates. RNN-LSTM based

model is implemented to predict the high-risk patients of sepsis [50]. Prediction of

Extreme Preterm Birth from HER is explained in [51]. A recurrent neural network

in the form of temporal deep learning is used to predict extreme preterm birth. From

25,689 data RNN ensemble model is trained.

3.3 Genomics

Genome sequence of the patients are used for detection of disease is explained

in [52]. Deep neural network-based genome sequencing for personalized cancer

treatment is feasible now. High dimensional datasets like RNA measurement and

DNA sequencing have been increasing in large quantity and the processing of data

needs high throughput deep learning models. The internal structure of biological

data and its features have to be interpreted properly so that it can be used for better

decision-making while doing clinical practice. Without modifying the input extrac-

tion features, general neural networks are replaced by deep neural architectures in

Augmented Intelligence: Deep Learning Models for Healthcare 179

genomics [52–56]. Prediction of splicing using a fully connected network is demon-

stratedin[57]. More recent researches are directly handled raw DNA sequences using

a convolutional neural network [58–60]. Detecting similar patterns in the DNA, the

convolutional network will be helpful when compared to a fully connected network.

Because a convolutional neural network uses limited parameters to compute small

regions.

For example, [59] proposed CNN-based DeepBind architecture that predicts

particular patterns of RNA- and DNA-binding proteins. DeepBind architecture was

able to recover the novel sequence and recover known sequence and detect single

nucleotide variations. Similarly, [58,60] are proposed to detect the single nucleotide

variations and their effects. Signiﬁcant improvement in the drug discovery for

genomic medicine is achieved through deep neural networks [61].

4 Augmented Intelligence

Although several deep learning models performed excellent in healthcare, it is suscep-

tible to errors. Unintentionally those ﬂaws may happen if the process is completely

automated using AI model. In health care, such ﬂaws must be avoided. Optimal

patient care is the expected outcome of any DL model. With enormous patient data,

AI model can be used to analyze and provide better clinical decision to clinicians. In

the current state, AI will not take over human function i.e., AI model should not be

fully automated in providing clinical decision. AI should make deep revolution and

the change must be augmented not artiﬁcial. Intelligence Augmentation is the new

buzzword in economic forum [62]. Hybrid intelligence is the combination of DL

models and clinician promises to attain extreme level of accuracy. When compared

to AI or human alone, this hybrid model makes right predictions. In [63], AI model

and group of humans worked together to produce an accurate result in the diagnosis

of radiology. Scientiﬁc validation and more knowledge about the safety standards

are taken care by human and prediction is taken care by AI model.

Example: Augmented Intelligence: Deep Learning based Malaria detection

using blood smear images

Image classiﬁcation for malaria detection using deep CNN is shown in Fig. 11.

Malaria is a deadliest disease if not identiﬁed at earlier stage. In rural areas, the process

to identify this disease requires well-built laboratories. This model is designed to

provide automatic image classiﬁcation results through Internet. Malaria parasites are

classiﬁed using blood smear images and the results will be displayed on a website

(Fig. 12).

With the DL models prediction, patients can approach nearby hospitals to get a

more accurate treatment. The performance comparison of DL model is given in Table

180 M. Paranthaman and S. P. Rajan

Fig. 11 Deep learning model for Malaria detection

Fig. 12 Malaria detection

using blood smear image

Tabl e 1 Performance

comparison References Sensitivity Speciﬁcity Accuracy

[64]0.9688 0.9665 0.9678

[65]0.975 0.969 0.963

[66]0.971 0.986 0.977

[67]0.971 0.984 0.995

DL model 0.995 0.992 0.992

Augmented Intelligence: Deep Learning Models for Healthcare 181

5 Challenges in Deep Learning

There are unsolved problems in clinical practice despite many deep learning models

produced excellent results. Volume, Quality of data, Complexity, and Interoperability

are the key issues while designing a deep learning model for healthcare applications.

Volume: High-complexity computational models such as deep neural network

architectures require huge data for training. Prediction or estimating in deep learning

relies on intense training and the data required for training. Hundreds of network

parameters have to be processed properly to achieve the goal. Since it is an emerging

technology no thumb rule or guidelines about the training of deep network archi-

tectures. In general, we can collect an enormous amount of data within a minimum

period. But healthcare data collection is a tedious and difﬁcult process. We cannot

directly access patients or patient records to train the deep neural network archi-

tecture. Understanding medical records are the next challenge because of their

complexity (Table 2).

Quality of data: Health care data are incomplete, noisy, and highly heterogeneous.

We cannot get well-structured and clean from any health records. Several types of

research are going on to predict the missing values in electronic health records. So

training an excellent deep learning model with this incomplete dataset is a complex

task.

Complexity: Complexity is another key challenge in deep learning. Because of

incomplete medical records, deep learning models will face a lack of training. The

best example of complexity is the covid-19 pandemic. The actual cause for this

disease is still not found and understanding this disease’s progress is very difﬁcult.

This will be a great opportunity to collect more clinical data easily and that will help

to train the model to predict the disease in the future. When compared to any other

domain, the complexity is high in healthcare.

Interoperability: Formation of a fully automated system inspired by a deep neural

network in any other domain is not an issue. But in the health care domain, it is not

acceptable by medical professionals. So deep learning methods in the health care

domain are treated as black boxes. The black box problem means; the computational

model cannot interpret the biological data without human intervention due to its

heterogeneous nature and lack of transparency. Even though the model is learned from

old medical records, the decisions made by the neural networks are not completely

accepted.

Despite the above challenges, there are some opportunities to improve the deep

learning models. Feature enrichment is the ﬁrst opportunity to build a large dataset.

We can capture as much data as possible. As the world is now digitized, collecting

electronic health records from individuals is also possible. For example, we can

collect data from wearable devices, through social media, online healthcare commu-

nities. Model privacy, interpretable modeling, temporal modeling, and incorporating

expert knowledge are the few opportunities to improve the models.

182 M. Paranthaman and S. P. Rajan

Tabl e 2 Summary of deep learning applications

Data Deep learning model Application

Clinical imaging CNN [13]Predict diabetic macular edema and diabetic

retinopathy in retinal fundus images

CNN [14] Automatic prediction of Skin cancer

SDAE [37]Diagnosis of pulmonary nodules in CT images

and breast lesions in ultrasound images

R-CNN [38]Diagnosis of onychomycosis

CNN [39] Pixel-Level Deep Segmentation

FCN [40]Segmentation of fat and abdominal muscle

areas on CT images

FCN [41]Segmentation to improve the assessment of

knee osteoarthritis

RNN and CNN [42]Automated fetal detection

RNN and CNN [43] 3D Biomedical Image Segmentation

SDAE [44]Measuring of similarity in multi-modal

medical images

GAN and CNN [45]Breast cancer detection

DBN [46] Diabetic retinopathy grading

Electronic Health Record CNN [47]Nonmelenoma skin cancer prediction

RNN–LSTM [48] Opioid overdose prediction

RNN–LSTM [49] Hospital readmission prediction

RNN–LSTM [50] Prediction of sepsis

RNN [51] Extreme preterm birth prediction

RBM [52] Automatic diagnosis from patient health

record

CNN [68]Prediction CHF from longitudinal EHR’s

Genomics RNN–LSTM [69] Personalized cancer treatment using genome

sequencing

CNN [60] Chromatin marks prediction from DNA

sequences

CNN [58]Quantify the effect of SNVs on DNA

sequence

CNN [59] DeepBind: Predict the functionalities of

RNA- and DNA-binding proteins

CNN [70] Sequencing studies on single cell bisulﬁte to

predict methylation states

CNN [71] Different chromatin marks prevalence

estimation

Augmented Intelligence: Deep Learning Models for Healthcare 183

6 Conclusion

Deep learning architectures are highly complex and excellent computational models

to provide better results in smart healthcare. Despite of its challenges, deep learning

models are proved that it can be implemented to build automated process in health-

care. The applications are broadly classiﬁed into three categories such as genomics,

electronic health records, and clinical imaging. Several deep learning models and

its applications along with key challenges and future research opportunities are

explained in this chapter.

References

1. Chaudhury, P., Mishra, S., Tripathy, H. K., & Kishore, B. (2016). Enhancing the capabilities

of student result prediction system. In Proceedings of the Second International Conference on

Information and Communication Technology for Competitive Strategies (pp. 1–6).

2. Jamaludin, A., Kadir, T., & Zisserman, A. (2016). Spinenet: Automatically pinpointing clas-

siﬁcation evidence in spinal MRIs. In International Conference on Medical Image Computing

and Computer-Assisted Intervention (pp. 166–175)

3. Mishra, S., Raj, A., Kayal, A., Choudhary, V., Verma, P., & Biswal, L. (2012). Study of cluster

based routing protocols in wireless sensor networks. International Journal of Scientiﬁc and

Engineering Research, 3(7).

4. Gulshan, V., Peng, L., Coram, M., Stumpe, M. C., Wu, D., Narayanaswamy, A., Venugopalan,

S., Widner, K., Madams, T., Cuadros, J. and Kim, R. (2016). Development and validation of

a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs.

JAMA, 316, 2402–2410.

5. Poplin, R., et al. (2018). Prediction of cardiovascular risk factors from retinal fundus

photographs via deep learning. Nature Biomedical Engineering, 2, 158–164.

6. Tripathy, H. K., Mishra, S., Thakkar, H. K., & Rai, D. (2021). CARE: A collision-aware

mobile robot navigation in grid environment using improved breadth ﬁrst search. Computers &

Electrical Engineering, 94, 107327.

7. Haenssle, H. A., et al. (2018). Man against machine: Diagnostic performance of a deep learning

convolutional neural network for dermoscopic melanoma recognition in comparison to 58

dermatologists. Annals of Oncology, 29, 1836–1842.

8. Mishra, S., Mahanty, C., Dash, S., & Mishra, B. K. (2019). Implementation of BFS-NB hybrid

model in intrusion detection system. In Recent Developments in Machine Learning and Data

Analytics (pp. 167–175). Springer.

9. Liu, S., Liu, S., Cai, W., Pujol, S., Kikinis, R., & Feng, D. (2014). Early diagnosis of Alzheimer’s

disease with deep learning. In International Symposium on Biomedical Imaging, (pp. 1015–18),

Beijing, China

10. Brosch, T., & Tam, R. (2013). Manifold learning of brain MRIs by deep learning. Medical

Image Computing and Computer Assisted Intervention, 16, 633–640.

11. Yoo, Y., Brosch, T., Traboulsee, A., Li, D. K., & Tam, R. (2014). Deep learning of image features

from unlabeled data for multiple sclerosis lesion segmentation. In International Workshop on

Machine Learning in Medical Imaging (pp. 117–24), Boston, MA, USA.

12. Prasoon, A., Petersen, K., Igel, C., et al. (2013). Deep feature learning for knee cartilage

segmentation using a triplanar convolutional neural network. Medical Image Computing and

Computer Assisted Intervention, 16, 246–253.

13. Gulshan, V., Peng, L., Coram, M., Stumpe, M.C., Wu, D., Narayanaswamy, A., Venugopalan,

S., Widner, K., Madams, T., Cuadros, J., & Kim, R. (2016). Development and validation of

184 M. Paranthaman and S. P. Rajan

a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs.

JAMA, 316, 2402–10.

14. Esteva, A., Kuprel, B., Novoa, R. A., Ko, J., Swetter, S. M., Blau, H. M., & Thrun, S. (2017).

Dermatologist-level classiﬁcation of skin cancer with deep neural networks. Nature, 542, 115–

118 (2017). https://doi.org/10.1038/nature21056

15. Krizhevsky, A. (2013). ImageNet classiﬁcation with deep convolutional neural networks.

16. Bengio, Y., Boulanger-Lewandowski, N., & Pascanu, R. (2013). Advances in optimizing recur-

rent networks. In IEEE International Conference on Acoustics, Speech and Signal Processing

(ICASSP) (pp. 8624–8628). IEEE, 2013.

17. Hinton, G. E., Osindero, S., & Teh, Y.-W. (2006). A fast learning algorithm for deep belief

nets. Neural Computation, 18(7), 1527–1554.

18. Sutskever, I., Martens, J., & Hinton, G. E. (2011). Generating text with recurrent neural

networks. In Proceedings of the 28th International Conference on Machine Learning

(ICML-11) (pp. 1017–1024).

19. Bengio, Y., Simard, P., & Frasconi, P. (1994). learning long-term dependencies with gradient

descent is difﬁcult. IEEE Transactions on Neural Networks, 5(2), 157–166.

20. Mikolov, T., Joulin, A., Chopra, S., Mathieu, M., & Ranzato, M. A. (2014). Learning longer

memory in recurrent neural networks. arXiv preprint arXiv:1412.7753.

21. Mikolov, T., Karaﬁát, M., Burget, L., Cernocký, J., & Khudanpur, S. (2010). Recurrent neural

network based language model. In Interspeech (Vol.2,p.3).

22. Pérez-Ortiz, J. A., Gers, F. A., Eck, D., & Schmidhuber,J. (2003). Kalman ﬁlters improve LSTM

network performance in problems unsolvable by traditional recurrent nets. Neural Networks,

16(2), 241–250.

23. Ma, S., & Ji, C. (1998). A uniﬁed approach on fast training of feedforward and recurrent

networks using EM algorithm. IEEE Transactions on Signal Processing, 46(8), 2270–2274.

24. Chan, L. W., & Szeto, C. C. (1999). Training recurrent network with blockdiagonal approxi-

mated levenberg-marquardt algorithm. In International Joint Conference on Neural Networks,

1999. IJCNN’99 (Vol. 3, pp. 1521–1526), IEEE, 1999.

25. Puskorius, G. V., & Feldkamp, L. A. (1994). Neurocontrol of nonlinear dynamical systems

with Kalman ﬁlter trained recurrent networks. IEEE Transactions on Neural Networks, 5(2),

279–297.

26. Larochelle, H., Mandel, M., Pascanu, R., & Bengio, Y. (2012). Learning algorithms for the

classiﬁcation restricted Boltzmann machine. Journal of Machine Learning Research, 13, 643–

669.

27. Welling, M., Rosen-Zvi, M., & Hinton, G. E. (2005). Exponential family harmoniums with an

application to information retrieval. In L. K. Saul, Y. Weiss, and L. Bottou (Eds.), Advances in

Neural Information Processing Systems 17 (NIPS’04) (Vol. 17). Cambridge, MA

28. Larochelle, H., Erhan, D., Courville, A., Bergstra, J., & Bengio, Y. (2007). An empirical evalua-

tion of deep architectures on problems with many factors of variation”. In Z. Ghahramani (Ed.),

Proceedings of the Twenty-fourth International Conference on Machine Learning (ICML’07)

(pp. 473–480). ACM, 2007.

29. Mishra, S., Tadesse, Y., Dash, A., Jena, L., & Ranjan, P. (2021). Thyroid disorder analysis

using random forest classiﬁer. In Intelligent and cloud computing (pp. 385–390). Springer.

30. Gehler, P. V., Holub, A. D., & Welling, M. (2006). The rate adapting Poisson model for infor-

mation retrieval and object recognition. In W. W. Cohen, and A. Moore (Eds.), Proceedings of

the 23rd International Conference on Machine Learning (ICML’06) (pp. 337–344), New York,

NY, USA, ACM. ISBN 1-59593-383-2. https://doi.org/10.1145/1143844.1143887

31. Xing, E. P., Yan, R., & Hauptmann, A. G. (2005). Mining associated text and images with

dual-wing harmoniums. In Proceedings of the 21st Conference in Uncertainty in Artiﬁcial

Intelligence (UAI’05) (pp. 633–641). AUAI Press. ISBN 0-9749039-1-4 (2005).

32. Schmah, T., Hinton, G. E., Small, S., Zemel, R., & Strother, S. (2009). Generative versus

discriminative training of RBMs for classiﬁcation of fMRI images. In D. Koller, D. Schuur-

mans, Y. Bengio, and L. Bottou (Eds.), Advances in neural information processing systems 21

(NIPS’08) (pp. 1409–1416). Curran Associates.

Augmented Intelligence: Deep Learning Models for Healthcare 185

33. Bengio, Y.,Lamblin, P., Popovici,D., et al. (2007). Greedy layer-wise training of deep networks.

Advances in Neural Information Processing Systems, 19, 153.

34. Arora, S., Ge, R., Liang, Y., Ma, T., & Zhang, Y. (2017). Generalization and equilibrium in

generative adversarial nets (gans). arXiv preprint arXiv:1703.00573

35. Salimans, T., Goodfellow, I., Zaremba, W., Cheung, V., Radford, A., & Chen, X. (2016).

Improved techniques for training GANS. In Proceedings of the Advances Neural Information

Processing Systems Conference, pp. 2226–2234.

36. Radford, A., Metz, L., & Chintala, S. (2016). Unsupervised representation learning with

deep convolutional generative adversarial networks. In Proceedings of the 5th International

Conference on Learning Representations Workshop Track, 2016

37. Cheng, J. Z., Ni, D., Chou, Y. H., Qin, J., Tiu, C. M., Chang, Y. C., & Chen, C. M. (2016).

Zomputer-aided diagnosis with deep learning architecture: Applications to breast lesions in

US images and pulmonary nodules in CT scans. Scientiﬁc Reports, 6, 24454.

38. Han, S. S., Park, G. H., Lim, W., Kim, M. S., Na, J. I., Park, I., & Chang, S. E. (2018).

Deep neural networks show an equivalent and often superior performance to dermatologists in

onychomycosis diagnosis: Automatic construction of onychomycosis datasets by region-based

convolutional deep neural network. PLoS ONE, 13(1), e0191493. https://doi.org/10.1371/jou

rnal.pone.0191493

39. Lee, H., Troschel, F. M., Tajmir, S., Fuchs, G., Mario, J., Fintelmann, F. J., & Do, S. (2017).

Pixel-level deep segmentation: Artiﬁcial intelligence quantiﬁes muscle on computed tomog-

raphy for body morphometric analysis. Journal of Digital Imaging, 30(4), 487–498. https://

doi.org/10.1007/s10278-017-9988-z. PMID: 28653123; PMCID: PMC5537099.

40. Park,H.J.,Shin,Y.,Park,J.,Kim,H.,Lee,I.S.,Seo,D.W.,Huh,J.,Lee,T.Y.,Park,T.,Lee,J.&

Kim, K. W. (2020). Development and validation of a deep learning system for segmentation of

abdominal muscle and fat on computed tomography. Korean Journal of Radiology, 21(1), 88–

100. https://doi.org/10.3348/kjr.2019.0470. PMID: 31920032; PMCID: PMC6960305. (2020)

41. Raj, A., Vishwanathan, S., Ajani, B., Krishnan, K., & Agarwal, H. (2018). Automatic knee

cartilage segmentation using fully volumetric convolutional neural networks for evaluation

of osteoarthritis. In 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI

2018), pp. 851–854. https://doi.org/10.1109/ISBI.2018.8363705

42. Chen, H., Dou, Q., Ni, D., Cheng, J. Z., Qin, J., Li, S., & Heng, P. A. (2015). Automatic fetal

ultrasound standard plane detection using knowledge transferred recurrent neural networks.

In International conference on medical image computing and computer-assisted intervention

(pp. 507–514). Springer.

43. Chen, J., Yang, L., Zhang, Y., Alber, M., & Chen, D. Z. (2016). Combining fully convolutional

and recurrent neural networks for 3d biomedical image segmentation. In Advances in neural

information processing systems (pp. 3036–3044).

44. Cheng, X., Zhang, L., & Zheng, Y. (2018). Deep similarity learning for multimodal medical

images. Computer Methods in Biomechanics and Biomedical Engineering: Imaging &

Visualization, 6(3), 248–252. https://doi.org/10.1080/21681163.2015.1135299

45. Guan, S., & Loew, M. (2019). Breast cancer detection using synthetic mammograms from

generative adversarial networks in convolutional neural networks. Journal of Medical Imaging,

6(3), 031411. https://doi.org/10.1117/1.JMI.6.3.031411

46. Tehrani, A. A., Nickfarjam, A. M., Ebrahimpour-komleh, H., et al. (2021). Multi-input 2-

dimensional deep belief network: Diabetic retinopathy grading as case study. Multimedia Tools

and Applications, 80, 6171–6186. https://doi.org/10.1007/s11042-020-10025-1

47. Huang, C. W., Nguyen, A., Wu, C. C., Yang, H. C., & Li, Y. C. J. (2021). Develop a prediction

model for nonmelanoma skin cancer using deep learning in EHR data. In V. Kreinovich, & N.

Hoang Phuong (Eds.), Soft computing for biomedical applications and related topics.Studies in

computational intelligence (Vol. 899). Springer. https://doi.org/10.1007/978-3-030-49536-7_2

48. Dong, X., Deng, J., Hou, W., Rashidian, S., Rosenthal, R. N., Saltz, M., Saltz, J. H. & Wang,

F. (2021). Predicting opioid overdose risk of patients with opioid prescriptions using elec-

tronic health records based on temporal deep learning. Journal of Biomedical Informatics,

116, 103725. ISSN 1532-0464. https://doi.org/10.1016/j.jbi.2021.103725

186 M. Paranthaman and S. P. Rajan

49. Ashfaq, A., Sant’Anna, A., Lingman, M., & Nowaczyk, S. (2019). Readmission prediction

using deep learning on electronic health records. Journal of Biomedical Informatics, 97,

103256. ISSN 1532-0464. https://doi.org/10.1016/j.jbi.2019.103256

50. Zhang, D., Yin, C., Hunold, K. M., Jiang, X., Caterino, J. M., & Zhang, P. (2021). An inter-

pretable deep-learning model for early prediction of sepsis in the emergency department.

Pat te rns, 2(2), 100196. ISSN 2666-3899. https://doi.org/10.1016/j.patter.2020.100196

51. Gao, C., Osmundson, S., Edwards, D. R. V., Jackson, G. P., Malin, B. A., & Chen, Y.

(2019). Deep learning predicts extreme preterm birth from electronic health records. Journal

of Biomedical Informatics, 100, 103334. ISSN 1532-464. https://doi.org/10.1016/j.jbi.2019.

103334

52. Liang, Z., Zhang, G., Huang, J. X., & Hu, Q. V. (2014). Deep learning for healthcare decision

making with EMRs. In IEEE International Conference on Bioinformatics and Biomedicine

(BIBM). IEEE, 2014.

53. Park, Y., & Kellis, M. (2015). Deep learning for regulatory genomics. Nature Biotechnology,

33, 825–826.

54. Mamoshina, P., Vieira, A., Putin, E., et al. (2016). Applications of deep learning in biomedicine.

Molecular Pharmaceutics, 13, 1445–1454.

55. Leung, M. K., Delong, A., Alipanahi, B., & Frey, B. J. (2016). Machine learning in genomic

medicine: A review of computational problems and data sets. Proceedings of the IEEE, 104,

176–97.

56. Mishra, S., Sahoo, S., & Mishra, B. K. (2019). Addressing security issues and standards in

Internet of things. In Emerging trends and applications in cognitive computing (pp. 224–257).

IGI Global.

57. Xiong, H. Y., Alipanahi, B., Lee, L. J., et al. (2015). The human splicing code reveals new

insights into the genetic determinants of disease. Science, 347, 1254806.

58. Kelley, D. R., Snoek, J., & Rinn, J. L. (2016). Basset: Learning the regulatory code of the

accessible genome with deep convolutional neural networks. Genome Research, 26(7), 990–

999.

59. Alipanahi, B., Delong, A., Weirauch, M. T., et al. (2015). Predicting the sequence speciﬁcities

of DNA- and RNA-binding proteins by deep learning. Nature Biotechnology, 33, 831–838.

60. Zhou, J., & Troyanskaya, O. G. (2015). Predicting effects of noncoding variants with deep

learning-based sequence model. Nature Methods, 12, 931–934.

61. Ma, J., Sheridan, R. P., Liaw, A., et al. (2015). Deep neural nets as a method for quantitative

structure – activity relationships. Journal of Chemical Information and Modeling, 55, 263–274.

62. Doraiswamy, M., & A. I. Forget. (2017). The real revolution could be IA. https://www.wef

orum.org/agenda/authors/murali-doraiswamy. Accessed November 18, 2017.

63. Nagar, Y., & Malone, T. W. (2011). Combining human and machine intelligence for making

predictions. MIT Center for Collective Intelligence Working Paper, 2 (2011), 1–6. http://cci.

mit.edu

64. Liang, Z., Powell, A., Ersoy, I., Poostchi, M., Silamut, K., Palaniappan, K., Guo, P., Hossain,

M. A., Sameer, A., Maude, R. J. & Huang, J. X. (2016). CNN-based image analysis for malaria

diagnosis. In Proceedings of the 2016 IEEE International Conference on Bioinformatics and

Biomedicine (BIBM), Shenzhen, China, 15–18 December, pp. 493–496 (2016).

65. Bibin, D., Nair, M. S., & Punitha, P. (2017). Malaria parasite detection from peripheral blood

smear images using deepbelief networks. IEEE Access, 5, 9099–9108.

66. Gopakumar, G. P., Swetha, M., Sai Siva, G., & Sai Subrahmanyam, G. R. K. (2018). Convolu-

tional neural network-basedmalaria diagnosis from focus stack of blood smear images acquired

using custom-built slide scanner. Journal of Biophotonics, 11, e201700003.

67. Rajaraman, S., Jaeger, S., & Antani, S. K. (2019). Performance evaluation of deep neural

ensembles toward malariaparasite detection in thin-blood smear images. PeerJ, 7, e6977.

68. Cheng, Y., Wang, F., Zhang, P., & Hu, J. (2015). Risk prediction with electronic health records:

a deep learning approach. In ACM International Conference on Knowledge Discovery and

Data Mining (pp. 705–714), Sydney, NSW, Australia.

Augmented Intelligence: Deep Learning Models for Healthcare 187

69. Suresh, A., Nair, R. R., Neeba, E. A., & Kumar, S. A. (2021). Recurrent neural network

for genome sequencing for personalized cancer treatment in precision healthcare. Neural

Processing Letters. https://doi.org/10.1007/s11063-021-10572-0

70. Angermueller, C., Lee, H., Reik, W., & Stegle, O. (2016). Accurate prediction of single-cell

DNA methylation states using deep learning. bioRxiv. https://doi.org/10.1101/055715

71. Koh, P. W., Pierson, E., & Kundaje, A. (2016). Denoising genome-wide histone ChIP-seq with

convolutional neural networks. bioRxiv. https://doi.org/10.1101/052118

Sentiment Analysis and Emotion

Detection with Healthcare Perspective

Sathish Kumar, Rama Prabha, and Selvakumar Samuel

Abstract The health sector has beneﬁted in many ways from the data science and

artiﬁcial intelligence (AI). One of the most promising applications of data science

and AI in the healthcare ﬁeld is sentiment analysis and emotion detection. Senti-

ment analysis is the automatic categorization of sentiment in a free text, whereas

emotional detection categorizes emotion on a human face using a sophisticated image

dispensation. This chapter aimed to focus on Sentiment Analysis and Emotion detec-

tion applications related to wellbeing healthcare through a systematic review of the

recent literature. With the support of AI methods and other mathematical models,

sentiment analysis can offer signiﬁcant assistance to healthcare professionals, espe-

cially psychiatrists to understand the mental health and psychological problems of

wellbeing. In general, people with certain intolerable problems, serious illnesses,

addictions to something, suicide victims, and caregivers use social networks, health

websites, and other web portals to share their sentiments. These are important data

sources for sentiment analysis related to health. Emotion detection and recognition

mechanisms use facial expressions for emotions such as joy, sadness, surprise, and

anger, and in addition, capture “micro-expressions” or controlled expression of body

language as the main source of data. Analysis outcomes help health professionals

to decide when patients need help or need medication. In conclusion, health profes-

sionals and community service volunteers or caregivers can use the results of the

sentiment and emotion detection analysis to help with wellbeing when they need it.

The accuracy of the analysis results can be improved by combining the analysis of

human expressions from a variety of forms such as texts, facial expressions, body

language, and speech.

Keywords Sentiment analysis ·Emotion detection ·And recognition ·

Healthcare ·AI applications ·Data science applications

S. Kumar ·S. Samuel (B)

Asia Paciﬁc University of Technology and Innovation, Kuala Lumpur, Malaysia

e-mail: selvakumar@staffemail.apu.edu.my

R. Prabha

Nehru Arts and Science College, Coimbatore, Tamil Nadu, India

S. Mishra et al. (eds.), Augmented Intelligence in Healthcare: A Pragmatic and Integrated

Analysis, Studies in Computational Intelligence 1024,

https://doi.org/10.1007/978-981-19- 1076-0_11

189

190 S. Kumar et al.

1 Introduction to Sentiment Analysis

The exactness of the emotions in the sentence or statement is determined by the

sentiment analysis. The personality of the person may be expressed in the way

of their speech, messages like statements posted in various social media that is;

Facebook, WhatsApp and Twitter. This methodology may be used in various ﬁelds

like psychology, forecasting, disease prediction, and agriculture to understand the

opinion.

The sentiment analysis algorithms identify the emotions of the human being accu-

rately. Totally the sentiment analysis is used to identify the invisible emotions of

human beings. The technological platforms may collect thousands of data about

behavior of humans and understand the opinion in fraction of seconds [1]. SA is

an essential tool to monitor and understand the feelings of the customers. The auto-

matic analysis of customer feedback, opinions in customers’ surveys, and social

media conversations may help to predict the market analysis in quick time and help

to take decision regarding the implementation of innovations to target the desired

proﬁt.

Now a day’s people are addicted to communicate over online media, so the opin-

ions of individuals and need for prediction through sentiments increases to under-

stand the likes and dislikes of one person. The opinion mining with sentiment analysis

is the two innovative approaches used in automated processes for recognizing and

predicting feelings of humans.

The sentiment analysis is classiﬁed into several levels as shown in Fig. 1.The

overall opinion-based analysis helps us to predict the general opinion of customers.

The outcome of the analysis may be positive or negative. If we go for mince analysis,

then we expand the analysis based on positive into very positive, neutral, negative,

and very negative. It may work well for ﬁve-star rating in client review process.

Thus, sentiment analysis includes not only emotion detection but also other factors

namely, valence, arousal, and dominance which play an important role in identifying

one’s emotional state of mind [2]. The evaluation of understanding one person’s

attitude and opinion using computational approaches at that instant of time is said to

be sentimental analysis, as shown in Fig. 2.

Moreover, the source of data for sentimental analysis is the Web 2.0 which has

introduced forums, blogs, various other social media or platforms, and networks that

allows any user to share their opinion or discuss any topic whether it is relevant or

irrelevant of their interest. Analyzing this information is quite challenging to explain

the human behavior and activity. Further, machine learning can be used to analyze

the same as the data available is abundant and issues arising due to data storage,

accessing of data, and processing them leads to big data approach.

Thus, social media is the envisioned data source for sentimental analysis and

the problem or challenges in the structural and data correlation, data presentation,

analysis, evaluation, inference, and decision making are to be addressed [3].

Sentiment Analysis and Emotion Detection … 191

Fig. 1 Sentiment analysis with emotion by AI driven [1]

Fig. 2 Sentimental analysis [2]

192 S. Kumar et al.

2 Emotion Detection

The evidence for behaviors, thoughts, and actions is called as emotions. It is situation-

dependent. According to the current situation, it may be expressed as positive or

negative emotions. The emotion may be classiﬁed as fundamentally different and

discrete then emotions may characterize as behavior of multidimensional groupings

(Fig. 3).

Thus emotional and mental healthcare paves a way to improve the individual’s

quality of life. Humans generally express emotions in the way felt thought and

behave, in terms of positive and negative emotions. Examples of positive emotions

are pleasure, satisfaction, and happiness, while examples of negative emotions are

fear, sadness, and anger. Our life is a blend of both of these emotions of positive and

negative, which affects the health mentally and emotionally.

Fig. 3 Basic emotions of human beings [4]

Sentiment Analysis and Emotion Detection … 193

3 Perspective of Basic Emotions with Different Researchers

By William James the emotions maybe fear, grief, love, and rage, based on bodily

involvement [5]. Anger, disgust, fear, happiness, sadness, and surprise. [6]arethe

six basic emotions identiﬁed by Paul Ekman: Wallace V. Friesen [5] (Table 1).

The expanded list of different kinds of emotions handled by Ekman in 1990s is

Amusement, Contempt, Contentment, Embarrassment, Excitement, Guilt, Pride in

achievement, Relief, Satisfaction, Sensory pleasure, and Shame [5]. These emotions

may relate to almost facial expressions. In 1996, a researcher [6] identiﬁed the vast

list containing 15 emotions like anger, anxiety, compassion, depression, envy, fright,

gratitude, guilt, happiness, hope, jealousy, love, pride, relief, sadness, and shame

and aesthetic experience in the book Passion and Reason. The researchers at the

University of California, Berkley observed vast list of emotions like anger, anxiety,

awe, awkwardness, boredom, calmness, confusion, craving, disgust, empathic pain,

entrancement, excitement, fear, horror, interest, joy, nostalgia, relief, romance,

sadness, satisfaction, sexual desire and surprise, admiration, adoration, aesthetic

appreciation, amusement. To elicit certain emotions the researchers may spend their

time to watch more than 2000.

Emotion detection as deﬁned through Robert Plutchik’s Wheel of Emotions is

shown in Fig. 4. According to the researcher [7], the emotion patterns are greatly

Tabl e 1 Positive and

negative emotions [5]Kind of emotion Positive emotions Negative emotions

Related to object

properties

Interest, curiosity,

enthusiasm

Indifference,

habituation,

boredom

Attraction, desire,

admiration

Aversion, disgust,

revulsion

Surprise, amusement Alarm, panic

Future appraisal Hope, excitement Fear, anxiety, dread

Event-related Gratitude,

thankfulness

Anger, rage

Joy, elation,

triumph, jubilation

Sorrow, grief

Patience Frustration,

restlessness

Contentment Discontentment,

disappointment

Self-appraisal Humility, modesty Pride, arrogance

Social Charity Avarice, greed,

miserliness, envy,

jealousy

Sympathy Cruelty

Cathected Love Hate

194 S. Kumar et al.

Fig. 4 Plutchik’s wheel of emotions [7]

inﬂuenced by the eight primary but bipolar emotions. The proposed emotions are

“joy and sadness, anger and fear, surprise and anticipation, and trust and disgust”.

Plutchik did plot these paired emotions on a wheel and the intensity of emotions

move from outside toward the center of the wheel, becoming milder or low intensity

to heavier or mixed emotions.

4 Performance Evaluation of Emotion Detection Using

Advanced Features in Health Care

Apart from the basic set of emotions as speciﬁed earlier, the performance evaluation

of much more complex emotions is always challenging to be processed by the vision

system, which contributes to the emotion strategy in health care systems.

The complex emotions do include attention, hale, attraction, and so on. Further,

there is always a debate on whether the ethnic aspects and cultural variations need

to be considered or not [2].

The advanced features in emotion detection contributing toward health care

system includes multi-modal system, micro-expressions, and compound emotions.

Sentiment Analysis and Emotion Detection … 195

Multi-modal system combines various modalities such as text, facial expressions,

and voice or body movements. The emotions that are obtained from these multi-

modalities are actually fused together in order to result in an increased efﬁciency.

This helps the health care system or the therapist to accurately offer the intended

guidance for diagnosis of any health-related issues.

Micro-expressions play a vital role in the contribution of emotion detection which

is detected as subtle or coarse. System must be able to identify the low-intensity

expressions such as skin texture, furrows, and so on in order to increase the accuracy

of the system in detecting the emotions.

Compound emotions are emotions that are said to occur in pairs, namely, the

dominant and complementary emotions. This includes “surprisingly-happy, angrily-

sad, surprisingly-happy, and fearfully-sad”.

Thus, the accuracy and reliability of the performance metrics that are provided by

machine vision system for the purpose of sentimental analysis and emotion recogni-

tion must be established based on large-scale ﬁeld and clinical trials. Further inaccu-

rate measurements might lead to dangerous predictions in making medical decisions

with false negatives and false positives that might result in overtreatment [8].

5 Role of Sentiment Analysis in Health Care

All the patients have strong sentiments about the healthcare they received from the

hospital because every interaction with doctor or hospital has a positive or nega-

tive action [9]. Patient sentiment analysis in healthcare helps the provider to get a

competitive advantage service improvement based on their opinions and feedback.

The results of the sentiment analysis provide the clear insights for better treatments

and the quality of the service the patient needs. The opinions collected from patients

may be, Helping and caring nurses—positive, customer care is excellent but auto-

mated appointment needed—neutral: I was waiting for hours to get a response from

receptionist-negative [9].

The fundamental methods used in sentiment analysis are, ﬁrst to analyze the data

collected to ﬁnd out the most common complaints and compliments regarding the

health service and classify the information based on doctors, nurses, and departments

and such other parameters to recognize opportunities for health care improvement

of patient. In Fig. 5[10], the information gathered from Twitter is fed into the cloud

server. The raw data need to be cleaned and then need to undergo preprocessing

stage, where the raw data should be transformed into valuable data for ready for

analytical process. Then it is segmented into tokens and passed to lemmatization

phase, where they are grouped based on noun, verb, adjective, and adverb. In such

a way, the sentiment analysis study predicts the opinion of the Twitter user and the

health care state (good, bad, and better) as shown in Fig. 5.

Figure 6shows the ﬂow chart of sentiment analysis [11], which involves eight

stages. The ﬁrst stage converts the words into tokens and in stage 2 stemming words

196 S. Kumar et al.

Fig. 5 Health care system architecture [10]

is done. Stemming is process of normalization which is used to remove redundant

and unnecessary information or data.

Stage 3 involves tagging of speech which is used to capture the user’s emotions

and intensity, while stage 4 involves chinking and chunking of data. Chunking is a

process of grouping the tokens while chinking is a process of removing unchunked

data.

Stage 5 is lemmatizing the data which is similar to stemming, where lemmas are

the actual words while stemming creates the non-existent words. Stage 6 is matching

of corpora, which helps to identify the words that are relevant to the real world based

on the data analyzed. Thus, useful words are identiﬁed after passing and comparing

the date.

Stage 8 involves Scikit-Learning, where the processed tokens are converted into

features, which consists of classiﬁers such as Gaussian and Support Vector Machine

(SVM). The ﬁnal stage 9 is the creation of the sentiment module.

The system data input is fed from blogs, websites, and android applications that are

related to the healthcare industries. The output of the system is sentiment index of the

reviews that are either positive or negative impression of the customers. The accuracy

obtained by this proposed system is only 65–70%, based on the experimental results

conducted. This indicates the accuracy of greater than 50% is ensured by the system

[11].

One such application of sentiment analysis was done in Canada’s Toronto General

Hospital [12]. In this case study, four reviews were done and public opinions were

analyzed on various aspects of the hospital related to the healthcare system such

Sentiment Analysis and Emotion Detection … 197

Fig. 6 Flowchart of sentiment analysis [12]

as hygiene, hospital and patient administration, waiting for timing, and so on. The

result of the same is shown in Fig. 7. Both positive and negative sentiments are

analyzed and evaluated. The inference made based on Fig. 7resulted in very high

positive sentiment for staff and very low positive sentiment for nurses. Hence, the

hospital can ﬁnd out the reason why the nurses have very low positive sentiment, and

addressing such a gap, will have a good reputation for the hospital and will result

in less nurse turnover, consistent quality of customer treatment, and other beneﬁts.

In addition, it could also be observed that the other hospital staff, such as doctors,

cleanliness staff, and administrative ofﬁcers are being held with good esteem.

Thus, the latest sentiment analytic technologies include feeds from Twitter, Face-

book, Google, or any other sources. Lexalytics [13] is one of the recent research

and technological aspect that aims to discover the feelings of people in relation

198 S. Kumar et al.

Fig. 7 Hospital healthcare aspects [12]

to the health care issues. The software platform used is Sailence and is used to

perform sentiment analysis, intention analysis, topic extraction, and entity extrac-

tion. Another platform that is used for the same purpose is IBM Watson Analytics

for the social media [14] which is mainly used for the research of sentiment analysis.

These mentioned techniques need further evaluation.

6 Predictive Techniques for Mental Health Status in Social

Media

Now a day’s social media is used to understand mental health outcomes. The scientists

are using quantitative techniques to predict the presence of mental disorders such as

depression, suicidality, and anxiety. By using behavioral and linguistics cues from

social media like Facebook and Twitter may predict the symptom of psychosocial

disorders. The accuracy of the prediction may be good from 80 to 90%. The signal

of symptom is taken from the posting of behavioral history on social media websites

and Apps [15]. Table 2depicts the various kind of words that expresses the degree of

depression. The frequency of words in tweets reﬂects the importance of the contents

of the tweets [16].

These are important data sources for sentiment analysis related to health. Emotion

detection and recognition mechanisms use facial expressions for emotions such as

joy, sadness, surprise, and anger, and in addition, capture “micro-expressions” or

controlled expression of body language as the main source of data. Analysis outcomes

help health professionals to decide when patients need help or need medication.

Sentiment Analysis and Emotion Detection … 199

Tabl e 2 Features used for

predicting depression [17]Name Description

Bag of words Frequencies of words used in the tweet

Topic Ratio of tweet topics found by LDA [5]

Positive Ratio of positive-affect words

Contained in the tweet

Negative Ratio of negative-affect words

Contained in the tweet

Hour Hourly posting frequency

Tweet frequency Tweets per day

Num. of words Average number of words per tweet

RT Overall retweet rate

Mention Overall mention rate

URL Ratio of tweets containing a URL

Followee Number of users following

Follower Number of users followed

7 Recent Trends of Sentiment Analysis and Emotion

Detection During COVID-19 Using AI

COVID-19 has affectedmore than 265 million people around the globe. This outbreak

has a signiﬁcant impact on the people’s mental health, which is aware by each indi-

vidual, who has lost their loved ones or has recovered from it. In this context,

an increased number of patients have posted on the social media communicating

with respect to the clinical and diagnosis done, along with the stress and anxiety

underwent.

There were many surveys conducted across the globe widely by many researchers.

It is revealed that more than 85% of the victims have used websites, blogs, and other

social platforms such as Facebook, Twitter, Instagram, and so on, to share their

views, experiences pre and post COVID-19. Various studies conducted in European

countries such as Belgium, Spain, and so on, have concluded that social media has a

positive impact of association in capturing the victims’ emotional health during this

pandemic. Thus, this data source is considered more valuable in order to curb the

barrier of data collection [18].

A recent system proposed makes use of AI to explicitly relate the victims’

emotional health. The researcher designed a triple layer task framework in order to

investigate the emotion detection based on Plutchik’s model that uses the COVID-19

tweets as shown in Fig. 8.

The three tasks involved in the system are:

1. Emotion detection of “#StayAtHome” tweets.

2. Emotion/Semantic-Trends of “#StayAtHome”.

3. Modeling Sentence and COVID-19 Emotion detection.

200 S. Kumar et al.

Fig. 8 Sentiment analysis and emotion detection framework during COVID-19 using AI [18]

The conclusion from the system analyzed is that machine learning methods are

the most appropriate tool to detect emotions of COVID-19 tweet. The most promi-

nent emotional detection is the “anticipation”. A CNN model with two layers of

convolutional resulted in the best outcome. This has attracted many researchers in

the ﬁeld of healthcare, machine vision, and medical worldwide, as the approach is

useful in identifying the reaction of public to any particular issue.

8 Design of Healthcare System

Design and implementation of healthcare systems have become notorious in this digi-

talized world of machine vision systems [19]. Figure 9shows the design of Healthcare

system, in which user gestures are detected using Kinect and web camera, respiration

is detected using ECG sensors, and emotion is detected using facial expression and

speech recognition. All the data collected are sent to the web-based system which in

turn interacts with all the smart electronic gadgets such as tablets, smartphones, and

computers with webcams and speakers.

Sentiment Analysis and Emotion Detection … 201

Fig. 9 Healthcare system design [19]

The designed system has the following analysis of behavior relating to healthcare

[2]:

1. Detection of the six universal expressions,

2. Detection of non-universal expressions,

3. Detection of facial muscle movements, and

4. Sentimental analysis.

9 Challenges and Issues in Sentiment Analysis

In this section, we discussed the challenges involved in processing sentiments from

text-based information. By our empirical observation, some challenges are very

obvious and easily observed. Some need to perform experiments like syntactic

and semantic analysis on data. Text normalization is essential to interpret the exact

meaning of the text which exposes the sentiment. They are converting the letters of

the text into similar types like all are in lower case or upper case letters. Converting

numbers into words and vice versa, eliminating improper punctuation, expanding

abbreviations and removing repeating charters, etc. For Example “I looooove it”

may be interpreted as “I love it”. From the observation of the process most of the

words must be normalized in Twitter and SMS messages. That is 31% from Twitter

and 92% from SMS [20].

202 S. Kumar et al.

Fig. 10 Challenges in sentiment analysis on text data [21]

The other kinds of errors may be unintentional, and intentional errors may deviate

from the interpreting of correct meaning of small text exchanged by the people. The

NLP tools are trained for consistent text for tweets and SMS. The classiﬁcation of

named entities in tweets may be insufﬁcient in exploring the context exactly. Such

different kinds of challenges may be in the process of identifying the sentiment

analysis in the ﬁeld of medical side also occur in emotional detection (Fig. 10).

10 Advantages, Limitations and Future Scope of Sentiment

Analysis and Emotion Detection

Sentimental Analysis has the following advantages and applications in the healthcare

industry [22]:

1. Manual methods are automated with the help of machine vision system, thereby

increasing the overall efﬁciency of the system.

2. Vital information can be extracted from patients.

3. The Analysis helps consider any unforeseen factors.

Sentiment Analysis and Emotion Detection … 203

4. Has been studied about communities around mental health problems, cancer,and

other chronic disorders. Further studies were done with the health care system

related to surgery, medications, physicians, ortho services, and vaccination in

general.

5. Thus, patient reviews help in making many decisions to improve the current

healthcare services.

The accuracy of sentiment analysis can be increased by extreme optimization.

Advancement of new techniques arises from the drawbacks or limitations of the

existing system or techniques.

Thus, detecting emotions not only from facial expressions and voice but also from

body movements is to be explored in future. Also, automatic system alerts can be

sent to the hospital, in case of negative features being extracted in any application

of healthcare. In future, sensing emotions remotely is challenging and is within the

future scope [23].

11 Conclusions

The technique used in sentiment analysis utilizes social media as sources to collect

the various opinions expressed by the persons. Nowadays, this analysis is the essential

to predict the recommendations regarding the expectations of patients in health care

system. The problem identiﬁed and highlighted by the analysis regarding patients

may convey to the authorities immediately to take necessary actions. Actually, the

data collected through the conversations and text shared through social media is not

exactly predicting the exact emotions of the patients. But the automation algorithms

in data mining and machine learning. The correct decisions may take by supervised

procedure with high accuracy is needed for ﬁnding emotions in health care system.

In the sense of predicting wellbeing of system. When the sample space for the

information is to be analyzed, then we use unsupervised techniques available in big

data analytics.

When we embed graphical devices then the data collected for problem prediction

may yield exact decisions to ﬁnd the required result. The various context used as

sub-domains like disturbance analysis, expression analysis, preference analysis, risk

analysis in sentiment analysis is used to identify the exact emotions exposed by

patients. The optimized methods used to identify emotions may increase the exactness

of the information derived by the researchers.

References

1. Chakriswaran, P., Vincent, D. R., Srinivasan, K., et al. (2019). Emotion AI-Driven sentiment

analysis: A survey, future research directions, and open issues. Applied Sciences, 9, 5462.

https://doi.org/10.3390/app9245462

204 S. Kumar et al.

2. Chakriswaran, P., Vincent, D. R., Srinivasan, K., & Sharma, V. (2019). Emotion AI-driven

sentiment analysis: A survey, future research directions, and open issues. Applied Sciences,

9(24). https://doi.org/10.3390/app9245462.

3. Kaur, S., & Kulkarni, N. (2021). Emotion recognition-a review. International Journal of Applied

Engineering Research, 16(2), 103–110.

4. Dang, N. C., Moreno-García, M. N., & De la Prieta, F. (2020). Sentiment analysis based on

deep learning: A comparative study. Electronics, 9, 483, 1–29.

5. (2021) Emotion classiﬁcation | Wikiwand. In: Wikiwand. https://www.wikiwand.com/en/Emo

tion_classification#/citenote28

6. James, W. (2021). (disambiguation). Wikiwand. In Wikiwand. https://www.wikiwand.com/en/

William_James_(disambiguation)

7. Emotion classiﬁcation. (2021). Wikiwand. In Wikiwand. https://www.wikiwand.com/en/Emo

tion_classification#/citenoteHandel34

8. Gutierrez, K. (2021). Improve learner engagement by using Plutchik’s wheel of emotions.

In Shiftelearning.com. https://www.shiftelearning.com/blog/using-plutchiks-emotion-wheel-

elearning. Accessed December 6, 2021.

9. Miotto, R., Danieletto, M., Scelza, J. R., Kidd, B. A., & Dudley, J. T. (2018). Reﬂecting health:

Smart mirriors for personalized medicine. Nature-Digital Medicine, 1–7.

10. The Importance of Sentiment Analysis In Healthcare. (2021). Well Health. In WELL Health.

https://wellapp.com/blog/sentiment-analysis-in-healthcare/

11. Htet, H., Khaing, S. S., & Myint, Y. Y. (2019). Tweets sentiment analysis for healthcare on

big data processing and IoT architecture using maximum entropy classiﬁer. In Zin, T., Lin JW.

(Eds.), Big data analysis and deep learning applications. ICBDL 2018. Advances in Intelligent

Systems and Computing, Vol. 744. Springer. https://doi.org/10.1007/978-981-13-0869-7_4

12. Reddy, N. S., Prabadevi, B., & Deepa, B. (2019). Heart rate encapsulation and response tool

using sentiment analysis. International Journal of Electrical and Computer Engineering, 9(4),

2585–2592.

13. Inc. R. (2021). NLP in healthcare: Applications and real world use cases. In Repustate.com.

https://www.repustate.com/blog/nlp-in-healthcare/

14. Lexalytics.com. (2021). https://www.lexalytics.com/resources/Lexalytics_Machine_Lea

rning_Natural_Language_Processing_Whitepaper.pdf

15. IBM 2017. (2021). A year of innovation. In Ibm.com. https://www.ibm.com/thought-leader

ship/best-of/2017/

16. Ellison, N. B., & Boyd, D. (2013). Sociality through social network sites. In The Oxford

handbook of internet studies, Oxford University Press.

17. Tsugawa, S., Mogi, Y., Kikuchi, Y., Kishino, F., Fujita, K., Itoh, Y., & Ohsaki, H. (2013). On

estimating depressive tendency of twitter users from their tweet data. In Proceedings of the 2nd

International Workshop on Ambient Information Technologies (AMBIT’12) (2013), pp. 29–32.

18. Tsugawa, S., Kikuchi, Y., Ohsaki, H. (2015). Recognizing depression from twitter activity.

https://doi.org/10.1145/2702123.2702280. Corpus ID: 11019114.

19. Tivatansakul, S., Ohkura, M., Puangpontip, S., & Achalakul, T. (2014). Emotional healthcare

system: Emotion detection by facial expressions using Japanese database. In 6th Computer

Science and Electronic Engineering Conference (pp. 41–46). University Essex, UK, 2014.

20. Jelodar, H., Orji, R., Matwin, S., Weerasinghe, S., Oyebode, O., & Wang, Y. (2021). Artiﬁcial

intelligence for emotion-semantic trending and people emotion detection during COVID-19

social isolation.

21. Xue, Z., Yin, D., Davison, B. D., & Davison, B. (2011). Normalizing microtext. Analyzing

Microtext, 11, 05.

22. Srivastava, R., & Bhatia, M. P. S. (2017). Challenges with sentiment analysis of on-line micro-

texts. International Journal of Intelligent Systems and Applications (IJISA), 9(7), 31–40. https://

doi.org/10.5815/ijisa.2017.07.04

23. Zunic, A., Corcoran, P., & Spasic, I. (2020). Sentiment analysis in health and well-being:

Systematic review. JMIR Medical Informatics, 8(1), 2020.

Augmented Intelligence in Mental Health

Care: Sentiment Analysis and Emotion

Detection with Health Care Perspective

Asmita De and Sushruta Mishra

Abstract Sentiment analysis or opinion mining has become one of the fastest

growing areas now. Though the journey started since 1990s but huge outbreak of

sentiment analysis occurred after 2004. With the increasing pressure of new era, new

technology, more complex and busy lifestyle, mental health issues are also becoming

more serious concerns. In this survey paper, a brief evolution history of sentiment

analysis has been discussed. Brieﬂy, emotion detection through facial expression has

also been addressed. Commonly used approaches and technology used for sentiment

analysis and emotion detection has been discussed with the comparison of available

technologies. With the methodology used for sentiment analysis, an insight view

of mental health concerns where sentiment analysis can play a vital role, has been

discussed. Nowadays, our youth generation is using social website in a large scale to

express their mental status, as a way of entertainment, to express their general opinion

regarding any topic or issue. Hence the large web data now has gained the ability

to show the overall mental condition for a large community. After corona outbreak

mental issues have been increased signiﬁcantly as well as use of social media also

has been increased incredibly due to lockdown and work from home lifestyle. So by

using the vast data of web platform, we can analyze the recent situation of mental

health issues and also predict near future concerns. Although in accuracy of senti-

ment analysis, we are still facing many challenges with our existing algorithms, but

there are a lot of future scope in this ﬁeld.

Keywords Sentiment analysis ·Facial emotion detection ·Mental health care ·

Augmented intelligence ·NLP ·Machine learning

Asmita De (B)·S. Mishra

School of Computer Engineering, Kalinga Institute of Industrial Technology, Deemed to be

University, Bhubaneswar, India

e-mail: deasmita07@gmail.com

S. Mishra

e-mail: sushruta.mishrafcs@kiit.ac.in

S. Mishra et al. (eds.), Augmented Intelligence in Healthcare: A Pragmatic and Integrated

Analysis, Studies in Computational Intelligence 1024,

https://doi.org/10.1007/978-981-19- 1076-0_12

205

206 Asmita De and S. Mishra

1 Introduction

1.1 Sentiment Analysis

Sentiment analysis (or opinion mining) is a use case of natural language processing

technique to ﬁnd programmatically the emotional tone behind the data (generally text

format) for understanding the attitudes, emotions and opinions expressed. Usually

sentiment analysis examined and categorized text into positive, negative or neutral

polarity.

i.e. ‘the process of computationally identifying and categorizing opinions

expressed in a piece of text, especially in order to determine whether the writer’s

attitude toward a particular topic, product etc. is positive, negative or neutral’.

Here, ‘Fig. 1’ shows 3 different opinions for a product has been taken and

according to the opinion the sentiment has been classiﬁed among positive, negative

and neutral.

Sentiment analysis also known as:

Opinion mining, opinion extraction, sentiment mining, emotion AI, subjectivity

analysis.

Lexicons and sentiment lexicons:

Lexicons means ‘vocabulary’ i.e. complete set of meaningful units in any

language.

A sentiment lexicon is collection of lexical units(words) along with their sentiment

orientation, i.e. positive or negative.

Examples: here, ‘Table 1’ shows few examples of lexicons classiﬁed into positive

and negative category.

Basic components of opinion:

Fig. 1 Example of opinion analysis for a product [1]

Augmented Intelligence in Mental Health Care … 207

Tabl e 1 Examples of

lexicons Positive Negative

Good Bad

Great Wor se

Creative Damage

1. Opinion holder.

2. Object.

3. Opinion.

An opinion is neutral, positive or negative sentiment, can be deﬁned with 5 tuples

(ei,e

ij,so

ijkl,h

k,t

where

eitarget entity

aij feature of ei

soijkl Is sentiment value of the opinion from the opinion holder ‘hk’ on aspect ‘aij’

of ‘ei’ at time ‘tl’

hkopinion holder

tltime when opinion was expressed.

1.2 Emotion Detection

Emotion detection or recognition refers to the emotional states by observing few

visual and auditory non-verbal cues including facial, vocal, acoustics, postural,

gestural, cues displayed.

Though for hundreds of years people assumed that there is a universal set of

expressions that we use to show our emotions and hence scientists assumed that

to be true and based on that hypothesis augmented intelligence started developing

emotion detection technology.

According to recent research papers, those standard expressions must not have

the speciﬁc meaning.

For example, if one smiles, possibility of being happy is although better than just

a chance, but sometimes it doesn’t allow us to know if the person genuinely smiling,

faking the smile, or she/he is happy but not smiling and so on.

May be for 40–50% cases these standard expressions work correctly.

Moreover, there are many remote cultures we don’t have much knowledge about

even their common expressions used.

Our initial algorithm can work good for sales, marketing etc. area but for mental

health care, to work with the patient, we must need more advanced algorithm as here

we cannot go with below 50% success rate.

208 Asmita De and S. Mishra

Fig. 2 Augmented Intelligence from human intelligence and artiﬁcial intelligence [2]

1.3 Augmented Intelligence

Human intelligence is capable to recognize and reason. 1st one helps to identify the

situation and 2nd one helps to further create, update and associate concepts.

‘Fig. 2’ shows how Augmented Intelligence performs with mixture abilities of

human brain and artiﬁcial intelligence [2]

Artiﬁcial intelligence (AI) is capable of only copy the ability to recognize, hence

learn from past experience and perform human like tasks. Most AI applications in

today’s time, are based on deep learning and natural language processing.

But AI cannot serve like complex tough process of human brain.

In everywhere, there are cause and effect relationship. Everything in our

environment is interrelated. Mental health care is also not an exception.

Though there are certain patterns, but our society, surroundings and hence health

care issues too changing constantly. Identifying near future event using only AI is a

challenge.

Here comes Augmented intelligence. It works beyond what AI can do.

Augmented intelligence:

1. Distilled knowledge

2. Artiﬁcial intelligence

1.4 In Mental Health Tracking

Tools like ginger.io uses data analytics for mindfulness training and cognitive behav-

ioral therapy. Here, Augmented intelligence tracks the using pattern of smartphone

by the customers and work accordingly.

Augmented Intelligence in Mental Health Care … 209

Although AI cannot replace therapist, psychologist or psychiatrist but AI can help

them to get insight of the mental behavior of a particular person by observing from

online platform how he/she reacted to a topic in a speciﬁc way.

His/her unique pattern of communication, posts, reviews, comments can reveal

the mental state (like happy, sad, angry etc.), even can reveal status of social

relationship(like breakup, mental illness etc.).

For example,

“I feel tired and sad all the time, cannot concentrate on my work”

Reveals the depressed state.

If a person is mentally sufferer, suicidal, addicted then most likely he/she will use

related sentiment lexicons.

This paper work focuses on contribution of AI on sentiment analysis for mental

health care.

Mental health care itself is a subjective state, it can be different for different

individuals even if their circumstances are similar.

Facial sentiment analysis is one of the most trending work areas now. Paul Ekman

ﬁrst introduced the term FER (Facial Expression Recognition) in 1980s.

Though getting high accuracy is still very challenging goal but it helps in various

application area like health care.

Sentiment analysis on voice is done focusing on pitch value, format, tone, speed,

power of voice.

Main stream social media is already used by maximum patients so data can be

easily collected and evaluated.

Well, due to privacy policy, data used in research work usually not publicly

disclosed for further research work.

An initiative to create, annotate corpus is already being used for research. The

corpus contained approximately 1280 suicidal notes, those each notes was reviewed

by atleast 3 annotators to mapped words, sentences to emotion schema.

2Evolution

Sentiment analysis and opinion mining often used as synonyms but we can view

sentiment as opinion that includes emotions in depth.

Probably the interest of knowing other sentiments is as old as verbal commu-

nication. Historically, leaders or kings used to deal with spy to know about what

their subordinate or people are thinking in order to prepare for any opposition, grow

popularity with needed activities or to understand their kingdoms strength, weakness

etc.

Since approximately 508B.C, Ancient Greece implemented voting method to

measure public opinion and earliest form of democracy got implemented.

A scientiﬁc journal or public opinion had published on 1937.

210 Asmita De and S. Mishra

Fig. 3 Graph showing huge increase in counts of works between 2004 and 2016

In ‘Fig. 3’, it is showing, According to a research paper, published at end of 2016,

(by Mäntylä, Graziotin, & Kuutila) nearly 7 k papers on opinion mining has been

published till the time and 99% of them published after 2004 and make this area as

one of the fastest growing ﬁelds [3].

Since world war 2, measuring public opinion started playing a big role but most of

them were on political view point after 2000 working on modern various application

of sentiment analysis has reached to a numerous different area.

Gradually, Natural Language processing and sentiment analysis are getting over-

lapped now. Augmented intelligence with working on sentiment analysis, has become

a new ray of hope for emotion detection, differentiating negative emotions and hence

helping in mental health problem.

At the start of twentieth century, sentiment analysis started growing through ques-

tionnaires and subjectivity analysis. But this research area was ignored until we got

huge data set of various opinions through web.

In 1995 a paper titled ‘Elicitation Assessment, and pooling of Expert judge-

ments using possibility Theory’ was published that used computer for expert opinion

analysis.

The Association for computational Linguistics (ACL) founded in 1962, works on

problems involving natural language and computation, proposed a standard to this

in 1999. Hence contributed a major part in birth at modern sentiment analysis.

But in 2005, only 101 major papers on SA published where in 2016 the number

was almost 7000.

Since the world wide web has come on the surface at success, the provision

of numerous data generated in web 2.0 had allowed people to share their opinion

emotions, review creations etc.

Since 2013, Big data analytics is becoming a major tool to handle massive and

high dimensional data. Now, Big data requires more computational works on huge

Augmented Intelligence in Mental Health Care … 211

data set without sensitive data loss. Here a highly efﬁcient a rapidly expanding ﬁeld

comes to overcome these problem: Machine learning.

2.1 Emotion Detection from Human Facial Movement

Our facial expression in day to day life can primarily reveal our mood of that particular

time. For example, smiling face means happy on satisﬁed, scowlingcan indicate anger

and other emotions like sadness, fear, surprise can be expressed through our facial

movement.

Thousands of emojis we use in electronic messages, which are formed from

proposed facial expressions from various emotion categories.

Agents of U.S FBI and TSA were trained to recognize intentions using facial

conﬁgurations with goal to identify terrorists.

In recent time, researchers are trying to develop this ﬁeld more so that it can be

used in mental health therapy.

Table 2is showing few papers presented by different authors between 2009

and 2020. Content is taken from A Literature Review on Application of Sentiment

Analysis Using Machine Learning Techniques (august 2020, by Anvar Shathik and

Krishna Prasad) [4].

3 Techniques and Methods

Figure 4shows different approaches, methods as well as algorithms for sentiment

analysis. This is given By Mehta and Pandaya [24]

Sentiment analysis can be carried out at different levels of granularity.

3.1 Levels of Granularity

1. Document level:

It is usually based on overall review to determine overall polarity of sentiment.

Here we assumed that there is a speciﬁc focused topic in the document that contains

opinion.

There are mainly 2 approaches to document level sentiment analysis:

1. Supervised learning

2. Unsupervised learning

In (Nissen et al. 2012) stated that most of researchers at document level follow 2

step-based approach ﬁrst one is relevance Retrieval and 2nd one is opinion Finding.

212 Asmita De and S. Mishra

Tabl e 2 Few papers presented by different authors between 2009 and 2020

S. No Author(s) Year Inventions/ﬁndings/results

1Boiy and Moens [5]2009 Classifying clients’ emotions by means of

approaches such as information processing,

natural language therapy and machine

learning systems, in English, French and

Dutch helped them produce successful results

2Boiy and Moens [5]2009 Integrated approach incorporating knowledge,

information retrieval, recovery approaches,

natural language processing and machine

learning have shown strong results

3Go et al. [6]2009 Describe the pre-processing steps to achieve

high precision or accuracy

4 Yang et al. [7]2010 This sentimental analysis compares and

describes the favorite goods that make

customers happy and looks at unsupervised

approaches to learning for sensational study

5LiandWu[8]2010 Empirical research relates the feeling of post

text to the distribution of hotspots. SVM is

used for hotspot prediction with exact data

tests

6 Kumar and Sebastian [9]2012 Examine the signiﬁcant study of emotions

and discusses their fundamental terms, tasks

and granularities, functional and future

applications and challenges

7Duric and Song [10]2012 Learn the features of feedback automatically

and identify them into positive, negative and

neutral comments

8Vohra and Teraiya [11]2013 Exploring and explaining its policies in this

ﬁeld by contrasting the concept of emotional

research in natural language processing

9 Medhat et al. [12]2014 Give the overview and review of the latest SA

algorithms and software updates

10 Axhiu et al. [13]2014 Stimulus studies are joined together for

tracking views and collecting qualitative

information, for calculating and

communicating in a palatable way

11 Suryawanshi et al. [14]2020 The methods of natural language processing

help to evaluate the tweet emotions, where

they are positive, neutral, negative and then

graded according to the emotions

12 Sentamilselvan et al. [15]2020 The classiﬁcation and logistic regression of

Naïve Bayes was used to evaluate feelings

and classify according to the better precision

of the scientiﬁc classiﬁcation

(continued)

Augmented Intelligence in Mental Health Care … 213

Tabl e 2 (continued)

S. No Author(s) Year Inventions/ﬁndings/results

13 Godara and Kumar [16]2020 Twitter dataset output is counted by applying

different efﬁciency metrics such as exactness

and accuracy, retrieval and f estimation,

including decision-tabbing and artiﬁcial

neural networks (ANN), naïve bays, fuzzy

techniques (FU) and vector support (SVM)

14 Yada v et a l . [17]2020 Sentiment analysis can be used with

supervised machine learning for fake positive

or negative review identiﬁcation. SVM

algorithm not only passes text detection but

can also be used for fake feedback

15 Gujar and Pardeshi [18]2020 Sets the path to social media opinion mining

through the common machine learning

approach by means of the Twitter API

16 Arote Rutuja et al. [19]2020 Different aspects of the sentiment analysis

were studied with a focus on Czech

17 Raju and Tarif [20]2020 Determining Bitcoin’s predictable price path

in USD by machine learning and emotional

research

18 Ardianto et al. [21]2020 In evaluating e-sports sentiment for student

learning curricula, calculating views, or

distinguishing positive and negative feelings

for e-sport education and Naïve Bayes

algorithms can best be forecasted

Document level sentiment Analysis can help us by providing summary of total

no. of positive and negative document.

It cannot be applicable to any document that compares or evaluates multiple

entities.

2. Sentiment level

This refers to determine sentiment of each sentence.

Sometimes when document, although discusses about a speciﬁc entity, it can be

possible that it contains multiple opinions about same entity. To have more ﬁne-

grained view of different opinion we need sentence level sentiment analysis.

Sentences are considered as short document here which makes the classiﬁcation

same for both document level and sentence level.

However, in document level, only positive and negative, the opinion so with

positive and negative the neutral class also must be considered.

We assume that we already know about the speciﬁc entity discussed in the sentence

and there is only single opinion in each sentence. By splitting sentence into different

phrases where single opinion is present in each phrase we can handle more complex

sentences (with multiple opinion, comparative, conditional, interrogative).

214 Asmita De and S. Mishra

Fig. 4 Different approaches of sentiment analysis

In ﬁrst step we must classify the sentence into subjective or objective. Generally

subjective sentence goes for further classiﬁcation. Objective sentence classiﬁcation

needs more complex approaches.

Most of the approaches to sentence level sentiment analysis are based on

supervised learning or unsupervised learning. In later approach, it uses modiﬁed

log-likelihood ratio instead of PMI and the no. of seed words used is much larger.

Recent ﬁndings say that we need different method for different kind of sentence

to get better result. Generally sarcastic sentence is extremely difﬁcult to classify.

3. Aspect-Based

This expresses opinion for different features of the target object.

In many cases, we discuss about multiple aspects of a speciﬁc entity. There can

be different opinion about each attributes.

Overall positive or negative feedback doesn’t imply positive or negative for

every aspects of the target. So here aspect-based SA performs better than other two

level sentiment analysis. According to tuples deﬁnition (mentioned in introduction),

aspect-based analysis is generally done by 2 tasks for ﬁrst 3 tuples: Aspect extraction

and Aspect sentiment classiﬁcation.

Augmented Intelligence in Mental Health Care … 215

Aspect extraction indicates extraction of both entities and those aspects can be

expressed implicitly or explicitly.

The 2nd task is similar to identify the polarity.

The extraction of remaining components is done as sub task the approach called

opinion holder extraction and time extraction.

Polarity of each sentiment is determined by sentiment lexicon, shifters, handling

of conjunctions (example: but). Final polarity is determined by weighted average

of all polarities, of all expressions inversely weighted by the distance between the

aspect and the sentiment expression (Ronen Feldman 2013).

The information of ‘Table 3’ has been taken from ‘sentiment analysis approaches

based on granularity levels’ by Rachid et al. [22]

3.2 Lexicon/Rule-Based Approach

This method has predeﬁned list of words with a valid score.

Example: The algorithm considers the words in the sentence and gives sum or

average of the values of those words as ﬁnal result (Table 4).

Example: Opinion: The book is good one (Table 5).

Hence, Total value: +1.

Lexicon- based approach can be further divided into Dictionary-based and corpus-

based approach.

Dictionary- based approach.

User gathers words of opinion and seed list is prepared. Then synonyms and

opposite words are searched from various source like phrases, books, thesaurus etc.

Now the new list of substitute words is added into seed list. Process continues until

no new terms left.

Finding domain-related opinion words is a tough work and as dictionary size

increases, this approach doesn’t serve well.

Corpus-Based Approach.

Here, corpus refers to a cluster of organized syntactic and semantic opinion

patterns. User just needs to draw out the required seed list.

3.3 Automatic/Machine Learning

This is basically a classiﬁcation problem where we use some historical data with

already available sentiments, we need to predict the sentiment of new piece of text.

Initially involuntary classiﬁcation can be done splitting into 2 classes, trained data

and test data further with the use of popular ML, classiﬁcation can be categorized

into.

216 Asmita De and S. Mishra

Tabl e 3 Uses of different approaches based on granularity levels

Level Approach Techniques Issue References

Document level Supervised

learning

Naïve bayes, svm,

maximum entropy,

stochastic gradient

descent

Movie review

classiﬁcation

[23]

Document level Unsupervised

learning

k-means clustering Mood swing analyzer [24]

Document level Deep learning Convolutional NN,

LSTM

Dual prediction of

word and document

sentiments

[25]

Document level Lexicon-based Dictionary-based Sentiment

classiﬁcation system

for social media

genre(SmartSA)

[26]

Sentence level Supervised

learning

Conditional random

ﬁelds

Context aware

approach for learning

sentiments

[27]

Sentence level Deep learning Recursive neural

network, LSTM

Increasing

phrase/sentence

representation

[28]

Sentence level Lexicon-based Dictionary-based Sentiment

classiﬁcation of

twitter messages

[29]

Aspect level Supervised

learning

NB, SVM, KNN,

Decision tree,

Bayesian network

Three ABSA

subtasks

[30]

Aspect level Deep learning LSTM Targeted

aspect-sbased SA

[31]

Aspect level Unsupervised

learning

Enriched LDA Aspect extraction [32]

Aspect level Lexicon-based Dictionary-based

and syntactic

dependency

Automating training

data labeling

[33]

Aspect level Ontology-based Retrieval and analysis

of social media

content

[34]

Tabl e 4 Showing a dummy

data of predeﬁned lexicon

with score

Lexicon Score

Nice 2

Good 1

Horrible −3

Augmented Intelligence in Mental Health Care … 217

Tabl e 5 Showing dummy

predeﬁned value of each word

in the given example

Wor d Va lu e

The 0

Book 0

Is 0

Good 1

One 0

1. Supervised 2. Unsupervised.

Supervised learning

Labeled training data set is used in supervised learning. Each opinion is categorized

under a label based on its type and related characteristics.

Used Algorithm:

Support vector Machine (SVM), Neural network(NN), Maximum Entropy (ME),

Naive Bakes(NB), Decision Tree (DT), Bayesian network(BN), Random Forest(RF),

logistics regression (LR), K-nearest Neighbour (KNN).

Unsupervised learning

Data sets are not classiﬁed on labeled here. Serve well for clustering and association

analysis.

Common Algorithm used:

K means clustering, Apriori Algorithm, KNN etc.

3.4 Hybrid-Based Approach

Few research techniques have proposed Hybrid-Based Approach that uses both ML

and lexicon-based approach in order to achieve best of both. Combination of lexicon

and learning increases accuracy (Jain & Dandannavar, 2016).

3.5 ML-Based Sentiment Analysis Steps

ML-based approach is most popular as well as efﬁcient for sentiment analysis till

now.

Figure 5[35], given by yogi and paudel, 2020, shows a simple ﬂow chart of

machine learning-based sentiment analysis/opinion mining methods

218 Asmita De and S. Mishra

Fig. 5 Flow chart of

machine learning-based

technique for sentiment

analysis [23]

Augmented Intelligence in Mental Health Care … 219

3.6 Brief Discussion on Techniques and Algorithms

Table 6describes various algorithms used for sentiment analysis, their brief principle

and beneﬁts.

3.7 An Example of Sentiment Analysis Using Textblob

TextBlob is a python library that actively uses Natural language Toolkit(NLTK)that

gives an easy access to a lot of lexical resources and helps us to work on

categorization, classiﬁcation and other tasks.

The sentiment property returns a tuple. 1. Polarity: it is measured on the scale

from −1to1.−1 indecates a very negative statement where +1 indicates a positive

statement 0 is neutral.

Text Blob also helps with ﬁne-grained analysis (emotions emojis etc.) as it has

semantic labels.

2. Subjectivity: It refers to the amount of personal opinion and factual information

contained in the text.

Subjectivity measured from 0 to1. 0 indicates very objective and indicates very

subjective. The higher subjectivity means the text contains personal opinion more

rather than factual information.

TextBolb calculates subjectivity with the help of intensity.

Intensity deﬁnes if a word modiﬁes the immediate next word. In English, adverbs

act as modiﬁers (Example: Very nice).

Example

Tex t =it was nice book.

1. Import textBlob

2. Valance =TextBlob (text)Valance sentiment

Result:

Sentiment (polarity =0.8, subjectivity =0.6000000002).

So the given example is positive and subjective.

If library returns 0,0 then.

1. Probably sentence doesn’t contain any word that has a polarity in NLTK set Or,

2. May be words in the sentences diffuses out the effect of each other.

For example,

This pen is good but I went to try another here ‘good’ gives +ve polarity ‘but’

gives −ve polarity.

They may diffuse each other’s effect and the result can be (0,0).

220 Asmita De and S. Mishra

Tabl e 6 Various algorithms for sentiment analysis

Name Principle Beneﬁts

1. Support vector machine

(SVM)

It is liner model supervision to

utilize for classiﬁcation and

regressions problems. This

algorithm creates a hyper plan

to separate data sets into classes

and hence enhanced separation

increases the training data

margin

Workswellevenwhenwedo

not have any proper idea on the

data

2. Naive Bayes This one is used to the

determine possibilities of an

entity with certain characteristic

to belong to a certain class

P(label/features) =p(label)*

P(features/label)/p(features)

Suitable one for predicting

multi-class classiﬁcation

3. Bayesian network It is used to understand relation

among different features but this

is very pricey algorithm so use

of this is lesser

It is compact, ﬂexible and

interpretable representation

4. Decision tree Training data classiﬁcation and

regression. These classiﬁers use

recursion techniques

Easy to prepare, read, interpret

5. KNN With testing dataset this is used

for classiﬁcation and regression

Quick, simple to interpret and

also versatile

6. k-means clustering Used on unlabeled dataset to

convert them into different

clusters. This unsupervised

learning algorithm repeatedly

divides dataset into k number of

cluster until the best cluster has

been found

Guarantees convergence. Easily

adapts to new examples

7. Maximum Entropy In encoding labeled feature sets

are converted into vectors.

These vectors are further

converted and utilized to

determine value of features that

will help to predict the label of

features set

Able to incorporate prior

information in the density

estimation

8. Random Forest This supervised classiﬁcation

and regression uses training data

set as input classiﬁcation trees

are generated by set of several

decision trees

Reduces overﬁtting in decision

trees. Improves accuracy

Automates missing values

present in the data

(continued)

Augmented Intelligence in Mental Health Care … 221

Tabl e 6 (continued)

Name Principle Beneﬁts

9. Vote It is a meta-algorithms that

combine various

classiﬁers-decision tree, naive

Bayes, logistic regression,

Random forest to enhance

classiﬁcation performance and

accuracy

As it combines more than one

algorithm, works better than

used individual algorithms

4 Comparative Analysis of Existing Models

Recent paper shows that, with the combination of Morphological Sentence Pattern

Model and Sentiment Analysis, many good outcomes can be achieved.

Algorithms like Naïve Bayes, Maximum Entropy, SVM, KNN work well for

Sentiment Analysis.

Combination of Naïve Bayes and SVM works better.

Vote performs slightly better than only NB or only SVM.

RNTN, CNN are very efﬁcient frameworks in Sentiment Analysis.

Now, in recent time, with powerful contextualized word embedding and network

like BERT the accuracy has been increased.

Table 7, given by Mehta and Pandaya [36], shows accuracy in output of various

paper works those have used various techniques for performing certain tasks.

Table 8shows beneﬁts and disadvantages of both Lexicon and Machine learning-

based approach. All over, ML-based approach is more preferred than lexicon-based

approach.

5 Application and Beneﬁts of Sentiment Analysis in Mental

Health Care

5.1 Depression Detection

Depression refers to a kind of mood disorder that is considered a serious issue that

can get worse without proper treatment.

Symptoms: a depression patient may experience.

1. Behavior: one can feel no interest left for anything, tiredness, suicidal thoughts,

consuming drugs, trying to get pleasure from unnecessary risky activity.

2. Mood: anger, irritability, empty feel, frustration, anxiousness.

3. Difﬁculties in cognitive ability, concentration problem, delay in works and

responses.

4. Physical problems: insomnia, excessive sleepiness, pains, headache etc.

222 Asmita De and S. Mishra

Tabl e 7 Comparative study of techniques of sentiment analysis

S. No Paper title Methodology used Review dataset Accuracy (%)

1 A feature-based

approach for

sentiment analysis

using SVM and

co-reference

resolution [37]

SVM and co-reference

resolution

Training dataset of

product review

73.6

2Neural networks for

sentiment analysis on

twitter [38]

Neural network with

feed forward method

Twitter dataset 74.15

3 Study of twitter

sentiment analysis

using machine

learning algorithms

on python [39]

Naïve Bayes,

SVM

Maximum entropy

Twitter dataset 86.4

73.5

88.97

4 Sentiment analysis

using neural

networks: a new

approach [40]

Convolutional Neural

Network

Product data review

Twitter data

74.15

64.69

5 Sentiment analysis of

twitter corpus related

to artiﬁcial

intelligence assistants

[41]

Vale n ce Awa r e

Dictionary and

Sentiment Reasoner

(VADER)

Reviews of electronic

product

87.4

6A framework for

sentiment analysis

with opinion mining

of hotel reviews [42]

Naive Bayes Hotel reviews from

OpinRank

83.5

7 Aspect-Level

Sentiment Analysis

on E-Commerce Data

[43]

Naïve Bayes

SVM

Amazon customer

review data

90.423

83.43

8Document level

sentiment analysis

from news articles

[44]

Machine learning

approaches

BBC news dataset 57.7

9 Polarity shift

detection approaches

in sentiment analysis:

Asurvey[45]

Lexicon-based and

supervised machine

learning-based

Product review 84.6

10 A sentiment analysis

method of short texts

in microblog [46]

Language technology

platform (LTP) for

dependency syntax

analysis

COAE2014(BBC

DataSet)

86.5

(continued)

Augmented Intelligence in Mental Health Care … 223

Tabl e 7 (continued)

S. No Paper title Methodology used Review dataset Accuracy (%)

11 SemEval-2016 task 4:

sentiment analysis in

twitter [47]

SVM Twitter dataset 84.5

12 A topic-based

approach for

sentiment analysis on

twitter data [48]

SVM Twitter dataset 74.09

13 Ensemble of

classiﬁcation

algorithms for

subjectivity and

sentiment analysis of

arabic customers’

reviews [49]

Naive Bayes,

SVM

Arabic reviews from

jeeran.com(service

and product reviews)

97.06

89.1

14 Cities: A Naive-Bayes

Strategy for sentiment

analysis on English

tweets [50]

Naïve Bayes Training dataset of

tweets by

SEMEVAL2014

76.54

15 Opinion mining on

social media data [51]

Naïve Bayes Twitter dataset 76.8

16 Sentiment knowledge

discovery in twitter

streaming data [52]

Multinomial Naïve

Bayes

Twitter API 82.45

17 Twitter as a corpus

for sentiment analysis

and opinion mining

[53]

SRF Twitter dataset 56.4

Tabl e 8 ML versus lexicon based approaches

Approaches Beneﬁt Disadvantages

Lexicon-based Domain independent,

Fast

No need of labeled dataset

Low accuracy

Needs dictionaries containing lot of

opinion expressing words, their related

terms

Machine learning-based High accuracy,

No need of vast dictionary,

High adaptability

Domain dependent,

Slow,

Needs labeled data

Nowadays social media has become a vital way to express our thoughts. Utiliza-

tion of various social media platform is expanding these days on a large scale. For

dataset we can use comment section and post captions etc. Use of words in sentence

like happy, good, well, nice indicates positive attitude while sad, worry, cry indi-

cates sadness; shit, kill, get lost, stop, may indicate disgust or anger. On this way a

224 Asmita De and S. Mishra

dataset containing predeﬁned linguistic cues and related emotion may be used for

classiﬁcation.

An article prepared by Md Raﬁqul Islam, Ashad Kabir, Hua Wang, Anwaar Ulhaq

in 2018 shows the depression detection from facebook comments. Algorithm uses

are SVM, Decision Tree, Ensemble, KNN.

5.2 Prediction on Suicidal Tendency

Suicide is becoming worse public health concern day by day. According to few

research, 15–24 age groups are prone to having suicidal thoughts or attempt suicide

more.

This young group frequently uses social media online web in order to commu-

nicate with outer world. Hence, we can easily analyze the situation and take aware

steps accordingly.

For handling a personal therapy session, the therapist can also get the help of

sentiment analysis by AI. Though the accuracy level of our available technologies is

not up to the mark to replace a psychologist but it can help for sure.

By analyzing available dataset determining the polarity regarding several predictor

of suicide such as depression, insomnia, hypersomnia, stress, anxiety etc., we can

build our model to detect suicidal tendency.

It has been found through recent papers that males are more prone to attempt

suicide than females and females are more prone to have suicidal thoughts than

male.

So here in case of analyzing comments, there can be a problem such as words

used by female may indicate more serious condition than the words used by males

where the actual condition may differ.

In this case previous suicide notes can provide material for natural language

processing. By examining and analyzing suicide notes using content analysis,

sentiment analysis, emotion detection our data set may be prepared.

5.3 Other Mental Issues Detection

Nowadays with the advancement of technology our young generation tends to spend

more time in the virtual world. Moreover, a sudden upgrade in society and technical

world has created a huge general gap. In India for the ﬁrst time, number of single

families have exceeded the joint families.

With all the condition our new generation is facing a whole different world where

guiding them and understanding their mental situation has become a tough task.

But as they are mostly involved in online web so sentiment analysis and emotion

detection can play a big role here.

Augmented Intelligence in Mental Health Care … 225

Sentiment analysis also can predict about one’s breakup, other mental breakdown

etc. current mental situation through the posts and activity of them in web.

6 Augmented Intelligence in Mental Health Care

Artiﬁcial Intelligence functions with methods, techniques and systems. But in health

care, the more appropriate term is ‘augmented intelligence’ because, it covers the

idea of human decision-making and highlights human capacities augmented with

necessary tools and technologies. Mental health care, being a very sensitive ﬁeld,

deﬁnitely cannot replace human intelligence but augmented intelligence speciﬁes

systems that augment human intelligence rather than replacing.

Combination of AI and human intelligence in health care will deﬁnitely accelerate

the treatment process in various diagnosis and decision-making.

6.1 AI-Based App for Mental Health Tracking

Ginger.io is an app that tracks or monitors the mental health of individual, uses

the mobile data for getting insight and is able to provide customized care for the

individual who needs mental health support. This app enables users having mental

health issues to contact with professional therapists. This app gathers and analyze

personal data and helps the therapists to provide customized support.

This app gathers information about how long a person texts, talks, does exercise

or how much time they sleep. For example, if a person is in depression, he/she may

show insomnia. Comparing past behavior with present one, one can get an idea about

the patient’s mental health.

Although ginger.io can face challenges like identifying accurate cause of change

in behavior, privacy in smartphone and protecting users’ data can hide important

information about the person’s mental health condition.

When machine intelligence combines with human intelligence, it can eliminate

human errors and make the process more efﬁcient. This collaboration is what we are

looking for at this era in term of ‘Augmented Intelligence’.

With the help of Augmented Intelligence, health care stuffs can be offered timely,

data-driven treatment recommendation that they may modify, accept, reject based on

their own situation and hence provide service at their best.

226 Asmita De and S. Mishra

6.2 Remote Patient Monitoring Devices

Recently, many remote patient monitoring devices are being used that utilize

Augmented Intelligence, that help the health care stuff to work with more care while

performing other tasks as they know their patients are being regularly evaluated.

‘MYIA’ collects data of patients who are suffering from chronic condition and

provides the information if he/she needs clinical intervention.

‘Innovaccer’ has developed a social vulnerability index that can be used to analyze

individual as well as population health condition.

The idea of having a digital health twin has also been proposed as future of

Augmented Intelligence in mental health care, where the ‘twin’ will contain all data

about the individual’s health condition and will provide guideline for betterment.

6.3 Reﬂection™—By Steve Ardire

It is a mental telehealth app based on augmented intelligence that helps to perceive

and interpret unconscious tactic behavioral and emotional intelligence. Untreated

behavioral health issues can cause other serious health issues.

There are few challenges in diagnosing mental health issues as human behavior

is way more complex than our current fundamental methods can deal with. For

example, CBT (cognitive Behavioral Therapy) focuses on eliminating symptoms

whereas patient can have multiple issues. DSM (diagnostic and Statistical Manual

of Mental Disorder) assumes the symptom to be for speciﬁc issue but in practical

patients have maximum of the time, very mix problems. Existing methods don’t

consider about the data our brain processes unconsciously as well as don’t really

account the fact that the ego biases the perception and past believe and behavior work

like glass for us through which we try to see our new experience. With the desire to

overcome these issues, Digital Behavioral Therapeutics is gaining recognition and

funding.

Augmented Intelligence, unlike simple machine learning technology, is able to

interpret unconscious behavior without bias to understand emotion, intent more

appropriately. In March, 2020, a digital therapeutic startup for behavioral mental

health, SignalActionAI has been established. The product Reﬂection™ analyzes

user data and provides clear view of patient’s psychological and emotional wellbeing

based on the emotion, intent and behavior related data in order to help psychologists,

psychiatrists make more informed treatment decisions.

Reﬂection can analyze contextual conversations, emotions and behavior in real

time and thus ‘can reveal the unseen’. It is able to detect subtle, nuanced patient

perceptions, anxieties and thoughts imperceptible to human eye. (Collected from

bringing augmented intelligence to mental health care, 03/10/2021) [54].

In Fig. 6shows the working of Reﬂection™ [54].

Augmented Intelligence in Mental Health Care … 227

Fig. 6 Reﬂection™ working model

An another important and wonderful possibility of utilizing Augmented Intel-

ligence in mental health care includes ‘Talk Therapy With AI’. In traditional talk

therapy, patient generally visits the therapist once in a week. With AI, this therapy

can be available for 24/7 for the patient. Talk therapy provided by augmented intel-

ligence will be surely non-judgmental where as in person this risk is not 100%

ignorable for few professionals [55].

However, there are few challenges augmented intelligence faces while in conver-

sation with patients. In therapeutic session the situation can arise while patient can

show extremely vulnerable emotions, where for some AI can behave as obvious but

for some it can be confusing. The AI designed for children may work very poor

for adults. WE cannot absolutely trust upon algorithms for situation like suicidal

tendency as machine can have errors. Hence we are in a journey where we are

trying to establish a well utilization of Augmented Intelligence in order to enhance

therapeutic care but NOT TO REPLACE IT.

228 Asmita De and S. Mishra

7 After Covid 19 Effect in Mental Health

This pandemic has not only affected us physically, economically but the mental

issues since last 1.5 year have been increased dangerously. It has been found that

among young adults report of substance use had been 25% there become pandemic

it has 13% suicidal thoughts now approximately 26% where previously it was 11%.

Due to economic downward graph job loss has increased the rate of depression

anxiety substance uses and even suicide thoughts. Essential front line workers are

more likely to report symptoms of anxiety and depressive disorder (42%) than non-

essential workers (30%). Domestic violence has been increased in a large scale.

Women have reported higher rate of anxiety and depression than men. Isolation has

caused loneliness in a large scale.

7.1 Sentiment Analysis Role

Due to stay at home period use of social media has increased in a large volume.

We have seen during pandemic it has become frequent to make something social

platform sensation within one night, so it is clear that we are so involved in social

media now that we are able to ﬂood a certain post overnight all over India [56].

Hence, this platform should be used to take productive steps to handle the mental

health of society in a better manner.

With the help of sentiment analysis, we can predict the near future concern about

mental health.

According to psychologists, degradation in mental health due to covid-19 period,

needs much time to go back in baseline again.

Researchers are analyzing thousands of reddit posts and providing several key

impacts on current mental health situation. According to recent researches, suici-

dality, loneliness has been more than doubled during covid-19 crisis. It is found that

in pandemic, ADHD, eating disorder also increasing.

Other social issues like racism, bullying, discrimination, verbal violence are also

increasing according to the ﬁndings of several sentiment analysis research paper.

So we can conclude, the general sentiment in community level is becoming more

negative.

The bad issue is due to lockdown, it has been difﬁcult to provide the necessary

treatment to people who need. However, many online platforms are now arising and

improving to provide mental health care service in online mode.

Figures 7,8and 9represents Word cloud related to covid-19 analysis, given by

By Mateus Broilo and Andrea Posada Cardenas [57]:

Augmented Intelligence in Mental Health Care … 229

Fig. 7 Covid 19 analysis of 2020, May

8 Challenges and Future Work

1. Work on sarcasm in sentiment analysis is a tough task. Though there are

some approaches that use classiﬁcation methods for identifying sarcasm but

improvement is needed here [58–62].

2. If a document discusses about several identities it becomes difﬁcult to do

classiﬁcation. Current accuracy in this case is not satisfactory yet.

3. We are still unable to understand the reason behind an expressed emotion.

4. With currently used algorithm we generally consider subjective sentiment and

ignore objective one, saying it’s a factual sentence so contains no emotions.

But a factual opinion too can carry important emotions. We are still unable to

handle those.

230 Asmita De and S. Mishra

Fig. 8 Covid 19 analysis of 2020, June

Fig. 9 Words cloud of covid 19 analysis

Augmented Intelligence in Mental Health Care … 231

5. Sometimes a mentally depressed person can use happiness expressing words,

can post joyful content in social media. We cannot actually say if document

contains his/her expression, expressing correct mental state or not.

6. We are still unable to achieve high accuracy where the document contains

negations, ambiguous words.

7. Resolving multipolarity is yet a tough task.

8. Sometimes comparatively an intellectual person expresses his/her negative

emotion through sarcasm that seems funny and at ﬁrst sight positive. Here we

can get a big failure to detect the reality.

9. People can use Irony or tone also to express their negative emotion. Here

algorithm can detect as positive opinion.

10. Sentiment analysis also can be selectively difﬁcult according to speciﬁc

language that is being considered.

11. We use AI for emotion detection from facial movement where many challenges

are being arrived.

(a) If a person smiles AI will detect that as happiness but it may be the case

that the person is actually sad and faking the smile.

(b) We don’t have sufﬁcient datasets. There are many remote areas for which

we are still unknown about the expression patterns of inhabitant from

those areas.

nize as angry or sad. But actually it is the person’s normal facial state

that expresses particularly no emotions.

12. We may use Driverless AI for building sarcasm detection classiﬁer for more

efﬁciently.

9 Discussion and Conclusion

In this paper, beneﬁts and application of sentiment analysis for mental health care

purpose have been discussed. With the various methods used for sentiment analysis

and comparison among them, ﬁnding the challenge and future scope also has been

tried to analyze. Although for classiﬁcation of facial sentiment analysis, it is found

that we do not have sufﬁcient dataset also we need more developed algorithm for

proper accuracy. It has been found that in covid-19 situation, sentiment analysis is

playing a vital role in analyzing the community mental health issues and it will be

doing more in near future. However, more research work is needed in future to gain

sufﬁcient accuracy in performance so that we can use sentiment analysis in mental

health issues diagnosis purpose.

232 Asmita De and S. Mishra

References

1. Mishra, S., Tripathy, H. K., Mallick, P. K., Bhoi, A. K., & Barsocchi, P. (2020). EAGA-MLP—

An enhanced and adaptive hybrid classiﬁcation model for diabetes diagnosis. Sensors, 20(14),

4036.

2. Mukherjee, D., Tripathy, H. K., & Mishra, S. (2021). Scope of medical bots in clinical domain.

Technical Advancements of Machine Learning in Healthcare, 936, 339.

3. Mäntylä, M. V., Graziotin, D., & Kuutila, M. (2018). The evolution of sentiment analysis—A

review of research topics, venues, and top cited papers, 16–32.

4. Anvar Shathik, J., & Krishna Prasad, K. (2020). A literature review on application of sentiment

analysis using machine learning techniques

5. Boiy, E., & Moens, M.-F. (2009). A machine learning approach to sentiment analysis in

multilingual Web texts. Information Retrieval Springer, 12(5), 526–558.

6. Go, A., Bhayani, R., & Huang, L. (2009). Twitter sentiment classiﬁcation using distant

supervision. Processing, CS224N Project Report Stanford,1(12), 2009–2016.

7. Duan, W., Yu, Y., Cao, Q., & Levy, S. (2015). Exploring the impact of social media on hotel

service performance: A sentimental analysis approach. Cornell Hospitality Quarterly, 57(3),

282–296.

8. Oscar, N., Fox, P. A., Croucher, R., Wernick, R., Keune, J., & Hooker, K. (2017). Machine

learning, sentiment analysis, and tweets: An examination of Alzheimer’s Disease Stigma on

Twitter. The Journals of Gerontology. Series B, Psychological Sciences and Social Sciences,

72(5), 742–751.

9. Kumar, A., & Sebastian, T. M. (2012). Sentiment analysis: A perspective on its past, present

and future. International Journal of Intelligent Systems and Applications, 4(10), 1–14.

10. Duric, A., & Song, F. (2012). Feature selection for sentiment analysis based on content and

syntax models. Decision Support Systems, 53(1), 704–711.

11. Vohra, S., & Teraiya, J. (2013). A comparative study of sentiment analysis techniques. Inter-

national Journal of Information, Knowledge and Research in Computer Engineering, 2(2),

313–317.

12. Medhat, W., Hassan, A., & Korashy, H. (2014). Sentiment analysis algorithms and applications:

Asurvey.Ain Shams Engineering Journal, 5(4), 1093–1113.

13. Axhiu, M., Veljanoska, F., Ciglovska, B., & Husejni, M. (2014). The usage of sentiment analysis

for hearing the voice of the customer and improving businesses. Journal of Educational and

Social Research, 4(4), 1–17.

14. Suryawanshi, R., Rajput, A., Kokale, P., & Karve, S. S. (2020). Sentiment Analyzer using

machine learning. International Research Journalof Modernization in Engineering Technology

and Science, 02(06), 1–12.

15. Sentamilselvan, K., Aneri, D., Athithiya, A. C., & Kumar, P. K. (2020). Twitter sentiment anal-

ysis using machine learning techniques. International Journal of Engineering and Advanced

Technology (IJEAT), 9(3), 1–9.

16. Godara, N., & Kumar, S. (2020). Twitter sentiment classiﬁcation using machine learning

techniques. Waffen-Und Kostumkunde Journal, 11(8), 10–20.

17. Yadav, N., Arora, M. S., & Tech, M. (2020). The performance of various supervised machine

learning classiﬁcation algorithms in sentiment analysis of online customer feedback in

restaurant sector of hospitality industry. International Journal for Technological Research in

Engineering, 7(11), 1–12.

18. Gujar, M. A., & Pardeshi, N. G. (2020). Review on a sentiment analysis and predicting winner

for Indian premier league using machine learning technique. International Research Journal

of Modernization in Engineering Technology and Science, 2(6), 963–967.

19. Arote Rutuja, S., Gaikwad Ruchika, P., Late Samidha, S., & Prof. Gadekar, G. B. (2020).

Online shopping with sentimental analysis for furniture shop. International Research Journal

of Modernization in Engineering Technology and Science, 02(05), 1–8.

20. Raju, S. M., & Tarif, A. M. (2020). Real-Time Prediction of BITCOIN Price using Machine

Learning Techniques and Public Sentiment Analysis. Retrieved on May 2020.

Augmented Intelligence in Mental Health Care … 233

21. Ardianto, R., Rivanie, T., Alkhaliﬁ, Y., Nugraha, F. S., & Gata, W. (2020). Sentiment analysis

on E-sports for education curriculum using Naive Bayes and support vector machine. Journal

of Computer Sciences and Information, 13(2), 109–122.

22. Rachid, B. A., Azza, H., & Henda, B. G. (2018). Sentiment analysis approaches based

on granularity level. In 14th International Conference on Web Information System and

Technologies.

23. Tripathy, A., Agrawal, A., & Rath, S. K. (2016). Classiﬁcation of sentiment reviews using

n-gram machine learning approach. Expert Systems with Applications, 57, 117–126.

24. Mishra, S., Panda, A., & Tripathy, K. H. (2018). Implementation of re-sampling technique

to handle skewed data in tumor prediction. Journal of Advanced Research in Dynamical and

Control Systems, 10, 526–530.

25. Lee, D., Ju, H., Yu, H., Park, J.-m., & Kim, K.-Y. (2018). Dualsentinet: Dual prediction of word

and document sentiments using shared word embedding. In Proceedings of the 12th Interna-

tional Conference on Ubiquitous Information Management and Communication, IMCOM ’18,

pp. 34:1–34:9, New York, NY, USA. ACM.

26. Muhammad, A., Wiratunga, N., & Lothian, R. (2016). Contextual sentiment analysis for social

media genres. Knowledge-Based Systems, 108, 92–101. (New Avenues in Knowledge Bases

for Natural Language Processing).

27. Yang, B. & Cardie, C. (2014). Context-aware learning for sentence-level sentiment analysis

with posterior regularization. In Proceedings of the 52nd Annual Meeting of the Association for

Computational Linguistics (Vol. 1: Long Papers, pp. 325–335). Association for Computational

Linguistics.

28. Huang, M., Qian, Q., & Zhu, X. (2017). Encoding syntactic knowledge in neural networks for

sentiment classiﬁcation. ACM Transactions on Information Systems (TOIS), 35(3), 26:1–26:27.

29. Musto, C., Semeraro, G., & Polignano, M. (2014). A comparison of lexicon-based approaches

for sentiment analysis of microblog, 1314, 59–68

30. Al-Smadi, M., Al-Ayyoub, M., Jararweh, Y., & Qawasmeh, O. (2018). Enhancing aspect-

based sentiment analysis of arabic hotels reviews using morphological, syntactic and semantic

features. Information Processing & Management

31. Ma, Y., Peng, H., & Cambria, E. (2018). Targeted aspect-based sentiment analysis via

embedding commonsense knowledge into an attentive lstm. In AAAI.

32. Shams, M., & Baraani-Dastjerdi, A. (2017). Enriched lda (elda): Combination of latent

dirichlet allocation with word co-occurrence analysis for aspect extraction. Expert Systems

with Applications, 80, 136–146.

33. Rathan, M., Hulipalled, V. R., Venugopal, K. R., & Patnaik, L. M. (2017). Consumer insight

mining: Aspect based twitter opinion mining of mobile phone reviews.

34. Thakor, P., & Sasi, S. (2015). Ontology-based sentiment analysis process for social media

content. Procedia Computer Science, 53, 199–207. (INNS Conference on Big Data 2015

Program San Francisco, CA, USA 8–10 August 2015).

35. Yogi, T. N., & Paudel, N. (2020). Comparative analysis of machine learning based classiﬁcation

algorithms for sentiment analysis. International Journal of Innovative Science, Engineering &

Technology, 7(6), 1–9.

36. Mehta, P., & Pandya, S. (2020). A review on sentiment analysis methodologies, practices and

applications.

37. Krishna, M. H., Rahamathulla, K., & Akbar, A. (2017). A feature based approach for sentiment

analysis using SVM and coreference resolution. In International Conference on Inventive

Communication and Computational Technologies, ICICCT2017.

38. Duncan, B., & Zhang, Y. (2015). Neural networks for sentiment analysis on twitter. In IEEE

14th International Conference on Cognitive Informatics & Cognitive Computing (ICCICC

2015)

39. Negi, M., Vishwakarma, K., Rawat, G., Badhani, P., & Gupta, B. (2017). Study of twitter

sentiment analysis using machine learning algorithms on python. International Journal of

Computer Applications, 165(9), 0975–8887.

234 Asmita De and S. Mishra

40. Dhar, S., Pednekar, S., Borad, K., & Save, A. (2018). Sentiment analysis using neural networks:

A new approach. In International Conference on Inventive Communication and Computational

Technologies (ICICCT 2018)

41. Park, C. W., & Seo, D. R. (2018). Sentiment analysis of twitter corpus related to artiﬁcial intel-

ligence assistants. In 5th International Conference on Industrial Engineering and Applications,

2018.

42. Zvarevashe, K., & Olugbara, O. O. (2018). A framework for sentiment analysis with opinion

mining of hotel reviews. In Conference on Information Communications Technology and

Society (ICTAS) 2018.

43. Vanaja, S., & Belwal, M. (2018). Aspect-level sentiment analysis on E-commerce data. In

International Conference on Inventive Research in Computing Applications (ICIRCA 2018).

44. Shirsat, V. S., Jagdale, R. S., & Deshmukh, S. N. (2017). Document level sentiment analysis

from news articles. In International Conference on Computing, Communication, Control and

Automation (ICCUBEA) 2017.

45. Zirpe, S., & Joglekar, B. (2017). Polarity shift detection approaches in sentiment analysis: A

survey. International Conference on Inventive Systems and Control, 2017

46. Li, J., & Qiu, L. (2017). A sentiment analysis method of short texts in microblog. In Interna-

tional Conference on Computational Science and Engineering (CSE) and IEEE International

Conference on Embedded and Ubiquitous Computing (EUC) 2017.

47. Nakov, P., Ritter, A., Rosenthal, S., Sebastiani, F., & Stoyanov, V. (2016). SemEval-2016 Task

4: Sentiment analysis in twitter. In Proceedings of SemEval2016

48. Ficamos, P., & Liu, Y. (2016). A topic based approach for sentiment analysis on twitter data.

International Journal of Advanced Computer Science and Applications 2016.

49. Omar, N., Albared, M., Al-Shabi, A. Q., & Al-Moslmi, T. (2013). Ensemble of classiﬁcation

algorithms for subjectivity and sentiment analysis of Arabic customers’ reviews. International

Journal of Advancements in Computing Technology (IJACT), 2013.

50. Gamallo, P., & Garcia, M. (2014). Citius: A Naive-Bayes strategy for sentiment analysis on

English tweets. In International Workshop on Semantic Evaluation (SemEval 2014).

51. Liang, P. W., & Dai, B. R. (2013). Opinion mining on social media data. In IEEE 14th

International Conference on Mobile Data Management, 2013.

52. Bifet, A., & Frank, E. (2010). Sentiment knowledge discovery in twitter streaming data. In

International Conference on Discovery Science 2010.

53. Pak, A., & Paroubek, P. (2010). Twitter as a corpus for sentiment analysis and opinion mining.

In International Conference on Language Resources and Evaluation, LREC 2010.

54. Mishra, S., Dash, A., & Mishra, B. K. (2020). An insight of Internet of Things applications in

pharmaceutical domain. In Emergence of Pharmaceutical Industry Growth with Industrial IoT

Approach (pp. 245–273). Academic Press.

55. Mishra, S., Tripathy, H. K., & Acharya, B. (2021). A precise analysis of deep learning for

medical image processing. In Bio-inspired Neurocomputing (pp. 25–41). Springer, Singapore.

56. Mishra, S., Mishra, B. K., & Tripathy, H. K. (2015). A neuro-genetic model to predict

hepatitis disease risk. In 2015 IEEE International Conference on Computational Intelligence

and Computing Research (ICCIC) (pp. 1–3). IEEE.

57. Jena, L., Mishra, S., Nayak, S., Ranjan, P., & Mishra, M. K. (2021). Variable optimiza-

tion in cervical cancer data using particle swarm optimization. In Advances in Electronics,

Communication and Computing (pp. 147–153). Springer.

58. Mishra, S., Dash, A., Ranjan, P., & Jena, A. K. (2021). Enhancing Heart Disorders Predic-

tion with Attribute Optimization. In Advances in Electronics, Communication and Computing

(pp. 139–145). Springer.

59. Tripathy, H. K., Mishra, S., Thakkar, H. K., & Rai, D. (2021). CARE: A collision-aware

mobile robot navigation in grid environment using improved breadth ﬁrst search. Computers &

Electrical Engineering, 94, 107327.

60. Mishra, S., Thakkar, H., Mallick, P. K., Tiwari, P., & Alamri, A. (2021). A sustainable IoHT

based computationally intelligent healthcare monitoring system for lung cancer risk detection.

Sustainable Cities and Society, 103079.

Augmented Intelligence in Mental Health Care … 235

61. Mishra, S., Mishra, B. K., Tripathy, H. K., & Dutta, A. (2020). Analysis of the role and scope

of big data analytics with IoT in health care domain. In Handbook of Data Science Approaches

for Biomedical Engineering (pp. 1–23). Academic Press.

62. Tripathy, H. K., Mallick, P. K., & Mishra, S. (2021). Application and evaluation of classiﬁcation

model to detect autistic spectrum disorders in children. International Journal of Computer

Applications in Technology, 65(4), 368–377.

NLP Applications for Big Data Analytics

Within Healthcare

Aadarsh Choudhary, Anurag Choudhary, and Shubham Suman

Abstract The signiﬁcance of integrating Natural Language Processing (NLP)

approaches in healthcare research has become more prominent in recent years, and

it has had a transformational impact on the state-of-the-art. In healthcare, NLPs are

developed as well as assessed on the basis of words, phrases, or record-level expla-

nations based on patient reports such as side-effects of medications, Medicines for

illnesses or semantic characteristics are prescribed (nulliﬁcation, seriousness), etc.

While some NLP projects take into account customer expectations at the level of

an individual or a group, these projects are still in the minority. A special focus

is placed on psychological wellness research, which is currently the subject of

little research in healthcare NLP research networks but where NLP approaches are

widely used. Although there have been signiﬁcant advancements in healthcare NLP

strategy improvement, we believe that in order for the profession to grow further,

more emphasis should be placed on comprehensive evaluation. To help with this, we

offer some helpful ideas, including one on a minor etiquette that may be used when

announcing clinical NLP strategy improvement and assessment.

Keywords Natural language processing ·Big Data ·Health care ·Semantic

similarities ·Electronic health records (EHRs) ·Classiﬁcation ·Mental health ·

Kawasaki disease ·Huntsman Cancer Institute ·Linguamatics NLP platform ·

Genomic ·Bio-specimen ·Morphology

A. Choudhary ·A. Choudhary

Xformics Inc., Bangalore, India

e-mail: aadarsh.choudhary@in.xformics.com

A. Choudhary

e-mail: anurag.choudhary@in.xformics.com

S. Suman (B)

Indian Institute of Technology (ISM), Dhanbad, India

e-mail: shubhamsuman16@gmail.com

S. Mishra et al. (eds.), Augmented Intelligence in Healthcare: A Pragmatic and Integrated

Analysis, Studies in Computational Intelligence 1024,

https://doi.org/10.1007/978-981-19-1076-0_13

237

238 A. Choudhary et al.

1 Introduction

The healthcare sector is fast to recognize the value of data, information gathering

through EHRs (Electronic Health Records), sensors, and data from other sources

[1]. The battle to comprehend, on the other hand, the data acquired throughout the

procedure may drag on for a long period. Appropriate use of big data sources, such

as EHRs, might have a signiﬁcant inﬂuence on healthcare research and delivery [2,

3]. As a result of the huge quantity of documentation in the form of free text, There

has been a rush in research to improve the EHRs that are now accessible in Natural

Language Processing (NLP) methodologies and applications in the healthcare sector.

The ﬁeld has grown in popularity in recent years, resolving many of the problems

mentioned by Chapman et al. and implementing Friedman et al. [4].

As an example, consider the following suggestions for dealing with the major

difﬁculties of restricted cooperation, a lack of shared resources, and approaches to key

jobs that are based on evaluation like medical concepts’ de-identiﬁcation, recognition,

classiﬁcation, semantic modiﬁers, and information about time. The arrangement of

various common responsibilities has helped to overcome these issues. The Semantic

Evaluation (SemEval) challenges, eHealth challenges from the Conference and Labs

of the Evaluation Forum (CLEF), and the Informatics for Integrating Biology and the

Bedside (i2b2) are two of them. These efforts have resulted in a signiﬁcant platform

for the development of international NLP methods [5].

Mental health is a growing ﬁeld that has seen a surge in the use of NLP approaches

and strategies in recent years, owing in part to the fact that the majority of clinical

documentation is in free-text format, but also to the other forms of documents are

becoming more widely available that provide indicators of behavior, emotion, and

intellectual, as well as signals on how people are dealing with various illnesses and

therapies To name a few, such text from social media and online communities, as

well as doctor-patient exchanges and online treatment, are all possible sources [6].

Despite the fact that there have been a few common mental health tasks, the area is

currently narrower than biological or clinical in general NLP.

Because of the majority of NLP technique evolution and state-of-the-art results,

more NLP solutions for the analysis of complex clinical outcomes have been success-

fully deployed [7]. However, the methodologies for evaluating and appraising NLP

procedures differ from those utilized in clinical research investigations, Although,

for data preparation and extraction, the latter frequently relies on the former.

These discrepancies must be clariﬁed, and new techniques and procedures must

be developed to close the gap.

The goal was to examine these assessment difﬁculties by highlighting existing

research initiatives in these domains, and the event gathered together clinicians and

researchers from the ﬁelds of NLP, informatics, mental health, and epidemiology. Our

goal is to present a comprehensive overview of current state-of-the-art information

and to give advice on future paths in this discipline, with a particular focus on internal

and extrinsic evaluation difﬁculties.

NLP Applications for Big Data Analytics Within Healthcare 239

2 Why Big Data in HealthCare?

Because of the growing expense of medical treatments in nations like India, Big data

is in growing market in the healthcare industry. In reality, and over the past 20 years,

expenses have been signiﬁcantly greater than they should have been. In this area,

the writers could use some clever, data-driven thinking [8]. Current motivations are

also changing: numerous insurance ﬁrms are moving away from chargeless plans

(which promote the use of expensive and sometimes unneeded therapies as well as

serving large numbers of patients rapidly) and toward programs that prioritize patient

outcomes. Previously, there was no clear incentive for healthcare practitioners to

exchange patient information with one another, This made it more difﬁcult to make

use of analytics’ potential. Now that more insurance ﬁrms are paid based on patient

outcomes, they have a ﬁnancial incentive to disclose data that can help patients while

also saving insurance companies money.

Big data in healthcare refers to massive quantity of data gathered by the use

of digital technology to collect patient records and aid in hospital achievements

management, which would or else be too huge, and conventional technologies ﬁnd

it difﬁcult [9]. The use of big data analytics in healthcare has a number of beneﬁcial

and even regenerative implications. Futuristic data, in essence, implies the massive

amounts of data generated by the digitalization of every single thing that is then

aggregated and analyzed by particular technology. When used to healthcare, it will

make use of detailed health data from a community (or an individual) to help prevent

epidemics, treat illnesses, cut costs, and so on. Doctors make an effort to learn as

much as they can about their patients as early as possible in their lives so that they

can spot early warning indications of serious illness. Early diagnosis of a condition

is far easier and less expensive.

For years, obtaining high volume of data for medical purposes has been expensive

and difﬁcult. With today’s ever-improving technology, Not only is it easier to obtain

such data, but it is also easier to analyze it, but also to construct comprehensive

healthcare reports and turn them into critical insights that may be used to improve

therapy [10]. This is what healthcare data analytics is all about: using data-driven

ﬁndings to not only predict and solve problems before they become too serious, but

also to evaluate methods and treatments more quickly, keep better track of inventory,

and engage patients more in their own health by providing them with the tools to do

so [11].

In addition, Physicians are becoming more evidence-based, relying on large

swaths of research and clinical data rather than just their education and professional

judgment.

There are several uses of Natural Language Processing in Big Data Analysis of

healthcare data [12–14]

•Natural language processing technologies can assist patients to comprehend their

health by bridging the gap between complicated medical words and their compre-

hension. NLP can be a powerful tool in the ﬁght against EHR stress. NLP is used

by many physicians as a substitute for typing and handwritten notes.

240 A. Choudhary et al.

•Even when patients have access to their health data through an EHR system, the

majority of them struggle to understand it. As a result, only a small percentage

of people can utilize their medical data to make health decisions. With the use of

machine learning in healthcare, this might alter.

•NLP technologies may make it easier to assess and enhance the standard of care.

Identifying gaps in care delivery and assessing physician performance would be

needed for value-based reimbursement. HCOs may use NLP algorithms to help

them achieve this, as well as identify possible mistakes in care delivery.

•NLP algorithms can extract important information from big datasets, giving

doctors the tools they need to treat patients with complicated problems.

Natural Language Processing can also be used for predictive analysis by the

author. It is critical for emergency rooms to have complete data on hand at all times

[15]. For example, if Kawasaki illness is misdiagnosed or treated incorrectly, it might

result in serious complications. In comparison to a human examination of clinician’s

notes, an NLP-based algorithm identiﬁed at-risk patients with Kawasaki illness with a

sensitivity of 93.6% and speciﬁcity of 77.5%, as scientiﬁc data show. The advantages

of employing NLP may certainly be extended to other ﬁelds of interest, and a variety

of algorithms can be used to identify and forecast certain situations among a large

number of variables patients. Even while the healthcare industry as a whole has to

improve its data skills before using NLP tools, it still has a lot of room to improve

care delivery and streamline procedures. Natural Language Processing and other

machine learning techniques will be critical in the better clinical decision support

and patient health outcomes in the future.

3 Challenges

Natural Language Processing (NLP) is a set of evaluation approaches that deal with

language in various forms, such as text, speech, and so on [16]. While clinical natural

language clariﬁcation has gotten more attention recently, it still lags behind the

general and biological sectors. This is due in part to a shortage of a clinical story

owing to privacy concerns corpora, However, it is also owing to the fact that clinical

narrative corpora are unusual, It is also because of the added challenges that come

with writing a clinical narrative piece. Biomedical literature is produced in a clear

and concise manner to communicate research ﬁndings to a broad audience. On the

other hand, clinical narrative text is authored by medical practitioners to inform

other medical practitioners or oneself about a patient’s present state and history.

The reason for the patient’s visit (i.e., the major complaint), the therapies given,

the results of some tests performed, and These notes contain all of the information

needed to make informed decisions about a patient’s care, Progress notes, which

document the treatment offered, consult reports, which communicate the results of

diagnostic testing, and other types of notes are common and discharge summaries,

which document a patient’s complete time in the hospital. These notes are quite

NLP Applications for Big Data Analytics Within Healthcare 241

varied in substance and amount of information due to their many objectives [17].

Furthermore, its layout is often unstructured, as a result of this, there is a wide variety

of haphazard structural elements.

Following common challenges have been identiﬁed while working on NLP in

Healthcare Domain.

Category Detail Example

Formatting possibilities are

extensive

Semantics of variable

formatting

“Admitting Diagnosis:

ANEMIA, splenomegalia

sintomi” states the section

header.

“Neuro: seizuer” is an

inseparable term

Amorphouses “Height: (in) 75 MASS(lb): 275

BODY SURFACE AREA (m2):

2.38 m2blood pressure (mm

Hg): 110/70 pulse (bpm): 108”

“CIPLOX 500 mg Pill Sig: Two

(2) Pill PO everyday (everyday)”

Points errors Short pause: “negative m/r/g b/l

pulmonary oedema w/associated

top airway sounds.”

Periods: “Patient with good

appearance no ailment distress

Chest is clear No

pneumothorax”

Uncommon grammar Words are missing that ought

to be present

Verb : “ Neg ati ve

Systolic failure”

Object: “Beta blockade of the

locpressor was administered.”

Artifacts: “echo appeared no

proof of [a] Systolic failure”

Opulent illustrations Wide range of textual concerns Patient: “Three times awake,

attentive, and oriented.”

Analysis: “LAE”

Examination: “Suboptimal

image quality—low echo

windows”

History: “CABG was performed

on my mother while CAD was

performed on my grandma”

Language speciﬁc to medical

context

Jargon: “multiple erosoins

present, multiple biopsies taken

for HP/E and gene expert for

TB”

Ad-hoc abbreviations: “His

CBC, FT3/FT4/TSH/FBH/LFT,

were ordinary”

Abbreviation: “current tension

PTX at OSH”

(continued)

242 A. Choudhary et al.

(continued)

Category Detail Example

Mismarks “Bicuspid stenosis is not present

and defanite [sic] vomiting is

not seen”

“s/p gsw now with fevers r/o

abcess [sic]”

“Sclarea [sic] anicteric”

“ventilator depending on

respiratoy [sic] failure”

4 Case Study

4.1 Huntsman Cancer Institute Optimizes Research

with Linguamatics NLP Platform

In Salt Lake City, HCI at the University of Utah is a nationally acclaimed cancer

hospital and research facility. The RISR team at HCI assists HCI by providing

comprehensive computing and information solutions that help researchers complete

their studies faster.

Samir Courdy is the director of RISR and Chief Research Information Ofﬁcer

for HCI. His team is responsible for developing tools and systems to capture data

that advance quality of care efforts and enables research. “We capture data in all

forms, whether it is genomic, clinical, bio-specimen, or population-based studies,”

explained Courdy. “In addition to collecting patient data for quality of care initiatives,

we improve access to data for research purposes so that users have high-quality data

for studies or to apply for research grants.”

Gaining access to high-quality data, however, can be a challenge [18]. While

ﬁnding demographic data or coded disease information is relatively straightforward,

researchers also need to access critical details such as morphology, topography,

tumor size and stage. Often these insights are not easily extracted and remain trapped

in narrative-style surgical or clinical notes, or in pathology and radiology reports.

Manual abstraction of data is very time-consuming and fallible. It may take a user

several hours to track down, review and transcribe the critical elements from indi-

vidual patient records, and because documents must be individually reviewed, even a

diligent researcher may miss critical details. Over the years, Courdy explored various

technologies to improve the search process and automate the capture of high-quality

data. “About 10+ years ago we began developing internal methods for using publicly

available natural language processing tools,” said Courdy. “We used the technology

to automate breast cancer research, but felt the process was too lengthy and tedious

and wasn’t yielding results at the scale we were hoping for. It was also fragile and did

not translate well to other disease areas.” Courdy continued looking for solutions and

NLP Applications for Big Data Analytics Within Healthcare 243

eventually encountered Linguamatics at an Informatics conference in 2011. “After

learning about the Linguamatics NLP tools and reviewing the […] solution, we

decided to abandon the old technology we had been using for NLP and started using

[the Linguamatics NLP platform]. And the rest is history.”

•Solution

Once HCI implemented the NLP platform, HCI’s ﬁrst project focused on breast

cancer. “We started with breast cancer because we had already built a collection

of pathology reports with patient cohort data and were able to use the same

gold standard for the […] toolset,” said Courdy. “Initially we trained [the NLP

platform] on the breast cancer data set while working closely with the folks at

Linguamatics. This project was intended as a proof-of-concept for using [the

NLP platform]—and the concept worked.” Following the success of the initial

project, the RISR team expanded into other disease areas. “For example, we

built a very robust process for prostate research and then focused on hematologic

malignancies, starting with nonHodgkin’s lymphoma,” said Courdy. Since ﬁrst

launching the Linguamatics NLP platform, HCI has expanded its use and advanced

NLP tools for different other conditions. Courdy noted that HCI’s work with the

NLP platform and Linguamatics has opened the doors for collaboration with other

cancer institutes [19]. “In 2014, I went to a big data conference and was talking

about the work we were doing with [the NLP platform],” said Courdy. “Someone

in the audience asked if we would be willing to share certain queries. Linguamatics

agreed to explore the idea and today all our queries are shareable, which beneﬁts

everyone.” In fact, once Linguamatics helped enable the sharing of NLP rules,

HCI began collaborating with other institutions. “We were able to work with

another group on a nonhodgkin’s lymphoma study and now, after sharing data

back and forth, we are ﬁnalizing a paper for publication,” said Courdy.

•Success

HCI’s use of the NLP platform has made its collection processes faster and more

efﬁcient and provided researchers with higher quality data to advance research

initiatives [20]. Two speciﬁc areas of improvement include:

•Faster data capture: Courdy noted that “Now we can capture more data faster

on many, many more documents than an individual could do manually. I do not

have a quantiﬁable number, but we signiﬁcantly improved our processes and the

quality and the access to the data.”

•Higher quality data: HCI’s use of NLP has improved access to higher quality data,

which in turn has advanced research efforts and enhanced disease understanding

in support of achieving better outcomes. Since adding the Linguamatics NLP

platform, HCI has successfully published several studies and presented research

ﬁndings on hematology malignancies at the 2018 ASH conference.

“We’re able to provide our principal investigators with the data they needed to

apply for or qualify for permits, write papers, or identify outﬁts for speciﬁc learnings,”

said Courdy. “For example, when applying for grants you have to prove that you have

the depth and breadth of data to support your research. With [the NLP platform] we

244 A. Choudhary et al.

are able to demonstrate that we have the tools needed to facilitate access to high-

quality data.” For other organizations implementing the NLP platform to advance

research efforts, Courdy offered the following advice [21]:

1. Begin by identifying the problem you are attempting to tackle. Once the problem

is well deﬁned, you will have a better understanding of the data you are looking

for, allowing you to narrow your ﬁeld of search.

2. Find the right people to perform the work and provide them with training oppor-

tunities. Give them time to learn the internal workings of the NLP platform so

that they understand how best to utilize it in a very efﬁcient way. Allow them

to attend formal training sessions and work on a small prototype project.

3. Have a team to provide support for users on the NLP platform framework. Verify

that data within an electronic warehouse can be accessed electronically and is

available in a format that the NLP platform can read and rules can be applied

[22].

“Research organizations like ours, that have massive amounts of data, need tech-

nology that automates the capture of relevant data,” said Courdy. “This facilitates

the delivery of high-quality data that is relevant; provides insights into patients’

treatments, outcomes and lifestyle habits; guides the development of therapies;

and provides researchers with the information they need for grant applications and

additional studies.”

4.2 Health System Applies Linguamatics NLP Platform To

Improve Care And Population Insights

Situation

To support the use of RWE in FDA ﬁlings, our customer, a health system in the

Midwest, specializes in contract research for medical device and pharmaceutical

companies. Because it is a huge health system with a sophisticated and centralized

Epic electronic health record (EHR) system, it has a large data collection of patient

treatments and outcomes. In addition, it has a precise supply chain management

system that tracks every device and treatment from manufacturer to patient. “Our

combination of a single mature EHR and ability to track exactly who received what

device is a key differentiator when we are working with life science companies,” said

a V.P. at the organization. A recent study, sponsored by a medical device company,

required analysis of 100,000 congestive heart failure (CHF) patients to assess the

impact of ﬁtting CRT devices, and capture patient outcomes. While identifying and

pulling records for this population is relatively trivial for this health system, a number

of the attributes needed by the study were trapped in clinical notes [23]. These

attributes indicate the severity of the disease and the impact of the device, for example:

•ejection fraction—a measure of how well the heart is pumping out blood;

•New York Heart Association classiﬁcation—a fourlevel indication of severity;

NLP Applications for Big Data Analytics Within Healthcare 245

•symptoms including dyspnea (shortness of breath), fatigue and dizziness; and

•the CRT model and whether it was explanted (removed), and why.

Manual chart review is impracticable at best; even if each chart were reviewed for

one hour, it would take more than 55 full-time equivalents (FTE) years to accomplish

[24]. Our client had tight deadlines to deliver the project and knew it would need

to use some innovative approaches to meet its target. The organization had been

evaluating the Linguamatics NLP platform in another part of the business, and a V.P.

attended an internal demo showing how ejection fraction could be extracted from

clinical notes. Once that connection had been made, he and his team began working

with Linguamatics to use NLP to deliver the project. “I could see that Linguamatics

could help us with our project, but we hadn’t used NLP techniques before so we

weren’t out of the woods quite yet. My team of data scientists needed to get up to

speed with the tool and integrate it into our workﬂow for us to be successful.”

•Solution

Following initial training, our customer was able to jump-start the project using NLP

resources built by Linguamatics. A major advantage for Linguamatics is having

access to 30 million medical transcripts—this supports the NLP platform’s data-

driven methods to explore large data sets for speciﬁc concepts, and design extraction

algorithms based on frequency and linguistic patterns [25].

To identify such a large patient population, the organization went back to 2011

to gather patients with a heart failure diagnoses. Once identiﬁed, clinical records

were extracted going back three years from the date of diagnosis, and forward to the

present day—resulting in approximately 34 million clinical notes. This comprised

eight years of data that was loaded into the Linguamatics platform. The platform

pre-builds large-scale indexes that incorporate clinical ontologies, and numeric and

linguistic patterns, which enables analysts to quickly and iteratively develop and

modify their NLP algorithms. Some issues with the EHR export formatting required

the use of pre-processing. This is a common issue that many groups run into when

pulling document sets using SQL queries. However, due to the complexity of the data

warehouse tables, this is often the only way to extract a focused set of documents

for analysis [26,27]. Learning the NLP platform was a rapid process for the data

scientists, but the delivery of results for the project was still a major undertaking. The

initial algorithms needed to be tuned to the local data set, a typical practice for NLP

to tailor the algorithms for the local writing style. “We looked at all the attributes

across each year to assess how well the algorithms were performing and iterated until

we were happy with the results,” said the Director of Data Science. “By sampling

sets of random records, we could see how the balance of sensitivity and speciﬁcity

changed with our modiﬁcations. We knew the data would be presented to the FDA,

so we wanted to triple-check everything. My team manually reviewed 100 records

for each year to get our ﬁnal accuracy scores. We especially wanted to check for

false positives, so we reviewed echo reports done by cardiologists that had no NLP

results; these were the most likely errors, but fortunately, we did not see any issues

there.”

246 A. Choudhary et al.

•Success

The health system’s staff completed the job on schedule and with outstanding accu-

racy of 95–99%. Most notably, the whole study was performed in three months by

two analysts with no prior training in NLP; a work that would have taken almost

55 FTE years to accomplish manually. “Linguamatics was really invested in helping

us meet our deadlines and deliver the accuracy level we wanted,” said the Director

of Data Science. Our customer subsequently presented the results to the FDA as

RWE for pacemaker devices and outcomes. Reaction to the project has been one of

amazement that such large-scale analysis is possible—let alone completed. Such use

of large data sets and NLP techniques has the ability to transform clinical trials and

post-market surveillance, cutting time to market and demonstrating true insights into

patient outcomes. The team is moving on to its next contract project with another

medical device company; it will continue to use the Linguamatics NLP platform

to support this work, but with more automation to support the processing of the

large document sets [28]. Looking to the future and broader application of NLP, this

health system is exploring how the Linguamatics platform can support patient safety

and care quality. Another key area is reducing the amount of clinical documentation

required by physicians, which continues to cause concern in the healthcare commu-

nity. “We were able to illustrate the life cycle of a heart failure patient, including

risk factors for heart failure, medicines, lab tests, the date of installation of a cardiac

device, and outcomes like ejection fraction ﬁndings”. According to the Director of

Data Science, “the experiment demonstrated that [the NLP platform] has the poten-

tial to relieve physicians’ EHR load”. “As we look to the future of healthcare, [our

organization] is well placed to support its patients and provide the outcomes analysis

that will lead to better understanding of treatment options and improved health,” said

a V.P. at the health system.

5 NLP Induction

According to IBM, natural language processing aims to produce robots that can

translate and respond to text or voice input in the same manner that people do, as

well as respond with text or speech of their own.

Natural language processing (NLP) is a branch of artiﬁcial intelligence in which

computers analyze, interpret, and infer meaning from human speech [29]. For tasks

like automatic summarization, translation, named entity identiﬁcation, connection

extraction, sentiment analysis, and more, developers can utilize NLP to order and

arrange knowledge.

NLP Applications for Big Data Analytics Within Healthcare 247

5.1 Short Demo

The author will use the NLTK Python Library to do Sentiment Analysis on text data

in this example.

Sentiment Analysis is the computer process of recognizing and classifying views

conveyed in a piece of text, with the goal of determining whether the writer’s attitude

toward a speciﬁc topic, product, or other object is positive, negative, or neutral.

Because the author just wanted to conduct sentiment analysis on this dataset, he

removed the superﬂuous columns, leaving only sentiment and text.

To begin, divide the dataset into two sections: training and testing. The test set

represents 10% of the entire dataset. This study is unique because the author’s

248 A. Choudhary et al.

objective was to distinguish just good and negative tweets, the neutral tweets were

removed.

The author then divided the training set’s Positive and Negative tweets to make

it easier to visualize the terms they included. After that, the author removed all

hashtags, mentions, and links from the text. They were now ready for a WordCloud

visualization that only displayed the most emphatic words from both the Positive

and Negative tweets.

NLP Applications for Big Data Analytics Within Healthcare 249

It is worth noting that the positive word set includes the following words and

expressions: truth, strong, legitimate, together, love, and job.

People, according to the author, tend to feel that their ideal candidate is honest,

legitimate, and above good and bad [30].

250 A. Choudhary et al.

Negative tweets, on the other hand, include phrases such as inﬂuence, news,

elevator music, disappointed, softball, makeup, cherry-picking, and trying.

People, according to the author, missed the decisive action and thought the

chastised candidates were too mild and cherry-picked.

The author removed hashtags, mentions, links, and stopwords from the training

set after the visualization.

Stop Words: Stop Words are words that have no signiﬁcant meaning and should

not be used in Search Queries [31]. These words are usually taken out of search

searches because they return a large volume of irrelevant data (the, for, this, and so

on.)

As a follow-up, The author used the nltk lib to extract the so-called features, ﬁrst

measuring a frequent distribution and then picking the resulting keys.

NLP Applications for Big Data Analytics Within Healthcare 251

Hereby the most widely spread words were plotted by the author. The most of the

words are about debate nights.

Using the nltk NaiveBayes Classiﬁer I classiﬁed the extracted tweet word features.

Finally, using some less-than-intelligent measures, the author attempted to assess

how well the classiﬁer system performed.

252 A. Choudhary et al.

5.2 Beneﬁts of NLP in HealthCare

NLP converts unstructured data from a range of sources (including electronic medical

records, music, and social groups) into structured data that can be understood by

analytics tools. Once the language has been converted to structured data, health

systems can use NLP to identify patients, extract insights, and summarize data.

Four areas where healthcare NLP could improve function—and, ultimately, care:

EHR usability, predictive analytics, phenotyping, and quality improvement:

•NLP Improves EHR Data Usability

The standard EHR organizes data by patient contact, making it difﬁcult to locate

crucial patient data (for example, social history, which is a substantial predictor of

readmissions). NLP may be used to create an EHR interface that makes it easier for

physicians to access information about patient encounters.

The interface populates the rest of the page with information about the term

by separating the interface into sections and inserting terms relating to difﬁculties

presented by patients during encounters. For example, all references to weary would

appear on a timeline at the top of the page, with notes regarding the term in a box

at the bottom. The interface can help clinicians locate data and diagnoses that might

otherwise go overlooked.

•NLP Enables Predictive Analytics

One of the more intriguing features of NLP is its ability to enable predictive analytics

to address major public health concerns. According to current statistics, suicide has

NLP Applications for Big Data Analytics Within Healthcare 253

been on the rise in the United States. Healthcare personnel is trying to ﬁgure out who

is in danger so they can intervene. A 2018 study used NLP to predict suicide attempts

by monitoring social media. Twitter users who used less emojis in text, limited the

kind of emojis they used (e.g., blue or broken heart symbols), or increased the amount

of angry or sad tweets before attempting suicide showed clear signals of impending

suicide attempts. Only 10% of the time was the approach incorrectly predicted, with

70% of the time being false positive.

•NLP Boosts Phenotyping Capabilities

A physical or physiological expression of a trait in an organism is known as an observ-

able phenotype. These attributes could be linked to physical characteristics, physi-

ological processes, or behavior. Clinicians can use phenotyping to group or classify

patients in order to receive a more in-depth, focused look at data (for example, listing

patients with similar characteristics) and compare patient cohorts. Most analysts and

practitioners today use structured data for phenotyping since it is easy to extract for

analysis. According to some experts, up to 80% of all patient data is saved on the

cloud. NLP allows analysts to extract and analyze unstructured data (such as follow-

up appointments, vitals, charges, orders, encounters, and symptoms). Phenotypes for

patient groups with signiﬁcantly more information can be created using unstructured

data.

NLP also makes it easier to create phenotypes with more variety. For instance,

pathology reports provide a lot of information on a patient’s condition, growth loca-

tion, cancer stage, procedure(s), drugs, and genetic status. Traditional analytics are

unable to extract data from pathology reports; however, NLP enables analysts to

extract this information in order to answer difﬁcult, particular inquiries. (For instance,

certain malignant tissue types are linked to a number of genetic alterations.)

•NLP Enables Health System Quality Improvement

The federal government and other groups require all hospitals to publish speciﬁc

outcome measures. The adenoma detection rate (ADR), which is the rate at which

professionals discover adenomas during a colonoscopy, is one of the most rele-

vant measurements. Paying someone to study a small sample of patient charts, look

through pathology data, and establish the ADR is the current reporting method. NLP

streamlines and automates this procedure, allowing for a bigger sample size and

real-time analysis of patient charts.

A report card has been constructed by a practitioner that uses natural language

processing to calculate ADR automatically. According to research, when physicians

can see quantitative beneﬁts of their labor, their behavior changes. Physicians who

received feedback on their ADR changed their behaviors in order to boost the detec-

tion rate in this circumstance. This is noteworthy since every 1% increase in ADR

results in a 3% decrease in colon cancer mortality.

254 A. Choudhary et al.

5.3 Application in Fields

Social Media Monitoring

•People are increasingly turning to social media to share their thoughts about a

product, policy, or issue. These can provide valuable insight into a person’s likes

and dislikes. As a result, analyzing this unstructured data can assist in the discovery

of relevant information. Natural Language Processing comes to the rescue once

more.

•Companies are now using a range of NLP approaches to analyze social media posts

and learn more about their consumers’ opinions on their products. Companies also

employ social media monitoring to gain a better understanding of the issues and

problems that their customers have as a result of utilizing their products. It is

used by both the government and industry to identify potential national security

threats.

Chatbots

•Customer service and experience are the most important parts of any organization.

It can help companies improve their products while assuring customer pleasure.

Directly connecting with each client and addressing difﬁculties, on the other hand,

can take time. In this case, chatbots come into play. Chatbots help companies

achieve their goal of offering a great customer experience.

•Many businesses now utilize chatbots to answer simple client inquiries via their

apps and websites. It not only saves businesses time and money, but it also relieves

clients of the burden of waiting for customer support agents.

•It may also enable the company to save money on the cost of hiring call center

personnel. Chatbots started off as a way to answer customers’ questions, but they

have since evolved into personal friends. Chatbots can be used for a variety of

tasks, including product promotion and customer feedback.

Survey Analysis

•It may also enable the company to save money on the cost of hiring call center

personnel. Chatbots started off as a way to answer customers’ questions, but they

have since evolved into personal friends. Chatbots can be used for a variety of

tasks, including product promotion and customer feedback.

•The problem arises when a high number of clients ﬁnish the survey, causing the

data quantity to grow. It is impossible to read them all and draw any conclusions.

Organizations use natural language processing to examine surveys and extract

insights, such as evaluating user opinions about an event based on comments and

analyzing product reviews to ﬁnd beneﬁts and cons. The bulk of organizations

currently adopts these tactics because they produce far more precise and usable

data.

NLP Applications for Big Data Analytics Within Healthcare 255

Voice Assistants

•A voice assistant is software that uses speech recognition, natural language under-

standing, and natural language processing to interpret and respond to a user’s

verbal commands. Despite the fact that voice assistants and chatbots are compa-

rable, I have classiﬁed them separately since they deserve a better rating on our

list. They are far more than a chatbot, and they can accomplish far more than a

chatbot.

•Today’s generation cannot imagine living without voice assistants. Over the years,

they have developed into a dependable and powerful friend. From setting our alarm

clock to ﬁnding us a dinner, a voice assistant can do it all. For both individuals

and corporations, they have opened up a whole new world of possibilities.

Email Filtering

•I am sure you have, and you have probably noticed that mail is separated into three

categories when it arrives: primary, social, and promotional. The ﬁnest feature is

that spam emails are instantly ﬁltered and sent to a different folder. Isn’t it lovely

and practical at the same time? Yes, and that is all there is to email ﬁltering. And

I do not think I need to tell you how critical this feature is in our daily tasks.

•The emails are ﬁltered using text categorization, a natural language processing

approach. It is, as you may have imagined. Text classiﬁcation is the process of

dividing a piece of text into distinct categories. Another great example of text

classiﬁcation is the categorization of news reports into various categories.

6 Conclusion

The author seeks to give a general outline of the most recent state-of-the-art, ques-

tions, and need in the utilization of natural language processing (NLP) in healthcare

research, with a pivot on techniques of assessment. From both a clinical and an NLP

standpoint, we have addressed methodological aspects, and we have identiﬁed three

key issues: data accessibility, assessment workbenches, and reporting standards. We

provide concrete advice for each problem that has been identiﬁed based on these

results. To enhance transparency and reproducibility, We provide a basic structured

approach that can be used to report on the creation and evaluation of clinical NLP

methodologies.

References

1. Ray, C., Tripathy, H. K., & Mishra, S. (2019). A review on facial expression based behav-

ioral analysis using computational technique for autistic disorder patients. In: Singh, M.,

Gupta, P., Tyagi, V., Flusser, J., Ören, T., Kashyap, R. (Eds.) Advances in computing and

data sciences. In: ICACDS 2019. communications in computer and ınformation science, vol

256 A. Choudhary et al.

1046, Springer, Singapore. https://doi.org/10.1007/978-981-13-9942-8_43.Uzuner, Ö., Solti,

I., Cadag, E. (2010). Extracting medication information from clinical text. Journal of the

American Medical Informatics Association, 17(5), 514

2. Mishra, S., Chaudhury, P., Mishra, B. K., & Tripathy, H. K. (2016, March). An implemen-

tation of feature ranking using machine learning techniques for diabetes disease prediction.

In: Proceedings of the Second International Conference on Information and Communication

Technology for Competitive Strategies, pp. 1–3.

3. Velupillai, S., Mowery, D., South, B. R., Kvist, M., & Dalianis, H. (2015). Recent advances in

clinical natural language processing in support of semantic analysis. IMIA Yearbook Medical

Informatics, 10, 183–193.

4. Friedman, C., Rindﬂesch, T. C., Corn, M. (2013). Natural language processing: state of the

art and prospects for signiﬁcant progress, a workshop sponsored by the National Library of

Medicine. Journal of Biomedical Informatics, 46(5), 765–773

5. Uzuner, Ö., Luo, Y., Szolovits, P. (2007). Evaluating the state-of-the-art in automatic

de-identiﬁcation. Journal of the American Medical Informatics Association, 14(5), 550.

6. Editors, Health System Applies Linguamatics NLP Platform to Improve Care And Population

Insights, Linguamatics, IQVIA, April 4, 2018.

7. Editors, Huntsman Cancer Institute optimizes research with Linguamatics NLP platform,

Linguamatics, IQVIA, August 28 2019.

8. Névéol A, Zweigenbaum P (2015) Clinical natural language processing in 2014: foundational

methods supporting efﬁcient healthcare. Yearbook of Medical Informatics, 10(1), 194–198.

9. Velupillai, S., Mowery, D., South, B. R., Kvist, M., & Dalianis, H. (2015). Recent Advances in

clinical natural language processing in support of semantic analysis. IMIA Yearbook of Medical

Informatics, 10, 183–193.

10. Chapman, W. W., Nadkarni, P. M., Hirschman, L., D’Avolio, L. W., Savova, G. K., Uzuner, O.

(2011). Overcoming barriers to NLP for clinical text: the role of shared tasks and the need for

additional creative solutions. Journal of 530 the American Medical Informatics Association,

18(5), 540–543.

11. Mukherjee, D., Tripathy, H. K., & Mishra, S. (2021). Scope of medical bots in clinical domain.

Technical Advancements of Machine Learning in Healthcare, 936, 339.

12. Mallick, P. K., Mishra, S., Mohanty, B. P., & Satapathy, S. K. (2021). A deep neural

network model for effective diagnosis of melanoma disorder. In Cognitive informatics and

soft computing (pp. 43–51). Springer, Singapore.

13. . Suominen, H., Salanterä, S., Velupillai, S., Chapman, W., Savova, G., Elhadad, N., Pradhan, S.,

South, B., Mowery, D., Jones, G., Leveling, J., Kelly, L., Goeuriot, L., Martinez, D., Zuccon, G.

(2013). Overview of the ShARe/CLEF eHealth evaluation lab 2013. Lecture Notes in Computer

Science (including subseries Lecture Notes in Artiﬁcial Intelligence and Lecture Notes in

Bioinformatics) 8138 LNCS (pp 212–231)

14. Elhadad, N., Pradhan, S., Gorman, S., Manandhar, S., Chapman, W., Savova, G. (2015).

SemEval-2015 task 14: Analysis of clinical text Proceedings of the 9th International Work-

shop on Semantic Evaluation (SemEval 2015) (pp. 303–310). Association for Computational

Linguistics, Denver, Colorado.

15. Wu, H., Toti, G., Morley, K. I., Ibrahim, Z. M., Folarin, A., Jackson, R., Kartoglu, I., Agrawal,

A., Stringer, C., Gale, D., Gorrell, G., Roberts, A., Broadbent, M., Stewart, R., Dobson, R.

J. (2018). SemEHR: a general-purpose semantic search system to surface semantic data from

clinical notes for tailored care, trial recruitment, and clinical research. Journal of the American

Medical Informatics Association, 25(5), 530–537.

16. Mishra, S., Thakkar, H., Mallick, P. K., Tiwari, P., & Alamri, A. (2021). A sustainable IoHT

based computationally ıntelligent healthcare monitoring system for lung cancer risk detection.

Sustainable Cities and Society, 103079.

17. Howes, C., Purver, M., McCabe, R. (2014). Linguistic ındicators of severity and progress in

online text-based therapy for depression. In Proceedings of the Workshop on Computational

Linguistics and Clinical Psychology: From Linguistic Signal to Clinical Reality. Association

for Computational Linguistics, Baltimore, Maryland, USA (pp. 7–16).

NLP Applications for Big Data Analytics Within Healthcare 257

18. Pestian, J. P., Matykiewicz, P., Linn-Gust, M., South, B., Uzuner, O., Wiebe, J., Cohen, K.

B., Hurdle, J., Brew, C. (2012). Sentiment analysis of suicide notes: a shared task Biomedical

Informatics. Insights, 5, 3–16.

19. Collins, G., Reitsma, J., Altman, D., Moons, K. (2015). Transparent reporting of a multivariable

prediction model for individual prognosis or diagnosis (tripod): the tripod statement. Annals

of Internal Medicine, 162(1), 55–63.

20. Rath, M., & Mishra, S. (2019). Advanced-level security in network and real-time applications

using machine learning approaches. In Machine learning and cognitive science applications

in cyber security (pp. 84–104). IGI Global.

21. Carter, G., Milner, A., McGill, K., Pirkis, J., Kapur, N., Spittal, M. J. (2017). Predicting

suicidal behaviours using clinical instruments: systematic review and meta-analysis of positive

predictive values for risk scales. British Journal of Psychiatry, 210(6), 387–395.

22. Mishra, S., Tripathy, H. K., & Panda, A. R. (2018). An improved and adaptive attribute selec-

tion technique to optimize dengue fever prediction. International Journal of Engineering and

Technology 7(3.34), 480–486.

23. Bell, J., Kilic, C., Prabakaran, R., Wang, Y. Y., Wilson, R., Broadbent, M., Kumar, A., Curtis,

V. (2013). Use of electronic health records in identifying drug and alcohol misuse among

psychiatric in-patients. The Psychiatrist, 37(1), 15–20.

24. Ford, E., Carroll, J. A., Smith, H. E., Scott, D., Cassell, J. A. (2016). Extracting information

from the text of electronic medical records to improve case detection: a systematic review.

Journal of the American Medical Informatics Association, 23(5), 1007–1015.

25. Jena, L., Mishra, S., Nayak, S., Ranjan, P., & Mishra, M. K. (2021). Variable optimiza-

tion in cervical cancer data using particle swarm optimization. In Advances in electronics,

communication and computing (pp. 147–153). Springer, Singapore.

26. Roy, S. N., Mishra, S., & Yusof, S. M. (2021). Emergence of drug discovery in machine

learning. Technical Advancements of Machine Learning in Healthcare, 936, 119.

27. Mishra, S., Mishra, B. K., & Tripathy,H. K. (2015, December). A neuro-genetic model to predict

hepatitis disease risk. In 2015 IEEE International Conference on Computational Intelligence

and Computing Research (ICCIC) (pp. 1–3). IEEE.

28. Sahoo, S., Das, M., Mishra, S., & Suman, S. (2021). A hybrid DTNB model for heart disorders

prediction. In Advances in electronics, communication and computing (pp. 155–163). Springer,

Singapore.

29. Mishra, S., Tadesse, Y., Dash, A., Jena, L., & Ranjan, P. (2021). Thyroid disorder analysis

using random forest classiﬁer. In Intelligent and cloud computing (pp. 385–390). Springer,

Singapore.

30. Tripathy, H. K., Mishra, S., Thakkar, H. K., & Rai, D. (2021). CARE: a collision-aware

mobile robot navigation in grid environment using ımproved breadth ﬁrst search. Computers &

Electrical Engineering, 94, 107327.

31. Sahoo, S., Mishra, S., Mishra, B. K. K., & Mishra, M. (2018). Analysis and implementation

of artiﬁcial bee colony optimization in constrained optimization problems. In Handbook of

research on modeling, analysis, and application of nature-ınspired metaheuristic algorithms

(pp. 413–432). IGI Global.

Cognitive Computing Driven Healthcare:

A Precise Study

Rohan Sharma and Uday Bhanu Ghosh

Abstract As medical and computer technology has developed rapidly, interest and

investment in the medical ﬁeld has grown tremendously. However, the majority of

healthcare systems do not take into consideration patients’ emergencies or could not

offer individualized resource assistance. The following discussion explores a smart-

healthcare system using Cognitive Computing approach in order to solve this issue.

Based on this cognitive computing view, (Electronic Medical Record) EMR systems

can be more proactively used not just for bookkeeping but as data dictionary which

can be used for analyzing and providing cures to not only patients with critical medical

conditions but can also be used as a precautionary analyst which can track patients

medical history and provide early diagnostic results. Cognitive computing is also an

active entity which if combined with EMR can provide healthcare providers with

access to vast amounts of information related to medical sciences, drug information,

and medical ontologies. This discussion focuses on understanding various different

methods in which cognitive approaches can be put together in healthcare. The later

section of the discussion throws light on how different modern technologies can be

merged with cognitive approaches to enhance the healthcare sector. An exploration

of past and current research advances in the ﬁeld of creating cognitive systems in

medical practice is presented. The comparison analysis section gives an overview

of different types of cognitive approaches in a generalized manner whereas the last

section (i.e.) results and analysis examines some experimental samples to show the

extent of cognitive agents.

Keywords Cognitive computing ·AI ·Computer-aided decision-making ·

Healthcare ·Information technology

R. Sharma (B)·U. B. Ghosh

HighRadius Corporation, Bhubaneswar, India

e-mail: sharma.rohan1028@gmail.com

S. Mishra et al. (eds.), Augmented Intelligence in Healthcare: A Pragmatic and Integrated

Analysis, Studies in Computational Intelligence 1024,

https://doi.org/10.1007/978-981-19- 1076-0_14

259

260 R. Sharma and U. B. Ghosh

1 Introduction

A cognitive computing system monitors and analyzes an individual’s physical health.

The computing network allocates computing resources based on the user’s health

risk level. As a result, cognitive computing based healthcare systems provide a

more user-friendly experience, optimize system resources effectively, and increase

patient survival rates in an emergency. In the healthcare ecosystem, there are vast

amounts of data, both structured and unstructured. Data can be used to gain valu-

able information through cognitive computing, which is a new computing paradigm.

Different domains, such as healthcare, have adapted cognitive computing systems.

The following discussion examines the necessity of cognitive computing systems,

utilizing the technology acceptance model as a guide. Information that is crucial

to healthcare may be hidden in Electronic Medical Record systems (EMR systems).

Healthcare professionals must make sense of this data in order to deliver the best care

to the patient. A Cognitive Agent, EMR systems can be transformed from merely

archiving data into intelligent systems that improve patient care. As an example,

when preparing for a consultation, the doctor should quickly review the patient’s

medical history from the EMR to put the patient’s concerns into perspective. As

part of the visit, a physician should verify, review, and further examine the patient’s

record with the EMR. A cognitive system that uses advanced analytics on patient

records can meet the patient’s information needs. In addition to providing problem-

speciﬁc summaries of patient records, these solutions can accurately answer natural

language questions about the content of patient records, identify urgent anomalies,

and determine accurate causes for such abnormalities.

The development of human society and environmental changes have resulted in

chronic diseases playing an increasingly greater role in the health of humans [1].

Traditionally, health systems have been classiﬁed into three categories: acquisition,

transmission, and analysis of data. The Body Area Network (BAN) [2] in its collection

layer collects sensing data based on the preset frequency. Afterward, the gateway

transmits it to the base station (BS) through the smartphone or the intelligent device

[3,4]. Through the Internet, the gateway or BS transmits it to the analysis layer (such

as a cloud data center). By utilizing machine learning and data mining algorithms,

cloud data centers store and analyze data. As a result, the system will take appropriate

medical treatment measures based on the users’ health status [5]. Patients have been

beneﬁted by the healthcare system, but there are a few problems:

•Medical data has multiple modalities, which make it difﬁcult for traditional data

mining and machine learning approaches to uncover it effectively [6]. It is there-

fore necessary to use an intelligent method for a complete analysis of disease for

all types of data.

•Health data collected by BAN sensors can be processed in the cloud, resulting in

a delay in the diagnosis and treatment in emergencies [7,8].

Cognitive Computing Driven Healthcare: A Precise Study 261

It wastes resources when network resources are deployed inﬂexibly. Further-

more, it does not offer customized resources for users based on their different health

statuses.

Many relevant studies have been conducted to overcome these problems. As an

example, in reference [9], authors propose a novel architectural approach, Body-

Cloud, which enables users to monitor and analyze cardiac data streams performed by

many different individuals, such as those engaged in medicine, emergency response,

exercise, and behavioral monitoring. Also, Ref. [10] presented cloud-based BAN

solutions for a number of human-centered applications such as medicine, sports, and

ﬁtness. The cognitive computing capabilities of medical professionals can also be

exploited [11]. It can diagnose a patient with the help of the medical and cognitive

systems, arrive at an optimal decision from the information available, and follow

through [12]. Among medical professionals, there is a risk of missing a vital piece

of data relation and pattern is very high. Patients may suffer long-term health conse-

quences or even die if crucial information is ignored or misunderstood. A multidis-

ciplinary subject such as cognitive computing, which integrates machine learning,

AI, and NLP [13], is able to determine disease patterns and relationships based on

the data [14] and analyze all the information, and thus help the medical community

determine suitable treatments [15,16].

Medicinal institutions gain more value from their data when humans and machines

cooperate in a cognitive system [17]. Moreover, recent advances in edge computing

are having an enormous impact on healthcare [18]. By deploying computing

resources in close proximity to end users, edge computing can offer better quality

of service (QoS) for tasks requiring large amounts of computation and low latency

[19]. There is a division between cognitive computing and edge computing, however

[20].

In recent years, data generation from various sources has increased dramatically

(Fig. 1shows medical data generation and computational workﬂows associated with

it). We now refer to this phenomenon as big data. Modern analytics technologies

Fig. 1 Health care impacts of cognitive computing systems

262 R. Sharma and U. B. Ghosh

enable a creative analysis of these data, resulting in vast opportunities for business.

Public records and social media data can be combined with data from these sources.

It is possible to create algorithms that can be used to predict, for instance, how

someone will respond to marketing messages. Additionally, personal information is

prized in the healthcare sector. The insurance plans and treatments they offer, for

example, can be tailored to the patient’s needs. It is unfortunate that most of the data

related to healthcare is unstructured, not following a set of standardized, predeﬁned

formats. Think of traditional sources of healthcare data, such as patient charts, records

written by hand, medical imaging, sensor information, audio recordings, etc. When

performing healthcare data analytics [21], ﬁnding valuable insights is extremely

challenging due to the large quantity of heterogeneous data coming from multiple

sources.

Using cognitive computing to tackle similar problems in data analysis, a new

paradigm is promising. Learning from structured and unstructured data, ﬁnding

important correlations, generating hypotheses for those correlations, and suggesting

actions to improve outcomes are some of the characteristics of cognitive computing

[22]. In cognitive computing, artiﬁcial intelligence is used to interpret, organize, and

explain patterns in context using sophisticated machine-learning algorithms. There-

fore, cognitive computing systems do not replace a subject matter expert; rather, they

act as decision support systems that work with humans to gather data on a particular

subject and, eventually, provide insight.

Applications of cognitive computing systems in healthcare are among the most

promising ones. EMRs (electronic medical records) are a good example. Several

patients’ EMRs along with their individual case ﬁles together can be loaded into a

cognitive computing system to uncover associations between symptoms and disor-

ders/diseases. Individual patients’ EMRs are not always reliable in capturing accurate

information on their recollections. Unless a researcher has access to comprehensive

data, such as that from their own ofﬁce/institution, it would likely be impossible

to uncover these relationships. Further, the computational system can analyze the

symptom-disease relationship and diagnose the ones likely to be further studied

by the scholar, thus enhancing the researcher’s productivity and working in close

collaboration (Fig. 2).

Information technology has been recognized for its potential to improve clinical

care and impact a health care delivery system for a long time [23]. Health Informa-

tion Technology for Economic and Clinical Health Act of 2009 and Meaningful Use

incentives beginning in 2013 have encouraged health information technology inte-

gration. Technology has brought some beneﬁts with the introduction of Electronic

Medical Record systems (EMR systems), physicians are experiencing productivity

decreases and workﬂow disruptions due to EMR systems [24]. Among the priori-

ties for improving electronic medical records usability, according to an American

Medical Association study, is reducing cognitive load [25]. A custom made cognitive

agent, such as one developed by Custom made Analytics can help physicians to use

EMRs more efﬁciently and effectively (Fig. 3).

Computing in the age of cognitive computing is a new process in which computers

interact naturally with users, learn continuously, and augment human cognition [26].

Cognitive Computing Driven Healthcare: A Precise Study 263

Fig. 2 Summary of patient records using Cognitive Agent creating a list of problems and displaying

the clinical data generated a summary of the relationship between each of them, as well as the

aggregate

Fig. 3 To gain knowledge, analyze data and use cognitive computing

Cognitive computing is a branch of computer science that draws on a variety of

computer technologies, including NLP, retrieval technologies, information repre-

sentation, artiﬁcial intelligence, and data-driven analytics. There is a convergence

of hardware development, software engineering, linguistic research, and many

decades of research on machine learning and natural language processing. Cogni-

tive computers are able to analyze, predict, reason, and interact naturally with

humans. Rather than eliminating humans from decision-making processes, cognitive

computing systems help augment human cognition and intelligence.

A physician’s routine when it comes to seeing a patient is mostly the same. The

concept of workﬂow was developed by Frederick Taylor and Henry Gantt in the late

nineteenth century [27]. Based on this work, time-motion studies were developed as

a systematic approach to optimizing service and manufacturing processes. Despite

distinct differences in speciﬁcs between ﬁelds and individuals within the same ﬁeld

of medicine, a few high-level procedures are routinely performed in regular patient

care. We can identify and understand physicians’ cognitive needs by studying their

workﬂow steps. These solutions affect the overall efﬁciency of physicians and the

patient care they provide. We can ﬁnd a practical solution by examining a physi-

cian’s workﬂow in an outpatient setting, since patient care happens in many different

264 R. Sharma and U. B. Ghosh

contexts. According to Shartzer, a physician’s clinical duties are divided into ﬁve

categories [28] into four distinct steps:

1. Visit Preparation

2. Patient history and physical examination

3. Assessment and Planning

4. Visit Wrap-up

An outpatient clinical setting generally follows these four steps. Identifying physi-

cians’ information needs is based on these tasks and the transitory steps. Using inter-

views with clinicians and nurses from two major hospitals, an ongoing study [29]

identiﬁes the information needed at various stages.

2 Background Study

Studies have been conducted on cognitive computing [30] using cloud computing,

big data, supervised learning [31], and some have applied to medical applications

[32,33]. One of the features of this study was the concept of a novel method of

cognitive system design. The study concentrated on the creation of a new model of

human knowledge, taking into account the use of a new data structure for organizing

and operating relative information. Researchers have developed an open-question–

answer system based on huge amounts of text data and NLP [31,34]. By applying an

understanding of human brain functioning, this study investigated how to complete

a broken picture using existing knowledge in a cognitive system. The development

of cognitive computing was surveyed in general terms. By using extensive data anal-

ysis techniques, the authors developed cognitive applications [35]. Security prob-

lems should be considered when designing cognitive systems in order to protect

conﬁdential user data.

It is possible to provide advanced healthcare services by leveraging cognitive

computing techniques in three different domains, namely physiological healthcare,

psychological healthcare, and medical analysis. Cloud-based technologies and exten-

sive analysis techniques, the discussion [36] proposed a patient-centric system.

According to Zhang et al. [37], the study provided methods for analyzing BP and

heart rate. In [38], researchers propose a mobile cloud computing service platform

utilizing effective computing techniques and cloud computing technology for service

provisioning that includes personalized emotional awareness. The authors discussed

the inference mechanism of a facial expression-based cognitive system and proposed

an emotional and cognitive system. Based on voice and body language, the author

designed a system to evaluate emotions and cognitive functions. To achieve real-time

emotional interaction between remote users [39], the study incorporated a pillow

robot, a 5G network, and a cloud platform. To provide both doctors and patients with

adequate medical service, cognitive computing technologies are used to recommend

drugs.

Cognitive Computing Driven Healthcare: A Precise Study 265

Cognitive computing is, from a practical standpoint, the revamping of well-funded

theories that had little practical application when they were ﬁrst devised due to the

limited computing capacity. This is the case, for example, of Artiﬁcial Intelligence

(AI) and neural networks, which are complex in nature and by the need to execute vast

amounts of data simultaneously in short intervals. In traditional artiﬁcial intelligence

techniques, expert systems are used and statistical models are utilized, which involves

large-scale processing of large-scale data sets for the training of expert systems. This

means that cognitive computing is, in this sense, AI’s revenge, since nowadays we

have computing architectures that are able to handle large data sets with signiﬁcant

dimensionality. Through continuous number crunching, new insights can be derived,

resulting in anticipating solutions to heterogeneous problems. Let’s take the search

engines on the Internet as an example. Using this type of AI, they aim to provide

users with relevant information based both on their data and on how patterns are

applied. In this regard, contextualizing a language, classifying entities, clustering, and

extracting entities are all important. Consider the famous e-commerce sites and their

recommendation systems as another example. A recommendation system engineered

to offer shopping advice to people who are seeking an item based on items they are

browsing, have marked as favorites or added to their wish list, as well as based on

their preferences [40].

In conclusion, cognitive computing systems should be seen as AI-driven tech-

nologies. By mimicking human reasoning methods, they show special abilities to

handle uncertainties and solve problems that require a lot of computation. Addi-

tionally, they demonstrate the capability to learn. Therefore, their knowledge base

continues to grow, and, consequently, their reasoning abilities continue to grow. The

cognitive computing revolution is also driving the development of new forms of

interaction between humans and computers, using Natural User Interface paradigms

(NUI), e.g., conversational systems [41]. In addition, by learning from experts’ way

of approaching problems and their problem-solving techniques, machines can also

be trained to teach beginners as well as humans, as simulated reasoning processes can

be accurate and reliable. Such intelligent systems can be used to improve products

and processes, for example, by training and customizing people [42].

The growth of the Internet of Things (IoT) is predicted to depend on cognitive

computing [43] and, consequently, on the underlying interconnection technologies

[44]. People, things, and their actions are forecast to interact naturally in the near

future [45], producing and consuming data while speaking natural languages. In

order to develop novel, somehow intelligent algorithms that can respond in real time

to unpredictable external factors with unknown origins, we need advanced analytics

to collect and analyze data, and vice versa. Accordingly, we observe that cognitive

computing’s greatest beneﬁts don’t originate with the cognitive systems themselves,

but rather with their coupling with their environment. As a result, engineering may

enter a new age of design driven more by behavior than by design constraints: making

machines rather than making machines [46] (Fig. 4).

As a result of cognitive computing’s potential resources and capabilities, this

technology may lead to the development of automated systems devoted to improving

the standard of living, dealing with major social issues [47], and aiding people in

266 R. Sharma and U. B. Ghosh

Fig. 4 Healthcare

implications of cognitive

computing as a triangle

Fig. 5 The concept of cognitive computing will be better utilized

overcoming minor complications as well as awkward situations. The main basis of

such a system will be to mimic human behavior and reasoning. Furthermore, cognitive

computing systems can be used to accomplish a variety of tasks such as text mining,

NLP, and sentiment analysis, as well as classiﬁcation, language processing, and image

recognition. Additionally, humans are becoming more conﬁdent that machines can

reliably provide answers, i.e., within a reasonable bound of conﬁdence, in sensitive

areas, as, for example, healthcare, academics, and ﬁnance [48] (Fig. 5).

Cognitive computing can contribute to evolving healthcare in a variety of ways,

as researchers. As of now, it is closest to being able to manage large quantities of

information. For extensive data management and decision support, cognitive systems

can quickly surpass existing solutions. Medicine has long been affected by this class

of problems, especially when it comes to analyzing statistics and assessing visual

patterns. Cancer diagnosis has achieved the most impressive and eye-catching results.

In this ﬁeld, Google, DeepMind, and Watson have all announced impressive results.

A cognitive system’s ability to recognize and classify images is another distinctive

feature [49], and this is an asset in the prevention of cancer pathologies, especially

when it comes to breast cancer, lung cancer, and prostate cancer [50]. A cognitive

system that sees is invaluable, both for relieving the physician of the task of analyzing

a large number of records about identical pathologies in a short period of time and for

providing semantic interpretation of diagnostic images [51]. However, it would be

remiss to fail to mention that there are multiple barriers and resistances in the medical

ﬁeld regarding the application of the cognitive system, mainly caused by the lack of

Cognitive Computing Driven Healthcare: A Precise Study 267

basic computer skills. Yet, cognitive systems can lead to a 50% improvement in results

where they are used for healing, as well as a 50% reduction in hospitalization costs

[52]. The number of diagnostic errors is also reduced, particularly in carcinogenic

diseases. In the United States, those who die due to diagnostic errors are the ﬁrst

cause of death.

3 Discussed Methodology

The term cognitive computing does not refer to one discipline in Computer Science—

it refers to the combination of several disciplines, like NLP and AI, and a combina-

tion of using industry knowledge to create solutions where humans and computers

continuously work together. Cognitive systems have, however, ﬁve core capabilities:

1. Their aim is to create a deeper human connection by attempting to understand

the attributes of the users interacting with them, resulting in a better value in

the long run.

2. Scale and expertise because they can automatically ingest, in context, vast

amounts of documents and help analysts and specialists explore large quantities

of information.

3. Through cognitive technologies, products and services improve themselves by

sensing and adjusting on their own, using their own consciousness.

4. Using them, businesses are able to pay closer attention to workﬂows, contexts,

and environments.

5. As a result, they enable increasingly complex patterns and opportunities to be

discovered, promoting exploration and discovery (Fig. 6).

Using the term taught rather than programmed, the study [53] describes how

the supercomputer has gone from pure mathematical simulation to practical appli-

cations in healthcare that emphasize the need to understand the complexities of it.

Cognitive computing is brieﬂy explained in history, followed by a list of practical

Fig. 6 Person-identifying information proﬁles of patients

268 R. Sharma and U. B. Ghosh

applications, including researching cancer, enhancing supply chains, and empow-

ering consumers. A number of important case studies are presented, including the

Memorial Sloan-Kettering Cancer Center (MSKCC) and the MD Anderson Cancer

Center (University of Texas at Dallas), where they realized that the vast amounts of

patient information stored in multiple heterogeneous systems cannot be merged to

produce actionable results. Oncology is another application of cognitive agents as

described in the paper [54], which also highlights the remarkable reasoning abili-

ties of cognitive computing systems using machine learning to determine the most

appropriate treatment options and to provide physicians with decision support. A

cognitive agent learns by observing more experienced agents, as a physician learns

by watching physicians with more experience.

4 Comparison Analysis

Programmers can develop a practical, efﬁcient, and reliable system tailored to their

particular needs by using speciﬁc platforms and tools designed to facilitate cognitive

computing adoption. There has been an explosion of cognitive computing kits in

recent years, resulting in an increase in both market share and all-inclusive software

available in a cost-effective manner. The following, in alphabetical order, are the

most important examples:

(i) Enterprise Cognitive Systems by Enterra. The company’s cognitive systems

framework, formerly known as Cognitive Reasoning Platform (CRP), is

deﬁned as a platform that leverages artiﬁcial intelligence in order to imple-

ment advanced analytics and ﬁndings that will provide organizations with

actionable business, customer, and value chain insights. The ﬁrm claims it

can also integrate the efﬁcacy and precision of computation with the analyt-

ical and predictive abilities of humans. It is capable of receiving enormous

amounts of unstructured and structured data, processing it, identifying rela-

tionships and relationships within the data, making a decision, and taking

action according to the outcome.

(ii) Deep Learning Technology Center. To use Microsoft’s Cognitive Toolkit,

which is made available in open source by GitHub for anyone to use, Microsoft

made it available as Open Source under the name Computational Networks

Toolkit (CNTK). A commercial toolkit for training deep learning algorithms

to use human-like thinking processes, it is described by its developers. Deep

learning networks are created with this tool so that you can use massive

datasets to leverage the intelligence of deep learning, the speed of this tool,

and the power of the ETL process, creating accurate results, and integrating

with coding languages and algorithms that you already use.

(iii) DeepMind. In 2014, Google acquired its namesake artiﬁcial intelligence

company in the United Kingdom. They claim to be able to solve intelligence

problems. Make the world a better place by using it. A different AI product

Cognitive Computing Driven Healthcare: A Precise Study 269

was developed by Google in 2015 to demonstrate the effectiveness of their

work. Using just raw pixels as input, they were able to teach themselves how

to play and win 49 completely different Atari games. For the ﬁrst time ever,

the AlphaGo program beat the world’s best Go player, which is an extremely

complex and intuitive game that has more positions than there are atoms.

(iv) IDOL (Intelligent Data Operating Layer). Hewlett-Packard offers this soft-

ware platform that uses AI for text analysis, speech analysis, image analysis,

and video analysis. With IDOL, HP, which acquired Autonomy in 2011, offers

a variety of products for, for example, data analysis and IoT, within its big data

software platform. With IDOL Natural Language Question Answers, organi-

zations can harness big data’s full potential by bridging the human–machine

distinction. Natural language-based human-centric exchanges are enabled by

using this system, allowing contextually relevant information to be delivered

by maximizing the power of machine learning.

(v) Watson, created by IBM. The technology has the potential to transcend AI.

This platform combines NLP with machine learning to share information

from massive datasets. In short, Watson analyzes and interprets data at scale,

takes natural human interactions into account, and learns and thinks naturally.

Your data might include non-structured text, images, audio, and video, which

you can analyze and interpret. Learn how machine learning is being used

in apps and systems to grow their subject matter expertise. When users’

personalities, tone, and emotions are understood, you can offer personalized

recommendations. Chatbots are capable of interacting with users.

(vi) In their publication Big Data and Analytics, they demonstrate that the digital

transformation that has taken place within healthcare and the handling of

patient information, as well as rapid adoption of electronic health records

within the healthcare industry, have resulted in the generation of a signiﬁcant

volume of data, difﬁcult to process and manage, and that must be managed

carefully, both for improving the efﬁcacy and efﬁciency of the healthcare

system, as well as for reducing costs [55]. The purpose of health analytics is to

provide fact-based decision making for planning, management, measurement,

and learning by analyzing health data and related business insights (e.g.,

statistical, contextual, quantitative, predictive, cognitive, other models) [56].

Therefore, health analytics utilize data analysis techniques from modeling,

data mining, and machine learning to make predictions based on current

and historical facts. Then, the author in their paper extends the scope of

cognitive computing to all areas of life science [57], analyzing challenges that

researchers in the life sciences face, and suggesting how cognitive innovations

can help to ﬁnd new solutions to large datasets so that they may be better

analyzed for latent information.

(vii) A survey of two peer-reviewed scientiﬁc indexing websites, PubMed and

IEEEXplore. In PubMed and IEEEXplore, the search terms were Blockchain

and Blockchain Healthcare. A total of ten publications related to the subject

270 R. Sharma and U. B. Ghosh

Fig. 7 Here is a graphic summary of the survey results based on text analytics

were identiﬁed (two papers presented at conferences and eight in jour-

nals). Figure 7shows the word analytics for these search results. Essen-

tially, blockchain technology is a network of decentralized nodes whose

purpose has been to enhance privacy, security, trustworthiness, and trans-

parency. A few avenues in which Blockchain technology can be applied in

the healthcare industry are discussed in [58]. Using Blockchain technology

and Pervasive Social Networks (PSN)-based networks, the modularity feature

of Blockchain-based infrastructure is combined with three applications to

enhance the wireless transfer and organization of medical records. Provides

insight into clinical trial data. In the clinical trials described by [59], more

than half fail to report data and results corresponding to the methods. Smart

contracts on the Blockchain, with their attached trust, are a potential game

where as a result of Blockchain, the quality of medical services may be signif-

icantly improved. The author of [60] looked at Blockchain across a broad

range of applications and theory for the time period of 2013–15, and reported

that more than 80% of the effort had been allocated to Bitcoin. A modiﬁed

Blockchain protocol with time-stamping has been empirically demonstrated

[61] to be a low-cost and secure alternative to existing clinical study audit

processes.

Cognitive Computing Driven Healthcare: A Precise Study 271

5 Result Discussion

5.1 Performance Analysis

When evaluating health care systems based on edge-environmental systems, it

is important to consider their resource-utilization rates. Optimizing computing

resources to an appropriate degree is necessary for quantifying the user experience.

Following is a qualitative method of assessing health system performance based on

QoS dimensions.

We can quantify user experience if we think about the edge node count as n and

the user count as m. In the next step, it is possible to classify users risk levels into

four categories: low, medium, high, and danger, with their respective quantiﬁcations

as 0, 1, 2, and 3. As a result, the health risk factor for an ith user is calculated by

multiplying s(i) by {0, 1, 2, 3}. In the case of the ith user, the resource occupancy rate

is equal to 1/connC (σ(i)), where σ(i)∈[1, n] indicates which edge node is currently

connected to user I, and connC(j), j∈[1, n] indicates how many users are currently

connected to the edge node j. Therefore, consumers can expect the following levels

of quality from the Overall Quality of Service:

Overall QoS =s(i)

connC(σ(i)) (1)

According to (1), resource occupancy rates of users who have a high risk of

vulnerability are related to their quality of service. Consequently, it is a method of

quantifying user experience that correctly represents the degree to which computing

resources can be optimized.

We are evaluating the quality of the service based on information found in Table

1and the health of the user. Experiments were conducted in two groups. Users

connecting to the closest edge node do not use resource cognition in the ﬁrst group.

In the second group, the resources are redistributed to the user in the dangerous

state using resource cognition. In Table 1a, we plot the experimental results for

two different scenarios, namely with and without the cognitive resource. Resources

cognizantly redistribute the edge resources when the system detects that the user is in

a dangerous state, resulting in enhanced QoS overall. This increases the utilization of

resources by the current danger user by transferring other less-risky users to the edge

nodes in which the current danger user is connected. Formula (1) states that the more

time a dangerous user occupies a resource, the greater the overall quality of service.

The overall quality of service has been improved with resource cognition. Therefore,

the allocation of resources in accordance with a user’s health status can improve

quality of service. The graph in Table 1b shows the number of potentially dangerous

users over time. By using Table 1a together with a zero number of dangerous users, it

can be seen that resources will not be redistributed. As a result, the overall quality of

service remains unchanged. The distribution of resources changes, however, when

the dangerous user number increases. The user with a higher disease level will get

272 R. Sharma and U. B. Ghosh

Tabl e 1 Results from a study

involving a healthcare system

based on cognitive computing

(a) Quality of service overall, whether cognitive resources are

available or not

Overall QoS

Experiment time (minutes) Resource cognitive No cognitive

01.58 1.37

10 0.87 0.84

20 0.77 0.63

30 1.63 1.34

40 1.53 0.97

50 0.85 0.82

60 0.67 0.64

70 0.78 0.78

80 0.76 0.75

90 1.23 1.19

100 1.41 1.37

(b) Count of users in danger

Experiment time (min) Count of users in danger

0 2

10 0

15 1

20 1

25 1

30 3

35 2

40 1

45 1

50 0

55 2

60 1

65 1

70 0

75 2

80 0

85 3

90 2

95 2

100 2

Cognitive Computing Driven Healthcare: A Precise Study 273

Tabl e 2 Approximately 140

potential problems for each

patient are identiﬁed based on

the data, with an occurrence

pattern that is close to normal

distribution

Number of candidate problems Number of EMRs

20 0

40 0

60 3

80 5

100 20

120 39

140 52

160 29

180 30

200 17

220 1

240 2

260 0

more resources. This results in signiﬁcantly improved QoS overall. Accordingly,

the proposed system recognizes and utilizes user data, performs resource allocation

according to the user’s preferences, enhances the user experience, and increases the

patient’s survival.

5.2 Problems with the Candidate

In Table 2, we present a breakdown of the number of candidate problems produced

for each EMR (each EMR included in our test set and training set). Averaging 130

candidate problems, we see a nearly normal distribution with a standard deviation of

32. In the machine learning model, the predicted ﬁnal problems are reduced by 93%

to an average of nine.

5.3 Problems Most Commonly Encountered

Table 3identiﬁes the 15 most common problems, together with their frequency (Table

3). Mentions this most common problem and the accuracy of cognitive agent’s predic-

tion. The cognitive agent predicts frequently occurring problems quite accurately,

therefore. This model, however, is not recommended for lower back pain. Our model

usually has a hard time dealing with issues like this. When this is acute or severe,

physicians will prescribe medications for this. Chronic pain, on the other hand, may

require the patient to take OTC medications not listed on the label. Rather than

speciﬁc reasons, medical experts used a more wide-ranging approach, taking into

274 R. Sharma and U. B. Ghosh

Tabl e 3 Following is a table

showing the top 15 frequently

occurring problems identiﬁed

by the cognitive agent

problem list

Occurrence of problem

detection (in percentage)

Types of diseases Gold standard Cognitive agent

Hypertensive disorder 8.0 6.2

Obesity 7.0 4.9

Hyperlipidemia 5.0 4.0

Gastroesophageal reﬂux 4.2 3.7

Depressive disorder 4.0 3.3

Asthma 3.8 2.5

Sleep apnea 3.6 3.5

Anxiety 3.0 2.8

Diabetes type 2 3.0 2.9

Migraine 2.6 3.3

Vitamin D deﬁciency 2.0 1.5

Degeneration of intervertebral 1.9 2.1

Hypothyroidism 1.7 1.4

Fatty liver 1.7 1.6

Lower back pain 1.6 2.1

account the severity of the problem and the absence of any other problem that could

explain it. Most problems are answered with excellent accuracy.

5.4 Overall Accuracy

This Gold Standard can be represented by the recall of 65% and precision of 62%

shown in Table 4. Accordingly, the list contains only about 68% of actual issues, and

about 67% of the entries in the list are correct. The method can be tuned to provide

an improved recall by reducing precision while maintaining accuracy, resulting in

more problems being identiﬁed while also introducing more noise into the problem

list.

Tabl e 4 A recall (sensitivity) of 80% is achieved when optimizing the Cognitive Agent list

generation for high sensitivity

Model prediction objective Recall (%) Precision (%) F1score F2score

Tuned for maximum F1score 65 62 0.687 0.685

Tuned for maximum F2score 76 50 0.632 0.729

Cognitive Computing Driven Healthcare: A Precise Study 275

6 Conclusion

The topic of cognitive computing in healthcare can provide promising results based

on this overview. The academic community and the private sector are both working

on improving current system performance and proposing novel methods for utilizing

big data. In practice, however, almost all the reported cases originate from the United

States, with a healthcare system that is quite unique, compared to other countries.

There is an insufﬁcient infrastructure setting and the lack of open big data hard-

ware speciﬁcations for the systems to run effectively are also demanding, due to

an insufﬁcient infrastructure setting and the lack of open big data. Nowadays, we

have sophisticated and modern cloud architectures available for use. However, cloud

computing is still expected to maintain its rapid growth in future years in order to

widen the availability of affordable services across a wide range of applications. In

this way, cognitive systems will be based on a solid foundation and make it easier

for the penetration of new services to be launched while disrupting many established

paradigms. The evolution of cloud technology will be inﬂuenced substantially by

mental health. In the future, cognitive data will need to be stored in cloud environ-

ments that are secure and hybrid. The heterogeneity of medical data necessitates

not only re-designing the data architecture, but the data architecture itself as well.

Almost 90% of the data in medicine is visual, and 80% of this data cannot be accessed

online because of security or privacy concerns. As a result of cognitive health, the

healthcare industry will experience a powerful revolution.

Intelligence-based healthcare system using cognitive computing is accomplished

by this system, which allows for the efﬁcient deployment of network resources.

Based on the edge of computing technology and cognitive computing architecture,

the smart-clothing system is ﬁrst monitoring and analyzing the physical health of

the wearers. The system then implemented an edge computing resource allocation

approach that is driven by data. Based on the experimental results, it was found that

the cognitive computing based healthcare system offered higher quality of service to

users when faced with an emergency, while utilizing reasonable computing resources.

In the light of these technological developments, cognitive systems are poised to

become more efﬁcient, effective, and sophisticated, which will be beneﬁcial both to

patients and medical professionals.

References

1. Jena, L., Kamila, N. K., & Mishra, S. (2014). Privacy preserving distributed data mining

with evolutionary computing. In Proceedings of the International Conference on Frontiers of

Intelligent Computing: Theory and Applications (FICTA) 2013 (pp. 259–267). Springer.

2. Fortino, G., Giannantonio, R., Gravina, R., Kuryloski, P., & Jafari, R. (2013). Enabling effective

programming and ﬂexible management of efﬁcient body sensor network applications. IEEE

Transactions on Human-Machine Systems, 43(1), 115–133.

3. Sahoo, S., Mishra, S., Mishra, B. K. K., & Mishra, M. (2018). Analysis and implementation

of artiﬁcial bee colony optimization in constrained optimization problems. In Handbook of

276 R. Sharma and U. B. Ghosh

research on modeling, analysis, and application of nature-inspired metaheuristic algorithms

(pp. 413–432). IGI Global.

4. Wang, H., Xu, F., Li, Y., Zhang, P., Jin, D. (2015). Understanding mobile trafﬁc patterns of

large scale cellular towers in urban environment. In Proceedings of the 2015 ACM Conference

on Internet Measurement Conference (pp. 225–238).

5. Mishra, S., Dash, A., & Mishra, B. K. (2020). An insight of Internet of Things applications in

pharmaceutical domain. In Emergence of Pharmaceutical Industry Growth with Industrial IoT

Approach (pp. 245–273). Academic Press.

6. Chen, M., Hao, Y., Hwang, K., Wang, L., & Wang, L. (2017). Disease prediction by machine

learning over big data from healthcare communities. IEEE Access, 5, 326–337.

7. Zhou, L. (2017). Qoe-driven delay announcement for cloud mobile media. IEEE Transactions

on Circuits and Systems for Video Technology, 27(1), 84–94.

8. Zhou, L. (2016). On data-driven delay estimation for media cloud. IEEE Transactions on

Multimedia, 18(5), 905–915.

9. Fortino, G., Parisi, D., Pirrone, V., & Fatta, G. D. (2014). BodyCloud: a SaaS approach for

community Body Sensor Networks. Future Generation Computer Systems, 35, 62–79.

10. Fortino, G., Fatta, G. D., Mukaddim, V., & Athanasios, V. (2014). Cloud-assisted body area

networks: state-of-the-art and future challenges. Wireless Networks, 20(7), 1925–1938.

11. Hossain, M. S., & Muhammad, G. (2017). Emotion-aware connected healthcare big data

towards 5G. IEEE Internet of Things Journal.https://doi.org/10.1109/JIOT.2017.2772959

12. Hossain, M. S. (2017). Cloud-supported cyber physical localization framework for patients

monitoring. IEEE Systems Journal, 11(1), 118–127.

13. Hou, X., Li, Y., Chen, M., Wu, D., Jin, D., & Chen, S. (2016). Vehicular fog computing: a

viewpoint of vehicles as the infrastructures. IEEE Transactions on Vehicular Technologies,

65(6), 3860–3873.

14. Mishra, S., Mishra, B. K., & Tripathy,H. K. (2015, December). A neuro-genetic model to predict

hepatitis disease risk. In 2015 IEEE International Conference on Computational Intelligence

and Computing Research (ICCIC) (pp. 1–3). IEEE.

15. Hossain, M. S., Muhammad, G., Alamri, A. (2017). Smart healthcare monitoring: a voice

pathology detection paradigm for smart cities. Multimedia Systems.https://doi.org/10.1007/

s00530-017-0561-x

16. Hossain, M. S. (2016). Patient state recognition system for healthcare using speech and facial

expressions. Journal of Medical Systems, 40(12), 1–8.

17. Savaglio, C., et al. (2017). Agent-based computing in the Internet of Things: a survey. In

International Symposium on Intelligent and Distributed Computing. Springer.

18. Salman, O., et al. (2016). Edge computing enabling the Internet of Things. In Internet of Things

(pp. 603–608). IEEE.

19. Sahoo, S., Das, M., Mishra, S., & Suman, S. (2021). A hybrid DTNB model for heart disorders

prediction. In Advances in electronics, communication and computing (pp. 155–163). Springer.

20. He, K., Chen, J., Du, R., Wu, Q., Xue, G., & Zhang, X. (2016). DeyPoS: Deduplicatable dynamic

proof of storage for multi-user environments. IEEE Transactions on Computers, 65(12), 3631–

3645.

21. Yang, C. C., & Veltri, P. (2015). Intelligent healthcare informatics in the big data era. Artiﬁcial

Intelligence in Medicine, 65(2), 75–77.

22. Jena, L., Mishra, S., Nayak, S., Ranjan, P., & Mishra, M. K. (2021). Variable optimiza-

tion in cervical cancer data using particle swarm optimization. In Advances in electronics,

communication and computing (pp. 147–153). Springer.

23. Institute of Medicine. (2001). Crossing the quality chasm: a new health system for the 21st

century. The National Academies Press.

24. Jena, L., Patra, B., Nayak, S., Mishra, S., & Tripathy, S. (2021). Risk prediction of kidney

disease using machine learning strategies. In Intelligent and Cloud Computing (pp. 485–494).

Springer.

25. Mishra, S., Dash, A., Ranjan, P.,& Jena, A. K. (2021). Enhancing heart disorders prediction with

attribute optimization. In Advances in electronics, communication and computing (pp. 139–

145). Springer.

Cognitive Computing Driven Healthcare: A Precise Study 277

26. Kelly, J. E., Hamm, S. (2013). Smart machines: IBM’s Watson and the era of cognitive

computing. Columbia Business School Publishing, New York.

27. Tripathy, H. K., Mishra, S., Thakkar, H. K., & Rai, D. (2021). CARE: a collision-aware

mobile robot navigation in grid environment using improved breadth ﬁrst search. Computers &

Electrical Engineering, 94, 107327.

28. Roy, S. N., Mishra, S., & Yusof, S. M. (2021). Emergence of drug discovery in machine

learning. Technical Advancements of Machine Learning in Healthcare, 936, 119.

29. Liang, J., Devarakonda, M., Mehta, N. (2015). Cognitive needs of physicians: a study based on

interviews at two major hospitals s.l. 2015. IBM Research Report (Manuscript in preparation).

30. Pozna, C., Precup, R. E. (2012). Novel design of cognitive system strategies. In Proceedings of

the 2012 4th IEEE International Symposium on Logistics and Industrial Informatics, Smolenice,

Slovakia, 5–7 Sept 2012 (pp. 205–214).

31. Liu, B., Wu, C., Li, H., Chen, Y, Wu, Q., Barnell, M., Qiu, Q. (2015). Cloning your mind:

security challenges in cognitive system designs and their solutions. In Proceedings of the 52nd

Annual Design Automation Conference, San Francisco, CA, USA, 8–12 June 2015 (p. 95).

32. Hossain, M. S., & Muhammad, G. (2016). Healthcare big data voice pathology assessment

framework. IEEE Access, 4, 7806–7815.

33. Bhati, R.; Prasad, S. (2016). Open domain question answering system using cognitive

computing. In Proceedings of the 2016 6th International Conference-Cloud System and Big

Data Engineering (Conﬂuence), Noida, India, 14–15 Jan 2016 (pp. 34–39).

34. Pan, X., Teow, L. N., Tan,K.H., Ang, J. H. B., Ng, G. W. (2012). A cognitive system for adaptive

decision making. In Proceedings of the 2012 15th International Conference on Information

Fusion (FUSION), Singapore, 9–12 July 2012 (pp. 1323–1329).

35. Zhang, Y., Chen, M., Mao, S., Hu, L., & Leung, V. C. (2014). Cap: community activity

prediction based on big data analysis. IEEE Network, 28, 52–57.

36. Zhang, Y., Qiu, M., Tsai, C. W., Hassan, M. M., & Alamri, A. (2015). Health-CPS: Healthcare

cyber-physical system assisted by cloud and big data. IEEE Systems Journal, 11, 88–95.

37. Andreoli, A., Gravina, R., Giannantonio, R., Pierleoni, P., Fortino, G. (2010). SPINE-HRV:

a BSN-based toolkit for heart rate variability analysis in the time-domain. In Wearable and

autonomous biomedical devices and systems for smart environment. Springer (pp. 369–389).

38. Mishra, S., Thakkar, H., Mallick, P. K., Tiwari, P., & Alamri, A. (2021). A sustainable IoHT

based computationally intelligent healthcare monitoring system for lung cancer risk detection.

Sustainable Cities and Society, 103079.

39. Chen, M., Ma, Y., Hao, Y., Li, Y., Wu, D., Zhang, Y., Song, E. (2016). CP-robot: cloud-

assisted pillow robot for emotion sensing and interaction. In Industrial IoT technologies and

applications. Springer.

40. Pazzani, M. J., Billsus, D. (2007). Content-based recommendation systems. In P. Brusilowsky,

A. Kobsa & W. Nejdl (Eds.), The adaptive web. LNCS 4321 (pp. 325–341). Springer.

41. Tripathy, H. K., Mallick, P. K., & Mishra, S. (2021). Application and evaluation of classiﬁcation

model to detect autistic spectrum disorders in children. International Journal of Computer

Applications in Technology, 65(4), 368–377.

42. Chattopadhyay, A., Mishra, S., & González-Briones, A. (2021). Integration of machine learning

and IoT in healthcare domain. In Hybrid artiﬁcial intelligence and IoT in healthcare (pp. 223–

244). Springer.

43. Zhang, M., Zhao, H., Zheng, R., Wu, Q., & Wei, W. (2012). Cognitive internet of things:

concepts and application example. International Journal of Computer Science, 9(6–3), 151–

158.

44. Orii, Y.,Horibe, A., Matsumoto, K., Aoki, T., Sueoka, K., Kohara, S., Okamoto, K., Yamamichi,

S., Hosokawa, K. and Mori, H. (2016), Advanced interconnect technologies in the era of

cognitive computing. In Proceedings of the Pan Paciﬁc Microelectronics Symposium (Pan

Paciﬁc) (pp. 1–6).

45. Mishra, S., Dash, A., & Jena, L. (2021). Use of deep learning for disease detection and diagnosis.

In Bio-inspired neurocomputing (pp. 181–201). Springer.

278 R. Sharma and U. B. Ghosh

46. Holtel, S. (2014). More the end of information overﬂow: how IBM Watson turn upside down

our view on information appliances. In Proceedings of the 2014 IEEE World Forum on Internet

of Things (WF-I.T) (pp. 187–188).

47. Mohideen, R., & Evans, R. (2015). Shaping our technological futures. IEEE Technology and

Society Magazine, 34(4), 83–86.

48. Coccoli, M., Maresca, P., & Stanganelli, L. (2017). The role of big data and cognitive computing

in the learning process. Journal of Visual Languages & Computing, 38(1), 97–103.

49. Teo, C. L., Yang, Y., Daumè, H., Fermuller, C., Aloimonos, Y. (2012). Towards a Watson

that sees: language-guided action recognition for robots. In Proceedings of the International

Conference on Robotics and Automation (ICRA) (pp. 374–381).

50. Strickland, E. (2013). Watson goes to Med School.IEEE Spectrum, 50(1), 42–45. Tan, S. S.-L.,

Gao, G., Koch, S. (2015). Big data and analytics in healthcare. Methods of Information in

Medicine, 54(6), 546–547.

51. Ogiela, L., Tadeusiewicz, R., Ogiela, M. R. (2006). Cognitive computing in intelligent medical

pattern recognition systems. In Intelligent control and automation (pp. 851–856). Springer.

52. Mukherjee, D., Tripathy, H. K., & Mishra, S. (2021). Scope of medical bots in clinical domain.

Technical Advancements of Machine Learning in Healthcare, 936, 339.

53. Lee, H. (2014). Paging Dr. Watson: IBM’s Watson supercomputer now being used in healthcare.

Journal of AHIMA, 85(5), 44–47.

54. Mishra, S., Mishra, B. K., Tripathy, H. K., & Dutta, A. (2020). Analysis of the role and scope

of big data analytics with IoT in health care domain. In Handbook of data science approaches

for biomedical engineering (pp. 1–23). Academic Press.

55. Tan, S.S.-L., Gao, G., & Koch, S. (2015). Big data and analytics in healthcare. Methods of

Information in Medicine, 54(6), 546–547.

56. Cortada, J. W., Gordon, D., Lenihan, B. (2012). The value of analytics in healthcare: from

insights to outcomes. IBM Global Business Services, Life Sciences and Healthcare, Executive

Report.

57. Chen, Y., Argentinis, E., & Weber, G. (2016). IBM Watson: how cognitive computing can be

applied to big data challenges. Life Sciences Research, Clinical Therapeutics, 38(4), 688–701.

58. Mettler, M. (2016).Blockchain technology in healthcare: the revolution starts here. In Proceed-

ings of 2016 IEEE 18th International Conference on e-Health Networking, Applications and

Services (Healthcom), 2016 (pp. 1–3).

59. Mishra, S., Chaudhury, P., Mishra, B. K., & Tripathy, H. K. (2016, March). An implemen-

tation of feature ranking using machine learning techniques for diabetes disease prediction.

In Proceedings of the Second International Conference on Information and Communication

Technology for Competitive Strategies (pp. 1–3).

60. Yli-Huumo, J., Ko, D., Choi, S., Park, S., & Smolander, K. (2016). Where is current research

on blockchain technology?—A systematic review.PLoS One, 11(10):e0163477.

61. Mallick, P. K., Mishra, S., Mohanty, B. P., & Satapathy, S. K. (2021). A deep neural

network model for effective diagnosis of melanoma disorder. In Cognitive informatics and

soft computing (pp. 43–51). Springer.

62. Iyengar, S., Bonda, F. T., Gravina, R., Guerrieri, A., Fortino, G., Sangiovanni-Vincentelli, A.

(2008). A framework for creating healthcare monitoring applications using wireless body sensor

networks. In Proceedings of the ICST 3rd international conference on Body area networks, ICST

(Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering),

2008 (p. 8).

63. Savaglio, C., Giancarlo, F. (2015). Autonomic and cognitive architectures for the Internet of

Things. In International Conference on Internet and Distributed Computing Systems.

64. Savaglio, C., Fortino, G.,& Zhou, M. (2016). Towards interoperable, cognitive and autonomic

IoT systems: an agent-based approach. In 2016 IEEE 3rd World Forum on Internet of Things

(WF-IoT). IEEE.

65. Chen, J., He, K., Yuan, Q., Xue, G., Du, R., Wang, L. (2017). Batch identiﬁcation game model

for invalid signatures in wireless mobile networks. IEEE Transactions on Mobile Computing,

16(6), 1530–1543.

Cognitive Computing Driven Healthcare: A Precise Study 279

66. Li, Y., Jin, D., Yuan, J., & Han, Z. (2014). Coalitional games for resource allocation in

the device-to-device uplink underlaying cellular networks. IEEE Transactions on Wireless

Communications, 13(7), 1536–1576.

Cognitive Techniques for Brain Disorder

Management: A Future Trend

Mihir Narayan Mohanty

Abstract Biomedical signal analysis has a great demand for effective clinical and

hospital services. Emerging techniques need to be developed and applied for diag-

nosis and treatment of the patients. Simultaneously it will be the better support to the

physicians. In current trend, processing is likely to be digital. Physiological signals

like ECG, EMG, EEG, and imaging like CT, MRI are to be well analyzed for better

accuracy, detection, and diagnosis. The research related to biosignals increases expo-

nentially. Electroencephalograph (EEG) is one of these signals and has a vital role

in the study of brain activity, as well as different brain-related diseases, disorders,

and treatments in the ﬁeld of medicine. This chapter aims to application of machine

learning techniques for electroencephalogram (EEG) analysis as varieties of brain

disorders are diagnosed by visual inspection of EEG signals. Initial phase provides the

basics of EEG, acquisition, and necessity of analysis. Next to it, different techniques

used earlier including computational intelligence are provided. Further, use of deep

neural networks as an emerging intelligent technique is provided for different modes

of EEG analysis by researchers. Finally, an example of classiﬁcation is depicted with

the future prospects.

Keywords Biomedical signal analysis ·EEG ·Classiﬁcation ·ANN ·SVM ·DNN

1 Introduction

Human bodies are continuously communicating information about our health.

Different instruments are used to capture information like heart rate, blood pres-

sure, saturation level of oxygen, glucose level, brain activity, etc. Physicians analyze

this information to start the diagnosis process. Accurate analysis of this information

is more important for the diagnosis process. Because of the traditional biomedical

data analysis approach, sometimes it is not easy to get important information from

the medical data. In biomedical signal processing, various techniques are used to

M. N. Mohanty (B)

ITER, Siksha ‘O’ Anusandhan Deemed to be University, Bhubaneswar, Odisha, India

e-mail: mihir.n.mohanty@gmail.com

S. Mishra et al. (eds.), Augmented Intelligence in Healthcare: A Pragmatic and Integrated

Analysis, Studies in Computational Intelligence 1024,

https://doi.org/10.1007/978-981-19- 1076-0_15

281

282 M. N. Mohanty

extract useful information from different clinical measurements like EEG and ECG.

Based on this information clinicians can make decisions [1,2].

An extensive range of applications need to analyze the EEG signals in intelligent

manner that involves brain-computer interface (BCI), EEG-based motor imaging

(MI), sleep deprivation, detection of seizure, brain disorders as the visual inspection

approach is not accurate and satisfactory as well as time-consuming process. BCI

systems need less number of activities to restore the sensory and motor functions of

human subjects. It distinguishes different types of mental tasks. Also, the BCI system

requires the EEG signal as input to diagnose the brain stroke suffering persons. In

similar manner, different sleep stages identiﬁcation is of great interest among all other

brain activities. For brain disorder and seizure cases the minute brain activities. For

brain disorder and seizure cases the minute evaluation of neural activity is required

which hinders the accuracy level in EEG analysis. All these signals are mixture of

brain sources along with noise due to many sources. It is a difﬁcult task to detect and

classify the signals with only spectral features and standard methods. The application

of intelligent techniques paves the way for solution to these problems. Researchers

are busy enough to increase the accuracy level that can be cost-effective as well

[3,4].

This chapter explains the EEG signal and acquisition in ﬁrst stage. Further, the

state-of-art is described to date. Deep learning algorithm is found to be one of the

effective techniques though different approaches have been used. Also, the neural

network plays a vital role to perform the task. In a systematic way, the esseve and

application of neural networks to deep neural networks is explained. Finally, one

example is considered for classiﬁcation.

2 EEG Signal

The electrical activity of brain is deﬁned as the EEG signal. It is an alternate record

from the scalp through the electrodes of the machine. Figure 1shows how the signals

can be acquired from the subject and the particular lead is positioned in particular

place of the head.

This may be of electrocardiogram or electrogram depending upon the EEG

measurement from cortial surface or using depth probes. The various states in

wakefulness are like sitting idle, doing some mathematical task, or thinking any

innovative idea and in case of sleepiness, it is drowsy or in deep sleep, etc. The

analysis of EEG signals paves the way to diagnose and determine the mental state

of human beings. A typical normal EEG signal is shown in Fig. 2. EEG data

is collected by using an EEG recording machine with model number Quest 32

1, ploysomnograph. Three different activity of the human brain is recorded for

10 subjects. The activities are like sitting idle, performing complex multiplication,

composition of song. Different types of lobes presented inside the human brain are

accountable for performing different types of activities. The responsibility of the

three different lobes is presented in Table 1.

Cognitive Techniques for Brain Disorder … 283

Fig. 1 Data acquisition system

Fig. 2 A typical seven-second normal EEG record

Tabl e 1 Responsible for various lobes inside brain

Lobe Responsibility

Frontal Reasoning, emotion, movement, planning, and problem solving

Parietal Recognition, perception of stimuli etc.

Temporal Recognition and perception of auditory stimuli, memory, and speech

284 M. N. Mohanty

Tabl e 2 The frequency bands in EEG and their standard interpretation

Bund Frequency range Acquisition location Effects

Delta <4 Hz Everywhere Coma, deep sleep

Theta 4–7 Hz Temporal and parietal Stress due to emotion

Alpha 8–12 Hz Occipital and parietal Sensory stimulation or mental imagery

Beta 12–36 Hz Parietal and frontal Intense mental activity

Mu 9–11 Hz Frontal (motor cortex) Movement or intention of movement

Lambda Sharp, jagged Occipital Visual attention

In the proposed work, the brain signal is recorded from different positions such as

F3, F4, T3, T4, P3, and P4 by following the international 10–20 system. The sampling

frequency of the recorded EEG signal is 256 Hz. Total six number of electrodes are

placed in the different positions of the brain for recording the brain activity.

In general, the EEG signal is speciﬁed with its frequency level. The various

types are designated according to frequency level. Simultaneously various types

of waveforms are meant for particular activities. Table 2shows the details of the

type, frequency range, location of electrodes, and the use of the EEG signal.

3 Related Literature

The EEG signal detection and classiﬁcation found from the earlier work as of two

important areas such as (i) Brain-computer interface (BCI), (ii) Physiological analysis

of the brain. For both the areas computer is used for patients even for disabled patients.

Details overview of BCI can be found in [5–13]. Neural commands are translated into

central signals, that can be well analyzed in BCI. Control applications like text input

wheelchairs and neuroprostheses are utilized in the analysis of BCI. It can provide

the best communication to the patients along with disabled persons to analyze the

brain signal, whereas man–machine systems are used for standard users. Traditional

approaches are required to train the system to adjust their brain signals by the system.

The Berlin Brain-Computer Interface project (BBCI) has developed the system that

overcomes this problem with the help of machine learning methods. BCI can get

more rate of information transfer from subject by adaption of classiﬁcation systems.

This chapter provides a comprehensive overview of classiﬁcation algorithm used

so that future researchers will have the idea to use speciﬁc methods for speciﬁc

problems. The newly planned algorithms for EEG-based BCIs are categorized into

ﬁve types such as (i) adaptive algorithm, (ii) matrix and tensor classiﬁers, (iii) transfer

learning, (iv) deep learning, and (v) few other miscellaneous classiﬁers. Adaptive

classiﬁers were found to be superior even with unsupervised adaption among all these

classiﬁers. Transfer learning similarly can also prove useful even though the transfer

learning is unpredictable. It deserves to be explored thoroughly with tensor and

matrix-based methods. Linear discriminant analysis and random forest algorithms

Cognitive Techniques for Brain Disorder … 285

are also shown to be relevant for small training samples. However, deep learning

methods have not yet revealed persuasive development over state-of-art BCI methods.

The deep learning network using Convolutional to train the features and reduce

the dimension was used for classiﬁcation. Thus an end-to-end model was developed

and analyzed the raw signals directly and was proposed by researchers for validation

and robustness. As a new advanced approach, transfer learning was also considered

to adapt the global classiﬁer to improve the accuracy. The models were trained for

3secondEEG data segments from different subjects. It was used for disabled people

[14].

Because of changes in the trademark properties of the non-stationary information,

arrangement can regularly be deﬁled. Issues happen in signal preparing and AI when

dynamical frameworks change their properties after some time. Issues can especially

emerge if the calculations depend on alignment information or the estimation of

parameters on little portions of the accessible information. The causes for the changes

in the dynamics are the following:

•The properties of the sensors, (for example, anodes and enhancement units) change

after some time.

•Neurophysiological conditions can show an enormous changeability. This can

likewise inﬂuence mental procedures for the correspondence with the gadgets.

Brain-computer interface (BCI) gives a channel to convey from a human PC that

empowers the mind to send messages. Likewise, BCI enables a person with engine

incapacities to have powerful authority over gadgets. In outline, BCI framework

perceives the examples of mind action and makes an interpretation of them into

groupings of control directions.

EEG can provide the direct measurement from the cortical activities. Since there

are no speciﬁc criteria for evaluation, visual analysis of EEG signals is not sufﬁcient

for analysis. Automated classiﬁcation of EEG signals is a great help for the physicians

to utilize the expert opinion in an effective manner.

Literature suggests several methods for modeling and classiﬁcation of the EEG

signals in different contexts. An optimal nonlinear element extractor for removing

vitality includes under two various types of works was proposed by Vaughan et al.

[15]. The concurrent diagonalization of two sign covariance grids has been completed

in a high-dimensional piece changed space to discover discriminant highlights. In

[16] creators have depicted the utilization of neural system models for grouping of

EEG signals.

Wavelet Transform (WT) was used for feature extraction and then a learning-

based neural network classiﬁer was considered for the classiﬁcation purpose. A

linear estimator between non-stationary signals based on the cross-correlation of

narrow-band ﬁltered signals was presented. This estimator was contrasted with an

increasingly old-style estimator dependent on the intelligence work. Authors have

shown better performances when a priori knowledge was known. A method of anal-

ysis of EEG signals using WT and classiﬁcation using Artiﬁcial Neural Network

(ANN) and Logistic Regression (LR) has been dealt with in [17]. Various types of

286 M. N. Mohanty

neural networks were considered for classiﬁcation of EEG signals and that can be

found in [18–23].

The methodology in [24] utilized AI procedures as profound neural systems to

group cerebrum signals obtained utilizing a mind PC interface (BCI). The outcomes

acquired through this basic element alteration accomplished a characterization exact-

ness of up to 79% as they have asserted. They have shown the predominance of

embracing a profound learning approach over other AI approaches for identifying

human energy when submerged in a vivid computer-generated simulation condition.

Deep learning has been applied much of the time to deliver cutting edge brings

about different difﬁcult issues, for example, in PC vision, characteristic language

handling, and sound acknowledgment for arrangement over a bigger gathering of

subjects. They have indicated that profound learning beats an assortment of other

AI classiﬁers for this EEG-based inclination characterization task, especially in an

exceptionally testing dataset with enormous between and intra-subject ﬂuctuation

[25].

Profound learning models with long short-term memory units (LSTMs) appli-

cations execution was great when contrasted with customary strategies. Likewise,

they have investigated the utilization of move learning via preparing over numerous

subjects and reﬁning on a speciﬁc subject. This was improved the order exactness of

the profound learning models [26].

By and large, huge scale explained EEG datasets are practically difﬁcult to gain

on the grounds that organic information securing is testing and quality comment is

expensive. To conquer this sort of issue a model was created dependent on intellectual

occasions EEG information. A profound exchange learning structure was discovered

appropriate for moving information by joint preparing.

Deep learning (DL) for occasion characterization utilizing electroencephalo-

gram (EEG) estimations of mind exercises was proposed and named as progressive

profound neural system, and CNN4EEG, another convolution neural system (CNN).

A deliberate literature study of EEG order utilizing profound learning was carried

out on Web of Science and PubMed databases. Those investigations were broken

down dependent on kind of undertaking, EEG preprocessing techniques, input type,

and profound learning engineering. Convolutional neural networks were used along

with recurrent neural networks, deep belief networks and outperformed the multilayer

perceptron neural networks in classiﬁcation accuracy.

Deep learning is also applied for sleep stage classiﬁcation. In this case, features

have been extracted and exploited using multivariate and multimodal analysis. For

such cause, the signals like EEG, EMG, and EOG of 30-s window were used.

One of the most popular classiﬁcation methods used for EEG signals is Support

Vector Machines (SVM) which was proposed by Vapnik [27]. SVM has been the

choice for classiﬁcation as it works nicely when the margin of separation is prominent

between classes, is effective for higher dimension data, and utilizes the memory more

efﬁciently. In [2] and SVM based method has been utilized based on the feature

extraction principles cited in [28]. In [29] SVM is used for classiﬁcation in Fourier

and Time–Frequency domains for classifying the source of current in the EEG signals.

A method of feature extraction and classiﬁcation has been achieved by using Principal

Cognitive Techniques for Brain Disorder … 287

Component Analysis (PCA) and Hidden Markov Models (HMM) [30]. Since linear

classiﬁers are prone to errors in presence of noisy and complex data containing

outliers, Kernel-based methods are mostly used as they retain the properties of linear

classiﬁcation in the feature space even if the input space is nonlinear.

Recent methods like [31] use machine learning techniques have used deep neural

networks (DNN) for classiﬁcation of brain signals which are obtained by employing

a Brain-Computer Interface (BCI). This method uses DNNs allowing occurrences of

dropouts in the NN architecture and thereby achieving 79% classiﬁcation accuracy

(CA). This study proves that the DNN classiﬁer with dropouts achieves 13–18%

increase in classiﬁcation accuracy than the conventional DNNs for EEG preference

classiﬁcation. In the same study, the users are exposed to a roller-coaster environment

via virtual reality goggles to entice the emotional stimuli and the raw brain signals are

acquired simultaneously by using an EEG headset. The use of DNN based approach

has resulted in CA of over 90% for the excitement detection rate. The study also

infers that the deep learning methods are essentially superior to the other machine

learning approaches for detection of human excitement through a VR environment.

A study of application of deep learning [32] was used to test the 3D object pref-

erence classiﬁcation. Here the EEG was recorded for a group of 16 individuals

while they were shown 60 bracelet-like objects as rotating visual stimuli on a visual

display unit. The resulting data were used to train various machine learning methods

including deep learning, the classiﬁcation of users’ preferences for the 3D visual

stimuli was attempted. The results showed that the deep learning methods have

better CA than the other machine learning approaches used in this complex dataset

which contains large inter- and intra-subject variability. It was also found that the

small Convolutional NN (CNN) architectures performed at par with the traditional

ML methods whereas deep learning architectures that use long short-term memory

units (LSTMs) shown have outdid the traditional ML methods. The case of transfer

learning is also attempted which showed improvement of CA using the deep learning

networks.

The two major reasons for the reduced CA for traditional ML methods are (a)

the traditional ML methods in no way exploit the multimodal information and (b)

large-scale EEG datasets with annotation are very difﬁcult and costly to acquire [33].

A novel deep transfer learning approach has been applied where ﬁrst the cognitive

methods are events are modeled based on the EEG dataset using EEG optical ﬂow.

Here it is intended to preserve the multimodal EEG information in a uniform repre-

sentation. The next step is to develop a deep transfer learning framework that should

be able to transfer knowledge by joint training with the help of an adversarial network

and a novel loss function.

A deep learning method for event classiﬁcation using EEG signals using a Hierar-

chical DNN (HDNN) and a special CNN is known as CNN4EEG. Both the networks

were used for prediction of image targets. The results establish the fact that the

CNN4EEG achieved an improvement of 13% over the best non-DL algorithm, an

improvement of 9% over the canonical CNN algorithm used in image processing,

and an improvement of 6% over the DNN [34].

288 M. N. Mohanty

The study revealed number of investigations for spatio-temporal data which is a

better trade-off for optimized classiﬁcation in terms of accuracy. As sleep experts,

the system was suitable for multivariate and multimodal nature of PSG signals [35].

In [22], authors have used machine learning techniques for automated epileptic

seizure detection. Two approaches were taken as CNN and ANN to provide a proba-

bility of seizure occurrence. In order to extract relevant features, time–frequency and

time–frequency domains are considered for the neural network input. The median

gain in accuracy of deep learning approaches over traditional baseline was better as

compared to other methods.

Nowadays IoT is one of a buzzword in information technology. The real-world

objects can be transformed into intelligent and most effective objects by the applica-

tion of IoT. It formulates an integrated communication atmosphere of interconnected

devices and performs by engaging both the virtual and physical world together. The

concept of internet of things was raised due to the union of several advanced tech-

nologies such as machine learning, real-time data analysis, commodity sensors, and

embedded systems. IoT systems can be enabled by the contribution of the wire-

less sensor network, automation, embedded systems, and other advanced techniques

[36]. To improve the diagnosis time it is required to implement a smart low-cost EEG

signal detection system that can transform data to hospitals from any place. In Fig. 3

an IoT-based EEG data transmission system for the patients affected by different

brain diseases is presented. Due to the development in wireless sensor networks,

Fig. 3 IoT-based EEG data transmission system

Cognitive Techniques for Brain Disorder … 289

the transmission of ECG signals can be done easily to the hospitals through wireless

transmission techniques such as Bluetooth or Zigbee [37]. In this system, the number

of electrodes is less than traditional EEG data collecting system and can collect basic

information about the heart. Long-term EEG signals can be a monitor with minimal

cost by these portable sensors.

4 Method of Classiﬁcation

EEG data is collected by using an EEG recording machine with model number

Quest 321, ploysomnograph. Three different types of EEG signal is considered for

the proposed work. After collecting the EEG signal the original data is divided into

training and test set. About 80% of data is considered for training purposes and rest

20% is for testing purposes. For classiﬁcation of three different types of EEG total

of ﬁve types of classiﬁer is used. In Figs. 4and 5the structure of the EEG signal

classiﬁcation system is presented.

Fig. 4 Proposed system

Fig. 5 EEG signal

classiﬁcation system

290 M. N. Mohanty

Fig. 6 Standard ANN

structure

For the classiﬁcation of the EEG signal total of ﬁve types of classiﬁers are used

and are presented in the next subsection.

Neural Network

Neural network is the most accepted machine learning model that works on biological

nervous system principle. Artiﬁcial neurons are the most fundamental elements in

a neural network and it works in a layered structure as shown in Fig. 6. The input

data is captured through the input layer and then it goes through the hidden layer.

After processing through the hidden layer the ﬁnal output is achieved through output

layer. Generally, in feed-forward neural network, the data ﬂow in forward direction,

i.e., input to output layer. This classiﬁer is well suitable for both linear and nonlinear

classiﬁcation problems with variations in structure as well as learning algorithms

[37,38].

Support Vector Machine

Among all neural network-based support vector machines (SVM) is another powerful

and efﬁcient feed-forward neural network used for classiﬁcation and regression prob-

lems. It can be used for both linear and nonlinear data classiﬁcation. It is basically a

binary classiﬁer where a nonlinear mapping is considered for transforming the orig-

inal training data into a higher dimension. The data from one class is separated from

another class in this new dimension by using a decision boundary (i.e., a hyperplane).

The hyperplane is found by using the support vectors (training tuples) [39]. In Fig. 7

the structure of the SVM classiﬁer is presented and the detailed derivation of this

binary classiﬁer is explained as follows;

Let DMbe a set of Mlabeled data points in an Ndimensional hyperspace:

DM=[(y1,a1), ....(yM,aM)]∈(Y×A)M(1)

where yi∈Y, where Yis the input space and ai∈A,A={−1,+1}.

Cognitive Techniques for Brain Disorder … 291

Fig. 7 Support vector

machine

It is formulated for designing ψsuch as;

ψ:Y→Ad is predicted from the input y.

Though, Ycan be changed to an equivalent or high-dimensional feature space to

make it linearly separable. The issue of ﬁnding a nonlinear decision boundary limit

in Yhas been mapped to ﬁnding an optimal hyperplane for separating two classes.

In the transformed domain the hyperplane or feature space can be parameterized

by (z,c)pairs such as:



i=1

ziφi(y)+c=0(2)

It is required to calculate the mapping function φ(.)explicitly.

φ(yi),φ

yj=Kyi,yj(3)

In the proposed SVM the kernel function is considered as radial basis function

(RBF). In the input space, the patterns after the completion of the transformation are

not able to separate linearly. In Fig. 8the structure of the SVM structure is presented.

K-Nearest Neighbor (KNN) Classiﬁer

KNN is one of the most used classiﬁcation models which is based on analog learning.

Here in this method, a comparison is done between the training and the testing tuples

for ﬁnding the similar. The input to the classiﬁer is consist of k-training samples in

the feature set. The output of the KNN classiﬁer is a type of class membership. The

292 M. N. Mohanty

Fig. 8 The modular SVM structure

Fig. 9 KNN classiﬁer

classiﬁcation is done by depending upon the majority of votes by its neighbors, with

the object allocated to the most common neighbors. Here k is a small integer value

and when k=1, then the class is allocated to the class of a single nearest neighbor

[40]. The basic structure of the KNN classiﬁer is presented in Fig. 9.

4.1 Random Forests

Random forest is a type of classiﬁer which is the combination of multiple classiﬁers. It

works by ensemble learning procedure and multiple learning mechanisms are used

for solving a particular problem. Here in this method number of assumptions are

constructed and by combining them the problem is solved. Let us consider θmis a

random vector and free from earlier vectors. The classiﬁer h(y,θ

m)is generated by

completing the training of the data. yis the input data vector in the classiﬁer. After

generating a large amount of trees the voting for most accepted class is happened

Cognitive Techniques for Brain Disorder … 293

to get the classiﬁcation result. The overall structure of this proposed classiﬁer is

called random forests where a group of tree like classiﬁers {h(y,θ

m),m=1, ....}

is designed for the classiﬁcation purpose [41]. The structure of the RF model is

presented in Fig. 10.

Deep Neural Network (DNN)

DNN is another type of advanced neural network that consists of multiple hidden

layers between input and output layers. From Fig. 11, it can be observed the differ-

ence between a simple neural network (SNN) and DNN. In SNN we cannot add

more number of hidden layers but it can be possible in case of DNN. Due to this

advanced property of DNN it is very popular in many problems that include high-

dimensional data classiﬁcation. Proposed DNN based EEG classiﬁcation model is

designed with two stages. In the ﬁrst stage, the model automatically learns the features

from the input dataset. After successful completion of the feature learning procedure,

a fully connected multilayer perceptron classiﬁes the initially learned features. After

this two-stage, there is a feature identiﬁer module that includes the convolutional

and pooling layers. The feature set from the preceding layer is convolved by using

the convolutional kernel presented in the convolutional layer. After the convolution

process, again it goes through the activation function in order to get the activation

map for the next layer. At the same time, the pooling (subsampling) layer creates the

activation map to be reduced but it increases.

In the convolutional layer, the activation map from the past layer is convolved

utilizing convolutional channel (or piece) which is included with predisposition and

in this way nourished to the actuation capacity to produce an initiation map for the

following layer. It is used after the convolutional network. The output of the output

layer can be calculated as

Ci,a

j=σda+



m=1

mx0a

i+m−1(4)

Fig. 10 Random forest classiﬁer structure

294 M. N. Mohanty

Fig. 11 SNN versus DNN

where x0

i=(x1,x2,x3,...,xm)is the input data vector and mis sum of ECG

segments. jis the layer index and dis the bias of the feature map. σis the activation

function. Mis the ﬁlter size. wa

mis the weight for mth ﬁlter index.

Pooling layer is another building block of DNN that gradually decreases the

spatial size of the activation map to decrease the amount of parameters as well as

computation time in the neural network. This layer operates on every feature map

separately. In DNN classiﬁer, max-pooling is the most used pooling layer in every

problem. The output of a max-pooling layer can be found by the maximum activation

over a non-overlapping section of input [16,42].

A fully connected layer has full link to all the activations in the prior layer. It converts

all the outputs of the neural network into sum up to one. Activation function creates

an output to a set of inputs. An activation function is commonly used after every

convolutional layer. It is a nonlinear transfer function used over the input data. The

transfer output is then transmitted to the next layer as the input. Generally, in DNN,

two types of activation functions are used: (a) Rectiﬁed linear unit (Relu) and (b)

Softmax.

(a) Rectiﬁed linear unit (Relu):

Relu is one of the most used activation functions in DNN. The main beneﬁt of using

this function is that all the neurons do not get activated at a time. It also converts

all the negative neurons into zero and due to this all neurons could not be activated.

Another advantage of using this activation function is faster training as compared to

other functions [33]. Mathematically it can be represented by

f(x)=max(0,x)(5)

Cognitive Techniques for Brain Disorder … 295

where xis the input data and f(x) is the output function that returns the maximum

value between 0 and input data.

(b) Softmax:

It is a popular activation function as observed from literature. The exponential values

of the input signal are considered in this function. Further, the sum of all these values

is computed. Next to it, the ratio of the exponential to sum of exponential is evaluated

as the output function. The beneﬁt of this function is the probabilities of output data.

The variety probabilities differ between 0 and 1. In case of multiclass classiﬁcation

problem, softmax activation function provides the probability for every class and the

ﬁnal output will get highest probability.

This activation function can be represented by:

sj=exj

n

i=1exn(6)

where the input is xand the output value of sis between 0 and 1 and their sum is

equal to 1.

5 Results and Discussions

EEG signals from different electrodes with 25,600 sample size are considered for the

classiﬁcation by using different classiﬁers. The EEG signals recorded for performing

the classiﬁcation task are shown in Fig. 12.

First, the original signals are separated into training and testing set, respec-

tively. The classiﬁcation results for different classiﬁer for different band is shown

in Table 3. Table 4explains the classiﬁcation performance for different mental states.

From the experimental result, it is observed that the DNN classiﬁer is giving better

results as compared to other four types of classiﬁers. It is because of the presence

of more number of hidden layers in the network. Around 95% accuracy is obtained

from the proposed DNN classiﬁer. Also, SVM is performing better as compared to

KNN, Random forest, and ANN. Around 90% classiﬁcation accuracy is obtained by

using SVM classiﬁer.

6 Conclusion

In this chapter, a brief review is given for brain signal detection and classiﬁcation.

Though the classiﬁcation is treated with statistical procedure it is found from literature

to enhance the robustness and accuracy for large amount of data, the deep learning

algorithms also have been applied. As a single case, it has been shown the efﬁcacy

of deep learning for brain signal classiﬁcation along with comparative results. So

296 M. N. Mohanty

Fig. 12 Types of EEG signals are considered for the proposed work

Tabl e 3 Classiﬁcation results for different EEG bands

Classiﬁers Accuracy for different bands

Delta (%) Theta (%) Alpha (%) Beta (%) Mu (%) Lambda (%)

ANN 89.58 88.99 89.41 87.66 89.69 89.97

K-NN 88.69 87.98 88.97 88.97 89.99 91.22

Random Forest 88 88.69 88.57 89.97 90.10 89

SVM 91.21 90.90 87.56 88.79 87.56 89.99

DNN 91.26 92.69 93.57 95.63 95.55 95

Tabl e 4 Classiﬁcation performance for different mental state

Classiﬁer Sitting relax (%) Problem solving (%) Innovation thinking (%)

ANN 89.58 88.99 89.41

K-NN 88.69 87.98 88.97

Random Forest 88 88.69 88.57

SVM 91.21 90.90 87.56

DNN 91.26 92.69 93.57

Cognitive Techniques for Brain Disorder … 297

this chapter may help the researchers in this area for future development on optimal

classiﬁers and efﬁcient optimization algorithms for speciﬁc data.

References

1. Rangayyan, R. M. (2015). Biomedical signal analysis (Vol. 33). Wiley.

2. Cohen, A. (2019). Biomedical signal processing: Volume 2: Compression and Automatic

Recognition. CRC Press.

3. Tripathy, H. K., Mishra, S., Thakkar, H. K., & Rai, D. (2021). CARE: A collision-aware

mobile robot navigation in grid environment using improved breadth ﬁrst search. Computers &

Electrical Engineering, 94, 107327.

4. Lakshmi, M. R., Prasad, T. V., & Prakash, D. V. C. (2014). Survey on EEG signal processing

methods. International Journal of Advanced Research in Computer Science and Software

Engineering, 4(1).

5. Lal, T. N., Schroder, M., Hinterberger, T., Weston, J., Bogdan, M., Birbaumer, N., & Scholkopf,

B. (2004). Support vector channel selection in BCI. IEEE Transactions on Biomedical

Engineering, 51(6), 1003–1010.

6. Muller, K. R., Anderson, C. W., & Birch, G. E. (2003). Linear and nonlinear methods for brain-

computer interfaces. IEEE Transactions on Neural Systems and Rehabilitation Engineering,

11(2), 165–169.

7. Mishra, S., Mishra, B. K., & Tripathy,H. K. (2015, December). A neuro-genetic model to predict

hepatitis disease risk. In 2015 IEEE International Conference on Computational Intelligence

and Computing Research (ICCIC) (pp. 1–3). IEEE.

8. Wolpaw, J. R., McFarland, D. J., & Vaughan, T. M. (2000). Brain-computer interface research

at the Wadsworth Center. IEEE Transactions on Rehabilitation Engineering, 8(2), 222–226.

9. Lotte, F., Congedo, M., Lécuyer, A., Lamarche, F., & Arnaldi, B. (2007). A review of classi-

ﬁcation algorithms for EEG-based brain–computer interfaces. Journal of neural engineering,

4(2), R1.

10. Farwell, L. A., & Donchin, E. (1988). Talking off the top of your head: Toward a

mental prosthesis utilizing event-related brain potentials. Electroencephalography and clinical

Neurophysiology, 70(6), 510–523.

11. Ko, L. W., Tsai, I. L., Yang, F. S., Chung, J. F., Lu, S. W., Jung, T. P., & Lin, C. T.

(2008, November). Real-time embedded EEG-based brain-computer interface. In International

Conference on Neural Information Processing (pp. 1038–1045). Springer.

12. Mishra, S., Thakkar, H., Mallick, P. K., Tiwari, P., & Alamri, A. (2021). A Sustainable IoHT

based computationally intelligent healthcare monitoring system for lung cancer risk detection.

Sustainable Cities and Society, 103079.

13. Wolpaw, J. R., Birbaumer, N., McFarland, D. J., Pfurtscheller, G., & Vaughan, T. M. (2002).

Brain–computer interfaces for communication and control. Clinical neurophysiology, 113(6),

767–791.

14. Das, R., Maiorana, E., &Campisi, P. (2017, August). Visually evoked potential for EEG biomet-

rics using convolutional neural network. In 2017 25th European Signal Processing Conference

(EUSIPCO) (pp. 951–955). IEEE.

15. Vaughan, T. M., Wolpaw, J. R., & Donchin, E. (1996). EEG-based communication: Prospects

and problems. IEEE Transactions on Rehabilitation Engineering, 4(4), 425–430.

16. Kingma, D. P., & Ba, J. (2014). Adam: a method for stochastic optimization. arXiv preprint

arXiv:1412.6980.

17. Subasi, A. (2005). Automatic recognition of alertness level from EEG by using neural network

and wavelet coefﬁcients. Expert systems with applications, 28(4), 701–711.

18. Singla, R., Chambayil, B., Khosla, A., & Santosh, J. (2011). Comparison of SVM and ANN

for classiﬁcation of eye events in EEG. Journal of Biomedical Science and Engineering, 4(01),

62.

298 M. N. Mohanty

19. Srinivasan, V., Eswaran, C., & Sriraam, N. (2007). Approximate entropy-based epileptic EEG

detection using artiﬁcial neural networks. IEEE Transactions on information Technology in

Biomedicine, 11(3), 288–295.

20. Sors, A., Bonnet, S., Mirek, S., Vercueil, L., & Payen, J. F. (2018). A convolutional neural

network for sleep stage scoring from raw single-channel EEG. Biomedical Signal Processing

and Control, 42, 107–114.

21. Hsu, Y. L., Yang, Y. T., Wang, J. S., & Hsu, C. Y. (2013). Automatic sleep stage recurrent

neural classiﬁer using energy features of EEG signals. Neurocomputing, 104, 105–114.

22. Srinivasan, V., Eswaran, C., & Sriraam, A. N. (2005). Artiﬁcial neural network based epileptic

detection using time-domain and frequency-domain features. Journal of Medical Systems,

29(6), 647–660.

23. Ozdamar, O., Yaylali, I., Jayaker, P., & Lopez, C. N. (1991, May). Multilevel neural network

system for EEG spike detection. In 1991 Computer-Based Medical Systems@ m_Proceedings

of the Fourth Annual IEEE Symposium (pp. 272–279). IEEE.

24. Guger, C., Schlogl, A., Neuper, C., Walterspacher, D., Strein, T., & Pfurtscheller, G. (2001).

Rapid prototyping of an EEG-based brain-computer interface (BCI). IEEE Transactions on

Neural Systems and Rehabilitation Engineering, 9(1), 49–58.

25. Bashivan, P., Rish, I., Yeasin, M., &Codella, N. (2015). Learning representations from EEG

with deep recurrent-convolutional neural networks. arXiv preprint arXiv:1511.06448

26. Alhagry, S., Fahmy, A. A., & El-Khoribi, R. A. (2017). Emotion recognition based on EEG

using LSTM recurrent neural network. Emotion, 8(10), 355–358.

27. Vapnik, V. (2013). The nature of statistical learning theory. Springer Science & Business Media.

28. Guyon, I., & Elisseeff, A. (2003, March). An introduction to variable and feature selection.

Journal of Machine Learning Research, 3, 1157–1182.

29. Huang, W. Y., Shen, X. Q., & Wu, Q. (2002, November). Classify the number of EEG current

sources using support vector machines. In Proceedings. International Conference on Machine

Learning and Cybernetics (Vol. 4, pp. 1793–1795). IEEE.

30. Garcia, G. N., Ebrahimi, T., &Vesin, J. M. (2003, March). Support vector EEG classiﬁcation

in the Fourier and time-frequency correlation domains. In First International IEEE EMBS

Conference on Neural Engineering, 2003.Conference Proceedings (pp. 591–594). IEEE.

31. Behnam, M., & Pourghassem, H. (2015, November). Power complexity feature-based seizure

prediction using DNN and ﬁreﬂy-BPNN optimization algorithm. In 2015 22nd Iranian

Conference on Biomedical Engineering (ICBME) (pp. 10–15). IEEE.

32. Lehman, J., Risi, S., & Clune, J. (2016, June). Creative generation of 3D objects with deep

learning and innovation engines. In Proceedings of the 7th International Conference on

Computational Creativity.

33. Kim, S. S., & Kasparis, T. (1998). A modiﬁed domain deformation theory on 1-D signal

classiﬁcation. IEEE Signal Processing Letters, 5(5), 118–120.

34. Phan, H., Andreotti, F., Cooray, N., Chén, O. Y., & De Vos, M. (2019). SeqSleepNet: End-to-

end hierarchical recurrent neural network for sequence-to-sequence automatic sleep staging.

IEEE Transactions on Neural Systems and Rehabilitation Engineering, 27(3), 400–410.

35. Mohanty, M. N., Routray, A., & Kabisatpathy, P. (2010). Optimisation of features using evolu-

tionary algorithm for EEG signal classiﬁcation. International Journal of Computational Vision

and Robotics, 1(3), 297–310.

36. Mishra, S., Panda, A., & Tripathy, K. H. (2018). Implementation of re-sampling technique

to handle skewed data in tumor prediction. Journal of Advanced Research in Dynamical and

Control Systems, 10, 526–530.

37. De Buyser, E., De Coninck, E., Dhoedt, B., &Simoens, P. (2016). Exploring the potential

of combining smart glasses and consumer-grade EEG/EMG headsets for controlling IoT

appliances in the smart home.

38. Haykin, S. (2010). Neural networks and learning machines. 3/E. Pearson Education India.

39. Hagan, M. T., Demuth, H. B., & Beale, M. (1997). Neural network design. PWS Publishing

Co.

Cognitive Techniques for Brain Disorder … 299

40. Tripathy, H. K., Mallick, P. K., & Mishra, S. (2021). Application and evaluation of classiﬁcation

model to detect autistic spectrum disorders in children. International Journal of Computer

Applications in Technology, 65(4), 368–377.

41. Mohapatra, S. K., & Mohanty, M. N. (2020). Big data analysis and classiﬁcation of biomedical

signal using Random Forest Algorithm. In New paradigm in decision science and management

(pp. 217–224). Springer.

42. Nagi, J., Ducatelle, F., Di Caro, G. A., Cire¸san,D.,Meier,U.,Giusti,A.,&Gambardella,L.M.,

et al. (2011, November). Max-pooling convolutional neural networks for vision-based hand

gesture recognition. In 2011 IEEE International conference on signal and image processing

applications (ICSIPA) (pp. 342–347). IEEE.

Relevance of Blockchain

in Revolutionizing Health Records

Amlan Mishra, Kashif Moin, Mayank Shrivastava,

and Hrudaya Kumar Tripathy

Abstract Healthcare breaches and fragmented data in the healthcare sector have

been a pressing issue for a long time. Though standards of data systems and infor-

mation security have been increasing with time, there has not been any established

approach to keep health records secure and easy to use. Blockchain technology

offers a solution—and healthcare organizations are increasingly taking notice of

its transformative potential. It has of late been one of the most promising ﬁelds

of study in the research sector. Though it is still in its early development stages,

improvements are being devised every day to make it better than before. In this

study, we aim to discuss the innovation of blockchain technology and the way it

may solve the current issues in the healthcare sector. A framework is presented that

could be used to implement blockchain technology in the healthcare industry for

Electronic Health Records (EHR), evaluating the various perspectives around health

data, including data protection, security, control, and storage. Blockchain, being one

of the most encouraging technologies lately has imparted immense beneﬁts to the

health care sector due to security, privacy, conﬁdentiality, and decentralization. It has

the potential to comprehensively manage patient records.

Keywords Health records ·Electronic health records ·Blockchain ·Data security

and storage ·Decentralization ·Dapps

1 Introduction

Data security in the healthcare industry is one of the major concerns in today’s world.

For a country like India with a population of almost 1.3 billion, the task of providing

A. Mishra ·K. Moin ·M. Shrivastava ·H. K. Tripathy (B)

Kalinga Institute of Industrial Technology, Deemed to be University, Bhubaneswar, India

e-mail: hktripathyfcs@kiit.ac.in

A. Mishra

e-mail: 1905594@kiit.ac.in

K. Moin

e-mail: 1905617@kiit.ac.in

S. Mishra et al. (eds.), Augmented Intelligence in Healthcare: A Pragmatic and Integrated

Analysis, Studies in Computational Intelligence 1024,

https://doi.org/10.1007/978-981-19- 1076-0_16

301

302 A. Mishra et al.

each citizen of the country with proper healthcare is a huge responsibility. According

to Harvard University research published in 2017, over 5 million people die in India

each year as a result of medical blunders caused by a lack of practical knowledge

among physicians and nurses when patients are brought to the hospital. Not having

accurate data about the patients is one of the key problems. The present condition

of healthcare records is disconnected and fragmented as a result of data breaches,

information asymmetries, and a lack of uniform protocols and design.

The graph (Fig. 1) indicates that between 2009 and 2019, there have been 3054

healthcare statistics breaches within the United States with over ﬁve hundred records

leading to loss, theft, and unauthorized disclosure of 230,954,151 health records,

representing more than 70% of the United States population.

Earlier, all the health records of the patients were kept in hard copies, generally

in paper format, which was quite a cumbersome way to store data. But since the

past few years, Electronic Health Records (EHR) have been introduced. An EHR

is a digital patient health data structure that is created and maintained throughout

the patient’s life and is typically stored and distributed across multiple hospitals,

clinics, and medical records. Electronic Health Records (EHR) often contain highly

conﬁdential and sensitive personal data for treatment and diagnosis [1].

In Fig. 2, a few uses of electronic healthcare records have been mentioned which

include medical tests and medical records also.

An EHR provides a user-friendly medical record service that makes traditional

paper patient records electronically available over the internet. This system is

intended to provide patients authority over the development, administration, and

sharing of EHRs with family, friends, healthcare practitioners, and approved data

consumers.

Fig. 1 Healthcare data breaches. Source hipaajournal.com

Relevance of Blockchain in Revolutionizing Health Records 303

Fig. 2 Electronic healthcare

records

However, as life events have disseminated their EHRs to new places, the EHRs

have migrated from one database to another [2]. This leads to patients losing access

to their own records, past data getting scattered and lost, and the hospitals getting

direct access to the data.

In this chapter, the discussion starts with a centralized database approach and its

problems and then an idea for secured health records using blockchain technology

has been proposed. Although several approaches are currently being developed, there

are no agreed standards or protocols governing the validation of titles, so an attempt

has been made to implement an efﬁcient prototype.

2 Centralized Database

2.1 Means of Resolution

A centralized database could be one of the solutions to the fundamental problem

we are facing. We can store the records of the patients in one place which can be

accessed by the delegated authorities. The way most of the servers and websites

generally operate, data can be similarly stored in a common database.

In the practical scenario, whenever a person gets admitted to any hospital, the

hospital can just check in the database and get all the information about them and also

update the data if required. In this manner, all the health institutions can work together

and get beneﬁted. For more than a decade, the exchange of health information has

been considered a key factor for effective and high-quality health care, and various

policy approaches have been used to build capacity in this area [3].

304 A. Mishra et al.

Fig. 3 Health information

exchange

HIE (Health Information Exchange) between hospitals and other health institu-

tions (Fig. 3) has shown to be a valuable asset and is already being implemented in

several countries. It permits doctors, nurses, pharmacists, and patients to have proper

electronic access to a patient’s important medical information, enhancing the safety,

quality, timeliness, and reducing the cost of patient treatment.

2.2 Problems Faced

Though a centralized database approach is easy to work on, there are problems that

occur in this approach for the Health Information Exchange system, which are:

•INTEROPERABILITY: Certain Hospitals might not want to share their recorded

data with any other health institution for business purposes.

•INFORMATION ASYMMETRY: Some health institutions have better access to

data than other parties.

•DATA BREACHES: Data might get corrupted in the midst of uploading or

tampered with within the database.

•Tests done by some hospitals might be incorrect for future use.

•Data might get outdated and unusable.

Relevance of Blockchain in Revolutionizing Health Records 305

3 Blockchain

3.1 Background

Blockchain is a special type of database. In other words, blockchain may be thought

of as a series of blocks linked together in such a manner that each block contains

some information as well as the addresses of the current and next connected blocks.

It is different from a typical database in the way it stores information. Whenever new

data has to be entered, it is entered into a new block. After the successful formation

of the new block, it is anchored to the previous block, which causes the data to be

linked in chronological order. The information stored in the blocks may vary from

blockchain to blockchain.

Let us walk through the stages of blockchain emergence to get a better idea of the

technology.

I. Blockchain 1.0

The concept of Distributed Ledger Technology (DLT) inspired the initial age of

blockchain [4]. A distributed ledger is a database that is jointly shared among

a number of participants as a consequence of some agreement, allowing public

witnesses to avoid duplicate spending instances.

The most demanding use of DLT was cryptocurrency, in which Bitcoin [5]

played a key role, and the crypto market was born. Since then, till now numerous

cryptocurrencies have emerged in the market.

A Bitcoin network as shown in (Fig. 4) generated a private and public key of the

owner and cryptographically hashed to each of the transactions made. Then all the

transactions were linked to one another through those hashes.

Despite its many accomplishments, Bitcoin has seen several signiﬁcant setbacks.

The Proof of Work (PoW) consensus method was primarily employed in the orig-

inal generation of blockchain, which needed the computation of hard mathematical

problems and high-performance computers. Because of its complexity, PoW takes a

long time and consumes a lot of energy, which is similar to the overall return.

II. Blockchain 2.0

Due to 1st generation blockchain’s lack of scalability and waste mining, the

blockchain idea was expanded beyond cryptocurrencies. As a result, the second

generation of blockchain, led by Ethereum, was born, based on novel smart contract

principles and the Proof of Work (PoW) consensus mechanism. Smart contracts [6]

are self-governing, self-regulating computer programs that run under present circum-

stances between two parties. These agreements cannot be hacked or altered in any

way. As a result, smart contracts [7] reduce the cost of veriﬁcation, enforcement, and

fraud prevention while also providing a clear contract speciﬁcation.

Ethereum has various advantages because it is mostly built on smart contracts

(Fig. 5). Smart contacts are quite accurate and explicitly store each phrase, which is

306 A. Mishra et al.

Fig. 4 Bitcoin network

Fig. 5 Smart contracts

Relevance of Blockchain in Revolutionizing Health Records 307

why Ethereum is so well deﬁned. Contracts are totally visible to all parties involved,

and their implementation is quick and efﬁcient.

However, smart contracts also cause a lot of difﬁculties for users, because they

sometimes prove to be extremely complex to write [8]. Any error in the contract’s

drafting might have unanticipated negative effects [9]. Once a problem in the code

has been exploited, the only effective method to prevent it is to establish a consensus

and rewrite the whole code. As a result, in order to get the most out of Ethereum,

smart contracts must be appropriately built and executed.

III. Blockchain 3.0

The 1st and 2nd generation blockchain technology was a huge advancement to the

data security sector. However, due to their proof of work-based work, they took hours

and hours to conﬁrm transactions and were therefore not scalable at all.

This gave rise to the current generation of blockchain, which introduced the notion

of Decentralized apps, or DApps. DApps are digital applications or programs that

are stored and executed on a blockchain or a peer-to-peer computer network, rather

than on a single computer (Fig. 6), and are therefore outside the reach and control of

the central authority.

As a result, utilizing techniques such as sharding, this generation can boost inter-

networking transactions. Sharding suggests that each blockchain node stores only a

piece of the data, rather than all of it.

It also uses Proof of Stake (PoS) and Proof of Authority (PoA) [10] consensus

processes to decrease time and boost processing power of smart contracts without

Fig. 6 Apps versus DApps

308 A. Mishra et al.

the need for additional transaction fees. There is no central authority responsible for

the third generation blockchain, which makes it very difﬁcult to ﬁnd a single source

of breakdown. Decentralized Applications (DApps) generally have a lightning-fast

transaction speed and are not bound to a single IP address, which in turn increases

the security that no single attacker can tamper with the data.

However, because of its decentralized nature, the third generation of blockchain

has few limitations, such as bug repairs and upgrades. The consensus procedures in

use are somewhat complicated.

IV. Blockchain 4.0

Though the 3rd generation has been in effect widely, still, some issues were observed

in it too, which has led to the ﬁrst stones of the fourth generation blockchain. It

would be the future of blockchain which could enhance the usage of blockchain with

other technologies like Artiﬁcial Intelligence (AI) and the Internet of Things (IoT).

This generation will allow for seamless integration across several platforms to be

disseminated under a single umbrella to fulﬁll industry and company requirements.

Table 1compares all the generations of blockchain [11] with respect to different

required parameters.

Tabl e 1 Generations of blockchain

Criteria’s Blockchain

1.0

Blockchain 2.0 Blockchain 3.0 Blockchain 4.0

Fundamental

concept

Distributed

ledger

technology

Smart contracts Decentralized

apps (DApps)

Blockchain with

Internet of things

and artiﬁcial

intelligence

Consensus

mechanisms

Proof of work

(PoW)

Assigned proof of

work (PoW)

Proof of stake

(PoS) and proof of

authority (PoA)

Proof of integrity

(PoI)

Validation

authority

Miners Miners and smart

contracts

In-built

veriﬁcation

mechanisms via

DApps

Computational

veriﬁcations via

sharding

Ease of use Hard Less moderate Moderate Easy

Cost Expensive Moderate Cheaper Cost effective

Implementation

areas

Economic

affairs

Non-economic

affairs

E-commerce

platforms

Smart-digital

technology

Relevance of Blockchain in Revolutionizing Health Records 309

3.2 Conclusive Solution

One promising solution to the security and accessibility of medical records is

blockchain technology. It can collect information, provide a secure and safe trans-

parent environment to store medical records, and improve the actual workﬂow of the

hospitals.

To begin with, blockchains are distributed ledgers. It connects hundreds of thou-

sands of computers (similar to the internet) which can all store/host encrypted copies

of those records. So, in preference to one record keeper, one is presented with

hundreds of thousands of record keepers called “nodes”. One report keeper can

tamper records, collude with third parties and commit fraud, however, now there are

1,000,000 nodes who’re maintaining an eye on each other.

As of now, health records are kept in electronic health records (EHRs) utilizing

specialized software and are centralized. It is a concern if a centralized ledger is

ever stolen, destroyed, or hacked in any manner. Keeping precise copies of ledgers

in several locations would be good protection, assuming that each duplicate was

a faithful, conﬁrmed, and identical replica of the original. This is referred to as a

distributed ledger.

Each of the many distinct digital “nodes” in a blockchain retains a full copy of the

ledger and validates the integrity of both freshly entered information and the copy it

stores on a regular basis. The information in the ledger is unaffected if one or more

of these nodes fail or are hacked.

As a result, as mentioned in Fig. 7the creation and introduction of a secure and

decentralized infrastructure for health records based on blockchain technology have

the potential to bring several advantages:

•It will give patients full access and rights to their data.

•Blockchain technology will provide reliability, convenience, and a level of security

commensurate with the importance of virtually free medical records.

Fig. 7 Blockchain technology

310 A. Mishra et al.

•The interoperability of the system will become more efﬁcient, allowing all

hospitals to obtain real and accurate historical medical data from their patients.

•All fragmented data will be symmetrical, linked, and readily available for future

research.

4 Proposed Method

Internally, the suggested solution takes advantage of blockchain technology, despite

the fact that the technique is not intended for large-scale storage. In the case of health

records, a decentralized storage solution would cast blockchain’s ﬂaws into sharp

relief.

A blockchain system can be thought of as an almost impenetrable encrypted

database in which sensitive medical information is stored. The system is managed

through a computer network that can be accessed by anyone using the program. It

works as a pseudo-anonymous system with accessibility concerns as all transactions

are visible to the public, despite the fact that it is inviolable in terms of data integrity.

Though this system (Fig. 8) might look a bit complicated at ﬁrst yet it is very easy

to understand. Let us ﬁrst walk through the steps of the process [12] and then discuss

the advantages.

Step 1: The users (patients, doctors, administrators, etc.) with the help of a user

application send an initial transaction to the Blockchain Handshaker.

Step 2: The Blockchain Handshaker creates a blockchain transaction and submits

it to the public blockchain network for veriﬁcation.

Fig. 8 Blockchain-based network

Relevance of Blockchain in Revolutionizing Health Records 311

Step 3: Smart contracts verify transactions in the public blockchain network and

miners add transaction data to the blockchain.

Step 4: After the transaction has been mined the validation is again sent back to

the Blockchain Handshaker for acknowledgment.

Step 5: The conﬁrmed transaction is sent to the cloud via the Blockchain

Handshaker.

Step 6: The transaction data is then stored in the cloud database.

And here are the major advantages of the above blockchain system:

•Data Access and Control

Blockchain provides the patient with the access key for all his records, giving him,

not the hospital, the right to all his data.

It has the capacity to fulﬁll the three key aspects of privacy [13] in the history of

medical records, which are:

(a) Data ownership: This framework focuses on making certain that customers

have the right to own and manage their private facts. As a result, the

system recognizes people as data owners and services as visitors with trusted

permissions.

(b) Fine-grained access control: Mobile applications often ask their users to allow

them a particular set of permissions to operate. This is a signiﬁcant issue

because these permissions are granted permanently, and the most convenient

method to change the settlement is by opting out. Instead, with this framework,

the individual may change the set of rights and remove access to their data at

any moment.

parent view of all the data being collected and shared. The roles and authority

over the data of course could be changed.

•Data Privacy and Security

In comparison to centralized systems, blockchain employs a decentralized and

immutable database system, which means there is no single point of failure or attack.

The data is not only encrypted using different hashing techniques but also sharing of

data is aimed to be kept as anonymous as possible [14].

Blockchain thus ensures the privacy of health records. The data might be public

on the server but the contents would still be anonymous.

•Data Updation

Though blockchain does not allow changes in the already existing blocks in the

chain, it allows new blocks to be added for new information. This might be quite a

cumbersome process at ﬁrst, but ultimately it allows regular updation of the health

care records which is a very important criterion while maintaining healthcare records.

•Data Storage

312 A. Mishra et al.

As we know, blockchain technology with Bitcoin was not designed to store data

initially. It was only designed to store information about the data. But, to maintain

healthcare records large chunks of data need to be stored. Using blockchain directly

to store all the healthcare records is not a feasible solution. Personal health data is

way larger in size than the amount of information recorded in Bitcoin or alternative

public blockchains [15].

Unlike Bitcoin, a sharding-based blockchain technology may grow its transaction

processing power as more users join the network by enabling various sets of nodes

to work concurrently [16–18].

As a result, a viable technique to this trouble could be to save the signiﬁcant

medical information in an off-chain storage system or on a cloud server, while the

hash reference is saved on-chain. It would be beneﬁcial if the storage system also

operated in a decentralized manner in order to have a truly decentralized system.

An established blockchain storage solution (Fig. 9) is completed by the use of

two inspiring technologies as sharding and swarming.

Sharding allows us to break up large amounts of data into small amounts of data.

These blocks of data are then encrypted and uploaded to the blockchain. They have

been distributed in such a way that this data is available even when part of the network

is operational. Even this small piece of data is encrypted and protected by a private

key that can only be accessed by the user who uploaded the data [19].

And swarming refers to the technology used to hold a group of fragments together.

A blockchain manages a network of nodes, while decentralized storage uses a

collective group of nodes known as swarms to store data.

Fig. 9 Blockchain storage

Relevance of Blockchain in Revolutionizing Health Records 313

5 Conclusion and Future Work

Blockchain despite having a few sets of problems and obstacles, is nonetheless, a

promising technology for allowing open and secure access to medical data. In terms

of knowledge and conventional technology, blockchain is still in its early phases of

development which is not yet robust enough for large-scale corporate application. We

examined how blockchain may beneﬁt the healthcare industry and how it could be

implemented for electronic health records in this analysis. Despite advancements in

the healthcare ﬁeld and technical advancements in electronic health record systems, a

few issues still persisted that were later solved with the help of this unique blockchain

technology.

The proposed architecture combines safe data storage with granular access to

data controls to provide a solution that is convenient to use and comprehend. It also

includes recommendations for how to verify that the system tackles the data storage

issue.

Plans for the future development include implementing a working prototype of

electronic medical records that follows speciﬁc norms and regulations such as contin-

ually developing low-level blockchain protocols like TCP/IP and HTTP, which are the

basis of today’s internet connectivity, as well as enhancing trust and decision-making

on the network to make the system more scalable with faster transaction throughput

without compromising security. Data storage is also a critical problem that must be

addressed in order to discover a reliable strategy for securely storing our sensitive

information. In the coming time and with further development, blockchain would

certainly offer us scalable and efﬁcient solutions for our concerns in the healthcare

industry.

References

1. Mayer, A. H., da Costa, C. A., & Righi, R. D. R. (2020). Electronic health records in a

blockchain: a systematic review. Health informatics journal, 26(2), 1273–1288.

2. Guo, R., Shi, H., Zhao, Q., & Zheng, D. (2018). Secure attribute-based signature scheme

with multiple authorities for blockchain in electronic health records systems. IEEE Access, 6,

11676–11686.

3. Kuperman, G. J. (2011). Health-information exchange: Why are we doing it, and what are we

doing? Journal of the American Medical Informatics Association, 18(5), 678–682.

4. Maull, R., Godsiff, P., Mulligan, C., Brown, A., & Kewell, B. (2017). Distributed ledger

technology: applications and implications. Strategic Change, 26(5), 481–489.

5. Tripathy, H. K., Mishra, S., Thakkar, H. K., & Rai, D. (2021). CARE: a collision-aware

mobile robot navigation in grid environment using improved breadth ﬁrst search. Computers &

Electrical Engineering, 94, 107327.

6. Macrinici, D., Cartofeanu, C., & Gao, S. (2018). Smart contract applications within blockchain

technology: a systematic mapping study. Telematics and Informatics, 35(8), 2337–2354.

7. Delmolino, K., Arnett, M., Kosba, A., Miller, A., Shi, E. (2016). Step by step towards

creating a safe smart contract: Lessons and insights from a cryptocurrency lab. In International

Conference on Financial Cryptography and Data Security, pp. 79–94. Springer

314 A. Mishra et al.

8. Chen, T., Li, X., Luo, X., Zhang, X. (2017). Under-optimized smart contracts devour your

money. In 2017 IEEE 24th International Conference on Software Analysis, Evolution and

Reengineering (SANER) (pp. 442–446). IEEE

9. Marino, B., Juels, A. (2016). Setting standards for altering and undoing smart contracts. In

International Symposium on Rules and Rule Markup Languages for the Semantic Web (pp. 151–

166). Springer.

10. De Angelis, S., Aniello, L., Baldoni, R., Lombardi, F., Margheri, A., Sassone, V. (2018). Pbft

vs proof-of- authority: applying the cap theorem to permissioned blockchain.

11. Mukherjee, P., & Pradhan, C. (2021). Blockchain 1.0 to blockchain 4.0—The evolu-

tionary transformation of blockchain technology. In Blockchain technology: applications and

challenges (pp. 29–49). Springer.

12. Mishra, S., Tripathy, H. K., Mallick, P. K., Bhoi, A. K., & Barsocchi, P. (2020). EAGA-MLP—

An enhanced and adaptive hybrid classiﬁcation model for diabetes diagnosis. Sensors, 20(14),

4036.

13. Zyskind, G., Nathan, O., et al. (2015). Decentralizing privacy: Using blockchain to protect

personal data. In Security and Privacy Workshops (SPW), 2015 (pp. pp. 180–184). IEEE.

14. Halpin, H., & Piekarska, M. (2017, April). Introduction to Security and Privacy on the

Blockchain. In 2017 IEEE European Symposium on Security and Privacy Workshops

(EuroS&PW) (pp. 1–3). IEEE.

15. Mishra, S., Thakkar, H., Mallick, P. K., Tiwari, P., & Alamri, A. (2021). A sustainable ioht

based computationally intelligent healthcare monitoring system for lung cancer risk detection.

Sustainable Cities and Society, 103079.

16. Zamani, M., Movahedi, M., & Raykova, M. (2018, October). Rapidchain: Scaling blockchain

via full sharding. In Proceedings of the 2018 ACM SIGSAC Conference on Computer and

Communications Security (pp. 931–948).

17. Mishra, S., Mishra, B. K., & Tripathy,H. K. (2015, December). A neuro-genetic model to predict

hepatitis disease risk. In 2015 IEEE International Conference on Computational Intelligence

and Computing Research (ICCIC) (pp. 1–3). IEEE.

18. Mishra, S., Panda, A., & Tripathy, K. H. (2018). Implementation of re-sampling technique to

handle skewed data in tumor prediction. Journal of Advanced Research in Dynamical Control

Systems, 10, 526–530.

19. Tripathy, H. K., Mallick, P. K., & Mishra, S. (2021). Application and evaluation of classiﬁcation

model to detect autistic spectrum disorders in children. International Journal of Computer

Applications in Technology, 65(4), 368–377.

A Systematic Review on Blockchain

Technology: Concepts, Applications,

and Prospects in Healthcare

Adarsh Tikmani, Saurabh Bilgaiyan, Bhabani Shankar Prasad Mishra,

and Santwana Sagnika

Abstract Blockchain technology is the “ﬁfth evolution” in the era of computing.

The characteristics of blockchain such as immutability, tamper-proof, decentral-

ized, private, and permissioned blockchain have attracted the world’s attention. This

paper briefs about the core concepts of blockchain, smart contracts, and architec-

ture behind the integration of IoT devices and blockchain. The proposed architecture

using the blockchain and Inter Planetary File System (IPFS) improves the way of

storing, handling, and sharing medical data and records. A comprehensive study on

blockchain-based smart applications across diverse sectors such as ﬁnancial, manu-

facturing, supply chain, healthcare sectors is also included in this paper. Finally,

the paper also enumerates challenges, limitations, and future trends in the area of

blockchain.

Keywords Blockchain ·Smart contract ·Ethereum ·Cryptography ·

Cryptocurrency

1 Introduction

As per the report of the World Economic Forum (2020) [1], by 2021 the signiﬁ-

cant increase of cybercrimes will potentially damage the economy to the scale of

US$6 trillion, which is nearly equal to the GDP of a country. Cybercrime cases

have surged during the Covid-19 pandemic [2]. Nowadays, data is more valuable

than oil [3], which means that privacy and transparency among the users will be

the most important requirements in the coming years. Users should have control

over their data and know where their data is being stored, eliminating the reliance

on third parties. Blockchain facilitates easy and trustworthy transactions across the

world, such as cross-border payments [4], smarter supply chains [5], dealing with

counterfeiting of products [6], decentralization of cloud storage [7], eHealth [8], and

much more [9]. Blockchain is a decentralized and distributed digital ledger where

A. Tikmani (B)·S. Bilgaiyan ·B. S. P. Mishra ·S. Sagnika

School of Computer Engineering, Kalinga Institute of Industrial Technology, Deemed to be

University, Bhubaneswar, Odisha 751024, India

e-mail: atikmani8@gmail.com

S. Mishra et al. (eds.), Augmented Intelligence in Healthcare: A Pragmatic and Integrated

Analysis, Studies in Computational Intelligence 1024,

https://doi.org/10.1007/978-981-19- 1076-0_17

315

316 A. Tikmani et al.

the blocks keep growing continuously, secured by cryptography and consensus algo-

rithms [10,11]. The exchange of money and digital goods between two peers over the

distributed systems using blockchain technology has gained much attention, where

the transaction once initiated, will be broadcast to the distributed network and after

the validation by the miners, a block would be found to complete the transaction.

Then the block would be appended in the existing chain which consists of the hash of

the previous block, making it tamper-proof [12]. To protect from malicious attackers,

a consensus mechanism is used as Proof of Work [13], Proof of Stake [14]. In simple

words, blockchain is a technology used in peer-to-peer architecture to maintain the

integrity [15]. The development of blockchain starts from Blockchain 1.0, based on

bitcoin. Blockchain 2.0 was about smart contracts, followed by Blockchain 3.0 which

focuses on blockchain applications in many sectors. In this paper, the authors have

covered minute details about the core concepts behind the blockchain. This paper

also proposes an integrated architecture of blockchain and IoT which is for storing,

handling, and sharing medical records and data. In addition, smart applications, and

smart initiatives by different countries with innovative research have been discussed

in detail. The authors aim to provide a comprehensive understanding of concepts of

blockchain to the novice reader and also put forth a novel architecture for blockchain

with the utility of IoT devices.

1.1 Why Distributed Systems?

The bottleneck issues of centralized systems are resolved by promising and trust-

worthy blockchain-based distributed systems. There are many challenges and limi-

tations of centralized systems, which are enough reasons for bringing distributed

systems to the real-world applications.

•Higher computing power: The computing power of distributed systems is more

than the centralized systems as it is the combination of the powers of the partic-

ipating peers in the network. It is also proven that the computational power

of the blockchain-based distributed system is relatively more than the isolated

supercomputers.

•No single point failure:AsinFig.1shown above, the distributed system does

not have one central authority, which means even if one of the computers failed

to operate, it would not affect the whole ecosystem. Also, the contributors to the

ecosystem are being rewarded [16], which attracts them to be constantly active

on the network.

•Greater transparency: The users do not know the exact location and status of

data. The blockchain technology over this network uses consensus mechanisms

and smart contracts to ensure the richness of data along with the greater privacy

of the users [17].

A Systematic Review on Blockchain Technology … 317

Fig. 1 Centralized system (left) versus distributed system (right)

•No third parties: Removal of third parties has increased the security and privacy.

Distributed systems using blockchain allow timestamping and validation of data

by the miners, to ensure the freshness and correctness of the data [18].

•Economical: The most prominent advantage of the distributed systems is their

low operating cost [19].

1.2 About the Study and Analysis

This section mentions the vision of writing this paper and its structure. It covers

how the authors have selected the paper on basis of inclusion and exclusion criteria

which have undergone a reﬁnement process further. The analysis of last ﬁve-year

publications on blockchain technology from reputed journals like Science Direct,

IEEE, etc. is also cited in this section.

1.2.1 Vision

In this paper, the authors have extensively comprehended the concepts of blockchain,

which have included every aspect of this technology from the history of the

blockchain to the latest trend going in the market. The authors have also focused

on what will be the impact on the daily lives of a people to the impact on the global

economy. Blockchain has become a revolutionary and promising technology. It is

318 A. Tikmani et al.

scope?

What is the state

of the art,

applications, and

current trends of

blockchain?

What is the

possibility of

blockchain

based secured

healthcare

system?

What are the global

application areas

of blockchain

technology and

future research

Fig. 2 Structure of the paper

widely adopted in ﬁnancial sectors, governmental services, and much more. The

main research questions that this paper addresses are

•RQ1—What is the state of the art, applications, and current trends of blockchain?

•RQ2—What is the possibility of blockchain-based secured healthcare system?

•RQ3—What are the global application areas of blockchain technology and future

research scope?

1.2.2 Structure of the Paper

Figure 2provides a quick view of the structure of this paper. Section 1introduces the

general idea of blockchain. Section 2discusses the state of the art, applications, and

current trends of blockchain, addressing RQ1. Section 3focuses on RQ2, a proposed

architecture integrating blockchain, IoT, and IPFS for healthcare sector. The global

application areas of blockchain technology and future research scope are discussed

in Sect. 4, answering RQ3. Section 5concludes the work.

1.2.3 Number of Published Papers

ThegraphshowninFig.3depicts the number of published papers related to

blockchain technology in the last ﬁve years. In this analysis, the authors include

journal and conference papers from Springer, ACM, IEEE, and Science Direct. As

shown in Fig. 3, the authors found that blockchain has attracted enormous interest

among the researchers in the last ﬁve years. This technology is increasingly becoming

a hot topic in every industry in recent times. The advancements in blockchain

technology are bringing smart applications and redeﬁning the traditional way of

transferring digital goods.

A Systematic Review on Blockchain Technology … 319

Fig. 3 Number of publishing in last ﬁve years

2 RQ1—What is the State of the Art, Applications,

and Current Trends of Blockchain?

2.1 History

In 1991 the vision of Haber and Stornetta [20] was the concept of timestamping the

data. In a distributed network there are many clients, each holding a unique identi-

ﬁcation number. The documents that are to be timestamped are sent by the clients

to the Timestamping Service or TSS, which appends the date and time. The TSS

records the data and time when the client receives the document and saves a copy

of that document to maintain integrity. This faces some backlash as it raises some

questions about the privacy of storing the data at the TSS and the time of docu-

ments being transmitted. Larger documents take a longer time to be timestamped.

The incorrect time stamping can also create distrust in the ecosystem. Using crypto-

graphically secure hash functions solves some problems. Instead of transmitting the

whole document, hash of the document h(x) =pwould be sent, and this reduces the

overall storage and transmission time of large ﬁles. The TSS would receive hashes

and append the date and time in request, then sign the document and send it to the

client, which removes the problem of storing records at TSS. To know the correct

sequence of requests, the involvement of bits from the previous sequence of client

requests was a solution. By using the hash function, the clients can generate the hash

for the documents and send it to the TSS, then the TSS sends the signed certiﬁcate

where the certiﬁcate

320 A. Tikmani et al.

Cn=(k,tn,idngn;sn)(1)

In Eq. 1,kis the sequence number, tnis the time, idnis the id number of the

client, gnis the hash value and snis the linking information of the certiﬁcates which

were issued previously. And when the next request has been processed the TSS sends

the identiﬁcation number for the next request. The timestamped document is later

checked by the challenger who would check if the timestamp is in the correct format

and call the next client (idn+1) asking him to produce his timestamp. This continues

as long as the challenger desires.

2.2 Market Trends

The characteristic of blockchain brings motivation among the companies and

consumers to adopt it. Blockchain technology is a revolution that not only elimi-

nates third parties but also creates trust among the people. As seen in Fig. 4,the

blockchain market size has seen a signiﬁcant increase [21]. Some more reports on

blockchain’s market from various sources are as follows. The global blockchain

technology market accounted for US$1977.1 million in terms of value in 2019 and

is expected to grow at CAGR of 58.7% for the period of 2019–2027. According

to Gartner 2019 [22], blockchain will facilitate the global tracking of goods and

services of US$2 trillion by 2023. According to the extensive research report by

MarketsandMarkets analysts [23], there is also seen a steep growth in agriculture

and food supply chains which is projected to grow from US$ 60.8 million in 2018

to US$ 429.7 million by 2023 at a CAGR of 47.8%.

The aviation market is expected to grow to nearly US$73 million by 2025 from

2019 at a CAGR of 27.1% during the forecast period. Global healthcare using

Fig. 4 Global market size of blockchain technology

A Systematic Review on Blockchain Technology … 321

blockchain technology is estimated to reach a new height to US$829 million by 2023.

According to a survey conducted by Deloitte [24] published in 2019, taking 1386

respondents from all over the world, blockchain technology is seen as a promising

technology in terms of greater security/lower risks. According to the Bloomberg

article [25], the ﬁnancial industry spends nearly US$1.7 trillion on blockchain

technology. According to another Bloomberg report, blockchain identity manage-

ment market is expected to touch US$11.46 billion by 2026. According to Allied

market research [26], the supply chain market using blockchain as an underlying

technology is to reach US$9.85 Billion by 2025 at 80.2% CAGR. One of the

topmost recruiting companies, Hired, has shown a signiﬁcant increase in demand

for blockchain engineers in 2018, which accounted for 517% from the previous year.

Importantly, these predictions show the progress of this innovative, emerging,

and promising technology that will make great contributions to the revolution of the

world industries in future.

2.3 Core Concepts

The blockchain is the underlying technology of bitcoin [27], which was invented

by Satoshi Nakamoto. In this section, the authors discuss the fundamental topics of

blockchain, such as cryptographic hash, Merkle tree, different types of cryptography,

content of the block, mining, different categories of blockchain, and its analysis.

2.3.1 Cryptographic Hash

Hashing and digital signatures are the foundations of blockchain. Hash generally

consists of digits and letters [28]. Hash functions are the programs that simply convert

the data into a hash value of ﬁxed length. SHA-256 is one of the well-known hash

functions [29]. These functions can quickly calculate and produce unique hash for

any type of data. There are many types of hashing that can be used for a bunch of data

containing a single hash. Repeated, combined, sequential, and hierarchical hashing

are some of the types. Hash reference basically navigates to the data as shown in

Fig. 5. It keeps track of whether the data is modiﬁed or not.

Figure 6gives clear evidence that the data has been manipulated in the past [30].

2.3.2 Merkle Tree

The structure shown in Fig. 7is called the Merkle tree, where a single hash is referring

to a couple of transactions. At ﬁrst, the hash reference of an individual transaction

is created and then H1 and H2 are coupled, referred by H12, shown on the right

side. The H12 and H34 pair is created and referred to by the root of the Merkle tree

denoted by Drescher [15]. This is the structure of a Merkle tree.

322 A. Tikmani et al.

Fig. 5 Hash reference

Fig. 6 Broken hash

reference

2.3.3 Different Types of Cryptography

The idea of cryptography is to make sure that only the authorized person can view

the data. There are basically two types of cryptography

•Symmetric Cryptography: First the data is encrypted using the key to convert it

into cipher text. Then the cipher text is decrypted using the same key which was

used while encrypting, as seen in Fig. 8. Here both the sender and receiver share

the same key for encryption and decryption [31].

•Asymmetric Cryptography: Asymmetric cryptography, shown in Fig. 9, consists

of two types of keys—public and private key for encryption and decryption of

the text, respectively [32]. In real scenario, node A wants to send a text that it

encrypts using the public key of node B. This message can only be decrypted by

A Systematic Review on Blockchain Technology … 323

Fig. 7 Merkle tree

Fig. 8 Symmetric

cryptography

324 A. Tikmani et al.

Fig. 9 Asymmetric

cryptography

node B from his private key. It becomes a lot safer and easier to communicate in

a large community.

2.3.4 Digital Signature

Simply speaking, blockchain stands on two pillars: hashing and digital signature.

Digital signature is a signature that employs asymmetric cryptography. By the

signature, one can verify the authenticity of the data [33].

(1) Signing the data digitally

Figure 10 tells about the message’s hash value 653EF882 which is created and

encrypted using the private key, thus obtaining the cipher text called as digital signa-

ture. The left box in the ﬁgure contains the cipher text and the message is called the

digitally signed message to the public.

(2) Verifying the data

The process of verifying starts with calculating the hash of the message received

and then decrypting the cipher text by its public key. If the hash obtained from the

A Systematic Review on Blockchain Technology … 325

Fig. 10 Digitally signed message

cipher text is the same as that of the calculated, then the message is not modiﬁed. It

is extremely useful in large paragraphs of text which contain thousands of letters.

2.3.5 Blocks

(1) Chain of transactions

Figure 11 talks about the layout of the blockchain data structure. As a new

transaction is generated, it is coupled with other transactions and packed in a

new block. BLOCK 1 is the ﬁrst block that does not have any previous reference

called genesis block [34] and BLOCK 2 has reference to the previous block

(BLOCK 1). This chain continues with the growing number of transactions.

Fig. 11 Blockchain data structure

326 A. Tikmani et al.

Fig. 12 Contents of the

block

H2 is termed as the head of the blockchain data structure as it is the reference

of the currently added block header. The manipulation in any transaction could

break the whole blockchain data structure [10].

(2) Detailed study about content of Block

The block is divided into two major parts (a) block header, (b) block body. In

blockchain, all the blocks are cryptographically linked, which is tamper-proof in

nature [33]. In Fig. 12, the contents of the block are shown—block number, the hash

of the previous block, timestamp, the root hash of the Merkle tree, the difﬁculty, and

the nonce. The block body contains the transactions that are validated [35].

2.4 Blockchain Categories

Blockchain is mainly divided into two types—public and private. Public or open

blockchain means here anyone can read and access the blockchain. Bitcoin is an

open blockchain. Private or closed blockchain means there are restrictions where the

authorized personnel could access the blockchain.

The blockchain is further divided into permissioned and permissionless. Permis-

sionless means anyone can validate and send the transactions. Permissioned

blockchain means only the authorized personnel are allowed to validate or execute

the transactions. Bitcoin is a public permissionless blockchain where anyone could

participate, and see the transactions logs, whereas Hyperledger [36] is a private

permissioned blockchain. Here only authorized personnel can make transactions or

see it. The admin decides who can participate in the blockchain system.

A Systematic Review on Blockchain Technology … 327

2.4.1 Analysis of Types of Blockchain

Here the authors have done an analysis of the different characteristics possessed by

the different types of blockchain. The characteristics such as authority, privacy of

the users, number of participants who can participate in the network, permission to

see the transactions logs, and who can participate in the consensus mechanism are

analyzed with the supporting examples which are currently trending in the market

(Table 1).

2.5 Mining

Nonce is the number which is to be solved by the miners to mine the block further.

There is a clear point that varying the nonce would change the hash of the block.

Tabl e 1 Different types of blockchain

Types Authority Privacy Participants Transactions

logs

Consensus

mechanism

Example

Public

permissionless

No Less Anyone All can see No

restriction

Bitcoin

Public

permissioned

No Less Anyone All can see Restriction Ripple, EOS

Private

permissionless

Yes High Restriction Restriction No

restriction

Holochain

Private

permissioned

Yes High Restriction Restriction Restriction Hyperledger

Fig. 13 Pool of hashes

328 A. Tikmani et al.

Fig. 14 Below the target is accepted

Figure 13 illustrates the pool of hashes from the smallest to largest region where

the ones which have many leading zeroes are situated in lower regions compared to

others.

The algorithm sets a target for the miners to accomplish the hash. Target is basically

difﬁculty level, considered in terms of leading zeroes of hashes [37]. The guessing

of nonce basically will make you reach to below the target. As illustrated in Fig. 14,

the hash below the target will be accepted and above the red line or target will be

rejected (not good for blockchain). After guessing the correct nonce, the miner will

be able to add the block and get rewards for doing it [38].

2.6 Distributed P2P Network

Once the block is mined, it is broadcasted in the network, further adding to every other

chain [39]. If one of the nodes is hacked in the network the peers would constantly

be checking, and this fraudulent activity would come into view. Most of the peers

having the non-hacked blockchain will be copied over that node’s blockchain. The

hackers should have above 51% of the nodes in the network to get control over the

network [40]. There are millions of nodes present in the bitcoin network, therefore

it is impossible to hack (Fig. 15).

2.7 Consensus Protocol

Consensus protocol primarily solves two major challenges—saving from attackers

and competing among the network [41].

If at the same time two nodes mine the next block, then there is consensus for

growing the blockchain without the fees being split among the two nodes [41]. There

A Systematic Review on Blockchain Technology … 329

Fig. 15 Distributed P2P network

are various types of consensus protocols. Two of the popular ones are Proof of Work

(PoW) and Proof of Stake (PoS).

1. Proof of Work (PoW)

It is the one described by bitcoin founder Satoshi Nakamoto [42] and was ﬁrst used

for bitcoin. Ethereum, IBM Blockchain, Ripple are using this consensus protocol.

The puzzle solved by the miners and investments of money and time is called

the Proof of Work. By adding the block successfully and validating the transactions,

miners are awarded accordingly. The electricity bill gets shot up due to high energy

consumption which obstructs anyone from doing malicious activities [43].

2.8 Blockchain Operation

The blockchain operation which is illustrated in Fig. 16 is as follows.

1. First the transaction is requested. Example Adarsh wants to send cryptocurrency

to Atul.

2. Secondly, the transaction is broadcasted to the P2P work for validation.

3. After the validation using public-key cryptography, the veriﬁed transactions

group together to form a new block.

4. Then a new block is appended with the existing blocks in blockchain.

5. The transaction is complete.

330 A. Tikmani et al.

Fig. 16 Illustration of blockchain operation

2.9 Smart Contract

Smart contracts are the codes stored in blockchain that execute when speciﬁc condi-

tions are met [44]. This was ﬁrst released by Nick Szabo as “computerized transaction

protocols that execute terms of a contract”.

In Fig. 17, Adarsh wants to sell his house on lease for ten years. So Adarsh demands

10,00,000 rupees. Now he is identiﬁed by his unique address in the network (983EF3)

and he uses a smart contract with his digital signature. Then Adarsh leaves his home

for sale and the house has his own address (784TRE) as public key which is kept in

the blockchain. Atul is a buyer searching on the blockchain network to ﬁnd a suitable

house and is interested to buy this house. Atul needs to sign the contract using his

Fig. 17 Smart contract

implementation

A Systematic Review on Blockchain Technology … 331

private key to transfer 10,00,000 rupees from his address (45TY6GH) to Adarsh’s

address.

The smart contract is broadcasted on the network to validate the smart contract

[45] for checking that Adarsh is the sole owner of the house and Atul has sufﬁcient

funds to pay. After the nodes validate the contract, Atul automatically gets the access

code of the building smart lock. The ledger is maintained that for now Atul is staying

over there or owning that particular house.

2.10 Beneﬁts of Blockchain

There are several beneﬁts of blockchain-like immutability, security, transparency.

•Immutability: Blockchain transactions, once recorded, are unchangeable, as the

blocks are cryptographically linked. The new block contains the hash of the

previous block which makes it stronger and more secure [46].

•Availability: Blockchain is available at all times. It does not have a downtime or

power off button.

•Transparency: All the transactions are viewable to all the nodes and are vali-

dated by the nodes. There is greater transparency which maintains the integrity

of blockchain [47].

•Secure: The blockchain system is secure because of its characteristics. Like public

and private keys, consensus protocols, smart contracts. Using smart contracts, one

can have a high level of security of his payments and the entities belonging to him

[48].

2.11 Cryptocurrency

Cryptocurrency is digital money or asset that is secured cryptographically. Once the

money is sent to someone, it cannot be brought back. Cryptocurrency eliminates

third-party involvement [49]. It facilitates cross-border payments in seconds and

with incredibly low fees.According to Wikipedia, the number was over 1600 and

by March 2020 it has grown to 5576 cryptocurrencies. Figure 18 shows that bitcoin

is the most dominant in the market has a market cap of US$173.99 as of June 2020

followed by Ethereum, tether, XRP, and Polkadot.

2.12 Other Key Terms

•Double Spending Problem: Suppose A is a person transferring owner rights to

B. This will be recorded on the ledger in P2P system. It needs to be updated

which takes time. Meanwhile, A quickly approaches another person and demands

332 A. Tikmani et al.

Fig. 18 The market share of

different cryptocurrencies

based on market cap

to transfer ownership. This situation is called Double Spending Problem [50].

Blockchain can solve the Double Spending Problem.

•Zero-Knowledge Proofs: Bitcoin is an open public ledger where it cannot

offer privacy to users [51]. Zerocoin uses zero-knowledge proofs. It uses zero-

knowledge succinct non-interactive arguments of knowledge. It promotes privacy

as it hides the transaction amounts and coins value held by the users.

2.13 Smart Blockchain Applications

Blockchain’s applications are not only limited to bitcoin and cryptocurrencies. It has

also helped existing ﬁelds to turn more secure [52]. In this section, the authors discuss

recent blockchain-enabled smart applications, their working principles, and some of

the best services offered by the companies. Figure 20 shows the smart applications

represented in an innovative ﬂowchart design (Fig. 19).

2.13.1 Blockchain-Based Smart Applications

This section explores all the smart and innovative applications based on blockchain

technology. The numerous applications in sectors such as automotive, healthcare,

supply chain, city, and agriculture are brieﬂy discussed.

1. Smart Vehicles

A Systematic Review on Blockchain Technology … 333

Fig. 19 Smart applications

Company

Fig. 20 Framework of the distribution of updates

334 A. Tikmani et al.

With the increase in demand for intelligent and autonomous vehicles, the compa-

nies need to prevent cars from misusing the data and manipulating the systems. Here

we discuss the various types of secured applications used to keep vehicles safe and

meet the customers’ growing demand in the nearer future.

•Blockchain-based driving behavior detection and warning to surrounding vehi-

cles

The smart terminals in this proposed framework [53] are the smartphones, which

collect the data and do the identiﬁcation and analysis. The smart terminals then

upload the driving information to the edge node, which is deployed throughout the

city as transactions and packs all of them into a block. For communicating with the

other trafﬁc, the edge node issues certiﬁcations and keys. The group managers who

are usually the edge nodes in the group create a secure pathway for communication.

After creating, the public key is broadcasted to the surrounding trafﬁc. Then the

vehicles submit the relevant authentication details through the blind signature [53].

In the next phase, the manager issues a certiﬁcate by which the status information

is received, and public key and relevant communication happens through the edge

nodes. Meanwhile, in communication, the edge nodes create blocks that are sent to the

veriﬁcation server adding to the blockchain. Through this approach, the malicious

vehicles can be traced easily by the private key which is embedded in the group

signature. The regulators feel convenient to track the vehicles and analyze the data

ﬂow seamlessly.

•A novel public blockchain for used motor vehicle record system

Masoud et al. [54] proposed a public blockchain to keep the records of used motor

vehicles. It basically removes the concerns of the customers buying second-hand

vehicles, like is it good enough, why is the owner selling, etc. In this framework

there are four types of smart contracts (not the Ethereum smart contract) used—(1)

First the owner of the vehicle generates public-private key pairs using the vehicle’s

unique identiﬁcation number and a unique number using touch pattern or mouse

pattern. The private key should be kept safe with the owners and the public key for

the organizations (insurance, pollution, repairing shops) (2) This report shows the

history of the vehicle’s relations with the organizations and the ID of the speciﬁc

organizations. It also shows the change of the ownership.(3) The organizations having

one pair of public keys serves as the digital signatures and becomes trustworthy to

share the details to update the record. (4) In this contract, the owner grants the new

user which results in the generating of the new pair of keys with the help of the

vehicle’s unique identiﬁcation number and new user public key. Secondly, when the

insurance organization issues a new insurance, it updates the record by making a new

transaction to the block with prior permission. At last, permission granting allows

the user to show the detailed report of the vehicle. To construct the chain, the nodes

in the network should broadcast to the other nodes present in the alive nodes list.

When one of the ﬁrst nodes solves the complex problems of ﬁnding the nonce value,

it broadcasts to the network. With the agreement of more than 50% of the nodes in

the organizational nodes, the node utilizes the value to generate blocks and update

A Systematic Review on Blockchain Technology … 335

the chain. Subsequently, the user nodes also update the chain on getting data from

any of the organizational nodes.

•Firmware updates in AVs

Baza et al. [55] have proposed a robust model for the new ﬁrmware updates for

the Autonomous vehicles (AVs) using the blockchain technology. A consortium

blockchain is created by the manufacturer which has access to write a smart contract

by including the hash code for the AVs. Here there are two types of AVs—Respon-

ders and Distributors. Every AVs has a set of cryptographic keys and public param-

eters embedded in the vehicles during the manufacturing by the manufacturer. The

distributors are rewarded by the manufacturers for their contribution in distributing

the new updates to the responders. Moreover, the smart contracts manage the reward

and are responsible for the reputation of distributors. Here the authors have also

used attribute-based encryption which ensures that only authorized AVs have access

to request and receive the new updates from the manufacturer. Additionally, zero-

knowledge proofs are also set up. The distributor and responder have an exchange

of the encrypted version of the latest ﬁrmware updates and in return, the distributor

gets the proof of distributions by the responder AVs. These proofs are submitted to

the smart contract by the distributor for its reputation. Finally, the responder gets the

decryption key from the smart contract. The framework shown in Fig. 21, talks about

the distribution of the new updates to the AVs. The blue vehicles are the distribu-

tors, and the red ones are the responders. The green arrow means the companies

sending the updates to the repudiated distributor and the red arrow shows the distri-

bution of decryption keys. Black arrow shows the creation of smart contracts by the

companies in the blockchain network, and the white arrow depicts the exchange of

communication between the vehicles.

•Automatic selection of charging stations based on blockchain technology

Pustišek et al. [56] proposed a unique platform for automobile charging where there

are three actors involved—(1) Driver, (2) Electric vehicle, and (3) Charging stations.

The drivers who select the route in the vehicle’s infotainment system, generate a list

of the charging stations in that route. This information is passed to the Ethereum

address of the charging stations to blockchain contract (vehicle). The blockchain

contract then sends the bid which includes a key and value pair that tells the offered

price to charge stations smart contract, ﬁnally two stations are seen and when the

durations of bid are over, the blockchain contract of the vehicle sends the request for

the conﬁrmation of best bid from the driver. This beneﬁts the driver for charging at

an affordable price and providers to manage the bookings effectively.

Summary

The main reason for the adoption of blockchain technology is due to the growing

demand for intelligent AVs and Energy vehicles. The cyber-attacks on the centralized

systems cause havoc in the ecosystem. The incident of remotely hacking the car over

the internet like control over the air conditioning systems, and brakes, could create

bad consequences for the driver. The deployment of blockchain can efﬁciently resolve

336 A. Tikmani et al.

Fig. 21 The blockchain-based framework is used to store and access data in healthcare sectors

the problem in terms of security [57], safely dispatching new updates to the vehicle,

and communication [58].

2. Smart Healthcare

The revolution of digital health records has brought huge beneﬁts to people where

they can easily store the records and share them with other institutions. This even-

tually poses many risks [59]. The insurance claims validated by the third-party are

more prone to risks of how they deal with the personal data of the people. Blockchain

technology can make tamper-proof and immutable healthcare systems.

•Anti-Fraud Health Insurance service

The key issue is tampering of bills and reimbursement of the health insurance. Liu

et al. [60] proposed blockchain-enabled anti-fraud healthcare insurance service. The

distributed network helps the patient to avail services from anywhere and the agencies

to detect ﬂaws in the insurance. The authors use the permissioned blockchain where

hospitals and insurance agencies are involved. The hospitals can upload prescriptions

and reports as per the patient’s unique id. As the healthcare insurance agencies receive

the application it will enquire the relevant information registered using the unique id.

A Systematic Review on Blockchain Technology … 337

The bill paid by the patient will be recorded in the blockchain, and can be enquired

by the agencies for insurance reimbursement.

•Health Data accessing and sharing

Sharma et al. [61] proposed an architecture where each beneﬁciary having a smart

card can avail services under PM-JAY Scheme. A six-digit security pin along with the

hash of their national identiﬁcation card will be used to generate public-private keys.

To gain access to the record, the hospitals submit the proof with re-encryption keys

and get the EHRs hashes stored in the IPFS. Here the access control layer is secured

by Zk-snarks setup [62]. By these hashes, it will get encrypted by symmetric keys

EHRs. After giving a request, the cloud server performs checks on the re-encryption

keys and grants access to the hospital.

•Blockchain-based secure storage

Ito et al. [63] proposed a private blockchain called i-blockchain. In the i-blockchain

architecture, researchers use a permissioned blockchain where hospitals, individuals,

insurance companies are involved. They have also used a fusion of cold and hot

storage systems. The cold storage is an off-blockchain storage. So, after diagnosis,

the healthcare institution encrypts the data using the individual’s public key on request

and then sends the encrypted data to healthcare hot storage. After that, the individual

gets data in the cold storage as shown in Fig. 22, further decrypting using the private

key.

Chakraborty et al. [64] proposed a framework for integration of Blockchain, IoT,

and Machine learning. Two types of blockchain are used, the personal healthcare

blockchain maintained by the individual which receives and senses the data from

wearable devices, and the external record management blockchain which records

data by hospitals and doctors. This is often stored in the external database controlled

by the blockchain network. Here the machine learning module works to detect any

anomaly, and a notiﬁcation is sent to the doctor.

Summary

The main motive of the companies shifting to blockchain technology is because of a

myriad of data breaches and millions of health records being freely accessible on the

internet. So, the blockchain-enabled smart healthcare can provide a secure platform

to store and share the patient history, given prior permission [65].

3. Smart Supply Chain

Supply chain is the most important element of globalization. Blockchain is now

being used to make it smart and transparent enough [66]. This will help the supplier,

manufacturers, and the end consumers. Various researchers and companies have

bagged attention of most of the shipping companies throughout the world to invest

in this emerging technology.

•Smart Container and using Blockchain

338 A. Tikmani et al.

Fig. 22 The integrated architecture

Hinckeldeyn and Jochen [67] proposed a smart and innovative approach to tackle

the growing problems in supply chains. They have built smart storage containers

equipped with blockchain and IoT. A smart storage container is well equipped with

the weight and IoT sensors. It can order the items autonomously when the number

of items is low. After getting the correct quantity of the order placed, the payment is

processed to the supplier, with all information recorded on the blockchain governed

by smart contract.

•Integrating Blockchain and ERP

Kaid and Eljazzar [68] have integrated blockchain and ERP Systems where there is

a transparency and trackability of the items’ quantity or any defects. There will be a

smart contract agreement between the distributors and the retailers for the payment.

As the sales exceed 50%, the distributors automatically get the money according to

the agreement. And the distributor can also track whether the inventory is sold or not.

Here the consumers can scan the QR code and access the item details, edit through

smart contracts if there are any defects. A user proﬁle stores all the transactions as

A Systematic Review on Blockchain Technology … 339

Tabl e 2 Analysis of research

papers Management

systems

Traceability and

transparency

Trade

Wu et al. [10]

Yue an d F u [70]

Ekawati et al.

[71]

Malik et al. [72]

Perez et al. [73]

Bhatnagar et al.

[74]

Kaid and Eljazzar [68]

Baig et al. [75]

El-Sayed et al. [76]

one user proﬁle data. The integration allows easy supply chain movement where

all the data updated in ERP will be recorded as transactions. It is developed over

Hyperledger Fabric.

•Analysis of research papers related to supply chain

The distinct features of blockchain technology provide a way to securely trade, make

a transparent management system, and ease of traceability. Table 2provides the most

notable academic papers for the topic mentioned. In the management systems section,

Wue et al. [ 69] propose a supply chain management system, giving a brief about its

functionality. Yue and Fu [70] discuss the medical supply chain management and

Ekawati et al. [71] propose a supply chain management system for sugarcane based

on blockchain technology. In the traceability and transparency section, Malik et al.

[72] propose Trustchain, a secured framework that uses a consortium blockchain

that upholds the integrity in the supply chain. Perez et al. [73] discuss the analysis

of the implementation of blockchain technology to address the problems of the food

industry in Peruvian social programs. Bhatnagar et al. [74] speciﬁcally discuss the

traceability systems. In the trade section, Kaid et al. [68] propose a framework to

automate the payment for the trade happening between two parties. Baig et al. [75]

cover the minute details about IoT and blockchain-based peer-to-peer energy trading

platforms. The growing demand in energy trading needs to be addressed. Sayed et al.

[76] present a peer-to-peer energy trading platform that gives a unique approach to

deal with the problems in this market.

Summary

The main motive is to make the supply chain a trusted and encrypted mechanism

to move the goods in a secured way [77]. The illegal procurement, stealing from

the inventory, non-trusted third-party vendors, hacking of the IoT sensor can be

controlled by innovative solutions based on blockchain technology in the near future.

4. Smart City

Increasing interest among the people to develop smart cities and bringing up large

projects related to smart cities poses risks like DDoS attacks [78]. Here the authors

discuss how the integration of blockchain will ensure an encrypted and robust system.

•Hybrid Architecture for Smart City

Sharma et al. [79] proposed a hybrid architecture where the IoT producing raw data is

processed by the edge network which has limited storage and computational power to

340 A. Tikmani et al.

do this activity. The processed encrypted data is transferred to the distributed network

where the miners validate the transaction. After getting the previous hash (zero, if

none) the miner adds the block to the blockchain. Finally, the updated blockchain is

sent to all the nodes followed by sending the requested services to the IoT devices

or the users.

•Smart city with 5G and blockchain

Noh and Kwon [80] have shown the potential of integration of 5G and blockchain

technology which provides transparent, dispersible, and secure architecture. Here

they have discussed the companies providing the authentication services using

blockchain as public-key infrastructure and the processing speed of blockchain

increasing by the deployment of 5G technology.

Summary

The main motive to include this application is the growing cyber-attacks on the

systems which disrupt the whole ongoing services. Blockchain technology brings the

most secure video surveillance, intelligent trafﬁc systems, etc. in reality to experience

the smart city [81].

5. Smart Agriculture

In India agriculture accounts for 15% of total Gross Domestic Product (GDP) [6] and

6.4% of total global economic production. Using blockchain technology with IoT

will help the government, farmers, lenders, buyers [7]. It will encourage the farmers

and create more opportunities for them.

•Smart ﬁsh farming

In the ﬁsh farming conceptual architecture [82], the end-users get an overview of

what is really being done on a farm. Here four components are involved—ﬁsh farm,

cloud storage, end-user, blockchain network. The researcher has used a permissioned

blockchain network (Hyperledger Fabric) to build the platform. The ﬁsh tanks are

well equipped with IoT sensors like water level sensors and actuators for controlling

the operations. The data collected by IoT sensors are organized and optimized by

the data processing module. Then the optimized data is added to the blockchain. The

access to the blockchain is limited to the farmers such as only reading the sensed

data, whereas the farm owner has full access. The smart contract can identify the

person who can write or read.

Summary

Excessive use of illegal pesticides in farms causes a huge loss in India. The blockchain

technology emerges as a driver in limiting this [83]. The engagement of IoT sensors

with the blockchain technology can sense the amount of pesticides and verify the

origin, processing, and manufacturing processes [84]. Table 3gives an overview of

various applications.

A Systematic Review on Blockchain Technology … 341

Tabl e 3 Blockchain-based smart application

Category Ref Title Main contributions Limitations Achievement

Smart healthcare Jiang et al. [85]Health data

Exchange

A BlocHIE, platform

blockchain enabled for

exchanging of the health

record

There are no such

limitations

A secured framework

is proposed and

explained precisely

Tripathi et al. [86] Secured and Smart Health

System (S2HS)

A whole framework for the

secured and smart health

care

A prototype of the

framework is not included

in this research

An S2HS proposed

blockchain-based

healthcare system is

included

Omar et al. [87]MediBchain A healthcare data

management system for

patient using blockchain

technology

The deployment of this

system is not mentioned

The high level of the

platform is included

Smart Supply chain Papathanasiou

et al. [88]

Blockchain-based supply

chain

A brief study on the

blockchain enabled shipping

industry

There are no such

limitations

Beneﬁts and inhibitors

of this adoption are

mentioned

Chiacchio et al.

[89]

Application for pharma

supply chain

An application for the

tracing of the goods in the

supply chain smartly and

securely

There are no such

limitations

Prototype of the Dapp

is included in this

research

Xu et al. [90]Management platform An Ethereum

blockchain-enabled

management platform for

manufacturing industry

supply chain

Performance evaluation of

the system lacks in this

research

The concepts are

mathematically

explained

(continued)

342 A. Tikmani et al.

Tabl e 3 (continued)

Category Ref Title Main contributions Limitations Achievement

Smart city Deshpande et al.

[91]

SaFe A blockchain framework to

protect vehicles

There are no such

limitations

Algorithms are

included in this

research paper

Ren et al. [92] Intelligent trafﬁc A blockchain network for

intelligent management of

trafﬁc

Performance evaluation

lacks in this paper

Security concerns are

addressed in this paper

França et al. [93] Waste management A blockchain platform for

the solid waste management

There are no such

limitations

Prototype snapshots

are included in this

paper

Smart agriculture Basnayake and

Rajapakse [94]

Organic food supply

chain

A blockchain-enabled to

ensure the food quality and

its origin

Scalability of the system is

not mentioned in detail

The screenshot of the

interface is included in

this paper

Xu et al. [95]Blockchain application

on agri-foods

A whole transparent

blockchain system for

agri-foods

There are no such

limitations

A detailed case study is

analyzed on the

agricultural foods

Bechtsis et al.

[96]

Containerized food

supply chain

Blockchain technology

enabled

containerized food supply

chain

More elaborative

implementation is needed

The blockchain

framework is included

in this paper

A Systematic Review on Blockchain Technology … 343

3 RQ2—What is the Possibility of Blockchain-Based

Secured Healthcare System?

Today almost everything is connected to the internet. The use of IoT has skyrocketed

in recent years [97]. The data generated from IoT devices need to be secured and

handled properly [98]. The raw data from IoT devices are stored in the blockchain

layer and can be used in future. In this section, the authors propose a novel architecture

that integrates IoT with blockchain technology. In this architecture, the data is stored

in the IPFS and can be shared seamlessly.

3.1 The Proposed Architecture

The proposed architecture shown in Fig. 18 consisting of three layers will provide the

platform for the latest blockchain applications. Integrating the IPFS with blockchain

will allow to store the data in a decentralized and secured manner with the blockchain

ecosystem.

1. IoT Layer: IoT sensors will collect the required information and the chips

installed in the sensors are able to transmit the data to the nearest router or

access points. The computers with high computational power will be able to

run the IoT network and can maintain a blockchain. Here the IoT device can

achieve communication under the management of a blockchain network.

2. Blockchain Layer: The processed data from the IoT sensors will be recorded

on the network and can be used for further assistance in smart applications.

The Blockchain ecosystem comprises of

•Shared Ledger: It is the space shared by the nodes to validate the transactions,

recording, and storage purposes.

•Smart Contract: It will leverage the security among the users who are busy, on

agreement. For example, in smart healthcare, there is a smart contract by the

patient to the hospitals or clinics for the use of the Electronic Health records.

•Consensus: This is necessary for the network to decide whether they are storing

the data or not. And also to ensure that the correct transactions are being validated.

•Cryptography: Here the public–private key plays a major role in the network.

Suppose while sending the tokens, in the blockchain layer, the data will be stored

in IPFS [99], where all the nodes will participate in renting their space to the

clients. The miner will get a reward on behalf of it.

•Application layer: Many sectors can beneﬁt from this architecture like health-

care, smart city, supply chain, transportation, and agriculture. The integration of

blockchain and IoT will maximize efﬁciency and reduce the losses. For example—

A Universal Platform where all the citizens can store their prescriptions, check-up

reports, receipts, and bills in one place securely.

344 A. Tikmani et al.

Case Study

It will beneﬁt both customers and the insurance company. The IoT sensor can harvest

the data of patients by wearable devices, instruments in hospitals and will generate

the report which is recorded in the blockchain. The patient can share the details with

the healthcare institution which is also a participating node in the blockchain. The

payments can be automated by integrating smart contracts. If the speciﬁc conditions

set by the patients are met, then the money will be deducted from their account

automatically.

4 RQ3—What Are the Global Application Areas

of Blockchain Technology and Future Research Scope?

4.1 Platforms, Services Offered by Companies

Today blockchain has become an important technology, the companies are focusing to

develop many solutions deployed on the network with help of blockchain technology.

In this section, the authors review some of the innovative services offered by the

companies across the world.

4.1.1 Blockchain-Based Cloud Storage

Cloud storage has become a vital part of the lives of people like storing images,

documents, identities, certiﬁcates, and many more [100]. But clouds possess high

risks such as single point failure, building huge architecture setup, and lack of user

control over their data. Blockchain is seen as a revolutionary technology in these

sectors [101].

1. IPFS

IPFS [99] is a distributed system to store ﬁles, even host websites. IPFS identiﬁes the

matter someone is looking at by the content not the location, i.e., content addressing.

Basically, if someone wants to get the data it will search in the whole network by

using the hash of that ﬁle.

2. Filecoin

Filecoin [102] is one level up from bitcoin, where they offer the rewards to miners not

only for mining the blocks but also for renting their unused storage. In the network,

ﬁrst, the storage miners pledge their space and when the client and storage providers

meet their agreements, the clients send the data to the provider. When the data is

received, they both sign a deal, and it is recorded in blockchain.

3. Storj

A Systematic Review on Blockchain Technology … 345

Storj [103] is also offering a similar kind of service where the people can rent their

space. Storj uses Ethereum blockchain which stores metadata. The data of the clients

are generally stored in three nodes and an application Metadisk veriﬁes that the data

is untampered. Proof of Space is used as the consensus protocol in Storj.

Many companies like Storj, BigChainDB, Swarm, Sia have also started offering

decentralized cloud storage platforms.

4.1.2 Blockchain-As-A-Service (BaaS)

Many companies are offering BaaS throughout the world to the consumers without

the infrastructure installations [104]. TCS, a global leading IT service, is hugely

investing in blockchain to offer services. TCS has launched Quartz blockchain

services. IBM Blockchain platform is built upon the Hyperledger Fabric [36]. It

allows users to grow their network and an easy-to-manage interface. Amazon,

Microsoft Azure Blockchain, Huawei, Baidu, Alibaba, Oracle, Hewlett Packard,

SAP, R3, Infosys, TCS, Wipro are some of the top companies working on the

blockchain-related services.

4.2 Development Across the World

The increasing frauds and cybercrimes throughout the globe have pushed nations to

adopt certain measures to curb this major issue. It generally hurts the overall nation’s

gross domestic product. Many innovative solutions from different parts of the world

are coming forward to address the issues by adopting blockchain technology [105].

4.2.1 Developments on Blockchain in Different Parts of the World

1. India

India, one of the fastest-growing countries in the world is also taking much interest

in the blockchain deployment. In January 2020, NITI Aayog [106], released a draft

discussing blockchain, named Blockchain: The India Strategy. The NITI Aayog

focuses on ease of business, governance, and living. They proposed that blockchain

can be used for organic farming, insurance, and much more.

Andhra Government is working on digitizing the land registry sector. Every land

paper will have blockchain QR code, and all the transactions will be recorded on

blockchain, which no one can tamper with. Some 40,000 lands have been recorded

till now. This will be greatly beneﬁcial for farmers as they do not want to pay any

fees.

2. Singapore

346 A. Tikmani et al.

Singapore, a leading global ﬁnancial center, is heavily investing in blockchain

technology to revolutionize digital payments, storage, and its existing infrastructure.

Project Ubin is one of the topmost priority projects of the Government of Singapore.

•Opercerts Platform

Opercerts platform beneﬁts employers, local educational institutions, companies, and

job candidates [107]. Opencerts platform is a blockchain-based developed platform

in which each certiﬁcate is stored digitally, each having a unique hash for secure

veriﬁcation. After completing and graduating from institutions, students will have

their digital certiﬁcates available on their platform. This digital certiﬁcate can easily

be veriﬁed by the institutions to detect any signs of tampering. This would reduce

administrative processes and paperwork for students. It will enable the Singaporeans

to track their current skill in a single repository.

•Project Ubin

Project Ubin is a noticeably big project aimed to solve the challenges faced by the

ﬁnancial industry using the blockchain and Distributed Ledger Technology. More

than forty ﬁnancial and non-ﬁnancial members are working on this project [108].

3. USA

DARPA is building one robust platform for the transmission of messages to and

from personnel using blockchain technology. This will also help in communication

between the military headquarters also.

JP Morgan Chase, one of the leading banks in the world, said that it would use

Quorum built using the Ethereum network to issue a digital currency JP Morgan coin

for instantaneous and secure payments [109]. And there have been talks to merge

with Consensys, a Brooklyn blockchain-based startup.

4.3 Challenges

•Daily Payments

Bitcoin which is running on the blockchain technology is not deferrable, which means

if someone has sent a sum of money it cannot be retrieved back, and it is anonymous

too. Using cryptocurrencies in daily shopping would cause such types of problems.

Several altcoins like Dai, Tether are ﬂourishing in the market which is pegged to

the US Dollar. Their price remains to be non-volatile to some extent which solves

the problem of volatility that becomes a roadblock for people to use cryptocurrency

in their daily shopping.

•Energy Usage

The energy usage to mine a block in blockchain-based cryptocurrency remains a

problem for the miner [110] and, for the countries as it becomes a huge burden

A Systematic Review on Blockchain Technology … 347

for production of energy. Huge amount of electricity bills are generated due to

consumption for blockchain processing.

The shifting to Proof of Stake consensus algorithm signiﬁcantly reduces the energy

consumption as it does not include much processing.

•Scalability

Scalability remains a weakness of the blockchain, where one block can be mine in

less than 10 minutes. In contrast to the average throughput which is nearly seven

transactions per second in bitcoin, Visa handles more than thousands of transactions

per second [111].

Polkadot and Avalabs are leading blockchain projects that claim to have greater

scalability and interoperability which could solve the existing issues.

•Standardization

It will create problems for the people and the service providers as all the providers

may not have the same architecture and protocols.

ISO is working on the standardization of the blockchain and DLT standards

ISO/TC 307. This will be a great achievement for people as it will be easy for

them to switch between other platforms.

5 Future Research Scope

5.1 AI

Growing fake news and rumors is a huge problem for our society. Upadhyay [112]

propose an AI blockchain platform for ﬁghting against the fake news to value and

promote the truth.

5.1.1 Bigdata

Zhaofeng et al. [113], have shown concerns about the existing big data trust system

as not perfect and to solve this problem, the authors have chosen blockchain for the

management and all the operations related to the data. The author has proposed a

decentralized trust management system of IoT bigdata.

5.1.2 UAVs

Nowadays, Unmanned Aerial Vehicles (UAVs) have become special and important

in the industry. From use in farming, delivery of medicines, pipe inspections, and

348 A. Tikmani et al.

much more have evolved over time. Applications of UAVs are increasing day by day

and secure communication between them is necessary [114].

6 Conclusion

There is a signiﬁcant increase in the number of research and developments on

blockchain due to its immutability, tamper-proof and peer-to-peer nature. Blockchain

has numerous applications beyond bitcoin and altcoins. Blockchain has the capa-

bility to change today’s payment systems into more secure and reliable. The motive

of writing this paper is to provide core concepts of blockchain technology such

as basics about the blocks, smart contracts, and some of the amazing and robust

applications. The proposed architecture of the IoT and blockchain with IPFS facil-

itates in storing, handling, and sharing of medical records and data securely. Smart

applications can cause a huge impact on the global economy. The authors discuss

the importance of the applications covering important sectors. The research work

done by the authors, platforms, and services offered by the companies, and vision

of different countries have also been surveyed. Finally, the authors summarize the

challenges, future trends, and research.

References

1. World Economic Forum. Wild Wide Web. [online] https://reports.weforum.org/global-risks-

report-2020/wild-wide-web/. Accessed Oct 2 2020.

2. Ronak, D. D. (2020). Cybercrime In India surges amidst coronavirus lockdown. https://www.

forbes.com/sites/ronakdesai/2020/05/14/cybercrime-in-india-surges-amidst-coronavirus-loc

kdown/#6631190d392e. Accessed Sept 21, 2020.

3. The Economist, The world’s most valuable resource is no longer oil, but data. https://www.

economist.com/leaders/2017/05/06/the-worlds-most-valuable-resource-is-no-longer-oil-but-

data. Accessed Sept 30, 2020.

4. Kommu, N., Aghila, G. (2020). Securing online bank’s big data through block chain tech-

nology: cross-border transactions security and tracking. Security, Privacy, and Forensics

Issues in Big Data, 247–263.

5. Azzi, R., Chamoun, K. R., Sokhn. M. (2019). The power of a blockchain-based supply chain.

Computers & Industrial Engineering, 135, 582–592.

6. Liu, Z., Li, Z. (2020). A blockchain-based framework of cross-border e-commerce supply

chain. International Journal of Information Management.

7. Loss, S., Cacho, N., Valle, M. J., Lopes, F. (2019). Orthus: a blockchain platform for smart

cities’. In 2019 IEEE International Smart Cities Conference (ISC2) (pp. 212–217).

8. Mishra, S., Tripathy, H. K., Mallick, P. K., Bhoi, A. K., & Barsocchi, P. (2020). EAGA-

MLP—an enhanced and adaptive hybrid classiﬁcation model for diabetes diagnosis. Sensors,

20(14), 4036.

9. Ueyama, J., Krishnamachari, B., Faiçal, B. S., Aguiar, E. J. D. (2020). A survey of blockchain-

based strategies for healthcare. ACM Computing Surveys, 53(2). https://doi.org/10.1145/337

6915.

A Systematic Review on Blockchain Technology … 349

10. Kumar, S., Shrivastava, V. (2019).Utilizing block chain technology in various application areas

of machine learning. In International Conference on Machine Learning, Big Data, Cloud and

Parallel Computing (COMITCon) (pp. 167–171).

11. Wu, M., Wang, K., Cai, X., Guo, S., Guo, M., & Rong, C. (2019). A comprehensive survey

of blockchain: from theory to IoT applications and beyond. IEEE Internet of Things Journal,

6(5), 8114–8154. https://doi.org/10.1109/JIOT.2019.2922538

12. Zheng, Z., Xie, S., Dai, H., Chen, X., Wang, H. (2017). An overviewof blockchain technology:

architecture, consensus, and future trends. In 2017 IEEE International Congress on Big Data

(BigData Congress) (pp. 557–564).

13. Tripathy, H. K., Mishra, S., Thakkar, H. K., & Rai, D. (2021). CARE: A collision-aware

mobile robot navigation in grid environment using improved breadth ﬁrst search. Computers &

Electrical Engineering, 94, 107327.

14. Righi, R. D. R., Alberti, A. M., & Singh, M. (2020). Blockchain technology for Industry 4.0

(1st ed.). Springer.

15. Nguyen, C. T., Hoang, D. T., Nguyen, D. N., Niyato, D., Nguyen, H. T., & Dutkiewicz, E.

(2019). Proof-of-stake consensus mechanisms for future blockchain networks: Fundamentals,

applications and opportunities. IEEE Access., 7, 85727–85745.

16. Drescher, D. (2017). Blockchain basics: A non-technical introduction in 25 steps (1st edn.).

Frankfurt am Main.

17. Zhang, K., Vitenberg, R., Jacobsen, A. H. (2018). Deconstructing blockchains: concepts,

systems, and insights. In DEBS ‘18: Proceedings of the 12th ACM International Conference

on Distributed and Event-based Systems (pp. 187–190).

18. Chang, V., Baudier, P., Zhang, H., Xu, Q., Zhang, J., Arami, M. (2020). How Blockchain

can impact ﬁnancial services—The overview, challenges and recommendations from expert

interviewee. Technological Forecasting and Social Change, 158.

19. Wilczynski, A., Kolodziej, J. (2020). Modelling and simulation of security-aware task

scheduling in cloud computing based on Blockchain technology. Simulation Modelling

Practice and Theory, 99.

20. Swierczewski, L. (2012). The distributed computing model based on the capabilities of the

Internet. Networking and Internet Architecture.

21. Haber, S., & Stornetta, S. W. (1991). How to time-stamp a digital document. Journal of

Cryptology, 3, 99–111.

22. Liu, S. (2020). Blockchain technology market size worldwide 2018–2025. https://www.sta

tista.com/statistics/647231/worldwide-blockchain-technology-market-size/. Accessed Sept

30, 2020.

23. Gartner Blockchain Technology: What’s Ahead?. https://www.gartner.com/en/information-

technology/insights/blockchain. Accessed Sept 23, 2020.

24. MarketsandMarkets. (2018). Blockchain in agriculture market.https://www.marketsandma

rkets.com/PressReleases/blockchain-agriculture-food-supply-chain.asp.

25. Pawczuk, L., Massey, R., Holdowsky, J. (2019). Deloitte’s 2019 global blockchain survey.

https://www2.deloitte.com/content/dam/Deloitte/se/Documents/risk/DI_2019-global-blockc

hain-survey.pdf.

26. Wolf, J. (2018). Financial services industry spends $1.7 billion on blockchain. Bloomberg.

https://www.bloomberg.com/news/articles/2018-06-12/ﬁnancial-services-industry-spends-

1-7-billion-on-blockchain

27. Allied Market Research (AMR), Blockchain Identity Management Market to Reach $11.46

Bn Globally by 2026 at 79.2% CAGR. https://www.bloomberg.com/press-releases/2019-12-

04/blockchain-identity-management-market-to-reach-11-46-bn-globally-by-2026-at-79-2-

cagr-allied-market-research.

28. Cho, S., Lee, S. (2019). Survey on the application of blockchain to IoT. In International

Conference on Electronics, Information, and Communication (ICEIC) (pp. 1–2).

29. Sosu, R. N. A., Quist-Aphetsi, K., Nana, L. (2019). A decentralized cryptographic blockchain

approach for health information system. In 2019 International Conference on Computing,

Computational Modelling and Applications (ICCMA), 2019 (pp. 120–1204). https://doi.org/

10.1109/ICCMA.2019.00027

350 A. Tikmani et al.

30. Juma, H., Shaalan, K., & Kamel, I. (2019). A survey on using blockchain in trade supply

chain solutions. IEEE Access, 7, 184115–184132.

31. Yassein, M. B., Shatnawi, F., Rawashdeh, S., Mardin, W. (2019). Blockchain technology:

Characteristics, security and privacy; issues and solutions. In 2019 IEEE/ACS 16th Interna-

tional Conference on Computer Systems and Applications (AICCSA) (pp. 1–8).

32. Chandra, S., Paira, S., Alam, S. S., Sanyal, G. (2014). A comparative survey of symmetric and

asymmetric key cryptography. In International Conference on Electronics, Communication

and Computational Engineering (ICECCE) (pp. 83–93).

33. Castaldo, L., & Cinque, V. (2018). Blockchain-based logging for the cross-border exchange

of eHealth Data in Europe. International ISCIS Security Workshop, 821, 46–56.

34. Mishra, S., Thakkar, H., Mallick, P. K., Tiwari, P., & Alamri, A. (2021). A sustainable IoHT

based computationally intelligent healthcare monitoring system for lung cancer risk detection.

Sustainable Cities and Society, 103079.

35. Vallois, V., Guenane F. A. (2017). Bitcoin transaction: from the creation to validation, a

protocol overview. In 1st Cyber Security in Networking Conference (CSNet) (pp 1–7).

36. Gabriel, T., Cornel-Cristian, A., Arhip-Calin, M., Zamﬁrescu, A. (2019). Cloud storage. a

comparison between centralized solutions versus decentralized cloud storage solutions using

Blockchain technology. In 54th International Universities Power Engineering Conference

(UPEC) (pp. 1–5).

37. HyperLedger. (2018). An Introduction to Hyperledger.https://www.hyperledger.org/wpcont

ent/uploads/2018/07/HL_Whitepaper_IntroductiontoHyperledger.pdf

38. Ishan, Bhulania, P., Raj, G. (2018). Analysis of cryptographic hash in blockchain for bitcoin

mining process. In International Conference on Advances in Computing and Communication

Engineering (ICACCE) (pp.105–110).

39. Ghimire, S., Selvaraj, H. (2018). A survey on bitcoin cryptocurrency and its mining. In 26th

International Conference on Systems Engineering (ICSEng) (pp. 1–6).

40. Zhang, L., Xie, Y., Zheng, Y., Xue, W., Zheng, X., & Xu, X. (2020). The challenges and

countermeasures of blockchain in ﬁnance and economics. Systems Research and Behavioral

Science, 37, 691–698.

41. Kaur, A., Nayyar, A., Singh, P., Blockchain: a path to the future. Cryptocurrencies and

Blockchain Technology Application.

42. Sharma, K., Jain, D. (2019). Consensus algorithms in blockchain technology: a survey. In

10th International Conference on Computing, Communication and Networking Technologies

(ICCCNT) (pp. 1–7).

43. Tripathy, H. K., Mallick, P. K., & Mishra, S. (2021). Application and evaluation of clas-

siﬁcation model to detect autistic spectrum disorders in children. International Journal of

Computer Applications in Technology, 65(4), 368–377.

44. Adeyemi, A., Yan, M., Shahidehpour, M., Botero, C., Guerra, V. A., Gurung, N., (Calvin)

Zhang, L., Paaso, A., & Paaso, A. (2020). Blockchain technology applications in power

distribution systems. The Electricity Journal, 33.

45. Gallersdörfer, U., Klaaßen, L., & Stoll, C. (2020). Energy consumption of cryptocurrencies

beyond bitcoin. Joule, 4, 1843–1846.

46. Mohanta, B. K., Panda, S. S., Jena, D. (2018). An overview of smart contract and use cases in

blockchain technology. In 9th International Conference on Computing, Communication and

Networking Technologies (ICCCNT) (pp. 1–4).

47. Tanwar, S., Parekh, K., Evans, R. (2019). Blockchain-based electronic healthcare record

system for healthcare 4.0 applications. Journal of Information Security and Applications, 50.

48. Frizzo-Barker, J., Chow-White, P. A., Adams, P. R., Mentanko, J., Ha, D., Green, S. (2020).

Blockchain as a disruptive technology for business:A systematic review. International Journal

of Information Management, 51.https://doi.org/10.1016/j.ijinfomgt.2019.10.014

49. Benisi, Z. N., Aminian, M., Javadi, B. (2020). Blockchain-based decentralized storage

networks: a survey. Journal of Network and Computer Applications, 162.

50. Ghosh, A., Gupta, S., Dua, A., Kumar, N. (2020). Security of cryptocurrencies in blockchain

technology: state-of-art, challenges and future prospects. Journal of Network and Computer

Applications, 163.

A Systematic Review on Blockchain Technology … 351

51. Reiff, N. (2020). How does a block chain prevent double-spending of Bitcoins? https://www.

investopedia.com/ask/answers/061915/how-does-block-chain-prevent-doublespending-bit

coins.asp

52. Miers, I., Garman, C., Green, M., Rubin, D. A. (2013). Zerocoin: anonymous distributed

E-cash from bitcoin. In IEEE Symposium on Security and Privacy, pp. 397–411.

53. Aggarwal, S., Chaudhary, R., Aujla, S. G., Kumar, N., Kim-Kwang, R. C., & Zomaya, Y.

A. (2019). Blockchain for smart communities: applications, challenges and opportunities.

Journal of Network and Computer Applications., 144, 13–48.

54. Bai, H., Wu, C., Yang, Y., Xia, G., Jiang, Y. (2019). A blockchain-based trafﬁc conditions and

driving behaviors warning scheme in the internet of vehicles. In 2019 IEEE 19th International

Conference on Communication Technology (ICCT) (pp. 1160–1164). https://doi.org/10.1109/

ICCT46805.2019.8947312

55. Masoud, Z. M., Jaradat, Y., Jannoud, I., Zaidan, D. (2019). CarChain: a novel public

blockchain-based used motor vehicle history reporting system. In IEEE Jordan Interna-

tional Joint Conference on Electrical Engineering and Information Technology (JEEIT)

(pp. 683–688).

56. Baza, M., Nabil, M., Lasla, N., Fidan, K. M. M., & Abdallah M. (2019). Blockchain-based

ﬁrmware update scheme tailored for autonomous vehicles. In IEEE Wireless Communications

and Networking Conference (WCNC) (pp. 1–7).

57. Pustišek, M., Kos, A., Sedlar, U. (2016). Blockchain based autonomous selection of elec-

tric vehicle charging station. In International Conference on Identiﬁcation, Information and

Knowledge in the Internet of Things (IIKI) (pp. 217–222).

58. Singh, M., Kim, S. (2017). Introduce reward-based intelligent vehicles communication using

blockchain. In 2017 International SoC Design Conference (ISOCC) (pp. 15–16).

59. Kim, K. T., Kim, I. Y. J. (2020). Blockchain-based information sharing between smart vehicles

for safe driving. In 2020 IEEE 91st Vehicular Technology Conference (VTC2020-Spring)

(pp. 1–2). https://doi.org/10.1109/VTC2020-Spring48590.2020.9128995

60. Tandon, A., Dhir, A., Islam, N. A. K. M., Mäntymäki, M. (2020). Blockchain in healthcare: a

systematic literature review, synthesizing framework and future research agenda. Computers

in Industry, 122.

61. Liu, W., Yu, Q., Li, Z., Li, Z., Su, Y., & Zhou, J. (2019). A blockchain-based system for

anti-fraud of healthcare insurance. In IEEE 5th International Conference on Computer and

Communications (ICCC) (pp. 1264–1268).

62. Sharma, B., Halder, R., & Singh, J. (2020). Blockchain-based interoperable healthcare

using zero-knowledge proofs and proxy re-encryption. In International Conference on

Communication Systems & NETworkS (COMSNETS) (pp. 1–6).

63. Pinto, A. M. (2020). An introduction to the use of zk-SNARKs in blockchains. Mathematical

Research for Blockchain Economy, 233–249.

64. Ito, K., Tago, K., Jin, Q. (2018). i-Blockchain: a blockchain-empowered individual-centric

framework for privacy-preserved use of personal health data. In 2018 9th International Confer-

ence on Information Technology in Medicine and Education (ITME), 2018 (pp. 829–833).

https://doi.org/10.1109/ITME.2018.00186

65. Chakraborty, S., Aich, S., Kim, H. C. (2019). A secure healthcare system design framework

using blockchain technology. In 21st International Conference on Advanced Communication

Technology (ICACT) (pp. 260–264)

66. Farouk, A., Alahmadi, A., Ghose, S., & Mashatan, A. (2020). Blockchain platform for

industrial healthcare: Vision and future opportunities. Computer Communications, 154,

223–235.

67. Dutta, P., Choi, T.-M., Somani, S., Butala, R. (2020). Blockchain technology in supply chain

operations: Applications, challenges and research opportunities. TransportationResearch Part

E: Logistics and Transportation Review, 142.

68. Hinckeldeyn, J., Jochen, K. (2018). Developing a smart storage container for a Blockchain-

based supply chain application. In 2018 Crypto valley conference on blockchain technology

(CVCBT) (pp. 97–100).

352 A. Tikmani et al.

69. Kaid, D., Eljazzar, M. M. (2018). Applying blockchain to automate installments payment

between supply chain parties. In 2018 14th International Computer Engineering Conference

(ICENCO) (pp. 231–235).

70. Wu, M., Wang, K., Cai, X., Guo, S., Guo, M., & Rong, C. (2019). A comprehensive survey

of blockchain: From theory to IoT applications and beyond. IEEE Internet of Things Journal,

6, 8114–8154.

71. Yue, Y., Fu, X. (2020). Research on medical equipment supply chain management method

based on blockchain technology. In 2020 International Conference on Service Science (ICSS)

(pp. 143–148).

72. Ekawati, R., Arkeman, Y., Suprihatin, S., Sunarti, C. T. (2020). Design of intelligent decision

support system for sugar cane supply chains based on blockchain technology. In 2020 2nd

International Conference on Industrial Electrical and Electronics (ICIEE) (pp. 153–157).

73. Malik, S., Dedeoglu, V., Kanhere, S. S., Jurdak, R. (2019). TrustChain: trust management

in blockchain and IoT supported supply chains. In 2019 IEEE International Conference on

Blockchain (Blockchain) (pp. 184–193).

74. Pérez, D., Risco, R., Casaverde, L. (2020). Analysis of the implementation of blockchain as a

mechanism for digital and transparent food traceability in Peruvian social programs. In 2020

IEEE XXVII International Conference on Electronics, ElectricalEngineering and Computing

(INTERCON) (pp. 1–4).

75. Bhatnagarm, S. R., Jha, M. S., Singh, S. S., Shende, R. (2020). Product traceability

using blockchain. In 2020 2nd International Conference on Advances in Computing,

Communication Control and Networking (ICACCCN) (pp. 891–895).

76. Baig, A. J. M., Iqbal, T. M., Jamil, M., Khan, J. (2020). IoT and blockchain based peer to peer

energy trading pilot platform. In 2020 11th IEEE Annual Information Technology, Electronics

and Mobile Communication Conference (IEMCON) (pp. 0402–0406).

77. El-Sayed, I., Khan, K., Dominguez, X., Arboleya, P. (2020). A real pilot-platform imple-

mentation for blockchain-based peer-to-peer energy trading. In 2020 IEEE Power & Energy

Society General Meeting (PESGM) (pp. 1–5).

78. Patelli, N., & Mandrioli, M. (2020). Blockchain technology and traceability in the agrifood

industry. Journal of Food Science, 85(11), 3670–3678.

79. Biswas, K., Muthukkumarasamy, V. (2016). Securing smart cities using blockchain tech-

nology. In 2016 IEEE 18th International Conference on High Performance Computing and

Communications; IEEE 14th International Conference on Smart City,IEEE 2nd International

Conference on Data Science and Systems (HPCC/SmartCity/DSS) (pp. 1392–1393).

80. Sharma, P. K., & Park, J. H. (2018). Blockchain based hybrid network architecture for the

smart city. Future Generation Computer Systems, 86, 650–655.

81. Noh, J., Kwon, H. (2019). A study on smart city security policy based on blockchain in 5G age.

In 2019 International Conference on Platform Technology and Service (PlatCon) (pp. 1–4).

82. Ministry of Statistics and Programme Implementation (2018–2019), Planning Commission,

Government of India (2004–05 series). (2019). Sector-wise contribution of GDP of India.

http://statisticstimes.com/economy/sectorwise-gdp-contribution-of-india.php

83. Umamaheswari, S., Sreeram, S., Kritika, N., Prasanth, J. R. D. (2019). BIoT: blockchain based

IoT for agriculture. In 2019 11th International Conference on Advanced Computing (ICoAC)

(pp. 324–327).

84. Hang, L., Ullah, I., & Kim, D.-H. (2020). A secure ﬁsh farm platform based on blockchain

for agriculture data integrity. Computers and Electronics in Agriculture, 170.

85. Mirabelli, G., & Solina, V. (2020). Blockchain and agricultural supply chains traceability:

Research trends and future challenges. Procedia Manufacturing, 42, 414–421.

86. Chun-Ting, P., Meng-Ju, L., Nen-Fu, H., Jhong-Ting, L., & Jia-Jung, S. (2020). Agricul-

ture blockchain service platform for farm-to-fork traceability with IoT sensors. In 2020

International Conference on Information Networking (ICOIN) (pp. 158–163).

87. Jiang, S., Cao, J., Wu, H., Yang, Y., Ma, M., He, J. (2018). BlocHIE: A blockchain-based

platform for healthcare information exchange. In 2018 IEEE International Conference on

Smart Computing (SMARTCOMP) (pp. 49–56).

A Systematic Review on Blockchain Technology … 353

88. Tripathi, G., Ahad, M.A., & Paiva S. (2020). S2HS—a blockchain based approach for smart

healthcare system. Healthcare 8.

89. Omar, A. A., Rahman, M. S., Basu, A., Kiyomoto, S. (2017). MediBchain: a blockchain based

privacy preserving platform for healthcare data. In International Conference on Security,

Privacy and Anonymity in Computation, Communication and Storage.

90. Papathanasiou, A., Cole, R., Murray, P. (2020). The (non-)application of blockchain

technology in the Greek shipping industry. European Management Journal.

91. Chiacchio, F., Compagno, L., D’Urso, D., Velardita, L., & Sandner, P. (2020). A decentralized

application for the traceability process in the pharma industry. Procedia Manufacturing, 42,

362–369.

92. Xu, Z., Liu, Y.,Zhang, J., Song, Z., Li, J., Zhou, J. (2019). Manufacturing industry supply chain

management based on the ethereum blockchain. In 2019 IEEE International Conferences on

Ubiquitous Computing & Communications (IUCC) and Data Science and Computational

Intelligence (DSCI) and Smart Computing, Networking and Services (SmartCNS) (pp. 592–

596).

93. Deshpande, V., George, L., Badis, H. (2019). SaFe: a blockchain and secure element

based framework for safeguarding smart vehicles. In 2019 12th IFIP Wireless and Mobile

Networking Conference (WMNC) (pp. 181–188).

94. Ren, Q., Man, K. L., Li, M., Gao, B. (2019). Using blockchain to enhance and optimize IoT-

based intelligent trafﬁc system. In 2019 International Conference on Platform Technology

and Service (PlatCon) (pp. 1–4).

95. França, A. S. L., Neto, A. J., Gonçalves, F. R., & Almeida, C. M. V. B. (2020). Proposing the

use of blockchain to improve the solid waste management in small municipalities. Journal of

Cleaner Production, 244.

96. Basnayake, B. M. A. L., & Rajapakse, C. (2019). A blockchain-based decentralized system

to ensure the transparency of organic food supply chain. In 2019 International Research

Conference on Smart Computing and Systems Engineering (SCSE) (pp. 103–107).

97. Xu, J., Guo, S., Xie, D., & Yan, Y. (2020). Blockchain: A new safeguard for agri-foods.

Artiﬁcial Intelligence in Agriculture, 4, 153–161.

98. Bechtsis, D., Tsolakis, N., Bizakis, A., & Vlachos, D. (2019). A Blockchain framework for

containerized food supply chains. Computer Aided Chemical Engineering, 46, 1369–1374.

99. Mishra, S., Mishra, B. K., & Tripathy, H. K. (2015, December). A neuro-genetic model

to predict hepatitis disease risk. In 2015 IEEE International Conference on Computational

Intelligence and Computing Research (ICCIC) (pp. 1–3). IEEE.

100. Humayun, M., Jhanjhi, NZ, Alsayat, A., Ponnusamy, V. (2020). Internet of things and

ransomware: Evolution, mitigation and prevention. Egyptian Informatics Journal, 22.

101. Benet, J. (2020). IPFS—Content addressed, versioned, P2P ﬁle system. https://ipfs.io/ipfs/

QmR7GSQM93Cx5eAg6a6yRzNde1FQv7uL6X1o4k7zrJa3LX/ipfs.draft3.pdf. Accessed

Sept 20 2020.

102. Agarwal, A., Siddharth, S., & Bansal, P. (2016). Evolution of cloud computing and related

security concerns. In Symposium on Colossal Data Analysis and Networking (CDAN) (pp. 1–

9).

103. Zheng, Q., Li, Y., Chen, P., Dong, X. (2018). An innovative IPFS-based storage model for

blockchain. In 2018 IEEE/WIC/ACM International Conference on Web Intelligence (WI)

(pp. 704–708).

104. Protocol Labs. (2017). Filecoin: a decentralized storage network.https://ﬁlecoin.io/ﬁlecoin.

pdf

105. Storj Labs, Inc. (2018). Storj: a decentralized cloud storage network framework. https://storj.

io/storj.pdf.

106. Zhao, J. L., Fan, S., Yan, J. (2016). Overview of business innovations and research

opportunities in blockchain and introduction to the special issue.Financial Innovation, 2,

28.

107. Weking, J., Mandalenakis, M., Hein, A., Hermes, S., Böhm, M., Krcmar, H. (2019). The

impact of blockchain technology on business models—a taxonomy and archetypal patterns.

Electron Markets 30, 285–305.

354 A. Tikmani et al.

108. NITI Aayog, Blockchain: The India Strategy. https://niti.gov.in/sites/default/ﬁles/2020-01/

Blockchain_The_India_Strategy_Part_I.pdf

109. Lee, T. (2019). Singapore to roll out blockchain-based education certiﬁcation system nation-

wide. https://www.techinasia.com/singapore-rolls-blockchainbased-education-certification-

system-nationwide.

110. Opare, E. A., & Kim, K. (2020). A compendium of practices for central bank digital currencies

for multinational ﬁnancial infrastructures. IEEE Access, 8, 110810–110847.

111. Mishra, S., Panda, A., & Tripathy, K. H. (2018). Implementation of re-sampling technique

to handle skewed data in tumor prediction. Journal of Advanced Research in Dynamical and

Control Systems, 10, 526–530.

112. Upadhyay, N. (2020). Demystifying blockchain: A critical analysis of challenges, applications

and opportunities. International Journal of Information Management, 54.

113. Rankhambe, B. P., Khanuja, H. K. (2019). A comparative analysis of blockchain platforms—

bitcoin and ethereum. In 2019 5th international conference on computing, communication,

control and automation (ICCUBEA), pp. 1–7.

114. Shae, Z., Tsai, J. (2019). Ai Blockchain platform for trusting news. In 2019 IEEE 39th

International Conference on Distributed Computing Systems (ICDCS) (pp. 1610–1619).

115. Zhaofeng, M., Lingyun, W., Xiaochang, W., Zhen, W., Weizhe, Z. (2020). Blockchain-enabled

decentralized trust management and secure usage control of IoT Big Data. IEEE Internet of

Things Journal, 7, 4000–4015.

116. Alladi, T., Chamola, V., Sahua, N., Guizani, M. (2020). Applications of blockchain in

unmanned aerial vehicles: a review. Vehicular Communications, 23.

Integrated Machine Learning Models

for Enhanced Security of Healthcare

Data

Shasank Periwal, Tridiv Swain, and Sushruta Mishra

Abstract Healthcare practice aims to improve and change each individual’s physical

and mental life cycle, helping individuals toward a longer life, as well as avoidance

of mental and physical sickness, illness, and infection. Hospitals and clinics are

examining for state-of-art Machine learning algorithm approaches that will suit the

expansion of information technology and evaluate a large quantity of complicated

information, according to a study in the ﬁeld of Healthcare. The proposed model

enhances the patient’s data’s security by analyzing the application user’s movement

and scrolling pattern and classiﬁes it as suspicious or unsuspicious using a Convolu-

tional Neural Network and Random Forest Classiﬁcation approach. If the outcome

indicates suspicious behavior, the user’s proﬁle will be locked and a warning notice

will be sent to higher authorities right away. The model also utilizes Time Series

Analysis to compute the scope of the clinical equipment used by the patient, and if it

is altered within that scope, it will be allowed to change the value; otherwise, it will

require authentication from two or more senior doctors to verify there is no harm.

The key advantage of utilizing the model is the added high-security function, which

protects data better than any other system now available. It can reduce human labor

to almost 0% in terms of monitoring security cameras all day, saving organizations

money by allowing the algorithm to handle the heavy lifting. As machine learning

algorithms gather experience, their accuracy and efﬁciency improve, allowing them

to make better decisions.

Keywords Machine learning ·Cloud storage ·Data security ·Healthcare ·

Convolutional neural network (CNN) ·Random forest classiﬁer ·Time series

analysis

S. Periwal ·T. Sw ai n ·S. Mishra (B)

Kalinga Institute of Industrial Technology, Deemed to be University, Bhubaneswar, Odisha, India

e-mail: sushruta.mishrafcs@kiit.ac.in

S. Mishra et al. (eds.), Augmented Intelligence in Healthcare: A Pragmatic and Integrated

Analysis, Studies in Computational Intelligence 1024,

https://doi.org/10.1007/978-981-19- 1076-0_18

355

356 S. Periwal et al.

1 Introduction

In today’s scenario, the health sector is transitioning from volume-based business to

value-based business which requires great efforts from doctors, nurses, and staff to

make treatment more effective, productive, and the patients’ data secure.

It has been more than a decade since healthcare storage and data have been more

perplex for a single cause that is an enormous amount of data being given freshly

along with constant changes of technology and mobile software and new problems

and diseases [1]. Nowadays people are suffering from mental trauma and depression,

so our healthcare sector has believed that machine learning in healthcare is deﬁnitely

required for proper management of an enormous amount of data that is perplex which

can direct us to reﬁne healthcare facilities and assist the medical team to move to a

higher extent of productivity and task will be accurate.

Medical experts are making promises for quality medical outcomes as well as

reducing the time needed to examine healthcare data by holding systems updated and

categorizing medical data in a rational structure along with retrieving and accessing

patients’ former data [2].

In a research in the ﬁeld of healthcare, hospitals and clinics are studying for new

Machine learning algorithm approaches that will suit the advancement of informa-

tion technology and survey a huge amount of perplex data. The proposed approach

is recommended rather than suggested, since it will ease and enhance healthcare

practice, by allowing systems to use cloud computing along with Machine Learning

approaches thus making the whole process secure such that sensitive patient data

don’t get leaked and the patient stays safe. This technique is promising for superior

results and is more beneﬁcial if it’s applied correctly and properly.

The objective of this paper is to study how to combine Machine Learning algo-

rithms with the application used in healthcare to store patients’ data and to make it

more secure and the use of the application developed with it. The model proposed

utilizes a Convolutional Neural Network. It is a Deep Learning algorithm that can

take an input picture, relegate signiﬁcance (comprehendable weights and biases) to

different viewpoints/objects in the picture, and be capable to differentiate one from

the another which will be used here to detect mobile/camera pointing towards the

screen and to examine client’s expression and classify with Random Forest classiﬁer.

It is a classiﬁcation algorithm consisting of many decisions trees. It utilizes sacking

and component arbitrariness when establishing individual tree to attempt to generate

an uncorrelated woodland of trees whose forecast by board of trustees is more precise

than that of any individual tree] strategy to dissect the user’s movement and looking

over design and group it to be dubious or unsuspicious. If suspicious behavior is

deduced from the result, then it will lock up the users’ proﬁle and will immediately

send a warning message to the higher authorities. The model will also calculate the

scope of the clinical device used by the patient using Time Series Analysis [Time

series analysis requires developing models that best capture or express an observed

time series to comprehend the fundamental causes. This ﬁeld of study looks for

the “why” behind a time series dataset. Predictions are made for new data when

Integrated Machine Learning Models … 357

the genuine result may not be known until some future date. The future is being

predicted, but all prior observations are almost always considered equally] and if

changed within the calculated scope of the device it will allow it to change the value

or else would require authentication of two or more senior doctors that would ensure

there’s no harm. The most sensitive piece of information is the patient’s data, and

using this model in healthcare such pieces of information can be made secure. The

model briefs the authority when the user opens the application, his/her credentials get

checked through data stored in the cloud, on successful veriﬁcation, the face of the

user is matched with the camera onboard, on unsuccessful veriﬁcation for 5 ×times

the user gets locked for 30 min and the higher authorities are informed about the

same and on successful veriﬁcation, the application starts and asks the user to enter

the application number/name for/of the patient, the data for the same is then made

ofﬂine so that user can access the ﬁles directly. Then the user’s activity is tracked

using ML models, for movement analysis, CNN Model is used and for scrolling

pattern, Random Forest Classiﬁcation is used to track that if a user is on a speciﬁc

page for a longer amount of time or jumping to a speciﬁc part of the document for

all the patients, etc., and if found suspicious, higher authorities are informed.

For patients using clinical gadgets the scope of the device is calculated and

if increased/decreased beyond that, requires two doctor’s authentications. For

calculating the scope of the device, Time Series Analysis is used.

Medical Science has grown exponentially over the past few years, with a global

average life expectancy of 72.6 years in 2019, the average today is higher than in

any country back in 1950 and with this advancement in Health Care, the data and the

details the medical ﬁeld captures have grown exponentially and with such minute

details about a patient’s life, the clinical gadgets he/she uses, there is a demand to use

it reasonably and the need to protect it from unfair usage, for it may contain crucial

data of the patient which if used incorrectly may hamper him/her. In the current

times, the data is stored locally in the computer, which in the case of a ransomware

attack may leak. Basic security features such as a computer lock are not sufﬁcient in

providing the protection needed, for the password and UID may be known to a lot of

people. Patients using clinical gadgets to survive have their life dependent on them,

and so a minute change in the settings of the device may have harmful effects on the

person which may even lead to demise of the patient.

In order to avoid or reduce the severity of such impact, there is a need to create

a system more secure that uses machine learning algorithms for added security and

cloud storage for better accessibility and security over ransomware attacks which

will provide the security that the data or the clinical gadget needs and hence be safe.

2 Evolution of Patient Records and Its Security

Patient records are kept for a variety of purposes, including as a memory aid for the

physician and as a resource for other physicians involved in the patient’s care. Clini-

cians such as nurses, physiotherapists, dieticians, psychologists, and others contribute

358 S. Periwal et al.

to the patient record’s authoring. Nurses create daily notes in the patient record

because they care for the patient constantly, whereas physician records at speciﬁc

intervals. The other reason for record a patient’s healthcare journey is legal, as it is

mandated by law in various nations. Patient records are referred to by a variety of

terms, including patient records, health records, case sheets, and case histories. The

primary repository for information about a patient’s health care is the patient record.

Almost everyone involved in giving, receiving, or reimbursing health care services

is affected in some way.

A paper-based record comprises some distinguishing elements, such as the

patient’s identiﬁcation, the purpose for the visit, and the patient’s historical back-

ground. The patient record for patients admitted to the hospital contains records

on the state and progress of their treatment. These records are typically taken by

nurses who are responsible for the patient’s daily care. In most circumstances, data

is missing, illegible, or erroneous in paper records, while in other cases, certain data

is excessive or redundant. The more trips the patient makes, the thicker the paper

record ﬁle becomes. Time restrictions may make it difﬁcult for the user to locate and

use relevant information. It becomes quite difﬁcult and cumbersome to analyze the

past data of the patient in paper-based patient records.

By the 1990s, computers had made their way into most medical ofﬁces, and

they were being used for record-keeping reasons to a limited extent. A PC-based

patient record is an electronic patient record that is stored in a framework that is

intended to help clients by giving total and accurate data. It can further develop

medical care conveyance by giving clinical specialists more straightforward access

to information, quicker information recovery, better information, and more prominent

information presentation ﬂexibility. PC-based patient records can likewise support

dynamic and quality-control undertakings, just as give clinical suggestions to assist

with patient care. Secondly, by electronically catching clinical data for examination,

computer-based patient records can further develop results of research programs.

Thirdly, computerized patient records can help hospitals run more efﬁciently by

lowering expenses and increasing staff productivity.

The Internet has revolutionized the way data is stored and maintained; patient

records are now more accessible than ever before, and medical knowledge has

advanced, allowing individuals to live longer than ever before. With new-age tech-

nologies, patients can now be treated even in the most remote parts of the globe.

To provide the best possible care, even the slightest information about the patient

is examined and stored. With so much information about a patient, it’s important to

keep it safe. Nowadays, patient records are saved locally on the computer, which is

only accessible by the software in use, adding to the data’s security.

3 Literature Review

Action recognition is a crucial ﬁeld of research in the computer vision domain, where

researchers have started numerous techniques [3], in this model CNN technique is

Integrated Machine Learning Models … 359

used to recognize the same. Khan Muhammad and Mustaqeem used dilated CNN

features to recognize Human action [4].

The authors created a framework for developing distributed software systems

based on the XML standard and Service-Oriented Architecture (SOA). To process

digital records, a set of numerous web services is constructed in this technique.

As a result, each node is in charge of completing a certain duty. Technically, picture

analysis is divided into the following jobs, each of which is assigned to different cloud

providers. During data processing, this strategy is intended to guarantee collaboration

across business components. However, because this technology solely analyzes raw

photos, the biggest downside is privacy protection. Similarly, image processing as a

service is being explored in [5].

Prableen Kaur Manik Sharma Mamta Mittal carried out utilizing Naive Bayes,

Support Vector Machine, Decision Tree, a hereditary calculation which may be modi-

ﬁed contingent upon the exactness of the ML algorithms may further be joined with

other delicate ﬁguring methods to improve results [6].

AKM Iqtidar Newaz Amit Kumar Sikder Mohammad Ashiqur Rahman used ML

detection technique such as ANN, KNN, Decision Tree, Random Forest Classiﬁ-

cation to identify malicious activities in smart healthcare systems (SHS) [7], and

effective security of the framework was found to be 91%.

Sandeep Pirbhulal Nuno Pombo Virginie Felizardo Nuno Garcia Ali Hassan

Sodhro Subhas Chandra Mukhopadhyay utilizes a machine learning-based biometric

security architecture, within which qualities are obtained from electrocardiogram

(ECG) data. However, throughout the testing phase, user authentication will be

veriﬁed using ECG-generated unique biometric EIs and polynomial approximation-

acquired coefﬁcients. The recommended structure offers both scientiﬁc and economic

value, making it suitable for real-time healthcare applications [8].

Gondalia et al. in [9] shows the extended and low power consumption qualities

of LoRaWAN and ZigBee. It would be sent and shown the wellbeing information

and the constant area utilizing KNN which is detected from remote warriors to crew

pioneers utilizing ZigBee and onto the base site utilizing the LoRaWAN module as

a remote communication gadget.

Elngar [10] proposed a latest interfere detection component for IoT-based medical

care framework called (IOT-TD) where a Genetic Algorithm is utilized to optimize

weight and bias values of Artiﬁcial Neural Networks (ANN), which help to boost the

detection accuracy, curtail the timing detection speed and the efﬁcient energy saving

of IoT-network modules.

Bharathi et al. [11], proposed how to determine healthcare data in the cloud server

and determine the seriousness of diseases, an artiﬁcial neural network (ANN)-based

classiﬁcation algorithm is used. To test the speculation, a deliberate understudy point

of view medical care information is made utilizing the UCI dataset and clinical

gadgets to anticipate the different degrees of illness seriousness among students.

Mishra et al. [12] emphasized the amount of security attacks, which have expanded

considerably in recent-days in healthcare. In diverse works, many violation detection

systems have been suggested to detect cyber risks in smart healthcare and to recognize

network-based attacks and privacy violations.

360 S. Periwal et al.

The author in [13], The job of choosing an ML algorithm includes the feature

matching of the data to be learned based on existing approaches. Machine Learning

has numerous algorithms types which are grouped into three classes: Super-

vised learning, Unsupervised learning, Semi-supervised learning, Reinforcement

Learning.

1. Unsupervised Learning: In unsupervised learning, ML methods use unlabeled

data which allows the system to identify patterns within data sets on its own as

seen in Fig. 1. Without the need of human involvement, these algorithms expose

hidden patterns or data groupings. It is the efﬁcient solution for exploratory data

analysis, cross-selling techniques, consumer segmentation, and image identiﬁ-

cation because of its extent to detect alikeness and variations in information.

Generally utilized examples of unsupervised learning methods are a clustering

of data points using an alike metric and dimensionality reduction to project high

dimensional data to lower-dimensional subspaces also known as feature selec-

tion. Clustering algorithms [14] are utilized to convert raw, unclassiﬁed data

objects into groups signiﬁed by structures or patterns in the information. Clus-

tering algorithms can be characterized into a few types, speciﬁcally exclusive,

overlapping, hierarchical, and probabilistic.

2. Supervised Learning: Supervised learning [15], often known as supervised

machine learning, is artiﬁcial intelligence and machine learning subcategory.

It utilizes named datasets to train algorithms that precisely categorize data or

forecast outcomes as seen in Fig. 2. The weights are changed when input data

is fed into the model until the model is successfully ﬁtted, which occurs during

the cross-validation phase. Supervised learning may be used to solve a variety

of real-world problems at scale, such as spam classiﬁcation in a separate folder

from your email.

Fig. 1 Unsupervised learning

Integrated Machine Learning Models … 361

Fig. 2 Supervised learning

3. Semi-Supervised Learning: It refers to a learning problem (and the algorithms

developed to solve it) in which a model must learn and make predictions on

following data from a small amount of labeled data and a large amount of

unlabeled data as seen in Fig. 3. Because obtaining an adequate amount of

labeled data for model training is tough in healthcare, semi-supervised learning

approaches can be especially effective for a range of healthcare applications. In

the literature, many features of semi-supervised learning employing various ML

algorithms have been projected. The clustering method using semi-supervised

learning for medical care data is presented in [16].

4. Reinforcement Learning: The training of machine learning models to make a

series of decisions is known as reinforcement learning. In an indeterminate,

Fig. 3 Semi-supervised learning

362 S. Periwal et al.

Fig. 4 Reinforcement learning

potentially perplex environment, the algorithm learns to attain a result. In rein-

forcement learning, a machine learning technology meets a game-like situation.

The computer employs trial and error to solve the problem. Artiﬁcial intelligence

receives either incentives or penalties for the actions it does in order to achieve

the programmer’s goals. It has made the decision to boost the overall award as

much as possible. Many healthcare applications have the potential to be trans-

formed by RL, and it has recently been employed for context-aware symptom

screening for illness diagnosis [17] (Fig. 4).

4 Proposed Model for Improving the Security

of HealthCare Data and Clinical Devices

See Fig. 5.

4.1 Explanation of Flow of Application and Model

The model shown in Fig. 5lets the authority examine the motion of the logged-in

user, scrolling pattern, and the time that the logged-in person spends within side

the affected person’s report. If the person spends much less than a minute, he/she

may be taking a photograph of the report that’s forbidden and assume the person

spends extra than 10 min he/she may be memorizing the records that are once more

forbidden, for he may use those sensitive records and the patient will need to suffer.

An incorporated front digicam jogging with the software will be of added beneﬁt, it

will be able to tune the motion of the person and draw conclusions with the aid of

schooling an ML version with it.

It is advised to keep information within side the cloud with the aid of using

offerings like AWS, Google Clouds, and so forth., so that even within side the case

Integrated Machine Learning Models … 363

Fig. 5 Flow of application (including model)

of any ransomware assault the touchy records concerning the affected person aren’t

tempered/stolen.

Considering the signiﬁcance and sensitivity of the person’s clinical gadgets such as

pacemaker, urine output tracking tool, multisensory pedometer, and so forth wherein

surprising increase/decrease of the scope of the tool may be dangerous for the affected

person, for this it is proposed to have an ML version skilled within side the unique

tool’s dataset that can calculate the scope of the tool wherein there’s no damage to the

affected person and while accelerated past that scope would require the authentication

of Doctor(s). Here it is assumed that the devices can only be controlled by the software

used by the particular hospital/clinic.

•CNN: To detect objects, recognize faces, and so on, CNN (Convolutional Neural

Network) uses pictorial recognition and sorting. They’re formed up of neurons

with learnable weights and biases. Each neuron takes a huge number of inputs

and computes a weighted sum, which it then passes through an activation function

before producing an output. CNNs are naturally used to classify images, cluster

them based on resemblance, and eventually analyze items. Many CNN-based

algorithms can classify faces, street signs, animals, and other objects.

364 S. Periwal et al.

•RF Classiﬁer: Many decision trees are integrated to create an RF classiﬁer or

Random Forest Classiﬁer. The RF classiﬁer’s goal is to combine many decision

trees into more relevant and accurate results. It determines the mean of each

decision tree and allocates the mean value to the forecasted variable for regression.

The RF uses a widely held voting method for classifying instances. If three trees

predicted yes and two trees predicted no, the predicted variable will be set to yes.

The root or parent node of the tree is determined using entropy and information

gain.

•Time Series Analysis: A Time Series Analysis is a technique for examining a

collection of data points over a period of time. Instead of randomly recording

data points, time-series monitors record datasets at regular intervals over a preset

period of time. This type of research, on the other hand, entails more than simply

collecting data over time. Time-series data is distinguished from other types of

data by its ability to represent how variables change over time. To put it another

way, time is a crucial element since it indicates how data evolves through time as

well as the effects.

4.2 Beneﬁts of the Model

The main beneﬁt of using the model is the added high-security feature that provides

a higher level of protection of the data than any of the current systems. It can reduce

human work to almost 0% in observing the security cameras all day long and hence

will save the organizations’ money, by letting the algorithm do the hard work for them.

With time ML algorithms gain insight, they continue improving in accuracy and efﬁ-

ciency which allows them to settle on better decisions. In our case of CNN, Random

Forest, and Time Series Analysis, as the amount of data grows, the algorithms learn

to make more accurate predictions faster.

5 Comparative Study

By promoting the total health of every individual within a worldwide network of fami-

lies and communities, an effective health care system responds to the expectations

and requirements of community members. The World Health Organization (WHO)

suggests several techniques for establishing an effective health system, including the

development of healthcare policies that address disparities, availability, and common

decision-making by concentrating on people-centered care.

Table 1describes the techniques, issues, and impact of healthcare security in

various working models including the proposed model which helps to choose the

best ﬁt model in the healthcare ﬁeld.

Integrated Machine Learning Models … 365

Tabl e 1 Comparison of different studies in the healthcare security ﬁeld [8–10]

Area focused ML techniques Security issues Impact in healthcare

Security for soldier K-means Clustering No Monitors health

parameters of soldiers,

tracks their position

Video-based human

action in healthcare

CNN Limited Suspicious human action

Semi-supervised

clustering for healthcare

clinical implications

Adaboost Algorithm and

SOM Clustering

Minimal Epidemiological disease

diagnosis in a low level

of risk

Context-aware

symptom

Reinforcement Learning Limited A group of doctors with

various specialties that

work together to

diagnose a patient

Security enhancement

in healthcare

SVM and Fuzzy

C-means

Limited Using solely cloud

resources, a cloud

framework is used to

analyze digital records

Machine learning-based

secure healthcare

framework

SVM, Naive Bayes,

Genetic Algorithm, and

Decision Tree

No Soft computing methods

to get better outcomes

with secure 4 layers

network

A machine

learning-based security

framework for smart

healthcare systems

ANN, Decision Tree,

Random Forest

Minimal Detects and prevents

serious medical issues

automatically

Efﬁcient tamper

detection mechanism

ANN-GA No Tamper detection

mechanism

Healthcare security

using digital devices

CNN, Random Forest

Classiﬁcation, and Time

Series Analysis

No Focuses on more

security for patients and

makes patient safer and

keeps data in a highly

secured cloud

6 Result Analysis

The following precisions are obtained after applying several Machine Learning

Algorithms to the works:

•Algorithm 1 (For detecting user’s movement, expression, and object detection):

The CNN algorithm is the best ﬁt in this model because it detects user movement and

facial and object detection more efﬁciently for which it gives better results than other

algorithms mentioned [18–21]. The graph below shows how accurate the planned

algorithm is.

Here as seen in Table 2, CNN gives the highest accuracy of 96.5%

366 S. Periwal et al.

Tabl e 2 Comparison of different ML techniques for Algorithm 1

Algorithms Accuracy (%)

KNN 78

SVC 80

CNN 96.5

Logistic Regression 94

LDA 92.5

Fig. 6 Visualization of data in Table 2

Visualizing these accuracies allows us to see the differences between them more

clearly (Fig. 6).

•Algorithm 2(For deducing information from the scrolling pattern of the user):

The Random forest classiﬁer algorithm is the best ﬁt in this model because in our

model whenever there will be suspicious movement by the user it will detect more

accurately and in a secure manner.

Here as seen in Table 3, Random Forest Classiﬁer gives the highest accuracy of

98%

Visualizing these accuracies allows us to see the differences clearly (Fig. 7).

•Algorithm 3 (For prediction of the scope of Clinical device):

The Time-series analysis best ﬁts in this model because it can calculate the range

which is suitable for the patient based on his/her medical history and doctor’s

suggestion detection for a clinical device accurately and efﬁciently.

Here as seen in Table 4, Time-Series Analysis gives the highest accuracy of 97.5%

Integrated Machine Learning Models … 367

Tabl e 3 Comparison of different ML techniques for Algorithm 2

Algorithms Accuracy (%)

Random Forest Classiﬁer 98

AdaBoost Classiﬁer 92

Genetic Algorithm 89

Decision Tree 94

Bagging 85

Perceptron 78

Logistic Regression 95

Fig. 7 Visualization of data in Table 3

Tabl e 4 Comparison of different ML techniques for Algorithm 3

Algorithms Accuracy (%)

Naive Bayes 95

ANN 88

Time-Series Analysis 97.5

Gaussian NB 92

PCA 76

Visualizing these accuracies allows us to see the differences clearly (Fig. 8).

368 S. Periwal et al.

Fig. 8 Visualization of data in Table 4

7 Conclusion

Spatiotemporal elements, such as human action detection, are critical in recognizing

distinct behaviors in surveillance video data. In this chapter, it is proposed to have

multiple Machine Learning Algorithms in the application, for monitoring the user’s

movement, expression, object detection along with understanding the pattern of

scrolling in the report that the user does and data storage in the cloud rather than

storing it locally. The CNN network is utilized to draw-out the high-level promi-

nent information from the video frames for this purpose. Random Forest to deduce

information from the pattern of scrolling and Time Series Analysis for predicting the

scope of the clinical device that is used by a patient.

In the near future with the dataset evolving the model can be trained to further

predict the output more efﬁciently, accurately, and faster thus making the data more

secure.

References

1. Mishra, S., Dash, A., & Jena, L. (2021). Use of deep learning for disease detection and diagnosis.

In Bio-inspired Neurocomputing (pp. 181–201). Springer.

2. Ray, C., Kumar Tripathy, H., & Mishra, S. (2019, June). Assessment of autistic disorder

using machine learning approach. In International Conference on Intelligent Computing and

Communication (pp. 209–219). Springer.

3. Wu, D., Sharma, N., Blumenstein, M. (2017). Recent advances in video-based human action

recognition using deep learning: a review. In 2017 International Joint Conference on Neural

Networks (IJCNN), 2017 (pp. 2865–2872). https://doi.org/10.1109/IJCNN.2017.7966210

Integrated Machine Learning Models … 369

4. Muhammad, K., Mustaqeem, A. U., Imran, A. S., Sajjad, M., Kiran, M. S., Sannino, G., de

Albuquerque, V. H. C. (2021) Human action recognition using attention-based LSTM network

with dilated CNN features. Future Generation Computer Systems, 125, 820–830. ISSN 0167-

739X

5. Marwan, M., Kartit, A., Ouahmane, H. (2018). Security enhancement in healthcare cloud using

machine learning. Procedia Computer Science, 127, 388–397. ISSN 1877-0509.

6. Kaur, P., Sharma, M., Mittal, M. (2018). Big data and machine learning based secure healthcare

framework. Procedia Computer Science, 132, 1049–1059. ISSN 1877-0509.

7. Newaz, A. I., Sikder, A. K., Rahman, M. A., Uluagac, A. S. (2019). HealthGuard: a machine

learning-based security framework for smart healthcare systems. In 2019 Sixth International

Conference on Social Networks Analysis, Management and Security (SNAMS) (pp. 389–396).

https://doi.org/10.1109/SNAMS.2019.8931716

8. Pirbhulal, S., Pombo, N., Felizardo, V., Garcia, N., Sodhro, A. H., & Mukhopadhyay, S. C.

(2019). Towards machine learning enabled security framework for IoT-based healthcare. In

2019 13th International Conference on Sensing Technology (ICST) (pp. 1–6). https://doi.org/

10.1109/ICST46873.2019.9047745

9. Gondalia, A., Dixit, D., Parashar, S., Raghava,V., Sengupta, A., Sarobin, V. R. (2018). IoT-based

healthcare monitoring system for war soldiers using machine learning. Procedia Computer

Science, 133, 1005–1013. ISSN 1877-0509.

10. Elngar, A. A. (2018). Employing an efﬁcient tamper detection mechanism for IoT-based

healthcare systems. International Journal of Internet of Things and Cyber-Assurance,1(2),

158–172.

11. Bharathi, R., Abirami, T., Dhanasekaran, S., Gupta, D., Khanna, A., Elhoseny, M., & Shankar,

K. (2020). Energy efﬁcient clustering with disease diagnosis model for IoT based sustainable

healthcare systems. Sustainable Computing: Informatics and Systems, 28, 100453. ISSN 2210-

5379.

12. Mishra, S., Mishra, B. K., Tripathy, H. K., & Dutta, A. (2020). Analysis of the role and scope

of big data analytics with IoT in health care domain. In Handbook of data science approaches

for biomedical engineering (pp. 1–23). Academic Press.

13. Iwendi, C., Anajemba, J. H., Biamba, C., & Ngabo, D. (2021). Security of things intrusion

detection system for smart healthcare. Electronics, 10, 1375.

14. Mishra, S., Raj, A., Kayal, A., Choudhary, V., Verma, P., & Biswal, L. (2012). Study of cluster

based routing protocols in wireless sensor networks. International Journal of Scientiﬁc and

Engineering Research,3(7).

15. Alloghani, M., Al-Jumeily Obe, D., Mustaﬁna, J., Hussain, A., & Aljaaf, A. (2020). A system-

atic review on supervised and unsupervised machine learning algorithms for data science.

https://doi.org/10.1007/978-3-030-22475-2_1

16. Sohail, M. N., Ren, J., Muhammad, M. U. (2019). A Euclidean group assessment on semi-

supervised clustering for healthcare clinical implications based on real-life data. International

Journal of Environmental Research and Public Health, 16.

17. Kao, H. C., Tang, K. F., Chang, E. Y. (2018). Context-aware symptom checking for disease

diagnosis using hierarchical reinforcement learning. In AAAI Conference, 2018.

18. Tripathy, H. K., Mishra, S., Thakkar, H. K., & Rai, D. (2021). CARE: A collision-aware

mobile robot navigation in grid environment using improved breadth ﬁrst search. Computers &

Electrical Engineering, 94, 107327.

19. Mishra, S., Mahanty, C., Dash, S., & Mishra, B. K. (2019). Implementation of BFS-NB hybrid

model in intrusion detection system. In Recent Developments in Machine Learning and Data

Analytics (pp. 167–175). Springer.

20. Mishra, S., Sahoo, S., & Mishra, B. K. (2019). Addressing security issues and standards in

Internet of things. In Emerging trends and applications in cognitive computing (pp. 224–257).

IGI Global.

21. Mishra, S., Tadesse, Y., Dash, A., Jena, L., & Ranjan, P. (2021). Thyroid disorder analysis

using random forest classiﬁer. In Intelligent and cloud computing (pp. 385–390). Springer.

Symptoms-Based Biometric Pattern

Detection and Recognition

Uday Bhanu Ghosh, Rohan Sharma, and Abhishek Kesharwani

Abstract Biometrics play a crucial role in today’s technology world, since it means

analyzing biological data. Biometrics is the analysis of the physiology, behavior, or

morphology of a person including their face, ﬁngerprint, iris, retina, voice, or signa-

ture; a person’s identity can be established through biometrics. In the past decade,

biometrics have successfully been applied to areas such as forensic science, security,

and identiﬁcation. In the last three decades, much research has been conducted for the

development of biometric systems utilizing ﬁngerprints, voice, iris, and facial recog-

nition, as well as new biometric techniques. Biometric recognition systems must take

into account the multiple aspects of processing variable data: obtaining biometric

information from a sensor, preprocessing, extraction of features, biometric iden-

tiﬁcation and labeling, veriﬁcation of labeling, and clustering. Processing images,

recognition of patterns, as well as ML (Machine Learning) and artiﬁcial intelligence,

are used for the analysis of sensor data from a number of perspectives. The aim of the

discussion is to provide a summary of the current state of biometric systems and to

present techniques for tackling challenges and identifying future research directions.

The proposed discussion focuses on evolution of various different types of biometric

pattern Detection and recognition approaches, the result and analysis section focuses

on two cases: the ﬁrst section proposes mathematical derivation for enhancement of

ﬁngerprint recognition and the second section provides an insight comparison of

different approaches to iris recognition. Followed by the section which discusses the

latest approaches of biometric recognition approaches in relation to different wear-

able as well as acquisition devices. The ﬁnal section gives us an overview of different

metrics and their respective accuracies and ability to differentiate individuals based

on features obtained from different biometric detection methods.

Keywords Biometrics ·Identiﬁcation ·Biometric technologies ·Biometric

authentication ·Image processing ·Recognition

U. B. Ghosh (B)

HighRadius Corporation, Bhubaneswar, India

e-mail: ubg0706@gmail.com

R. Sharma ·A. Kesharwani

Vellore Institute of Technology, Vellore, India

S. Mishra et al. (eds.), Augmented Intelligence in Healthcare: A Pragmatic and Integrated

Analysis, Studies in Computational Intelligence 1024,

https://doi.org/10.1007/978-981-19-1076-0_19

371

372 U. B. Ghosh et al.

1 Introduction

Biometrics is the measurement and analyzing of biological data to ensure identity

or authenticity. Biological characteristics or behavioral traits of a person can be

determined by biometrics. Ancient Egyptians used biometric technology. Therefore,

biometrics is deﬁned as the analysis and measurement of certain physical and behav-

ioral traits that can uniquely identify an individual. A biometric is a measure of one’s

life. It comes from the Greek words bios meaning life and metric meaning measure-

ment. Behavioral traits are related to a person’s behavior, according to the structure

of the human body, including—Face, Fingerprint, Iris, Retina, DNA, Ear, and Hand

geometry [1].

During the past ten years, we have seen an exponential increase in computer

technology and electronics, which has resulted in sensors with unique capabilities [2].

The low cost, low size, and high computing power enables people to use these gadgets

every day. Biometrics has grown into a very exciting ﬁeld, especially for defense,

healthcare, and safety applications. As opposed to PIN numbers or login credentials,

biometrics can provide some relatively secure, collectible, and standardized methods

of user identiﬁcation. An individual can be characterized using pattern identiﬁcation

techniques based on the multiple responses of sensory devices with a wide range and

partially overlapped discriminations. An entire pattern identiﬁcation process may be

divided into three phases (Fig. 1): (i) preprocessing, dimensionality reduction, (ii)

classiﬁcation, and (iii) clustering and validation. In the initial phase, there is a sensor

and a computer for securely handling the biometric information and performing

necessary preprocessing, like aligning of the information, ﬁltration, and normalizing.

This is followed by a de-dimensionalization phase to reduce the dimensionality and

avoid complications that may occur with high dimensional sets of data and improve

data classiﬁcation. This low-dimensional feature vector is then used for a detection

Fig. 1 Ideation of pattern identiﬁcation for biometric recognition

Symptoms-Based Biometric Pattern Detection and Recognition 373

or estimation task, general categorization, and clustering. In classiﬁcation cases,

previously learned samples are used to recognize an unfamiliar instance, and in

clustering, the primary goal is to identify patterns among the biometric information.

Some skills are overlooked in the training process, such as selecting the parameters

and determining the accurate errors by means of validation [3] (Fig. 1).

Nowadays, Biometric technology is used in many practical applications.

Biometric technology is characterized by its stability. The most important thing is

to know that the biometric attributes of an object don’t change signiﬁcantly during a

speciﬁc period of time. A person’s handwriting and signature change throughout the

day based on their psychological state factors. A similar observation can be made

about keystroke dynamics analysis, however, it must be added that other factors are

also taken into account. A steady interest in ﬁngerprint and eye identiﬁcation methods

is evident from a position of stability. Biometric technology has a number of unique

properties that make it useful throughout an individual’s lifetime (Fig. 2).

Researchers have been exploring the use of biometrics for identifying individuals

with features such as the facial features, ﬁngerprints, sound, and iris for a long time.

One of the most precise and most trustworthy biometric authentication systems is iris

detection. The iris recognition technology is commonly used in airports, laboratories,

ATMs, and other security systems. Iris recognition can also be employed in clinical

settings [4–6]. The technique of iris diagnosis, also known as iridology or iris diag-

nosis [7,8], uses the color, patterns, and structure of the iris to diagnose a person’s

health status. Biometric technology applications are dynamically expanding. The

current stage of IT development may partly explain why information security issues

have become a priority. The use of biometric technology is often successful, and

the beliefs about it are frequently justiﬁed. The current biometric technology appli-

cations mainly focus on the following areas: healthcare, criminal justice, ﬁnancial

services, immigration laws, voice and video communications, and security (Fig. 3).

Fig. 2 Biometrics in the context of modern technology

374 U. B. Ghosh et al.

Fig. 3 Various modern biometric technology systems

2 Background Study

Biometrics can be used either to assess physiological or behavioral characteristics.

Physiological biometrics is the study of measures and data obtained directly from

human body parts. Furthermore, behavioral biometrics includes the measurement

and data generated during an action. Figure 4summarizes a few common features

of biometrics.

Fig. 4 Classiﬁcation of biometric features

Symptoms-Based Biometric Pattern Detection and Recognition 375

2.1 Fingerprint

A ﬁngerprint is a pattern of valleys and ridges on the tip of your ﬁnger that can be

used to verify your identity. There are several reasons why ﬁngerprint recognition is

the most popular and reliable technique, including its ubiquity, durability, distinctive-

ness, effectiveness, and affordability. Currently, it is the most widely used biometric

technology [9]. Archaeological evidence indicates that ﬁngerprints have been used

by Assyrians and ancient Chinese civilizations for identifying since 7000 to 6000 BC

[10]. Henry Faulds introduced minutiae features for ﬁngerprint matching in 1880,

which laid down the scientiﬁc basis for modern ﬁngerprinting. A number of types of

ﬁngerprint recognition techniques are currently available, including minutiae-based

or correlation-based [11], techniques and gradient-based ones [12].

In 1926, Dr. Harold Cummins invented ﬁngerprint patterns, but they had already

existed for several hundred years. Traditionally, ﬁngerprint patterns have been used

to identify individuals. Nowadays, every organization or even many Government

institutions in India use ﬁngerprint veriﬁcation to identify people uniquely. Besides

being used to identify gender and age, it has also been used as a biometric modality.

With this catchy principle in mind, biometrics has [13], evolved into an attractive

alternative to traditional methods of identiﬁcation such as tokens and passwords.

He was the ﬁrst to propose the singularity and uniqueness of a ﬁngerprint in 1880.

Herschel [14], helped establish current ﬁngerprinting identiﬁcation methods in the

sixteenth century.

3Face

Face recognition is one of the most popular biometrics due to its convenience as well

as non-intrusion, Ravindran et al. [15], summarizes the technologies available for

facial recognition. Furthermore, a review of existing facial recognition technologies is

presented [16]. The recognition of faces has been carried out using several algorithms.

The two main types of algorithms are feature-based algorithms and appearance-

based algorithms. Appearance-based algorithms include Eigen-faces [17], Fisher-

faces [18], ICA (Independent Component Analysis) [19], KPCA (Kernel Principal

Component Analysis) [20], KFDA (Kernel Fisher Discriminant Analysis) [21], GDA

(General Discriminant Analysis) [22], Neural-Networks [23], and SVM (Support

Vector Machine) [24].

Appearance-based algorithms can recognize a face under light, and position may

be reliably predicted when the person has previously been observed under similar

conditions. Additionally, appearance-based approaches capture global characteris-

tics of images of the face, and occlusion is frequently detected as challenging to

handle in these methods. It is known that geometric approaches are resilient to vari-

ations in illumination and viewpoint, but they are highly dependent on the process

of feature extraction. Methods based on geometry feature data analyze local features

376 U. B. Ghosh et al.

in explicit geometric relationships. Activated Shape Modeling [25], Elastic Bunch

Graph Matching [26], and LFA (Local Feature Analysis) [27], are feature-based

options.

3.1 Iris

Located outside the cornea and inside the lens in the eye, the iris is a transparent

diaphragm. An iris recognition technology survey is provided in [28]. Initially, Victor

et al. [29], discussed the idea of iris recognition in an automated manner. John

Daugman developed a functioning iris recognition system [30]. Daugman’s system

is the best known and most effective, and there are a variety of other systems as well.

On the basis of the circular Hough transform, Daugman [31], uses an automatic

segmentation algorithm. The 1-D wavelet transform [32] was used to extract iris

features. Using iris images, the method was further developed by Boles et al. [33,

34], Using the Haar wavelet transform, iris features were extracted. From ﬁlter banks

Lim et al. [35], extracted directional energy. To do so Park et al. [36], used correlation

ﬁlters. Earlier this year, Ma et al. presented a pair of methods for recognizing iris

images: one is based on multiple channel Gabor ﬁlters [37], and the other based on

circular symmetry ﬁlters [38]. Later, the authors revised their method by deﬁning

a wavelet class for describing local variations, and recording the positions of local

sharp variations as features [39]. Several other techniques have been developed to

identify iris patterns. There are other methods for iris recognition as well. It was

proposed by Daugman to use 2-D Gabor ﬁlters enhanced by quality measures [10].

Both Golfarelli et al. [40], and Mishra et al. [41] proposed local texture patterns in

1-D, and iris blob matching using a moment-based method.

3.2 Hand Geometry

The geometry of the hand describes the hand’s geometric structure, which is deter-

mined by the lengths, widths, and widths of the ﬁnger and the palm. Hand geometry

systems are relatively simple, utilize low resolution images, and are highly efﬁ-

cient with high acceptability [42]. There is a summary of hand geometry veriﬁcation

systems reported in [43] and a detailed survey of hand geometry veriﬁcation in [44].

Almost all hand recognition systems rely heavily on geometrical features of the

hand. Geometric properties are approximately constant with respect to the hand’s

global orientation and the respective ﬁnger plane orientation. Geometric measure-

ments are used to determine the length, width, area, and perimeter of hands, ﬁngers,

and palms. We have demonstrated that hand geometrical characteristics alone cannot

be sufﬁcient for discrimination. Because of this, highly sophisticated software will

need to rely on alternatives such as global hand shapes, appearances, and textures.

Symptoms-Based Biometric Pattern Detection and Recognition 377

By using ﬁve pegs, 16 axes are predetermined. Based on a set of identical geomet-

rical features, Kong et al. [45] uses four ﬁngers measured at different latitudes,

three ﬁngers measured at different lengths, as well as the palm deformation. The

ﬁngertip regions are included along with tips, measurements, and between-ﬁnger

distances [46]. Reference [47] describes a pegless algorithm that extracts approxi-

mately 30 geometrical measurements from images of hands. They include not only

the circumferences, areas, and perimeters of ﬁngers, but also the radius of tracing

circles.

3.3 Palmprint

The palmprint is located in between the ﬁngers and wrist. Several characteris-

tics including furrows, singular points, minutiae markings, central-lines, freckles,

and skin tone might be used to verify a person’s identity [48]. Palmprint veriﬁca-

tion systems fall into two categories: high resolution and low resolution. Furrows,

singular points, and minutiae points provide characteristics in images with high

resolution. The main features of low-resolution images are lines, wrinkles, and

texture. In general, there are four types of palm prints veriﬁcation: (1) Line-Based

[49]; (2) Texture-Based [50]; (3) Orientation-Based [51]; and (4) Appearance-Based

[52]. The line in a palm print is its most distinguishing feature. Therefore, line-

based approaches are essential for palmprint veriﬁcation. In order to extract prin-

cipal line features, the author applied extended wavelet-expansion and directional-

context modeling techniques [53]. Using Sobel and morphological techniques, the

author proposed extracting palmprint features that resemble lines [54]. A hierar-

chical decomposition technique was applied to obtain primary palmprint character-

istics, including directional and multi-resolution decompositions [55]. Also, a pair

of distinct approaching palm lines were proposed, where palm lines are taken to be

roof edges, which are based on zero-cross points of the ﬁrst derivative and the magni-

tude of the second order derivative [56,57]. The most common methods of texture

extraction utilize the 2-D Gabor ﬁlter. A texture-based approach called Palm-Code

for palm print authentication exploits zero-crossing data on an image of palm print

using Gabor ﬁlter [58]. Fusion rules were then used to further improve PalmCode,

called FusionCode [59].

3.4 Speaker/Voice

Voice/speaker veriﬁcations comprise physiological and behavioral factors in order to

create patterns of speech that can be captured using voice recognition software. For

speech authentication, fundamental frequencies, tone of voice, pacing, articulation,

etc., are considered. A speech recognition system can be categorized as text dependent

(ﬁxed text) or text independent (free text). In general, text dependent systems are

378 U. B. Ghosh et al.

more efﬁcient than text independent systems since foreknowledge of speech can

be used to classify the speaker. On the other hand, text dependent systems need the

subject to pronounce certain phrases, which usually contain exactly the same text that

accompanied the training data. Sahoo et al. [60], presents a survey of text dependent

validation techniques.

3.5 Signature

Handwritten signatures are among the oldest civilian and scientiﬁc biometric authen-

tication techniques in our society [61]. Verifying signatures by hand is usually very

accurate. Signature recognition utilizes the behavioral characteristics of a signature

(factors such as movement, force, and order of strokes) as a way to conﬁrm a user’s

identity, rather than relying on a physical signature alone and also cross-checks two

signatures. There are two categories of signature veriﬁcation: ofﬂine and online. In

the ofﬂine approach, signatures are identiﬁed through an image-processing process,

where the signatory writes down the signature and then images it using a scanner or

CCD camera. Online certiﬁcation of signatures entails capturing real-time signature

information, such as pen tip pressure, the duration of a full signature, and speed as

the signature moves.

In online systems, tablet devices are used, whereas ofﬂine methods are more

challenging since the only data available is a static two-dimensional image obtained

by scanning pre-written signatures on paper; the dynamic speciﬁcations of pen-tip

(stylus) movement including pressure, velocity, and acceleration may not be captured

by an image scanner in real-time. Due to this, a traditional technique must use sophis-

ticated algorithmic methods to segment and analyze signature shapes to extract

features. Therefore, online signature veriﬁcation is more likely to be successful.

However, ofﬂine systems are more advantageous than online systems in that they

require no special processing tools to produce signed documents (Table 1).

4 Generalized Working Model

As shown in Fig. 5, the general biometric system consists of four compo-

nents: Sensor component, Feature Extraction component, Matcher component, and

Decision-Making component.

A biometric system consists of two phases: learning and recognition.

(i) Sensor component: capture of real-time biometrics from an individual, either

video, audio, or a combination of these.

(ii) Feature extraction component: automated extraction of distinct biometric

features based on computer vision, machine learning, and pattern detection.

Symptoms-Based Biometric Pattern Detection and Recognition 379

Tabl e 1 Development timeline table of biometric technology

Time period Descriptions

1903–1936 • Fingerprint system used by New York State Prison Department

• The Hungarian government developed a Palm System for criminal identiﬁcation

(CID)

• FBI ﬁngerprint analysis

1936–1970 • Iris detection for Identiﬁcation was proposed by ophthalmologist Frank Burch

• Woodrow W. Bledsoe invented the ﬁrst semi-automated face recognition system

• Goldstein, Harmon, and Lesk developed Face Recognition for automation

• Dr. Perkell modeled the ﬁrst behavioral components of speech

1970–1985 • The US Air Force and the MITRE Corporation tested the ﬁrst prototype for

speaker recognition developed by Texas Instruments

• Leonard Flom and Araan Saﬁr, ophthalmologists, proposed that no two irises

are identical

• David Sidlauskas is awarded a patent for his Hand Identiﬁcation system

1985–1995 • Eigenfaces technique developed by Kirby and Sirovich for face recognition

• Face detection for real-time face recognition was invented by Turk and Pentland

• FERET (Face Recognition Technology) was initiated by the Defense Advanced

Research Products Agency (DARPA)

• Dr. Daugman’s First Iris Recognition Algorithm is patented

1995–2004 • Hand geometry is implemented at the Atlanta Olympic Games

• To counter national security threats, the Department of Defense (DOD), US

government implemented ABIS (Automated Biometric Identiﬁcation System)

• Federal employees are required to obtain identity cards through the Homeland

Security Presidential Directive 12 (HSPD-12)

2004–2010 • The USA and EU issue biometric passports

• Hitachi introduces a ﬁnger vein scanner

• To identify terrorists The United States, the government uses biometrics

2011–2017 • The body of Osama bin Laden was identiﬁed using DNA and facial recognition

technology

• A mass Iris Recognition System is deployed in India

• A new wearable biometric technology allows for handwritten signatures to be

authenticated with smartwatches and ﬁtness trackers

• Speakers like the Amazon Echo and Google Home, which use Alexa voice

recognition, can recognize speech, control parameters, and track voice

biometrics

(iii) Database component: stores biometric information of registered/enrolled

users, as well as various templates.

(iv) Matching component: Analyzes the extracted features and the template to ﬁnd

the match value, or match score, that determines how similar two biometric

samples are.

(v) Decision-making component: compares the matching scores with a given

threshold value to determine whether the decision should be accepted or

rejected.

380 U. B. Ghosh et al.

Fig. 5 General biometric system ﬂow diagram

5 Discussed Methodology (Working of Different Biometric

Recognition Techniques)

5.1 Biometrics Using Fingerprints

The ﬁngerprint is the most ancient and widely known biometric authentication

method. Law enforcement agencies use this modern, automated upgrade of the tradi-

tional ink-and-paper system for recognition. Based on analyzing an individual’s

ﬁngerprint, it recognizes them. Every individual’s ﬁngerprint is distinguishable

and immovable, and its properties do not change over time. Even identical twins

have different ﬁngerprints. Additionally, the ﬁngerprints on each ﬁnger of the same

individual differ.

In order to identify an individual’s ﬁngerprint, the following two aspects are

required: unchangeability (the ridge pattern never changes) and uniqueness (ridge

patterns are unique on each ﬁnger of the same individual). Three types of ﬁngerprint

matching techniques exist.

1. The minutiae-based approach involves identifying minutiae points and locating

them on the ﬁnger.

2. A correlation-based method is based on much grayscale data. It can handle

low-resolution data.

3. In ﬁngerprint pattern and image matching, each claimant’s ﬁngerprint pattern

is compared with a ﬁngerprint template (Fig. 6).

5.2 Face Recognition

Face recognition is an emerging subject that is constantly improving. Scientists in the

ﬁelds of psychology, optics, neural networks, machine learning, image processing,

Symptoms-Based Biometric Pattern Detection and Recognition 381

Fig. 6 Fingerprints and their Arches, Loops, and Whorls

computer vision, and pattern recognition have studied face recognition. The ﬁeld

has been developed not only by engineers, but also by neuroscientists. Image

interpretation is one of its major applications (Fig. 7).

The following methods are available for recognizing faces:

1. Face Factor: A facial measurement measures the characteristics of the face. For

instance, the space between the nose and the lip or the pupil and the chin.

2. Eigenfaces: An image of an entire face is processed, i.e., a set of weights

representing the face in its entirety.

3. Evaluation of skin texture: A new method for recognizing faces and other details

of the skin. Identifying the distinctive spots, lines, and patterns on a subject’s

skin (Fig. 8).

Fig. 7 Attributes/features of the face images

382 U. B. Ghosh et al.

Fig. 8 General facial recognition process

Fig. 9 Retina scanning steps biometrics

5.3 Retina Biometrics

Biometrics based on retinal pattern recognition is signiﬁcantly different from iris

recognition. In addition to Iris scanning, it’s older which also utilizes the eye’s part.

In the eyeball’s inner surface, there is a complex pattern of blood vessels which cover

about 65% of the eyeball’s surface. As the innermost layer of the eye, the retina is

made up of neural cells (Fig. 9).

5.4 Iris Biometrics

Located within the pupil, Irises are strips of thin, colorful, elastic, ﬁbrous connective

tissue that controls pupil size and diameter as well as how much light reaches the

Symptoms-Based Biometric Pattern Detection and Recognition 383

Fig. 10 Iris biometrics

Fig. 11 Different stages of proposed system

retina. When it is in the early stages of its life cycle, it is called morphogenesis. Iris

patterns are different for everyone, and identical twins are no exception [62]. Also,

a person’s left and right iris are different, not identical. Cornea covers the iris, thus

providing a protective layer which is visible from the outside. Eye color is determined

by the color of the iris, which may be blue, brown, or green (Fig. 10).

Iridology and iris recognition algorithms present the possibility of creating an

automatic computer model that can be used to diagnose an individual’s health status.

Generally, there are ﬁve stages.

Figure 11 illustrates the Iris recognition-based diagnostic model. In the following

subsections, these stages have been brieﬂy discussed.

5.5 DNA Biometrics

Recognition of faces, ﬁngerprints, iris scans, retinal scans, voice dynamics, and

handwriting recognition has become famous and has made rapid progress. A variety

of features-points measurement techniques have been integrated into the system, and

these provide inaccurate results for a universal identiﬁcation system based on feature-

point measurement. Deoxyribonucleic acid (DNA) is the best form of identiﬁcation.

It is a form of genetic material present in every living being and in every cell. Each

384 U. B. Ghosh et al.

Fig. 12 DNA biometrics

human being possesses hereditary traits in their DNA. A digital record remains the

same throughout a person’s lifetime and even after death. Identical twins share the

same DNA, which is a genetic code that identiﬁes each individual.

DNA proﬁling involves the following steps:

•Blood, saliva, hair, semen, or tissue samples are isolated and separated for DNA

analysis.

•Segmenting DNA samples into smaller pieces (identical DNA sequences)

•DNA segments/fragments arranged by size

•Different DNA segments or fragments are compared (Fig. 12).

5.6 Gait Biometrics

Gait Biometrics is the recognition of a person’s walking style. A Gait is a coordi-

nated and cyclic sequence of movements caused by human locomotion. Compared

to traditional methods such as ﬁngerprints, faces, etc., it has 90% accuracy and is a

relatively new technique. A familiar person can usually be recognized by the way he

or she walks, including how he or she walks, how often he or she sways and how far

two feet separate.

Gait cycles are called strides, and they consist of two phases—one foot on the

ground (standing phase) and another not on the ground (swing phase). Gait biometrics

includes both shape and dynamics. A video image is ﬁrst captured and recorded using

a camera. Walking people can be separated from the background using appropriate

segmentation and motion detection techniques, after which mathematical models are

constructed to extract their gait characteristics (Fig. 13).

5.7 Biometrics of Keystrokes

In World War II, intelligence agencies employed a method known as Fist of the

Sender, which uses a rhythmic typing pattern to determine how a Morse code message

originated from an ally or an enemy. Rhythms of typing at a keyboard can be used

Symptoms-Based Biometric Pattern Detection and Recognition 385

Fig. 13 Gait biometrics

for biometrics, namely keystroke dynamics. A keystroke is a behavioral biometric

technique that provides enough information to differentiate between users when it is

typed on a keyboard in a unique way. In this biometric assessment, a person’s typing

speed, rhythm, and pattern is analyzed.

Many researchers may not ﬁnd the typing dynamics appealing when it comes to

identiﬁcation. However, studies indicate that dwell time, the time spent pressing a

key, together with ﬂight timing, or when the key is released and the next or character

pressed is timed and calculated, which can provide 99% accuracy in identifying who

is typing. [63]. Individual differences in keyboard typing are determined by how long

it takes you to ﬁnd the correct key, how long you dwell on the key, and how long

you actually type. Additionally, there are differences in typing speed and rhythm. In

the context of an interaction, a keystroke can be recognized in two different ways:

statically at the beginning, and continuously during the interaction (Fig. 14).

Fig. 14 KeyStroke dynamics

386 U. B. Ghosh et al.

5.8 Brainwave Biometrics

Emerging research and promising ﬁeld of biometrics related to identifying human

brainwaves. Brainwave biometrics is a form of cognitive biometrics. According to

research, everyone’s brain waves are different. It may be possible to use these features

as biometric identiﬁcation. Electroencephalograms (EEG) track the activity of the

brain, and those features can enhance security. Currently, measuring brain waves is a

laborious task. Nevertheless, it may be used in a variety of situations such as security

speciﬁc areas in the future. A few beneﬁts offered by brainwave biometrics include:

(i) Increased Security resulting in difﬁculty of eavesdropping on personal brain

wave data

(ii) Identifying mental activity of an individual.

A person’s identity is identiﬁed by analyzing templates that reﬂect brain activity

as a cognitive process. Three types of brainwave biometrics exist. A ﬁrst group

of brainwaves based on Alpha (α) and Gamma (γ)waves[64]; a second group of

brainwaves based on motor imagery [65]. Finally, near-infrared spectroscopy (NIRS)

(Fig. 15).

Fig. 15 Brainwave biometrics

Symptoms-Based Biometric Pattern Detection and Recognition 387

5.9 Voice-Based Biometrics

In modern world, sound recognition biometrics dominates research. Speaker recog-

nition biometrics is also known as voice biometrics. Figure 16 illustrates it. Phone-

based applications rely on them. When recognizing a voice, the focus is on the way

a person speaks, not on the pronunciation or sounds. Therefore, it is unnecessary to

install special and more expensive hardware. Acoustic patterns of the voices serve as

a basis for voice analysis that distinguishes the people, whose characteristics include

a combination of behavioral features (articulation, pitch) and anatomical character-

istics (throat structure and mouth size) [66]. Because the vocal tract is not affected

by cold, accuracy remains intact.

Voice recognition can be speaker-speciﬁc or speaker-independent. The speaker-

speciﬁc system takes into account each individual’s characteristic voice. It is neces-

sary for the system to learn and train on vocal characteristics, namely accents and

tones. Speaker independent voice recognition recognizes speech without restriction

based on the speech context, requiring no training. The voice system supports three

vocal input methods: text-dependent, text-prompted, and text-independent. Speech

recognition that is speaker-dependent is difﬁcult to design. Speech recognition and

Speaker recognition differ greatly. Voice or speaker recognition identiﬁes people

based on their voice pitch, tone, accent, and other indicators of WHO is speaking.

Fig. 16 Voice biometrics

388 U. B. Ghosh et al.

Speech recognition is used in navigating menus and in hands-free computing to

comprehend WHAT is said.

6 Comparison Analysis

6.1 Fingerprint Image Enhancement an Algorithm Approach

To begin, we must remove any background information that is not pertinent to deter-

mining the ﬁngerprint and the quality of the image. It is not challenging as we have

the necessary resources. Avoid using dactyloscopic cards, which can contain a lot

of background noise. The quality of Jain et al. [67], is determined by the thickness

and degree of steepness of the valleys and ridges (papillary lines). For quality assess-

ment, the methodology used [67], takes advantage of the slopes of ridges and valleys

(papillary lines) as well as the thickness of the ridges and valleys. We use the sine

function to estimate image quality (this is only one option available) [67] (Fig. 17):

DD=AFP

Asin

−1·100% (1)

Fig. 17 Fingerprint enhancement process

Symptoms-Based Biometric Pattern Detection and Recognition 389

where,

AFP =



f(x)dx

Asin =



sin(x)dx(2)

An image pixel’s intensity and orientation can be represented using a function

called f(x). According to this deﬁnition, the papillary line is also a criterion for

deﬁning the thickness:

DTh =Th

0.033 −1·100% (3)

with

Th =2.54

RDPI

·NPix [cm](4)

Sensor resolution is measured in RDPI, where NPix is the pixel density in a

deﬁned span on the papillary line. Experimentally, it has been established that the

papillary line is 0.33 mm thick on average. Lastly, the criteria of papillary lines are

determination of their steepness.

As follows, we only specify the steepness of the upward papillaryline (a simple

inference can be made regarding the downward line):

Dα=|a−60◦|

60◦·100% (5)

with

α=arcsinPxl

Pxl +Py

6.2 Iris Recognition an Algorithmic Approach

A model based on iris recognition that uses 2D DWT to determine diabetes, is a

procedure for reducing data dimensionality, thus increasing the likelihood of feature

extraction. The DWT algorithm divides the image into four subbands (i.e., LL, LH,

HL, HH) as well as SVM as a classiﬁer has been implemented using the MATLAB

390 U. B. Ghosh et al.

Image Processing Toolbox 7.1. Four-fold cross-validation was performed on the

experiments. During partitioning, every single dataset is divided randomly into four

equally-sized partitions. Three of the partitions are used for testing and the remaining

three for training the classiﬁer. Training the SVM and testing it requires four iter-

ations. We measure model accuracy independently for each subsequent iteration.

Thus, the accuracy of the proposed model is calculated from the average of the

accuracies obtained from individual iterations.

NOTE-1: LL is an approximation of input image, it belongs to the lower bandwidth

subbands so that it can be used for decomposition. Under the LH subband, the image

extracts the horizontal features of the original image. Vertical features are provided

by the HL subband. Diagonal features are provided by the HH subband (Fig. 18).

To analyze a potential impact of wavelet coefﬁcients on system accuracy, a study

was conducted. We have generated feature vectors by combining combinations of

LL, HL, and LH components. A study is conducted on the impact of kernel function

on the performance of the proposed model. SVM was tested using three types of

kernels, namely, polynomial, Gaussian, and radial basis function (RBF).

In Table 2the overall accuracy of diagnoses based on iris recognition is compared

for various feature vector combinations and kernel functions. According to Table

2the RBF vector kernel function yields the greatest level of accuracy for a given

feature. Additionally, these results indicate the proposed model performs better when

using LL and HL features together. Table 2represents accuracy in tabular format.

It is explicit in the above analysis that the most accurate output of 87.5% is

achieved using the RBF kernel along with a feature vector derived from the combi-

nation of LL and HL information from the decomposed image. Speciﬁcity (1 false

negative rate) and sensitivity (1 false positive rate) for each possible combination of

Fig. 18 A two-dimensional decomposition of an image for iris-based feature extraction

Tabl e 2 Accuracy comparisons of various kernel functions using a variety of feature vectors

Feature vector LL HL LH LL +HL LL +LH HL +LH LL +HL +LH

Gaussian (%) 81.25 76.25 76.25 86.25 85.00 81.25 81.25

Radial basis

function (%)

81.25 81.25 80.00 87.50 86.25 85.00 81.25

Polynomial (2nd

Order) (%)

76.25 75.00 75.00 81.25 81.25 80.00 77.50

Polynomial (3rd

order) (%)

75.00 77.50 75.00 77.50 76.25 78.60 78.60

Symptoms-Based Biometric Pattern Detection and Recognition 391

feature vectors and kernel functions were measured. A maximum of 0.95 is obtained

for the sensitivity and a maximum of 0.90 for the speciﬁcity of the system.

6.3 A Deep Learning Approach to Biometrics

By using deep learning, features can be learned from the data, and indirect features,

when trained, will help them distinguish vast populations of people [68]. Further, if

there are sufﬁcient samples representing all these factors, a deep learning method can

uncover them during the learning of feature representations. With this approach, we

may be able to handle large differences within classes as well as unclean biometric

data. On the other hand, collecting data that have progressive variations over time

requires tremendous effort (e.g., voice data for gender determination, facial data for

age estimation). In situations like these, deep learning will probably be useful since

such variations can be blended.

In light of growing security concerns, privacy concerns, and the rising incidence of

cybercrime, scientists are also looking into behavioral biometrics as a way to verify

identities. Biometrics with a temporal element can be effectively captured with deep

learning. In addition to those that rely on local features, there are also methods based

on global representations. Deep learning is used to recognize faces using a framework

that includes both global and local characteristics. Typical deep learning architectures

for biometrics use Neural-Network, MLPs (typically Multi-Layer Perceptrons) [69].

MLPs are built by layering different input layers, a hidden layer, and an output

layer. The overﬁtting problem of MLP must be avoided, especially with non-

stationary and noisy biometric data. SoftMax activation functions are used in the

output layer of MLP to predict the posterior p(y,x). A number of biometric Neural-

Network architectures are available, including the RBF (Radial Basis Function)

Neural-Network and ANN (Artiﬁcial Neural-Network) [70], DBN (Deep Belief-

Networks) [71], CNN (Convolutional Neural Networks) [72], and RNN (Recurrent

Neural-Networks) [73]. In Table 3a number of approaches to pattern classiﬁcation

are listed.

7 Result Discussion

Biometrics technology is important for evaluating their performance. Various

biometric authentication techniques are reported by a number of parameters,

including FAR (false acceptance rate), FRR (false rejection rate), CER (crossover

rate), and EER (equal error rate) [5,52,53]. False Rejection refers to the rejection

of a true identity claim. False acceptance, on the other hand, refers to a wrongful

acceptance of an identity claim. FAR and FRR are used to restrict access to autho-

rized users. Essentially, False Rejection Rate (FRR) is the probability of incorrectly

rejecting an authorized user as invalid. The following method can be used to calculate

392 U. B. Ghosh et al.

Tabl e 3 Biometric recognition literature review of classiﬁcation techniques

Classiﬁer Application Reference

KNN Signature veriﬁcation, ﬁngerprint recognition,

iris recognition

M. Faundez-Zanuy

Bayes quadratic Facial expression recognition, hand geometry

recognition, gait recognition

P. S. R. C. Murty

SVM ECG, EEG, EMG biometric authentication O. C. Kurban

ANN Fingerprint recognition, ﬁnger vein recognition,

bioacoustics recognition

V. A. Akpan

MLP Vein identiﬁcation, ﬁngerprint recognition, gait

identiﬁcation

L. Ma

RBFNN Face recognition, ECG biometric authentication,

wrist vein recognition

V. Vasilakakis

CNN EEG biometric authentication, ﬁngerprint

recognition, iris recognition!

A. Czajka

DBN Face recognition, multimodal biometrics,

audio–video based biometrics

M. R. Alam

RNN Handwriting recognition, EEG, ECG based

biometrics

X.-Y. Zhang

it.

FRR =Number of False rejections

Number of Identiﬁcation Attempts

FAR (False Acceptance Rate) represents whether or not unauthorized users are

considered valid users, and it is calculated as follows. If a biometric system has a

low FAR, it is highly secure. Figure 19 illustrates FAR and FRR.

FAR =Number of False Acceptances

Number of Identiﬁcation Attempts

(a) (b)

Fig. 19 a Biometric system error rates, bEER and ROC

Symptoms-Based Biometric Pattern Detection and Recognition 393

The other recognition error rates are FTE (Failure to Enroll) and FTC (Failure

to Capture). An FTE represents the percentage of inputs that do not go through the

recognition process. FTC is that percentage of times that a biometric characteristic

is unable to capture when it is presented correctly.

A pair of matched samples contains the same individual; a pair of samples

containing a different individual contains Sand T.S—Similarity score, T— Accep-

tance threshold. The distribution consists of pairs of samples generated from different

individuals, whereas true distributions consist of pairs of samples derived from

the same individual (Fig. 19; Tables 4and 5)aIllustrates the intersection where

FAR equals FRR, also known as the EER (Equal Error Rate) or CER (Crossover

Rate), which demonstrates that false acceptances and false rejections have the same

proportion.

Tabl e 4 Evaluation of outcomes

Identiﬁcation/Criteria Fingerprint

(%)

Face

(%)

Iris

(%)

Retina

(%)

DNA

(%)

Voice

(%)

Keystroke

(%)

FAR 1.98 10.938 0.998 –2.00 2.00

FRR 2.00 10 0.987 0.039 –9.98 0.099

CER 1.99 –0.099 0.80 –5.97 1.80

FTE 1.97 –0.598 0.79 – – –

Tabl e 5 Performance based on signiﬁcant factors

Identiﬁcation

/Criteria

Fingerprint Face Iris Retina DNA Voi c e Keystroke

Socially

introduced

1981 2000 1995 1999 1965 1998 2005

Price Low Medium High High High Low Medium

Popularity High High Medium Low High High Low

Ease of use High High Medium Low Low High Low

Accuracy Medium Low High High High Medium Low

Stability High Medium Medium High High Medium Low

Speed High Medium Medium Medium Low High Medium

Safety Medium Medium High High High High Low

Error of

incidence

factors

Dryness,

Dirt, Age

Age,

Hair,

lighting

Glasses,

Poor

lighting

Glasses,

Contact

lens

Devices Noise,

cold

Device,

weather

394 U. B. Ghosh et al.

7.1 Latest Biometric Trends

(a) Smartphones: The smartphone market has the most potential for biometrics.

This phone utilizes biometric recognition including ﬁngerprint, voice, and face

recognition to unlock and lock. By doing so, it improves the efﬁciency of the

operation, as well as the data protection.

(b) Wearable devices: The current boom revolves around wearables that use

biometrics. Individuals’ heart rate, sweat, and brain activity are measured.

Thus, wearable devices can provide information about an individual’s health.

to online payment due to its insecurity. Therefore, biometrics, iris scans, and

facial recognition can be used to protect logins instead of credentials.

(d) Cloud biometrics: Cloud computing has gained much traction since it ensures

security, highly convenient, and a lot of space for all valuable information,

but there are still concerns about security. The security of intelligent environ-

ments, intelligent spaces, and access control applications can be enhanced by

deploying biometrics.

8 Conclusion

Based on physiological and behavioral characteristics, individuals are authenticated

automatically by biometrics. Despite the fact that the industry is still evolving and

developing, biometrics is gaining momentum as a reliable method of authentica-

tion [74–76]. Presently, relatively few applications use unimodal biometric recog-

nition systems because of the numerous challenges they face. By integrating data

at multiple levels, multimodal biometrics are able to address some limitations of

unimodal biometric systems. Consequently, biometrics in the future can be cate-

gorized as multimodal biometrics. Pattern recognition is the basis of biometrics.

Biometrics is an evolving technology widely used in forensics, security, ATMs,

smart cards, PCs, and networks. Biometrics is also more secure than the traditional

method of identiﬁcation. The report provides an overview of the literature review on

various identifying technologies and emphasizes the application of biometric recog-

nition systems. Biometric recognition systems are used to combat the drawbacks of

traditional methods of identiﬁcation systems. With the advancement of biometric

technology, biometric-based systems can also be improved.

We only know a tiny fraction about living things through the modern application of

biometric technology. Distance methods of identifying a person are used in elemen-

tary variants and are currently being researched in laboratories. As of today, the

primary forms of Biometric technology are ﬁngerprints, audio/vocal, signatures and

penmanship, facial features, hand biometric geometry, faces, keystroke dynamics,

and so on. Biometrics can allow for the identiﬁcation of individuals and can be used

in a variety of practical applications by automating the process. Biometric technology

brings many exciting effects to the educational process, the most important of which

Symptoms-Based Biometric Pattern Detection and Recognition 395

is that it stimulates students’ curiosity about methods of image analysis and pattern

identiﬁcation. A greater number of people will anticipate an increase in interest in

biometric technology. The forecast is based on the fact that biometric technology

supplements traditional means of identifying a person and is more effective in a

number of situations.

References

1. Wibawa, A. D., Purnomo, M. H. (2006). Early detection on the condition of the pancreas

organ as the cause of diabetes mellitus by real time iris image processing. In IEEE Asia Paciﬁc

Conference on Circuits and Systems.APCCS-2006 (pp. 1008–1010).

2. Ma, L., & Li, N. (2007). Texture feature extraction and classiﬁcation for iris diagnosis Lecture

notes in computer science (pp. 168–175). Springer.

3. Mukherjee, D., Tripathy, H. K., & Mishra, S. (2021). Scope of medical bots in clinical domain.

Techn Adv Mach Learn Healthcare, 936, 339.

4. Ramlee, R. A., Ranjit, S. (2009) Using iris recognition algorithm, detecting cholesterol pres-

ence. In: IEEE: Proceedings of international conference on information management and

engineering

5. Mallick, P. K., Mishra, S., Mohanty, B. P., & Satapathy, S. K. (2021). A deep neural

network model for effective diagnosis of melanoma disorder. In Cognitive informatics and

soft computing (pp. 43–51). Springer.

6. Lesmana, I. P. D., Purnama, I. K. E., & Purnomo, M. H. (2011). Abnormal condition detection

of pancreatic beta-cells as the cause of diabetes mellitus based on iris image. In International

Conference on Instrumentation, Communication, Information Technology and Biomedical

Engineering.

7. Chattopadhyay, A., Mishra, S., & González-Briones, A. (2021). Integration of machine learning

and IoT in healthcare domain. In Hybrid artiﬁcial intelligence and IoT in healthcare (pp. 223–

244). Springer.

8. Jensen, B. (1985). The science and practice of iridology (Vol. 1). California Bernard Jensen

Co.

9. Jain, A. K., & Prabhkar, S. (2001). Fingerprint matching using minutiae and texture features.

In Proceeding of International Conference on Image Processing (ICIP) (pp. 282–285).

10. Golfarelli, M., Maio, D., & Maltoni, D. (1997). On the error-reject trade-off in biometric

veriﬁcation systems. IEEE Transations on PAMI, 19(7), 786–796.

11. Tripathy, H. K., Mallick, P. K., & Mishra, S. (2021). Application and evaluation of classiﬁcation

model to detect autistic spectrum disorders in children. International Journal of Computer

Applications in Technology, 65(4), 368–377.

12. Mishra, S., Thakkar, H., Mallick, P. K., Tiwari, P., & Alamri, A. (2021). A sustainable IoHT

based computationally intelligent healthcare monitoring system for lung cancer risk detection.

Sustainable Cities and Society, 103079.

13. Tripathy, H. K., Mishra, S., Thakkar, H. K., & Rai, D. (2021). CARE: A collision-aware

mobile robot navigation in grid environment using improved breadth ﬁrst search. Computers &

Electrical Engineering, 94, 107327.

14. Fernandes, J. M., Soares, F. O., Minas, G. (2013). RH phenotypes analysis by spec-

trophotometry in human blood typing. In Proceedings of the IEEE 3rd Portuguese meeting

(pp. 133–136).

396 U. B. Ghosh et al.

15. Ravindran, G., Joby, T., Pravin, M., & Pandiyan, P. (2017). Determination and classiﬁcation

of blood types using image processing techniques. International Journal of Computers and

Applications, 157(1), 12–16.

16. Chellappa, R., Wilson, C. L., & Sirohey, C. (1995). Human and machine recognition of faces:

Asurvey.Proceedings of IEEE, 83(5), 705–740.

17. Abate, A. F., Nappi, M., Riccio, D., & Sabatino, G. (2007). 2D and 3D face recognition: A

survey. Pattern Recognit Letters, 28, 1885–1906

18. Turk, M., & Pentland, A. (1991). Eigenfaces for recognition. Journal of Cognitive Neuro-

science, 3, 71–86.

19. Belhumeur, P. N., Hespanha, J. P., & Kriegman, D. J. (1997). Eigenfaces vs. Fisherfaces:

recognition using class speciﬁc linear projection. IEEE Transactions on Pattern Analysis and

Machine Intelligence, 19, 711–720.

20. Bartlett, M. S., Movellan, J. R., & Sejnowski, T. J. (2002). Face recognition by independent

component analysis. IEEE Transactions on Neural Networks, 13, 1450–1464.

21. SCHOLKOPF B., MIKA S., BURGES C. J. C., KNIRSCH P., MULLER K. R., RATSCH G.

AND SMOLA A. J., 1999. Input space versus feature space in kernel-based methods. IEEE

Transactions on Neural Networks, Vol. 1, pp. 1000–1017.

22. Liu, C. J. (2004). Gabor-based kernel PCA with fractional power polynomial models for face

recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 26, 572–581.

23. Baudat, G., & Anouar, F. E. (2000). Generalized discriminant analysis using a kernel approach.

Neural Computation, 12, 2385–2404.

24. Lawrence, S., Giles, C. L., Tsoi, A. C., & Back, A. D. (1998). Face recognition: A convolutional

neural network approach. IEEE Transactions on Neural Networks, 8, 98–113.

25. Phillips, P. J. (1999). Support vector machines applied to face recognition. In Proceedings of

the 1998 Conference on Advances in Neural Information Processing Systems II.MITPress

(pp. 803–809).

26. Cootes, T. F., Taylor, C. J., Cooper, D. H., & Graham, J. (1995). Active shape models—Their

training and application. Computer Vision and Image Understanding, 61, 38–59.

27. Wiskott, L., Fellous, J. M., Kruger, N., & Malsburg, C.V. (1997). Face recognition by elastic

bunch graph matching. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19,

775–779.

28. Sahoo, S., Mishra, S., Mishra, B. K. K., & Mishra, M. (2018). Analysis and implementation

of artiﬁcial bee colony optimization in constrained optimization problems. In Handbook of

Research on Modeling, Analysis, and Application of Nature-Inspired Metaheuristic Algorithms

(pp. 413–432). IGI Global.

29. Victor, B., Bowyer, K.W., & Sarkar, S. (2002). An evaluation of face and ear biometrics. In

Proceedings of International Conference on Pattern Recognition (pp. 429–432).

30. Mishra, S., Mahanty, C., Dash, S., & Mishra, B. K. (2019). Implementation of BFS-NB hybrid

model in intrusion detection system. In Recent Developments in Machine Learning and Data

Analytics (pp. 167–175). Springer.

31. Daugman, J. G. (1993). High conﬁdential visual recognition by test of statistical independence.

IEEE Transactions on Pattern Analysis and Machine Intelligence, 15, 1148–1161.

32. Sanchez-Avila, C., & Sanchez-Reillo, R. (2002). Iris-based biometric recognition using dyadic

wavelet transform. IEEE Aerospace and Electronic Systems Magazine, 17, 3–6.

33. Wildes, R. P. (1997). Iris recognition: An emerging biometric technology. Proceedings of the

IEEE, 85, 1348–1363.

34. Boles, W. W., & Boashash, B. (1998). A human identiﬁcation technique using images of the

iris and wavelet transform. IEEE Transactions on Signal Processing, 46, 1185–1188.

35. Lim, S., Lee, K., Byeon, O., & Kim, T. (2001). Efﬁcient iris recognition through improvement

of feature vector and classiﬁer. ETRI Journal, 23, 1–70.

Symptoms-Based Biometric Pattern Detection and Recognition 397

36. Park, C., Lee, J., Smith, M., & Park, K. (2003). Iris-based personal authentication using a

normalized directional energy feature. In Proceedings of the Fourth International Conference

on Audio- and Video-Based Biometric Person Authentication (pp. 224–232).

37. Mishra, S., Tadesse, Y., Dash, A., Jena, L., & Ranjan, P. (2021). Thyroid disorder analysis

using random forest classiﬁer. In Intelligent and cloud computing (pp. 385–390). Springer.

38. Roy, S. N., Mishra, S., & Yusof, S. M. (2021). Emergence of drug discovery in machine

learning. Technical Advancements of Machine Learning in Healthcare, 936, 119.

39. Du, Y., Ives, R. W., Etter, D. M., & Welch, T. B. (2006). Use of one-dimensional iris signatures

to rank iris pattern similarities. Optical Engineering, 45, 037110–037201.

40. Mishra, S., Dash, A., Ranjan, P., & Jena, A. K. (2021). Enhancing Heart Disorders Predic-

tion with Attribute Optimization. In Advances in electronics, communication and computing

(pp. 139–145). Springer.

41. Sanchez-Reillo, R. (2000). Hand geometry pattern recognition through Gaussian mixture

modeling. In 15th International Conference on Pattern Recognition (Vol. 2) (pp. 937–940).

42. Wong, A. L. N., & Shi, P. (2002). Peg-free hand geometry recognition using hierarchical geom-

etry and shape matching. In Proceedings of IAPR Workshop on Machine Vision Applications,

Japan (pp. 281–284).

43. Jena, L., Mishra, S., Nayak, S., Ranjan, P., & Mishra, M. K. (2021). Variable optimiza-

tion in cervical cancer data using particle swarm optimization. In Advances in Electronics,

Communication and Computing (pp. 147–153). Springer.

44. Kong, A., Zhang, D., & Lu, G. M. (2006). A study of identical Twin‘s palm prints for personal

veriﬁcation. Pattern Recognition, 39(11), 2149–2156.

45. Wu, X. Q., Wang, K. Q., & Zhang, D. (2005). Palmprint authentication based on orientation

code matching. In AVBPA. Lecture Notes in Computer Science (Vol. 3546, pp. 555–562).

46. Ray C., Tripathy, H. K., Mishra, S. (2019). A review on facial expression based behavioral

analysis using computational technique for autistic disorder patients. In: M. Singh, P. Gupta,

V. Tyagi, J. Flusser, T. Ören, R. Kashyap (Eds.), Advances in computing and data sciences.

ICACDS 2019.Communications in Computer and Information Science (Vol. 1046). Springer.

https://doi.org/10.1007/978-981-13-9942-8_43

47. Tripathy, H. K.,Mishra, S., Suman, S., Nayyar, A., & Sahoo, K. S. (2022). Smart COVID-shield:

an IoT driven reliable and automated prototype model for COVID-19 symptoms tracking.

Computing, 1–22.

48. Mishra, S., Sahoo, S., & Mishra, B. K. (2019). Addressing security issues and standards in

Internet of things. In Emerging trends and applications in cognitive computing (pp. 224–257).

IGI Global.

49. Monrose, F., & Rubin A. (1997). Authentication via keystroke dynamics. In Proceedings of

4th ACM Conference on Computer and Communications Security, Switzerland, Apr 1997,

(pp. 48–56).

50. Biel, I., Pettersson, O., Philipson, L., & Wide, P. (2001). ECG analysis, a new approach in human

identiﬁcation. IEEE Transactions on Instrumentation and Measurement, 50(3), 808–812.

51. Yongjin, W., Agraﬁoti, F., Hatzinakos, D., Plataniotis, K. N. (2008). Analysis of human elec-

trocardiogram for biometric recognition. EURASIP Journal on Advances in Signal Processing,

January 2008.

52. Miyamoto, C., Baba, S., & Nakanishi, I. (2009). Biometric person authentication using new

spectral features of electroencephalogram (EEG). In: Proceedings of International Symposium

on Intelligent Signal Processing and Communications Systems (ISPACS), 2009 (pp. 1–4).

53. Mishra, S., Mishra, B. K., & Tripathy,H. K. (2015, December). A neuro-genetic model to predict

hepatitis disease risk. In 2015 IEEE International Conference on Computational Intelligence

and Computing Research (ICCIC) (pp. 1–3). IEEE.

54. Palaniappan, R., Mandic, D. P. (2017). Biometrics from brain electrical activity, A machine

learning approach. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29, 738–

742.

398 U. B. Ghosh et al.

55. Riera, A., Soria-Frisch, A., Caparrini, M., Grau, C., Rufﬁni, G. (2008). Unobtrusive biometric

system based on electroencephalogram analysis. EURASIP Journal on Advances in Signal

Processing, 1–8.

56. Hu, J. (2009). New biometric approach based on motor imagery EEG signals. In Proceedings of

International Conference on Future BioMedical Information Engineering FBIE 2009 (pp. 94–

97).

57. Eriksson, A., Wretling, P. (1997). How ﬂexible is the human voice? A case study of mimicry.

In Proceedings of European Conference on Speech Technology, Rhodes, Greece, Sept 1997

(pp. 1043–1046).

58. Furui, S. (1997). Recent advances in speaker recognition. In Proceedings of First Interna-

tional Conference on Audio and Video based Biometric Person Authentication, UK, Mar 1997

(pp. 859–872).

59. Sahoo, S., Das, M., Mishra, S., & Suman, S. (2021). A hybrid DTNB model for heart disorders

prediction. In Advances in electronics, communication and computing (pp. 155–163). Springer.

60. Bharati, A., Singh, R., Vatsa, M., & Bowyer, K. W. (2016). Detecting facial retouching using

supervised deep learning. IEEE Transactions on Information Forensics and Security, 11(9),

1903–1913. https://doi.org/10.1109/TIFS.2016.2561898

61. Thomaz, C. E., Gillies, D. F., & Feitosa, R. Q. (2004). A new covariance estimate for Bayesian

classiﬁers in biometric recognition. IEEE Transactions on Circuits and Systems for Video

Technology, 14(2), 214–223. https://doi.org/10.1109/TCSVT.2003.821984

62. Azmi, A. N., Nasien, D., & Omar, F. S. (2017). Biometric signature veriﬁcation system based

on freeman chain code and k-nearest neighbor. Multimedia Tools and Applications, 76(14),

15341–15355. https://doi.org/10.1007/s11042-016-3831-2

63. Sousedik, C., & Busch, C. (2014). Presentation attack detection methods for ﬁngerprint recog-

nition systems: A survey. IET Biometrics, 3(4), 219–233. https://doi.org/10.1049/iet-bmt.2013.

0020

64. Tolba, A. S., El-Baz, A. H., & El-Harby, A. A. (2006). Face recognition: A literature review.

International Journal of Signal Processing, 2(2), 88–103.

65. Rath, M., & Mishra, S. (2019). Advanced-level security in network and real-time applications

using machine learning approaches. In Machine learning and cognitive science applications

in cyber security (pp. 84–104). IGI Global.

66. Jain, L. C., Halici, U., Hayashi, I., Lee, S. B., & Tsutsui, S. (1999). Intelligent biometric

techniques in ﬁngerprint and face recognition (Vol. 10). CRC press.

67. Faundez-Zanuy, M. (2005). Signature recognition state-of-the-art. IEEE Aerospace and

Electronic Systems Magazine, 20(7), 28–32. https://doi.org/10.1109/MAES.2005.1499249

68. Mishra, S., Dash, A., & Mishra, B. K. (2020). An insight of Internet of Things applications in

pharmaceutical domain. In Emergence of pharmaceutical industry growth with industrial IoT

approach (pp. 245–273). Academic Press.

69. Ma, L., Minett, J. W., Blu, T., & Wang, W. S. Y. (2015). Resting state EEG-based biometrics

for individual identiﬁcation using convolutional neural networks. In Proceedings of EMBC,

Milan, Italy (pp. 2848–2851). https://doi.org/10.1109/EMBC.2015.7318985.

70. Jena, L., Kamila, N. K., & Mishra, S. (2014). Privacy preserving distributed data mining

with evolutionary computing. In Proceedings of the International Conference on Frontiers of

Intelligent Computing: Theory and Applications (FICTA) 2013 (pp. 259–267). Springer.

71. Mishra, S., Tripathy, H. K., & Acharya, B. (2021). A precise analysis of deep learning for

medical image processing. In Bio-inspired Neurocomputing (pp. 25–41). Springer.

72. Czajka, A., Bowyer, K. W., Krumdick, M., & VidalMata, R. G. (2017). Recognition of image-

orientation-based iris spooﬁng. IEEE Transactions on Information Forensics and Security,

12(9), 2184–2196. https://doi.org/10.1109/TIFS.2017.2701332

73. Zhang, X.-Y., Xie, G.-S., Liu, C.-L., & Bengio, Y. (2017). End-to-end online writer identiﬁ-

cation with recurrent neural network. IEEE Transactions on Human-Machine Systems, 47(2),

285–292. https://doi.org/10.1109/THMS.2016.2634921

74. Murty, P. S. R. C., Reddy, E. S., & Babu, I. R. (2009). Iris recognition system using fractal

dimensions of Haar patterns. International Journal of Signal Processing, Image Processing

and Pattern Recognition, 2(3), 75–84.

Symptoms-Based Biometric Pattern Detection and Recognition 399

75. O. C. Kurban, Ö. Nıyaz, and T. Yildirim, “Neural network based wrist vein identiﬁcation using

ordinary camera,” in Proc. INISTA, Sinaia, Romania, 2016, pp. 1–4, doi:https://doi.org/10.

1109/INISTA.2016.7571860.

76. Vasilakakis, V., Cumani, S., Laface, P., & Torino, P. (2013). Speaker recognition by means of

deep belief networks. In Proceedings of Biometric Technologies in Forensic Science, Nijmegen,

Netherlands, 2013 (pp. 52–57). Available:https://cls.ru.nl/staff/dvleeuwen/btfs-2013/procee

dings-btfs2013.pdf

Time Series Analysis of COVID-19

Waves in India for Social Good

Lakshmi Swarna Durga Nallam, Sindhu Sankati, Hiren Kumar Thakkar,

and Priyanka Singh

Abstract In the past decade, the world has seen rapid advancements in the ﬁeld

of healthcare services due to the state of the arts in technologies. Several real-time

health monitoring applications and products are designed to assist the human to

take the timely precautionary measures to avoid the unseen abnormalities. How-

ever, current healthcare monitoring infrastructures are not ready to provide efﬁ-

cient health services during the sudden and unknown pandemic situations such as

COVID-19. The COVID-19 started in the later part of 2019, rapidly spread across

the countries and labeled as a pandemic in the very early part of the 2020. Several

people died due to the lack of the healthcare infrastructure and lack of access to

health facilities. This book chapter explores the various technologies such as aug-

mented reality, connected e-health along with the time series analysis of COVID-

19 waves in India to know the implication of COVID-19 on society for a social

good.

Keywords Time series analysis ·COVID-19 waves ·Augmented reality ·

Connected e-health ·Data smoothening

L. S. D. Nallam ·S. Sankati ·P. Singh

Department of Computer Science and Engineering, SRM University, Andhra Pradesh,

Amaravati 522502, India

e-mail: swarna_durga@srmap.edu.in

S. Sankati

e-mail: sindhu_sankati@srmap.edu.in

P. Singh

e-mail: priyanka.s@srmap.edu.in

H. K. Thakkar (B)

Department of Computer Engineering, Marwadi University, Rajkot, Gujarat 360006, India

e-mail: iamhiren@gmail.com

S. Mishra et al. (eds.), Augmented Intelligence in Healthcare: A Pragmatic and Integrated

Analysis, Studies in Computational Intelligence 1024,

https://doi.org/10.1007/978-981-19- 1076-0_20

401

402 L. S. D. Nallam et al.

1 Introduction

In the past few decades, the healthcare monitoring becomes the need of the hour due

to the changing lifestyle, unhealthy food habits and irregular sleeping hours [1–5].

Several real-time and continuous health monitoring systems are being designed to

support the healthy livings using various intelligent and machine learning models [6–

10]. Due to the emergence of Artiﬁcial Intelligence and Machine Learning, health

monitoring systems become efﬁcient and accurate in early warning of vital signs

generation [11–14]. In some instances, researchers engage the mobile robot navi-

gation in hospital environments for effective healthcare delivery [15]. In addition to

that, several contagious diseases are spreading due to the lack of hygiene and proper

care. In past one year, several countries experience the sudden spread of the novel

coronavirus disease (COVID-19), which is an infectious disease caused by “severe

acute respiratory syndrome coronavirus 2” (SARS-CoV-2). Due to its rapid spread

and lack of the medicine and treatments, the World Health Organization (WHO) has

declared COVID-19 as a pandemic on March 11, 2020. After a more than year by

June 2021, an estimated population of more than 180 million people are infected by

the virus globally (as of June 30, 2021) resulting into several thousands of deaths

including severe ﬁnancial and mental burden on those who survived. With rising

variants of SARS-CoV-2, many regions worldwide are experiencing several waves

of COVID-19 and the handiest tools most governments across the nations using is

to impose the lockdown. Due to the prolonged periods of subsequent waves, it will

continue to pose an imminent threat to society leading to devastating social and

economic impacts.

Considering the emergency situations, researchers have started exploring the

emerging technologies such as artiﬁcial intelligence, machine learning to analyze

the healthcare data [3,4]. Additionally, technologies such as augmented reality and

connected e-health are also explored to identify the possibilities of efﬁcient collec-

tion of data for further remote processing. Most of the current healthcare monitoring

systems are dependent on wireless communication technologies and employ tiny

sensors to retrieve and communicate the vital body sign for remote processing [9].

Since those body sensors are connected among each other by means of wireless

technologies and work together, they generate an e-connected health monitoring plat-

form. Several e-connected health monitoring platforms are invented to support the

remote health monitoring and continuous health parameter monitoring. COVID-19

is a contagious disease and can be identiﬁed by prominent symptoms such as fever,

dry cough, tiredness, sore throat, loss of taste and smell [16]. In order to identify

the presence of COVID-19 infection, it is essential to monitor the above-mentioned

symptoms on regular basis, which requires an efﬁcient COVID-19 symptom mon-

itoring e-connected health system. In this book chapter, we analyze the time series

COVID-19 health data to understand the spread of the disease in the ﬁrst and second

waves in the Indian context.

Time Series Analysis of COVID-19 Waves in India for Social Good 403

1.1 Indian Circumstances During the First Wave

and the Second Wave

According to the report from [17], there are several evident differences in the ﬁrst

and the second wave. Several factors could have led to a rise in the reproduction rate

(R0) in India. For COVID-19 patients, there is a shortage of hospital beds, oxygen

and ventilators in India. Although there is no signiﬁcant percentage increase in the

death rate in the second wave due to an alarmingly high number of infections, the

total death numbers are disappointingly high.

1.1.1 First Wave

The ﬁrst case of COVID-19 in India was reported on January 30, 2020. On June

10, recoveries exceeded the number of active patients for the ﬁrst time in India. The

number of active cases peaked in mid-September with 90,000 active cases per day and

dropped to 15,000 conﬁrmed cases by January 2021. During the ﬁrst wave, COVID-

19 primarily affects the respiratory system, by causing breathlessness and acute

respiratory problems. In this period, the information of COVID-19 is unsubstantial

such as its effects, origin and treatment. The shortage of drugs made people buy

in black markets [17]. It affected mainly the older population and patients with

comorbidities [17]. Healthcare workers are less trained, panicked about the spread

of infection and are not vaccinated. Bed capacity, ventilator beds, laboratory testing

and PPE kits were scarce, and the test price was high [17]. There is no availability

of vaccination in the ﬁrst wave. Several mutants of the SARS-Cov-2 virus mutated,

caused new variants and caused the spread of COVID-19 infection in the second

wave. The transmission rate of disease is fast compared to the ﬁrst wave.

1.1.2 Second Wave

A second wave beginning in March 2021 was much more immense than the ﬁrst,

with shortages of vaccines, hospital beds, oxygen cylinders and other medicines. On

April 30, 2021, India became the ﬁrst country to report over 400,000 active cases

in 24 h in the second wave. The Indian government began the vaccination program

on January 16, 2021. Indian government authorized the British Oxford-AstraZeneca

vaccine (Covishield), the Indian BBV152 (Covaxin) vaccine and the Russian Sputnik

V vaccine for emergency use [18]. In the second wave, new symptoms like gastroin-

testinal symptoms are noticed [17]. Compared to the ﬁrst wave, younger people are

much more affected. There is an increase in cases of breathlessness in the second

wave. Supplementary amounts of drugs are available in the second wave. Bed capac-

ity, ventilator beds, laboratory testing and PPE kits are enhanced compared to the

ﬁrst wave, and they are not sufﬁcient for the patients [17]. The government of India

approved three vaccinations in the second wave.

404 L. S. D. Nallam et al.

2 Effect of Variants

India has been experiencing a recent surge in cases from March; this period is com-

monly referred to as the second wave. Though the factors for the steep rise in cases

in the second wave include the lack of preparedness of the Indian government, relax-

ation of social distancing policies, shortage in medical facilities, high population and

population density, one can still say that an important factor was the delta (B.1.617.2)

variant which is characterized by high transmissibility and lower incubation periods.

The delta variant has a 55% increase in the effective reproduction number compared

to the alpha variant (Weekly Epidemiological Update on COVID-19—6 July 2021,

2021). Due to this high transmissibility of the delta variant, we can observe that 101

countries have already reported cases about delta variant as of the 6th July weekly

report of WHO (2021). The geographical spread of the four major variants alpha,

beta, gamma and delta is evident. It is also estimated that the delta variant will outpace

the other variants and may become the dominant variant in the next few months.

2.1 SARS-CoV-2 Variants of Interest and Variants

of Concern

According to the Weekly Epidemiological Update on COVID-19—6 July 2021,

WHO deﬁnes Variants of Concerns as “the SARS-CoV-2 Variant” which is associ-

ated with one or more of the following changes at a degree of global public health

signiﬁcance:

•Increase in transmissibility or detrimental change in COVID-19 epidemiology.

•Increase in virulence or change in clinical disease presentation.

•Decrease ineffectiveness of public health and social measures or available diag-

nostics, vaccines, therapeutics.

And Variants of Interest as “A SARS-CoV-2 variant”:

•With genetic changes that are predicted or known to affect virus characteristics

such as transmissibility, disease severity, immune escape, diagnostic or therapeutic

escape.

•Identiﬁed to cause signiﬁcant community transmission or multiple COVID-19

clusters, in multiple countries with increasing relative prevalence alongside an

increasing number of cases over time, or other apparent epidemiological impacts

to suggest an emerging risk to global public health.

As of July 29, 2021, the VOC and VOI variants which describe the efﬁciency of

available vacancies on the different variants of SARS-CoV-2. RNA viruses including

SARS-CoV-2 can mutate faster due to their error-prone coping mechanism. This can

lead to the emergence of multiple variants in a single individual, some of which

have a survival advantage in terms of greater afﬁnity to the host receptor, faster

Time Series Analysis of COVID-19 Waves in India for Social Good 405

replication rates and the ability to evade the host immune response. In a cohort

study [19], 417 people who have received the second dose of vaccination of either

BNT162b2 (Pﬁzer-BioNTech) or mRNA-1273 (Moderna), at least 2 weeks earlier.

The researchers identiﬁed two women who were tested positive for SARS-CoV-2

despite showing the signs of vaccine efﬁcacy [19]. This indicates that there is a

potential for reinfection even after a successful vaccination. These characteristics of

COVID-19 have made it a very difﬁcult task to completely eradicate the pandemic,

though local elimination of the virus has been successful to some extent in some

regions of the world. While places like India are facing a conjugacy-like state [20].

3 Methodology

3.1 Dataset Description

The data is taken from [21], which are brieﬂy described and summarized in Fig. 1,

and the variables used for this study are as follows:

•Date: It is described in the format dd/mm/yyyy. The difference between periods

is for 1 year.

•New cases: It is also known as a conﬁrmed case. They are observed by performing

a viral test. If one test is positive, then they are positive. The number of positive

cases in a day is described in the data.

•New tests: They are performed to determine whether SARS-CoV-2 antibody exists

in a person or not. They are often referred to as serology. Tests look for antibodies

in a sample to ﬁnd if an individual has a past infection with a virus that causes

COVID-19. The number of tests taken in a day is described in the data.

•New deaths: If a conﬁrmed case is deceased because of COVID-19, then that case

is counted as new death. The number of deaths that happens in a day is described

in the data.

•New vaccination: The vaccine is a substance used to stimulate the production of

antibodies and boost the immune system. The number of vaccines taken in a day

is described in the data.

•Stringency index: It is a composite measure of nine response indicators such as

school closure, workplace closure, travel bans, etc., and is rescaled from 0 to 100

(100 is the strictest). The data consisted of the stringency index of a day [21].

The majority of the methodology follows the study of [22] which analyzes the

ﬁrst and second waves in Italy, while in this study we applied a similar methodology

for Indian data. Figure 2shows the basic trend of COVID-19 variables (daily cases,

daily deaths, daily tests and daily vaccinations).

406 L. S. D. Nallam et al.

Fig. 1 Data summary and description

Fig. 2 Trend of fatality rates of the ﬁrst and second waves of COVID-19 in India; time series is

presented in log scale

3.2 Deﬁning the Time Sets for the First Wave

and Second Waves

The beginning of the ﬁrst wave in India is considered from April 1, 2020, since the

cases have signiﬁcant rise after that period and the end of the ﬁrst wave is considered

on the February 3, 2021; around this period, there are a minimum number of cases.

After this date, i.e., February 4, 2021, we take it as the second wave, till the latest

available date July 14, 2021. In summary,

Time Series Analysis of COVID-19 Waves in India for Social Good 407

•The ﬁrst wave of COVID-19 is from April 1, 2020 to February 3, 2021.

•The second wave of COVID-19 is from February 4, 2021 to July 14, 2021.

Note: The second wave of COVID-19 in India is still ongoing as of writing this

report, i.e., July 16, 2021.

3.3 Data Standardization

The data reports usually have an intrinsic delay, i.e., they are usually reported after

some time the event has passed. Therefore, we standardize the data for this study to

get more appropriate results.

•Daily case–test ratio: NewCases (t)

NewTests (t−2).

•Daily fatality rate: NewDeaths (t)

NewCases (t−14).

The lag of two indicates the amount of time it takes for the result to be given after

the testing [22].

3.3.1 Descriptive Analysis of Data

Data is analyzed using descriptive statistics; this is by comparing ﬁrst and second

waves by using mean, standard deviation, minimum and maximum values.

3.3.2 Smoothening of Data

The reporting of COVID-19 data usually comprises weekly ﬂuctuations which are

caused due under-reporting on a few days and over-reporting on others. For elim-

inating the original time series from weekly seasonal variation, we apply a simple

moving average of the period of length r=7 days (week). It is calculated by using

the following formula.

¯yt=yt−3+yt−2+yt−1+yt+yt+1+yt+2+yt+3+yt+4

r(1)

Here, rrepresents the period for a moving average, which is set to 7. ytrepresents

the original series, and ¯ytrepresents the smoothened series. For the rest of the study,

we use SMA to represent the time series with a simple moving average of 7 periods

calculated using the above formula.

408 L. S. D. Nallam et al.

3.3.3 Model of Data

For examining the relationship between various variables, we analyze their correla-

tion and association, for which we use a regression model based on a linear relation-

ship, in which we model the variables as a linear function of time. The model used

for analysis is as follows.

log(yt)=α+βt+(2)

Here, ytrepresents the time series. trepresents the time. represents an error

term, and αand βrepresent the constants which need to be estimated using the linear

model. We estimate the constants using the ordinary least square (OLS) method. For

the study, the OLS is performed in Python using the statsmodel.api library.

4 Observations and Results

The descriptive statistics of cases, test–case ratio and fatality rate of COVID-19 from

April 2020 to July 2021 are shown in Fig.3. On the contrary, descriptive statistics

of vaccinations, stringency index, of COVID-19 from April 2020 to July 2021, is

shown in Fig. 4. The ﬁrst wave of the pandemic has an average of about 0.03 million

cases daily with a test–case ratio of 5%. In the second wave, the average case rose to

0.12 million with a test–case ratio of 8%, indicating that there have been lower tests

performed in comparison to the transmission rates in the second wave. Though there

were signiﬁcantly higher cases, the fatality rate has gone down. The fatality rate was

4% during the ﬁrst wave and 1% during the second wave. There were no vaccines

during the ﬁrst wave. Making a comparison of the vaccination rates of the ﬁrst wave

and the second wave would result in insigniﬁcant results on the effect of vaccination

on the overall impacts of COVID-19. The steeper curve in the second wave as shown

in Fig. 5may be explained by the decrease in the average stringency index (Fig.3).

From Figs. 5and 6, we can say that in the conﬁrmed cases there is an upward trend

during the onset of spring during both the ﬁrst wave and the second wave, while the

ﬁrst wave has a more ﬂattened curve, the second wave we can observe that there is a

steeper curve. There is a declining trend toward winter in the ﬁrst wave. This decline

may be due to stricter preventive policies and measures. The upward trend of both

the COVID-19 waves in India took place during the summer months. This trend is

not consistent with some studies that indicate that the novel coronavirus shows a

decreased transmission dynamic that is likely due to hot and dry temperatures in the

summer season. As a whole, the stringency index of the second wave is lower than

that of the ﬁrst wave, which could have contributed to higher cases in the second

wave than in the ﬁrst wave (Fig. 7).

Figure 8shows that there is a lower fatality rate in the second wave than com-

pared to the ﬁrst wave. The general trend of fatality rate in the second wave seems

to be positive (uptrend). A potential cause for this could be related to the higher

hospitalization due to higher cases.

Time Series Analysis of COVID-19 Waves in India for Social Good 409

Fig. 3 Descriptive statistics of cases, test–case ratio and fatality rate of COVID-19 (from April

2020 to July 14, 2021)

Fig. 4 Descriptive statistics of vaccinations and stringency index of COVID-19 (from April 2020

to July 14, 2021)

Fig. 5 Trend of daily cases of the ﬁrst and second waves of COVID-19 in India; time series is

presented in log scale

410 L. S. D. Nallam et al.

Fig. 6 Trend of the case–test ratio of the ﬁrst and second waves of COVID-19 in India; time series

is presented in log scale

Fig. 7 Trend of stringency index of the ﬁrst and second waves of COVID-19 in India; time series

is presented in log scale

Fig. 8 Trend of fatality rates of the ﬁrst and second waves of COVID-19 in India; time series is

presented in log scale

Time Series Analysis of COVID-19 Waves in India for Social Good 411

Fig. 9 Descriptive statistics of cases, test–case ratio and fatality rate of COVID-19 (from April

2020 to July 14, 2021)

Fig. 10 Bivariate correlation matrix of variables under the study of the second wave of the COVID-

19 pandemic

From Fig. 9, vaccinations have a moderate negative association with case–test

ratio (r=−0.48) and low negative association with fatality rate (r=−0.29) and

stringency index (r=−0.27). The correlation between fatality rate and stringency is

r=0.624 indicating a moderate-high positive association, though few studies have

shown higher stringency rate (preventive measures like lockdowns, social distanc-

ing, masks) leads to lesser cases. We may well observe this because there was a lack

of understanding about pathological features of the virus. While stringency test and

case–test ratio have no moderate positive association (r=5.024) with each other.

Fatality and case–test have a low positive association (r=0.33). The observation

indicates that the cases decrease due to vaccinations. We need to note that vaccina-

tions are only available for the ﬁnal few weeks; therefore, we cannot form a causal

relationship between vaccination and daily cases.

Figure 10 shows the bivariate correlation matrix of variables under the study of

the second wave of the COVID-19 pandemic. Fatality rates have a high positive asso-

ciation with vaccinations (r=0.72) and low-moderate positive correlation between

412 L. S. D. Nallam et al.

Fig. 11 Summary of the parameters of analysis of different time series using OLS

case–test ratio (r=0.37) and stringency index (r=0.45). The correlation coefﬁ-

cient between case–test and vaccination is r=0.24, which appears to be lower than

the expected values, this can be explained by comparing the vaccination to the popu-

lation, and also the fact that the second wave is caused due to the delta variant which

showed resistance to vaccination. A greater percentage of the population should be

vaccinated for the correlation between vaccination and case–test ratio to be neg-

ative. Figure 11 shows the estimation of parameters of log-linear models between

dependent variables and time as explanatory variables.

Figure 12 shows that the ﬁrst wave of novel coronavirus has declining evolutionary

trends of daily case–test ratio and fatality rate along with almost stationary stringency

index, i.e., the preventive measures taken were more or less same throughout the ﬁrst

wave. The fatality rate and case–test ratio have similar trends. Figure 13 shows that the

second wave of novel coronavirus has a stationary trend in vaccination rates. While

there seems to be signiﬁcant growth in cases which can be said due to the onset of

delta variation and also relaxation of lockdown measures. While the decline also can

be correlated with reinforcement of lockdown partially, and also understanding of

the delta variation. For more details, the context comparison of ﬁrst wave and the

second wave of COVID-19 in India is shown in Fig. 14.

Time Series Analysis of COVID-19 Waves in India for Social Good 413

Fig. 12 Trend of fatality rate, the case–test ratio of the ﬁrst wave of COVID-19 in India; time series

is presented in log scale

Fig. 13 Trend of vaccination and fatality rate in India; time series is presented in log scale

5 Conclusion

Connected e-health is an essential requirement to design an efﬁcient health moni-

toring platform. COVID-19 is one of the life-threatening diseases, which requires

continuous body parameter monitoring for its effective early identiﬁcation. With this

study, we tried understanding the trends between the second wave and ﬁrst wave

of COVID-19 in India, which is a country characterized by tropical monsoon, high

temperatures and dry winters. We found that the fatality of the ﬁrst wave is higher

than the second wave though there are signiﬁcantly more cases during the second

414 L. S. D. Nallam et al.

Fig. 14 Context comparison of the ﬁrst wave and the second wave of COVID-19 in India

wave. But the total number of deaths is higher during the second wave correspond-

ing to high cases. Preventive measures like lockdown make social distancing bring

declining progression of COVID-19 cases. A summary of the observation can be

shown in the table: Which is consistent with global studies of COVID-19. This study

reveals that a country with a background like India will continue to have spikes in

COVID-19 cases due to variants unless there is a signiﬁcant population vaccinated.

These spikes can have higher increases due to relaxation of preventing measures.

Multiple strategies are required to handle the current outbreak including computa-

tional modeling, statistical tools and quantitative analyses to control the spread and

the outbreak of the pandemic. The ﬁrst wave of the covid pandemic caused the death

of millions of people in the world. The lack of awareness about novel coronavirus

and specialized equipment added severity to the ﬁrst phase. Though implementation

of lockdowns and safety measures were fastidious, the second wave is much more

extreme.

References

1. Vedaei, S. S., Fotovvat, A., Mohebbian, M. R., Rahman, G. M., Wahid, K. A., Babyn, P.,

Marateb, H. R., Mansourian, M., & Sami, R. (2020). COVID-SAFE: An IoT-based system for

automated health monitoring and surveillance in post-pandemic life. IEEE Access, 8, 188538–

188551.

2. Swayamsiddha, S., & Mohanty, C. (2020). Application of cognitive internet of medical things

for COVID-19 pandemic. Diabetes & Metabolic Syndrome: Clinical Research & Reviews,

14(5), 911–915.

3. Rai, D., Thakkar, H. K., Singh, D., & Bathala, H. V. (2020). Machine learning assisted automatic

annotation of isovolumic movement and aortic valve closure using seismocardiogram signals.

In IEEE 17th India Council International Conference (INDICON) (pp. 1–6). IEEE.

Time Series Analysis of COVID-19 Waves in India for Social Good 415

4. Thakkar, H. K., Shukla, H., & Patil, S. (2020). A comparative analysis of machine learning clas-

siﬁers for robust heart disease prediction. In IEEE 17th India Council International Conference

(INDICON) (pp. 1–6). IEEE.

5. Rai, D., Thakkar, H. K., Rajput, S. S., Santamaria, J., Bhatt, C., & Roca, F. (2021). A compre-

hensive review on seismocardiogram: Current advancements on acquisition, annotation, and

applications. Mathematics, 9(18), 2243.

6. Wang, L., Wang, L., Zhang, Y., Pan, J., Li, S., Sun, X., Zhang, B., & Peng, H. (2018). Weaving

sensing ﬁbers into electrochemical fabric for real-time health monitoring. Advanced Functional

Materials, 28(42), 1804456.

7. Thakkar, H. K., Liao, W.-W., Wu, C.-Y., Hsieh, Y.-W., & Lee, T.-H. (2020). Predicting clinically

signiﬁcant motor function improvement after contemporary task-oriented interventions using

machine learning approaches. Journal of NeuroEngineering and Rehabilitation, 17(1), 1–10.

8. Dagtas, S., Pekhteryev, G., & Sahinoglu, Z. (2007). Multi-stage real time health monitoring via

ZigBee in smart homes. In 21st International Conference on Advanced Information Networking

and Applications Workshops (AINAW’07) (Vol. 2, pp. 782–786). IEEE.

9. Thakkar, H. K., & Sahoo, P. K. (2019). Towards automatic and fast annotation of seismocar-

diogram signals using machine learning. IEEE Sensors Journal, 20(5), 2578–2589.

10. Mishra, S., Thakkar, H., Mallick, P. K., Tiwari, P., & Alamri, A. (2021). A sustainable IoHT

based computationally intelligent healthcare monitoring system for lung cancer risk detection.

Sustainable Cities and Society, 103079.

11. Sahoo, P. K., Thakkar, H. K., & Lee, M.-Y. (2017). A cardiac early warning system with multi

channel SCG and ECG monitoring for mobile health. Sensors, 17(4), 711.

12. Nandy, S., & Adhikari, M. (2021). Intelligent health monitoring system for detection of symp-

tomatic/asymptomatic COVID-19 patient. IEEE Sensors Journal.

13. Sahoo, P. K., Thakkar, H. K., Lin, W.-Y., Chang, P.-C., & Lee, M.-Y. (2018). On the design

of an efﬁcient cardiac health monitoring system through combined analysis of ecg and scg

signals. Sensors, 18(2), 379.

14. Rai, D., Thakkar, H. K., & Rajput, S. S. (2020). Performance characterization of binary clas-

siﬁers for automatic annotation of aortic valve opening in seismocardiogram signals. In 2020

9th International Conference on Bioinformatics and Biomedical Science (pp. 77–82).

15. Tripathy, H. K., Mishra, S., Thakkar, H. K., & Rai, D. (2021). Care: A collision-aware mobile

robot navigation in grid environment using improved breadth ﬁrst search. Computers & Elec-

trical Engineering, 94, 107327.

16. Zens, M., Brammertz, A., Herpich, J., Südkamp, N., & Hinterseer, M. (2020). App-based

tracking of self-reported COVID-19 symptoms: analysis of questionnaire data. Journal of

medical Internet research, 22(9), e21956.

17. Jain, V. K., Iyengar, K. P., & Vaishya, R. (2021). Differences between ﬁrst wave and second

wave of COVID-19 in India. Diabetes & Metabolic Syndrome.

18. Sharma, P., & Pardeshi, G. (2021). Rollout of COVID-19 vaccination in India: A SWOT

analysis. Disaster Medicine and Public Health Preparedness, 1–4.

19. Hacisuleyman, E., Hale, C., Saito, Y., Blachere, N. E., Bergh, M., Conlon, E. G., Schaefer-

Babajew, D. J., DaSilva, J., Muecksch, F., Gaebler, C., & Lifton, R. (2021). Vaccine break-

through infections with SARS-CoV-2 variants. New England Journal of Medicine, 384(23),

2212–2218.

20. Kofman, A., Kantor, R., & Adashi, E. Y. (2021). Potential COVID-19 endgame scenarios:

Eradication, elimination, cohabitation, or conﬂagration? JAMA, 326(4), 303–304.

21. Gros, D., Ounnas, A., & Yeung, T. Y.-C. (2021). A new COVID policy stringency index for

Europe. Covid Economics, 115.

22. Coccia, M. (2021). The impact of ﬁrst and second wave of the COVID-19 pandemic in soci-

ety: comparative analysis to support control measures to cope with negative effects of future

infectious diseases. Environmental Research, 197, 111099.

Detection of COVID-19 Using a

Multi-scale Deep Learning Network:

Covid-MSNet

S. V. Aruna Kumar, S. Nagashree, and B. S. Mahanand

Abstract Currently, the novel coronavirus disease-19 (COVID-19) has become a

worldwide serious public health problem. Screening methods based on chest radi-

ological imaging (e.g., X-ray and computed tomography) have been widely used

as a primary tool for early diagnosis and treatment of COVID-19. Many studies

have used single-scale deep learning models for COVID-19 detection from chest

radiological images using global features. In chest radiological images, the identiﬁ-

cation of regions of interest using single scale may be difﬁcult because the disease

often appears in different positions at varying scales. For a more accurate and early

detection of COVID-19, it is necessary to consider local spatial (location attention)

features along with the global features. In this paper, we propose such a deep learn-

ing network referred to as Covid-MSNet for early detection of COVID-19 based on

multi-scale learning that considers both attention and global features. Covid-MSNet

comprises three main layers, namely a convolutional layer, a multi-scale stream layer

and a fusion layer. In the convolutional layer, convolutional and max pooling layers

are integrated together. In the multi-scale stream layer, features are extracted from the

convolutional layer output images on multiple scales. To select the most discriminant

features, a saliency-based learning is employed in the fusion layer. Performance eval-

uation of the Covid-MSNet is presented using the publicly available COVIDx dataset

consisting of chest X-ray images of normal, pneumonia and COVID-19 patients. The

results are compared with that of other well-known deep learning models, namely

Inception-V4, Xception and ResNet50, and the results indicate that an improvement

in overall accuracy of 3–14% is achieved along with an improvement in speciﬁcity

S. V. Aruna Kumar

Department of Computer Science and Engineering, Malnad College of Engineering, Hassan,

Karnataka, India

S. Nagashree

Department of Information Science and Engineering, JSS ATE, Bangalore, Karnataka, India

e-mail: nagashrees@jssateb.ac.in

B. S. Mahanand (B)

Department of Information Science and Engineering, Sri Jayachamarajendra College of

Engineering, JSS Science and Technology University, Mysuru, Karnataka, India

e-mail: bsmahanand@sjce.ac.in

S. Mishra et al. (eds.), Augmented Intelligence in Healthcare: A Pragmatic and Integrated

Analysis, Studies in Computational Intelligence 1024,

https://doi.org/10.1007/978-981-19- 1076-0_21

417

418 S. V. Aruna Kumar et al.

of 3–10% using Covid-MSNet. Additionally, the critical regions in the lungs that

are getting affected due to COVID-19 are highlighted using the heat maps generated

from the discriminating features obtained using Covid-MSNet.

Keywords COVID-19 ·Medical image classiﬁcation ·Multi-scale learning ·

Pneumonia

1 Introduction

Novel coronavirus disease-19 (COVID-19) is one of its kind that has put the lives of

the world’s population at stake in a very short time. The world has considered the

situation as an emergency and is set to confront it in all possible ways. The origin

is said to be from Wuhan, China, in December 2019. As the nature of the virus

is zoonotic, it is presumed to be contained in bats and transmitted to humans. The

contamination is very rapid from person to person through primary (direct touch of an

infected person to a normal person) and secondary contacts (by touching the surfaces

used by an infected person like door knobs, railings, etc.) and have led to a worldwide

pandemic. COVID-19 originally belongs to the family of coronaviruses like severe

acute respiratory syndrome (SARS) and Middle East respiratory syndrome (MERS)

that attacks the respiratory systems of the humans leading to blockages in the lungs.

Initial symptoms of the attack of coronavirus that are commonly found in patients

are cold, dry cough, fever and respiratory problems or congestion in the lungs. Some

of the uncommon symptoms of the coronavirus are aches and pains, headache, diar-

rhea, sore throat, conjunctivitis, loss of taste, smell or a rash on skin or discoloration

of ﬁngers or toes. These symptoms are milder and begin gradually. Some people

become infected but show very mild symptoms according to World Health Organiza-

tion (WHO). Some of the serious conditions of the coronavirus attack are pneumonia,

congestion in the lungs, shock, failures of the organs and death. According to a John

Hopkins University study, 224 countries are infected, causing 5,125,033 deaths and

255,054,826 conﬁrmed cases all over the world as on November 18, 2021.1

Wang [19] has presented a deep convolutional neural network model for COVID-

19 detection. This model consists of ﬁve steps, viz. ﬁrst-stage projection, an expan-

sion, a depth-wise representation, a second-stage projection and ﬁnally an extension.

In the ﬁrst-stage projection, the input features are projected to lower dimensions

using 1 ×1 convolutions. Next it is expanded to a higher dimension (different from

input dimension) in the expansion stage using 1 ×1 convolution. In the depth-wise

representation, 3 ×3 depth-wise convolution is used to learn the spatial features,

which is further projected to a lower dimension in the second projection stage using

1×1 convolution. Finally, 1 ×1 convolution is used in the extension stage to get

the ﬁnal features. In their study, a COVID-Net model was proposed to classify nor-

mal, pneumonia, and COVID-19 chest X-ray images and achieved an accuracy of

1https://coronavirus.jhu.edu/map.html.

Detection of COVID-19 Using a Multi-scale Deep Learning Network: Covid-MSNet 419

92.6%. Ozturk [13] has developed a DarkCovidNet deep learning architecture to

detect COVID-19 in chest X-ray images. The DarkCovidNet comprises fewer layers

and fewer ﬁlters compared to the original DarkNet-19, which was mainly used for

real-time object detection.

Narin [11] has presented a comparative analysis of pretrained Inception-V3,

ResNet50 and Inception–ResNetV2 for COVID-19 detection. The experimental

results indicate that ResNet50 outperforms the other two methods, with an accu-

racy of 98% using 50 normal and 50 COVID chest X-ray images. Zhang [21] has

developed a deep anomaly detection model consisting of three components, viz. a

backbone architecture, a classiﬁcation head and an anomaly detection head. The

backbone architecture extracts the global features from input X-ray images, which

are then fed as input to the other two components. Based on the global features, clas-

siﬁcation score and scalar anomaly score are computed in classiﬁcation head and

anomaly detection head, respectively. Finally, the model is optimized using a cross-

entropy loss function for classiﬁcation and deviation loss for anomaly detection.

Abbas [1] developed a decompose, transfer and compose (DeTraC) deep convolu-

tional neural network for detection of COVID-19. DeTraC model comprises three

phases. In the ﬁrst phase, global features are extracted using the backbone architec-

ture, and then the local structure of the data distribution is simpliﬁed in the class

decomposition layer. In the second phase, the model is trained using sophisticated

gradient descent optimization. Finally, the class composition layer is used to reﬁne

the ﬁnal classiﬁcation. This model obtained a 95.12% accuracy to classify normal

and COVID-19 chest X-ray images.

Panwar [14] developed an CNN-based model named nCOVnet which analyzes

the X-ray images of lungs to detect the COVID-19 cases. This model consists of

24 layers among which 18 layers are from VGG-16, and they are combination of

convolution+ReLU and max pooling layers. The last ﬁve layers are from transfer

learning model. This model achieved 97.62% true positive rate with limited data.

Hussain [7] developed a deep learning model called CoroDet. This model is based

on a 22-layer network which consists of convolutional, max pooling, dense layer,

ﬂatten layer and three activation functions. This model has achieved an accuracy of

99.1% for two-class classiﬁcation, 94.2% for three-class classiﬁcation and 91.2% for

four-class classiﬁcation. Nayak [12] evaluated the effectiveness of eight pretrained

convolutional neural network (CNN) models such as AlexNet, VGG-16, GoogleNet,

MobileNet-V2, SqueezeNet, ResNet34, ResNet50 and Inception-V3 for classiﬁca-

tion of COVID-19 from normal cases. Among eight models, best performance is

obtained by ResNet34 with an accuracy of 98.33%. Several studies have also been

reported in the literature on detection of COVID-19 cases from CT images using

various deep learning models [2,5,8].

From the above review, it was found that most of the studies based on deep learning

models for COVID-19 detection have used the global features extracted at a single

scale from the chest radiological images. Recently, single-scale CNN visualization

studies have shown that at higher layers, the model captures global features with

limited spatial information, and in fully connected layer, spatial information is lost,

and it has only global information [4,10]. However in chest radiological images,

420 S. V. Aruna Kumar et al.

the identiﬁcation of regions of interest using a single scale may be difﬁcult because

they often appear in different positions at varying scales. Thus, it is necessary to

train the deep learning models using multiple scales to capture both spatial (location

attention) and global features.

Recently, multi-scale learning models have been developed successfully to extract

spatial locations along with global features for problems in the area of person re-

identiﬁcation and medical image classiﬁcation problems such as mammography,

diabetic macular edema and myopic macular degeneration [9,15,16,18,20]. Tang

[18] developed a method based on multi-scale learning for mammographic images

classiﬁcation. In this model, an autoencoder was used to capture the scale-invariant

features. Li [9] proposed a multi-instance multi-scale CNN model for diabetic macu-

lar edema and myopic macular degeneration classiﬁcation. In this model, shared set

of convolutional kernels with different sizes were used to get the information from the

receptive ﬁelds. Most of the existing multi-scale learning models are computation-

ally expensive due to the use of an excessive number of convolutional kernels with

different receptive ﬁelds. Also, there is no automatic weight learning schema in the

existing models to combine the multi-scale features. To alleviate these limitations,

Qian [15] proposed a multi-scale deep learning (MuDeep) method for person re-

identiﬁcation. This method learns the discriminative features at multiple scales and

provides an automatic weighting schema to combine features from multiple scales.

Drawing inspiration from [15], in this paper, we propose a deep learning network

called Covid-MSNet for detection of COVID-19 based on multi-scale learning that

extracts both spatial (location attention) and global features and automatically learns

the weights to combine the multi-scale features. The Covid-MSNet comprises three

main layers, namely a convolutional layer, a multi-scale stream layer and a fusion

layer. In the convolutional layer, convolutional and max pooling layers are integrated

together. In this layer, we used two convolutional layers followed by one max pooling

layer. In multi-scale stream layer, the features are extracted from a preliminary layer

output X-ray image on multiple scales. This layer comprises two multi-phase layer

and one reduction layer. In order to select the most discriminant features, a saliency-

based learning is employed in fusion layer. The saliency-based learning automatically

learns the weights to combine the features from multiple scales. Performance of the

proposed Covid-MSNet is evaluated using the publicly available COVIDx dataset

[19] consisting of normal, pneumonia and COVID-19 chest X-ray images and based

on widely used metrics, namely sensitivity (recall), speciﬁcity, precision, accuracy

and F-score. The performance of Covid-MSNet is also compared with other well-

known deep learning models such as Inception-V4 [17], Xception [3] and ResNet50

[6]. The results indicate that an improvement in overall accuracyof 3–14% is achieved

along with an improvement in speciﬁcity of 3–10% using Covid-MSNet. In addition,

we used heat maps to identify critical regions in the lungs responsible for COVID-

19. The discriminating features obtained from Covid-MSNet are mapped to original

chest X-ray images using heat maps. It reveals that discriminating features obtained

by the Covid-MSNet could potentially help clinicians to identify critical lung regions.

The remainder of this paper is organized as follows: Sect.2provides a detailed

description of the Covid-MSNet. Section 3presents the details about the COVID

Detection of COVID-19 Using a Multi-scale Deep Learning Network: Covid-MSNet 421

dataset, implementation details and the performance comparison results of Covid-

MSNet. Finally, the conclusions are summarized in Sect.4.

2 Description of Covid-MSNet

The overall structure of the proposed COVID multi-scale deep learning network

(Covid-MSNet) is shown in Fig. 1. The input to the network, namely the chest X-ray

images, is fed into convolutional layer. In the multi-scale stream layer, features are

extracted in multi-scales which are further combined in the fusion layer based on

saliency learning. The discriminate features are then fed to the fully connected layer

for classiﬁcation. The details of the convolutional layer, multi-scale stream layer and

fusion layer are described below:

Convolutional Layer: The input X-ray images are processed by applying two con-

volutional layers, followed by a max pooling layer. In this layer, two convolutional

layers with ﬁlter size 3 ×3×3 and 3 ×3×48 are used. The ﬁlter of size 3 ×3×96

is used in max pooling. The layers, output shapes and number of parameter informa-

tion of convolutional layer are presented in Table1.

Multi-Scale Stream Layer: In this layer, features are extracted based on multi-scale

learning. Multi-scale stream layer comprises two multi-phases and a reduction layer.

In the multi-scale phase-1, the data stream with size 1 ×1, 3 ×3 and 5 ×5ofthe

receptive ﬁeld is analyzed.

Further, the ﬁlter size of 5 ×5 is split into two 3 ×3 streams to increase the depth

and width. In the reduction layer, feature map of size 78 ×28 from the multi-scale

phase-1 layer is reduced to 39 ×14. These reduced feature maps are then passed on

to the multi-scale phase-2 layer. In the multi-scale phase-2, high-level features are

extracted in multiple scales of size 1 ×1, 3 ×3 and 5 ×5. To increase the depth

and width of multi-scale phase-2 layer, 5 ×5 stream is split into two 3 ×3 streams.

Further, to reduce the number of computations, 3 ×3 ﬁlter is decomposed into 1 ×3

and 3 ×1 ﬁlters. Tables 2,3and 4present the layers, output shapes and number of

parameter information of multi-scale stream layer.

Fusion Layer: In this layer, saliency-based learning is employed to obtain both the

global features and location attention.

Let the input feature maps of ith stream in each scale be Ti∗, where 1 ≤i≤4 and

Lij represent the jth channel of Ti∗where 1 ≤i≤256. The output feature maps are

obtained by fusing the features from four streams. Let Rjrepresent the jth channel

map, and it is computed as shown in Eq.1

Rj=



i=1

Tijβij (1)

where βij is the scalar for jth channel of Ti∗.

422 S. V. Aruna Kumar et al.

Fig. 1 COVID multi-scale network (Covid-MSNet). The input chest X-ray images are processed using convolutional and max pooling layers. In multi-scale

stream layer, the features are extracted in multi-scale which further combined in fusion layer based on saliency learning. The discriminate features are then fed

to fully connected layer for classiﬁcation

Detection of COVID-19 Using a Multi-scale Deep Learning Network: Covid-MSNet 423

Tabl e 1 Convolutional layer parameters

Layer (type) Output shape No. param

Conv2d [48, 256, 128] 1344

BatchNorm2d [48, 256, 128] 96

ConvBlock [48, 256, 128] 0

Conv2d [96, 256, 128] 41,568

BatchNorm2d [96, 256, 128] 192

ConvBlock [96, 256, 128] 0

MaxPool2d [96, 128, 64] 0

ConvLayers [96, 128, 64] 0

Tabl e 2 Multi-scale phase-1 parameters

Layer (type) Output shape No. param

Conv2d [96, 128, 64] 9312

BatchNorm2d [96, 128, 64] 192

ConvBlock [96, 128, 64] 0

Conv2d [24, 128, 64] 20,760

BatchNorm2d [24, 128, 64] 48

ConvBlock [24, 128, 64] 0

AvgPool2d [96, 128, 64] 0

Conv2d [24, 128, 64] 2328

BatchNorm2d [24, 128, 64] 48

ConvBlock [24, 128, 64] 0

Conv2d [24, 128, 64] 2328

BatchNorm2d [24, 128, 64] 48

ConvBlock [24, 128, 64] 0

Conv2d [16, 128, 64] 1552

BatchNorm2d [16, 128, 64] 32

ConvBlock [16, 128, 64] 0

Conv2d [24, 128, 64] 3480

BatchNorm2d [24, 128, 64] 48

ConvBlock [24, 128, 64] 0

Conv2d [24, 128, 64] 5208

BatchNorm2d [24, 128, 64] 48

ConvBlock [24, 128, 64] 0

Multi-Scale-A [96, 128, 64] 0

424 S. V. Aruna Kumar et al.

Tabl e 3 Reduction layer parameters

Layer (type) Output shape No. param

MaxPool2d [96, 64, 32] 0

Conv2d [96, 64, 32] 83,040

BatchNorm2d [96, 64, 32] 192

ConvBlock [96, 64, 32] 0

Conv2d [48, 128, 64] 4656

BatchNorm2d [48, 128, 64] 96

ConvBlock [48, 128, 64] 0

Conv2d [56, 128, 64] 24,248

BatchNorm2d [56, 128, 64] 112

ConvBlock [56, 128, 64] 0

Conv2d [64, 64, 32] 32,320

BatchNorm2d [64, 64, 32] 128

ConvBlock [64, 64, 32] 0

Reduction [256, 64, 32] 0

In this layer, the most discriminative features with their scale and location are

discovered from saliency features of multi-scale stream layer. Finally, discriminative

features obtained from fusion layer are fed as input to fully connected softmax layer to

detect COVID cases. The layers, output shapes and number of parameter information

of fusion layer and fully connected layer are presented in Table5.

During training, the input X-ray images are fed into Covid-MSNet to extract

the most discriminant features. Further, the features are fed into a fully connected

layer to obtain the class labels. Based on the predicted class label, we compute the

cross-entropy loss, which is further used to train the network.

3 Performance Evaluation of Covid-MSNet

This section presents the details about the dataset used to conduct the experiments,

implementation settings, training process and ﬁnally performance comparison with

other existing methods.

3.1 Dataset

The experiments are conducted using the publicly available COVIDx dataset [19].

This dataset was created by combining X-ray images from three publicly available

datasets, and it contains 13,700 chest X-ray images which consist of three classes,

Detection of COVID-19 Using a Multi-scale Deep Learning Network: Covid-MSNet 425

Tabl e 4 Multi-scale phase-2 layer parameters

Layer (type) Output shape No. param

AvgPool2d [256, 64, 32] 0

Conv2d [256, 64, 32] 65,792

BatchNorm2d [256, 64, 32] 512

ConvBlock [256, 64, 32] 0

Conv2d [64, 64, 32] 16,448

BatchNorm2d [64, 64, 32] 128

ConvBlock [64, 64, 32] 0

Conv2d [128, 64, 32] 24,704

BatchNorm2d [128, 64, 32] 256

ConvBlock [128, 64, 32] 0

Conv2d [256, 64, 32] 98,560

BatchNorm2d [256, 64, 32] 512

ConvBlock [256, 64, 32] 0

Conv2d [256, 64, 32] 65,792

BatchNorm2d [256, 64, 32] 512

ConvBlock [256, 64, 32] 0

Conv2d [64, 64, 32] 16,448

BatchNorm2d [64, 64, 32] 128

ConvBlock [64, 64, 32] 0

Conv2d [64, 64, 32] 12,352

BatchNorm2d [64, 64, 32] 128

ConvBlock [64, 64, 32] 0

Conv2d [128, 64, 32] 24,704

BatchNorm2d [128, 64, 32] 256

ConvBlock [128, 64, 32] 0

Conv2d [128, 64, 32] 49,280

BatchNorm2d [128, 64, 32] 256

ConvBlock [128, 64, 32] 0

Conv2d [256, 64, 32] 98,560

BatchNorm2d [256, 64, 32] 512

ConvBlock [256, 64, 32] 0

Multi-Scale-B [[256, 64, 32], [256, 64, 32],

[256, 64, 32], [256, 64, 32]]

426 S. V. Aruna Kumar et al.

Tabl e 5 Layers and layer parameters of fusion layer

Layer (type) Output shape No. param

AvgPool2d [256, 16, 8] 0

Fusion [256, 16, 8] 0

Linear [4096] 134,221,824

BatchNorm1d [4096] 8192

ReLU [4096] 0

Linear [3] 12,291

namely normal, pneumonia and COVID-19. In this work, the experiments are con-

ducted using the training and testing split mentioned in [19]. The proposed Covid-

MSNet is trained using 7966 normal, 5451 pneumonia and 152 COVID-19 chest

X-ray images. The remaining chest X-ray images of the dataset are used for testing

the Covid-MSNet.

3.2 Implementation Details

The Covid-MSNet is implemented using the PyTorch framework on a i5-8600K

CPU@3.60 GHz with GTX2080ti GPUs. The original input chest X-ray images are

resized to 256 ×128 ×3. During training, Adam optimization approach is used with

following parameters: learning rate = 0.0003, 100 epochs, learning rate decay = 0.1,

and weight decay = 5e−4.

3.3 Performance Evaluation of Covid-MSNet

Performance of Covid-MSNet is also compared with other well-known deep learning

models such as Inception-V4 [17], Xception [3] and ResNet50 [6]. The performance

is evaluated based on widely used metrics, namely sensitivity (recall), speciﬁcity,

precision, accuracy and F-score. These metrics are computed as follows:

Sensitivity (Recall)=TP

TP +FN (2)

Speciﬁcity =TN

TN +FP (3)

Precision =TP

TP +FP (4)

Detection of COVID-19 Using a Multi-scale Deep Learning Network: Covid-MSNet 427

Fig. 2 Confusion matrix on the test set for Covid-MSNet. 0, 1 and 2 indicate normal, pneumonia

and COVID-19 classes, respectively

Tabl e 6 Performance comparison of COVID-19 class

Method Sens. (%) Spec. (%) Prec. (%) Acc. (%) F-score (%)

Inception-V4 39.21 98.11 90.07 80.09 55.11

Xception 40.43 97.32 83.13 80.95 54.01

ResNet50 77.03 98.19 87.22 94.81 82.41

Covid-MSNet 79.33 98.21 87.24 95.24 83.15

Accuracy =TP +TN

TP +TN +FP +FN (5)

F-Score =2∗Precision ∗Recall

Precision +Recall (6)

where TP represents true positive, TN represents true negative, FN represents false

negative, and FP represents false positive.

Figure 2shows the confusion matrix of Covid-MSNet on the test set. Performance

comparison of the proposed Covid-MSNet with other models on COVID class is

showninTable6. From Table 6, it is found that Covid-MSNet produces an sensitivity

79.33% while those of Inception-V4, Xception and ResNet50 are 39.21%, 40.43%

and 77.03%, respectively, on COVID class. Performance of the Covid-MSNet is

improved by 40% (Inception-V4), 39% (Xception) and 2% (ResNet50) in terms

of sensitivity. The Covid-MSNet produces an speciﬁcity of 98.21% while those of

Inception-V4, Xception and ResNet50 are 98.11%, 97.32% and 98.19%, respectively,

on COVID class. From Table7, we can observe that Covid-MSNet achieved similar

speciﬁcity value. From Table 6, it is found that Covid-MSNet produces an accuracy

of 95.24%, precision of 87.24% and F-score of 83.15% on COVID class. In Table 6,

we can clearly see that accuracy, precision and F-score of Covid-MSNet are 2–10%

more than other methods in comparison.

Table 7shows the performance comparison of Covid-MSNet with other models on

pneumonia class. From Table 7, it is found that Covid-MSNet produces an sensitivity

96.25% while those of Inception-V4, Xception and ResNet50 are 90.31%, 76.27%

428 S. V. Aruna Kumar et al.

Tabl e 7 Performance comparison on pneumonia class

Method Sens. (%) Spec. (%) Prec. (%) Acc. (%) F-score (%)

Inception-V4 90.31 84.31 78.32 87.01 84.15

Xception 76.27 84.15 80.25 80.95 78.06

ResNet50 85.19 88.03 85.41 87.45 85.33

Covid-MSNet 96.25 90.00 86.15 92.64 91.45

Tabl e 8 Performance comparison on normal class

Method Sens. (%) Spec. (%) Prec. (%) Acc. (%) F-score (%)

Inception-V4 87.31 77.25 64.41 80.09 74.09

Xception 87.32 73.32 55.32 77.49 68.42

ResNet50 88.11 89.01 86.40 89.18 87.31

Covid-MSNet 87.61 96.51 95.30 92.21 91.42

and 85.19%, respectively, on pneumonia class. The performance of the Covid-MSNet

is improved by 6% (Inception-V4), 20% (Xception) and 11% (ResNet50) in terms

of sensitivity. The Covid-MSNet produces an speciﬁcity of 90% while those of

Inception-V4, Xception and ResNet50 are 84.31%, 84.15% and 88.03%, respectively,

on pneumonia class. From Table 7, we can observe that the performance of the Covid-

MSNet in terms of speciﬁcity is improved by 5% (Inception-V4), 5% (Xception) and

2% (ResNet50). From Table 7, it is found that Covid-MSNet produces an accuracy of

92.64%, precision of 86.15% and F-score of 91.45% on pneumonia class. In Table7,

we can observe that accuracy, precision and F-score of Covid-MSNet are 2–5% more

than other methods.

Performance comparison of the Covid-MSNet with other models on normal class

isshowninTable8. From Table 8, it is found that Covid-MSNet produces a sensitivity

87.61% while those of Inception-V4, Xception and ResNet50 are 87.31%, 87.32%

and 88.11%, respectively, on normal class. In Table 8, we can see that Covid-MSNet

achieved similar sensitivity value like other three methods. Covid-MSNet achieved

speciﬁcity value of 96.51% while those of Inception-V4, Xception and ResNet50 are

77.25%, 73.32% and 89.01%, respectively, on normal class. From Table8, we can

observe that, in terms of speciﬁcity, Covid-MSNet shows the improvement of 19%

(Inception-V4), 22% (Xception) and 6% (ResNet50) on normal class. Covid-MSNet

obtained an accuracy of 92.21%, precision of 95.30% and F-score of 91.42% on

normal class. In Table 8, we can clearly see that accuracy, precision and F-score of

Covid-MSNet is 3% more than other methods in comparison.

Further, we also compared the overall performance of the proposed Covid-MSNet

with other methods on all three classes. Table 9presents the performance comparison

of the Covid-MSNet with other methods on overall class. From Table 9, it is found

that Covid-MSNet produces an sensitivity 88.09% while those of Inception-V4,

Xception and ResNet50 are 72.03%, 69.42% and 84.62%, respectively, on overall

Detection of COVID-19 Using a Multi-scale Deep Learning Network: Covid-MSNet 429

Tabl e 9 Overall performance comparison

Method Sens. (%) Spec. (%) Prec. (%) Acc. (%) F-score (%)

Inception-V4 72.03 87.16 77.11 82.39 71.54

Xception 69.42 85.21 73.05 79.79 67.13

ResNet50 84.62 93.51 86.15 90.48 85.54

Covid-MSNet 88.09 95.15 89.61 93.36 88.51

class. The performance of the Covid-MSNet is improved by 15% (Inception-V4),

19% (Xception) and 4% (ResNet50) in terms of sensitivity. The Covid-MSNet pro-

duces an speciﬁcity of 95.15 while those of Inception-V4, Xception and ResNet50

are 87.16%, 85.21% and 93.51%, respectively, on overall class. From Table 9,we

can observe that the performance of Covid-MSNet on overall class in terms of speci-

ﬁcity is improved by 8% (Inception-V4), 10% (Xception) and 3% (ResNet50). From

Table 9, it is found that Covid-MSNet produces accuracy of 93.36%, precision of

89.61% and F-score of 88.51% on overall class. Future work will focus on further

improving sensitivity as more COVID-19 patient data becomes available. The results

clearly indicate that combining both global and location attention feature using multi-

scale learning can classify COVID-19 cases more accurately as compared to other

deep learning models.

Additionally, we used heat maps to identify critical regions in the lungs that are

affected by COVID-19. The discriminating features obtained from Covid-MSNet are

mapped back to the original chest X-ray images using heat maps. For comparison,

we also mapped the features used by ResNet50 model. For illustration, two patients’

chest X-ray images along with heat maps are shown in Fig.3. It can be observed that

ResNet50 model has used global features from different regions of the lung. The

Covid-MSNet has used both global and spatial (location attention) features and can

identify speciﬁc regions (shown in red) within the lungs in COVID-19 patients. It

reveals that discriminating features obtained by the Covid-MSNet could potentially

help clinicians to identify critical regions associated with COVID-19, which they

can then leverage to early diagnosis of the disease and treatment.

4 Conclusions

This paper has presented a deep learning network based on multi-scale learning

called Covid-MSNet for early detection of COVID-19. In the Covid-MSNet, multi-

scale learning is employed to extract both spatial (location attention) and global

features. Performance evaluation of Covid-MSNet was carried out on publicly avail-

able normal, pneumonia and COVID-19 chest X-ray images from COVIDx dataset.

An overall sensitivity of 88.09%, speciﬁcity of 95.51%, accuracy of 93.36%, preci-

sion of 89.61% and F-score of 88.51% were obtained. The results clearly indicate

430 S. V. Aruna Kumar et al.

Fig. 3 Heat map identifying critical regions in the lung

Detection of COVID-19 Using a Multi-scale Deep Learning Network: Covid-MSNet 431

better performance of the proposed approach as compared to Inception-V4, Xception

and ResNet50 deep learning models especially in the area of reduced misclassiﬁca-

tion. The discriminating features obtained from Covid-MSNet also identify critical

regions in the lungs getting affected by COVID-19.

References

1. Abbas, A., Abdelsamea, M. M., & Gaber, M. M. (2021). Classiﬁcation of COVID-19 in chest

X-ray images using DeTraC deep convolutional neural network. Applied Intelligence, 51(2),

854–864.

2. Butt, C., Gill, J., Chun, D., & Babu, B. A. (2020). Deep learning system to screen coronavirus

disease 2019 pneumonia. Applied Intelligence, 1.

3. Chollet, F. (2017). Xception: Deep learning with depthwise separable convolutions. In Proceed-

ings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 1251–1258).

4. Gatys, L. A., Ecker, A. S., & Bethge, M. (2015). Texturesynthesis and the controlled generation

of natural stimuli using convolutional neural networks. arXiv preprint arXiv:1505.07376,12,

5. Gozes, O., Frid-Adar, M., Greenspan, H., Browning, P. D., Zhang, H., Ji, W., Bernheim, A.,

& Siegel, E. (2020). Rapid AI development cycle for the coronavirus (COVID-19) pandemic:

Initial results for automated detection & patient monitoring using deep learning CT image

analysis. arXiv preprint arXiv:2003.05037

6. He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition.

In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp.

770–778).

7. Hussain, E., Hasan, M., Rahman, M. A., Lee, I., Tamanna, T., & Parvez, M. Z. (2021). Corodet:

A deep learning based classiﬁcation for COVID-19 detection using chest X-ray images. Chaos,

Solitons & Fractals, 142, 110495.

8. Li, L., Qin, L., Xu, Z., Yin, Y., Wang, X., Kong, B., Bai, J., Lu, Y., Fang, Z., Song, Q., & Cao, K.

(2020). Artiﬁcial intelligence distinguishes COVID-19 from community acquired pneumonia

on chest CT. Radiology, 200905.

9. Li, S., Liu, Y., Sui, X., Chen, C., Tjio, G., Ting, D. S. W., & Goh, R. S. M. (2019). Multi-

instance multi-scale CNN for medical image classiﬁcation. In International Conference on

Medical Image Computing and Computer-Assisted Intervention (pp. 531–539). Springer.

10. Mahendran, A., & Vedaldi, A. (2015). Understanding deep image representations by inverting

them. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition

(pp. 5188–5196).

11. Narin, A., Kaya, C., & Pamuk, Z. (2021). Automatic detection of coronavirus disease (COVID-

19) using X-ray images and deep convolutional neural networks. Pattern Analysis and Appli-

cations, 1–14.

12. Nayak, S. R., Nayak, D. R., Sinha, U., Arora, V., & Pachori, R. B. (2021). Application of deep

learning techniques for detection of COVID-19 cases using chest X-ray images: A compre-

hensive study. Biomedical Signal Processing and Control, 64, 102365.

13. Ozturk, T., Talo, M., Yildirim, E. A., Baloglu, U. B., Yildirim, O., & Acharya, U. R. (2020).

Automated detection of COVID-19 cases using deep neural networks with X-ray images.

Computers in Biology and Medicine, 103792.

14. Panwar, H., Gupta, P., Siddiqui, M. K., Morales-Menendez, R., & Singh, V. (2020). Application

of deep learning for fast detection of COVID-19 in X-rays using nCOVnet. Chaos, Solitons &

Fractals, 138, 109944.

15. Qian, X., Fu, Y., Jiang, Y. G., Xiang, T., & Xue, X. (2017). Multi-scale deep learning archi-

tectures for person re-identiﬁcation. In Proceedings of the IEEE International Conference on

Computer Vision (pp. 5399–5408).

432 S. V. Aruna Kumar et al.

16. Shi, C., & Pun, C. M. (2019). Adaptive multi-scale deep neural networks with perceptual loss

for panchromatic and multispectral images classiﬁcation. Information Sciences, 490, 1–17.

17. Szegedy, C., Ioffe, S., Vanhoucke, V., & Alemi, A. A. (2017). Inception-v4, inception-resnet and

the impact of residual connections on learning. In Thirty-First AAAI Conference on Artiﬁcial

Intelligence.

18. Tang, Q., Liu, Y., & Liu, H. (2017). Medical image classiﬁcation via multiscale representation

learning. Artiﬁcial Intelligence in Medicine, 79, 71–78.

19. Wang, L., Lin, Z. Q., & Wong, A. (2020). COVID-Net: A tailored deep convolutional neural

network design for detection of COVID-19 cases from chest X-ray images. Scientiﬁc Reports,

10(1), 1–12.

20. Wang, Y., Cheung, S. W., Chung, E. T., Efendiev, Y., & Wang, M. (2020). Deep multiscale

model learning. Journal of Computational Physics, 406, 109071.

21. Zhang, J., Xie, Y., Li, Y., Shen, C., & Xia, Y. (2020). COVID-19 screening on chest X-ray

images using deep learning based anomaly detection. arXiv preprint arXiv:2003.12338

Immersive Technologies

in the Healthcare Space

Selvakumar Samuel

Abstract The ﬁeld of life sciences and healthcare sector have always beneﬁted from

various digital innovations. One of the most beneﬁcial technologies for healthcare is

immersive technology. Immersive technologies such as VR, AR, and MR provide a

full immersive experience by blending real and virtual worlds. Extended Reality (XR)

is the new term to represent all immersive technologies. Currently, these technologies

are used in almost all ﬁelds with the convergence of other relevant technologies. The

role of these immersive technologies is inevitable in the healthcare sector as well. It

provides many use cases with the support of other digital technologies and software

techniques such as AI-based software methods, smart devices, sensors, robots, high

performance computers (quantum computers), 6G networks and other compatible

technologies. This chapter aimed to relate the state of the art in Immersive technolo-

gies related to healthcare by conducting a systematic review of the recent literature.

With the support of digital technologies and artiﬁcial intelligence (AI), immersive

technologies can offer signiﬁcant assistance to medical professionals, surgeons, and

even medical students in various ﬁelds. For instance, it can help a surgeon plan

a complex surgery procedure by allowing the doctor to analyze layer by layer of

a 3D holographic image of the patient. Without any physical limitation, medical

students can study anatomy by examining a holographic body. Importantly, immer-

sive technologies challenge age-related declines, especially by increasing morale. In

conclusion, immersive technologies have already developed a strong identity in the

healthcare sector and will be key components of healthcare in the future.

Keywords Virtual reality ·Augmented reality ·Mixed reality ·Extended reality ·

Immersive technologies ·AI ·DARQ ·6G ·Healthcare sector

S. Samuel (B)

Asia Paciﬁc University of Technology and Innovation, Kuala Lumpur, Malaysia

e-mail: selvakumar@staffemail.apu.edu.my

S. Mishra et al. (eds.), Augmented Intelligence in Healthcare: A Pragmatic and Integrated

Analysis, Studies in Computational Intelligence 1024,

https://doi.org/10.1007/978-981-19- 1076-0_22

433

434 S. Samuel

1 Introduction

Immersive technologies such as virtual reality (VR), augmented reality (AR), mixed

reality (MR) and now extended reality (XR) provide a full immersive experience by

mixing the real and virtual worlds. VR is a virtual or simulated experience created by

computers that doesn’t really exist. AR is a technique for projecting digital content in

a real environment. Virtual reality which excludes the outside world and augmented

reality which only projects digital information onto the real world, but MR projects

content that interacts with the real-world environment. XR is the newterm to represent

all immersive technologies that can combine physical and virtual environments. It

will essentially be a computer-generated reality experience.

Currently, immersive technologies are used in almost all ﬁelds with the conver-

gence of other relevant technologies. Immersive technologies are already a key part of

healthcare and they have produced many use cases with various software techniques

and other compatible digital technologies. This will reshape the future of healthcare

and eventually become a permanent key part of the healthcare domain.

Immersive technologies use other technological advances to produce useful appli-

cations in various ﬁelds of health. XR is combined with AI-based software methods,

smart devices, sensors, robots, high-performance computers (quantum computers),

6G networks, and other compatible technologies to produce use cases.

Among these technological convergences of XR, the combination of XR with AI

is inevitable. Here, AI assists XR technology, so this can be referred to as augmented

intelligence or assisted intelligence, this is brieﬂy discussed in Sect. 2(Fig. 1).

The main beneﬁts of XR in healthcare are improving mental health of well-being,

examining medical data more effectively, and improving medical treatments and

surgeries [2].

Fig. 1 Augmented reality (AR) in the health care [1]

Immersive Technologies in the Healthcare Space 435

The following sections brieﬂy explain the different applications of XR in health-

care. XR’s trending technology collaborations such as DARQ (Distributed Ledgers,

AI, XR, and Quantum Computing) and 6G healthcare networks are also brieﬂy

discussed in this chapter.

2 Augmented Intelligence

Artiﬁcial intelligence (AI) and human intelligence together produce augmented intel-

ligence. Artiﬁcial intelligence was introduced to augment human intelligence and not

to replace humans [3]. Researchers and scientists mainly produce augmented intel-

ligence [4]. Most of the time what we have are augmented intelligence apps, not

artiﬁcial intelligence apps. The main goal of augmented intelligence is to improve

cognitive performance and new experiences [5].

Many of the challenges of AI are not problems for augmented intelligence.

Augmented intelligence does not necessarily have to participate in the decision-

making process, but simply provide the results or outcomes of the application and let

human intelligence take over [4]. Augmented applications based on immersive tech-

nology are effectively applied in the healthcare industry. It can be of great help

to medical professionals, surgeons and even medical students in various ﬁelds.

Augmented intelligence in healthcare is also referred to as augmented medical

intelligence.

With the use of different AI methods, immersive technologies capable of providing

many use cases, in particular to discover health decisions quickly and with more

precision [6] (Fig. 2).

Fig. 2 Immersive technology with AI in the healthcare space [7]

436 S. Samuel

3 Immersive Technology Applications

Immersive technologies applied to various aspects of healthcare. One of the impor-

tant advantages of this technology is the user-friendliness, without much technical

support, healthcare professionals and healthcare personnel can use this technology.

Many applications are already available to support healthcare in several ways. In

Table 1, some examples of clinical applications are summarized, followed by some

of the important applications are brieﬂy discussed.

3.1 XR in Planning Complex Surgeries

For example, a neurosurgeon planning a complex surgery, the surgeon receives the

scanned images, and a MR machine renders them as a three-dimensional holographic

Tabl e 1 Sample XR healthcare applications with devices and other details [8]

XR Hardware

examples

User interface Technical

strengths

Technical

limitations

Clinical

applications

VR Oculus rift

and HTC

vive

Handheld

motion-tracked

controllers

Superior 3D

graphics

performance

and highest

resolution

User has no

direct view of

physical

environment

and requires

controller

inputs

The Stanford

virtual heart, the

body VR, and mind

maze

2D-AR

(indirect)

iPhone,

iPad, and

Android

devices

Touch screen Widely

available and

inexpensive

Phone or

tablet must be

held or

mounted and

requires touch

input

ECG probe

orientation

2D-AR

(direct)

Google

glass

Side-mounted

touchpad and

voice

Lightweight

head

mounted

display

2D display

and UI does

not interact

with physical

environment

First-in-man use in

interventional Cath

3D AR Microsoft

HoloLens,

magic leap,

and real

view

holoscope

Voice, gaze,

and gestures

Touch-free

input, 3D

displays, and

full visibility

surroundings

Narrow ﬁeld

of view for 3D

graphics

Holo anatomy,

echo pixel, real

view,

intraprocedural

scar visualization,

and enhanced

electro-physiology

visualization and

interaction system

Immersive Technologies in the Healthcare Space 437

Fig. 3 Surgeons examine the brain layer by layer [9]

image. The surgeon can examine the problem area, examine it layer by layer, and

prepare for the complete procedure. An example is given in Fig. 3.

3.2 XR in Physiotherapy Exercises

This technology can be used to distract the patient’s concentration in different direc-

tions, for example doctors can distract the patient from pain during physiotherapy

exercises. An example is given in Fig. 4.

3.3 XR in Medical Students Training

Immersive technologies can overcome the limitations of traditional training. A major

advantage of this technology is the three-dimensional experience. This immer-

sive experience will allow students to easily and clearly understand complicated

subjects. For example, medical students study anatomy in groups while dissecting

the holographic body without physical limitations.

The human heart is one of the complex anatomies, with the help of the AR heart

can be visualized as a three-dimensional model without any major limitation. These

models can be incorporated into digital medical books as moving pictures and addi-

tional content can be linked as well. A sample model is shown in Fig. 5. Likewise, any

438 S. Samuel

Fig. 4 Treating a paediatric burn patient with VR by distracting pain at Shriners Children Hospital

in Galveston, Texas [9]

Fig. 5 A healthcare professional interacts with a 3D holographic heart image [11]

individual organ can be visualized. It would really help students understand complex

anatomies without any physical limitation.

The use of virtual reality creates a fully simulated environment and is not tied

to anything in the real world. With the simulated patients and VR equipment such

as headsets, handheld controllers, and others, produce use cases such as emergency

trauma reception care, risk-free training, etc. [10].

Immersive Technologies in the Healthcare Space 439

Fig. 6 AR based training system for sonologists designed by GE, USA [3]

3.4 Augmented Reality Application to Train Sonographers

Researchers at General Electric Company in the United States have designed an

augmented reality (AR)-based system with their own proprietary AI algorithms to

train sonographers and provide advice. They used Microsoft HoloLens glasses with

a scanner. With this environment, trainees are able to see the exact locations and

shapes of human organs using the instructions provided by headsets and ultrasonic

waves to completely scan the body [3] (Fig. 6).

3.5 XR in Rehabilitation

The application of XR in rehabilitation has proven to be very effective. The virtual

environment allows patients to dive into an alternative reality.

Patients can interact with the simulated virtual objects and perform activities that

otherwise would not be accessible to them in the real world [12]. During Covid-

19, this immersive environment enabled effective rehabilitation through remote

treatment in patients’ homes [13].

Figure 7shows virtual reality-based rehabilitation programs for stroke recovery.

This approach has been very effective and has produced outstanding results. This

improved the psychological state and motivated the patients remarkably.

440 S. Samuel

Fig. 7 Rehabilitation program using virtual reality for stroke recovery [12]

3.6 XR in Mild Dementia and Mild Cognitive Impairment

(MCI) Recovery

Mild dementia and MCI are reported health problems for older people. MCI is an

intermediary stage in the cognitive transition from normal senility to dementia [15].

In order to avoid cognitive loss, appropriate and timely screening is necessary [2]

(Fig. 8).

The extended reality-based approach is considered to be one of the promising

alternative approaches or additional method for treating MCI and mild dementia

problems with current medical methods. Some useful research has been done on this

application. One of the researches in this area [16] has shown that XR technology is

able to slow the progression of mild cognitive impairment to dementia, on the other

hand able to improve the cognitive functions of patients. This obviously proves that

XR can play a positive role for patients with cognitive impairment [17]. According

to a pilot study [18] conducted with 21 people, showed that training based on mixed

reality is able to signiﬁcantly improve memory impairments.

Jennifer Zhang [14] of Oxford had invented a virtual reality-based applica-

tion called Dancing Mind. This application has been able to slow the progres-

sion of dementia and help stroke patients overcome the already arduous path of

rehabilitation.

Immersive Technologies in the Healthcare Space 441

Fig. 8 Jennifer Zhang and a

patient using Dancing Mind

[14]

3.7 XR in the Treatment of Small Animal Phobia

Augmented Reality (AR) can be used to effectively treat small animal phobias.

Numerous research works have already proven this. AR using optical see-through

(OST) or video see-through (VST) systems for the treatment of small animal phobia.

People who suffer from phobias of small animals worry about the circumstances

in which these animals may appear. Because they are always afraid to look at the

scared animal, they experience unrealistic and undue fear, which makes life worse.

In HITLab, Washington [19] was conducted the very ﬁrst treatment for a phobic

small animal patient during a 12-h therapy session with a virtual reality system. A

signiﬁcant improvement has been shown compared to other medical methods [20].

Subsequently, many researchers have shown the remarkable results of AR systems

for the treatment of small animal phobias.

Figure 9a, b showed the AR-based exposure therapy session for a patient with

small animal phobia.

442 S. Samuel

Fig. 9 a Patient (right side)

and therapist (left side)

during therapy session [21].

bView of the patient through

the head-mounted display

[21]

4 XR with DARQ and 6G Network in Healthcare

Augmented intelligence can be applied in almost all ﬁelds and industries. The most

effective application of augmented intelligence is in healthcare, primarily to improve

the quality of patient care and reduce human medical errors. This application requires

more than the available processing power and high-speed network connectivity.

According to recent initiatives in this area, the promised 6G network and DARQ

(Distributed Ledgers, AI, XR, and Quantum Computing) can meet the requirements.

In these two areas, augmented intelligence with immersive technologies plays a key

role. The following sections will therefore address the role of DARQ and the 6G

network in the healthcare space.

Immersive Technologies in the Healthcare Space 443

4.1 DARQ in Healthcare Space

Technological convergence where several technologies work together to bring inno-

vations in any ﬁeld. Computed tomography (X-rays and computer) is a suitable

example in the ﬁeld of healthcare. DARQ Power is an emerging technological conver-

gence. Healthcare sectors in many developed countries are preparing to take advan-

tage of DARQ’s innovations. Extended reality is at the heart of this technological

convergence.

According to Accenture Digital Health Tech Vision, this DARQ power will shape

the healthcare space over the next few years [22]. According to the UK’s National

Health Services (NHS), to solve emerging health challenges, healthcare services

should be restructured with the blend of physical care, digital services, self-service

and virtual care. DARQ innovations is an optimized solution to meet this demand

[23].

XR (extended reality) naturally connects technologies (AI, Blockchain, Quantum

computers) and users [23]. XR and augmented intelligence have been sufﬁciently

introduced in the previous sections. Distributed Ledger or Blockchain and Quantum

Computers are brieﬂy introduced in terms of healthcare in the following sections.

4.1.1 Blockchain

The blockchain is made up of a collection of blocks. A block is a record or a unit of

data. Each block contains a collection of transactions related to many other blocks

organized in a particular order. Blockchain can refer to an information recording

system in a way that makes it complicated or impossible to modify, hack, or deceive

the system [24]. The systematic process of Blockchain infrastructure is depicted in

Fig. 10.

Fig. 10 The process ﬂow of a transaction in the blockchain platform [24]

444 S. Samuel

The distributed ledger (blockchain) provides secure data transfer in the healthcare

sector. For instance, to process invoices and claims, patient records can be transferred

securely. In addition, this platform is capable of tracking drugs and medical supply

chains.

Harvard Business Review [25] recently looked at blockchain from a health records

management perspective, stating: “With our health records, issues of access, security,

privacy, monetization and advocacy can be resolved”.

4.1.2 Quantum Computers

A quantum computer is a high-performance machine based on quantum mechanics.

This machine is capable of performing several thousand times faster than current

digital computers and capable of performing the most complex tasks that cannot be

performed by binary computers [26].

As mentioned in Fig. 11, quantum computers are able to solve drug design prob-

lems in seconds, which can take several years for classical computers. Quantum

computers are capable of solving many insoluble problems even in the ﬁeld of health

by classical computers [27]. Quantum computers are the answer to the performance

demands of immersive technologies to effectively support healthcare.

According to Accenture, quantum computers can “analyze complex data sets,

such as DNA data, to enable more personalized medicine and interactions”. This

could enable great strides in drug discovery and therapeutic innovations [28].

Fig. 11 A simple comparison between classic and quantum computers [29]

Immersive Technologies in the Healthcare Space 445

4.1.3 Summary of DARQ Power in Healthcare

Blockchain, AI, XR, and Quantum (DARQ) computers complement each other to

solve complex and intractable healthcare sector problems.

In summary, it can be said that DARQ is beneﬁcial for the experience of all

healthcare services, professionals and patients in various aspects, such as blockchain-

based data exchange will help improve the conﬁdence of all stakeholders, AI will

ensure high accuracy in diagnosis, while XR will provide immersive interaction for

remote surgery, telemedicine, and enhanced training of healthcare professionals and

the Quantum computer will be responsible for the rapid processing of complex

tasks and disease screening. With the innovations of DARQ, if we automate health

services, it will certainly transform the entire health sector by removing robotic tasks

and exponentially improving overall productivity [25].

5 6G Network with Immersive Technologies in Healthcare

Many companies established in communication technologies have been working on

6G, which is soon to revolutionize the world of communication.

Countries such as the United States, China, Japan, Finland and other countries

have already invested in the development of 6G. The synergy between 6G and immer-

sive technologies will be able to augment the healthcare sector in several ways, in

particular it can help in improved ways to support clinical decision-making.

Stoica and Abreu [30] clearly recorded the weaknesses of 5G and the important

factors for the development of 6G [31]. They [30,31] also concluded that artiﬁcial

intelligence and extended reality (XR) applications will be the main driver of 6G.

This collaboration could solve complex challenges and complex problems [31]. As

highlighted in Table 2, the 6G network will provide full support for AI and XR over

5G.

Basically, most AI-powered healthcare applications struggle with massive connec-

tivity, big data, and ultra-low latency. On the other hand, AR/VR applications are the

most data intensive and require high bandwidth. Figure 12 presents an overview of

the circumstances in view of various KPIs.

As shown in Fig. 2, a use case related to telepresence, telehealth with AR/VR

are discussed in Sect. 5.2. Immersive technology-based devices will need a short

data transfer of up to 10 Gbps and also XR services require near zero latency

to improve QoS [32]. Therefore, essentially, current AI-based applications and

immersive technology services require beyond what is offered by 5G networks [30].

Moreover, in 6G, another important promise is battery-less devices where they

will be powered remotely and intelligently by the network itself, thus eliminating

the need for batteries in various devices [33].

It is estimated that a wireless connectivity driven by the 6G system would be 1000

times more efﬁcient than 5G. Thus, the 6G-based network will be able to transform

and revolutionize healthcare services.

446 S. Samuel

Tabl e 2 Comparison between 5 and 6G [33]

Characteristic 5G 6G

Operating frequency 3–300 GHz Upto 1 THz

Uplink data rate 10 Gbps 1 Tbps

Downlink data rate 20 Gbps 1 Tbps

Spectral efﬁciency 10 bps/Hz/m21000 bps/Hz/m2

Reliability 10−510−9

Maximum mobility 500 km/h 1000 km/h

U-plane latency 0.5 ms 0.1 ms

C-plane latency 10 ms 1ms

Processing delay 100 ns 10 ns

Trafﬁc capacity 10 Mpbs/m21–10 Gpbs/m2

Localization precision 10 cm on 2D 1cmon3D

Uniform user experience 50 Mpbs 2D 10 Gbps 3D

Time bu ffer Not real-time Real-time

Center of gravity User Service

Satellite integration No Fully

AI integration Partially Fully

XR integration Partially Fully

Haptic communication integration Partially Fully

Automation integration Partially Fully

Fig. 12 Representation of various KPIs of 6G use cases [34]

Immersive Technologies in the Healthcare Space 447

Other breakthrough technologies likely to be deployed on 6G enable networks

with XR and AI in healthcare would be surgical robots and wireless medical implants.

The following sections would brieﬂy explain these technological convergences.

5.1 Wireless Implantable Medical Devices

Monitoring patients at home and screening for disease are intrinsic requirements

today. Implantable medical devices (IMDs) with wireless connectivity,low power and

low range wireless sensor networks, low power medical radios are the essential part of

this healthcare infrastructure for remote monitoring of patients and remote diagnosis.

Basically, it will capture energy from the body and transfer it to the physician’s device

[35].

Implantable medical devices (IMDs) including pacemakers, cardiac deﬁbrillators,

insulin pumps, neurostimulators and others mentioned in Fig. 13. This system will

help healthcare professionals provide timely treatment to patients.

The role of the 6G network will be inevitable, when we include additional func-

tionality with artiﬁcial intelligence methods and mathematical models like the CT

scan functionality.

Fig. 13 Wireless implantable medical devices. Source [35]

448 S. Samuel

Fig. 14 Remote surgery use case [32]

5.2 XR and Remote Surgery

The promise of 6G may provide extremely sensitive use cases for remote surgery

[35]. To facilitate the extremely data intensive technological convergence of remote

surgery with extended reality, AI, surgical robots and nano-devices must work

together in good synergy.

This immersive environment use case requires ultra-low latency (less than 1 ms)

for real-time interaction, system capacity greater than 1 Tbps for processing appli-

cations, especially for encoding and decoding Gbps data rate per user. 5G is not

feasible to meet these requirements [3], although there are use cases of immersive

environments with 5G [34,36,37].

Therefore, this immersive environment use case requires the 6G system to meet

the requirements [32]. A simple representation of this use case is shown in Fig. 14.

This immersive environment supports major and minor surgeries that can be

performed remotely by surgeons. Through remote controls in immersive environ-

ments, surgeons are able to operate inside the human body also using nano-devices

and communication using the small wavelength band (THz) [34].

Figure 15 shows a surgeon performing surgery with a remote surgical robot called

the “Da Vinci robot surgeon” [38] by viewing the 3D image of the patient.

6 Summary

In this chapter, the important use cases of immersive technologies in the healthcare

space are explained. No technology can offer the maximum beneﬁt and support to

the ﬁeld that is being applied without the support of other technologies. Immersive

technologies also provide many useful applications with various other technologies.

Immersive technologies have made the best partnership with AI to deliver intelli-

gent immersive experience to user. Apart from this, as a part of DARQ power, XR

Immersive Technologies in the Healthcare Space 449

Fig. 15 A surgeon performing a surgery on a patient who is 400 km away [38]

will augment various services in the healthcare space. The synergy between the 6G

network and XR will transform immersive interactions with users to an unimagin-

able level. In the future, immersive technologies will be an essential and permanent

citizen in the healthcare space.

References

1. MedCity News. (2021). The beneﬁts of AR in healthcare. MedCity News. [Online]. Available

at: https://medcitynews.com/2019/09/the-beneﬁts-of-ar-in-healthcare/

2. Marr, B. (2021). Extended reality in healthcare: 3 reasons the industry must get ready for AR

and VR. Forb e s. [Online]. Available at: https://www.forbes.com/sites/bernardmarr/2021/06/

14/extended-reality-in-healthcare-3-reasons-the-industry-must-get-ready-for-ar-and-vr/?sh=

1ea39bdb73a4

3. Tripathy, H. K., Mishra, S., Thakkar, H. K., & Rai, D. (2021). CARE: A collision-aware

mobile robot navigation in grid environment using improved breadth ﬁrst search. Computers &

Electrical Engineering, 94, 107327.

4. Mishra, S., Panda, A., & Tripathy, K. H. (2018). Implementation of re-sampling technique

to handle skewed data in tumor prediction. Journal of Advanced Research in Dynamical and

Control Systems, 10, 526–530.

5. Gartner. (2021). Deﬁnition of augmented intelligence—Gartner information technology glos-

sary. [Online]. Available at: https://www.gartner.com/en/informationtechnology/glossary/aug

mented-intelligence

6. MobiHealthNews. (2021). Contributed: Top 10 use cases for AI in healthcare. [Online].

Available at: https://www.mobihealthnews.com/news/contributed-top-10-use-cases-ai-health

care

7. Mishra, S., Thakkar, H., Mallick, P. K., Tiwari, P., & Alamri, A. (2021). A sustainable IoHT

based computationally intelligent healthcare monitoring system for lung cancer risk detection.

Sustainable Cities and Society, 103079.

450 S. Samuel

8. Andrews, C., Southworth, M. K., Silva, J. N. A., & Silva, J. R. (2019). Extended reality in

medical practice. Current Treatment Options in Cardiovascular Medicine, 21, 18. https://doi.

org/10.1007/s11936-019-0722-7

9. NBC News. (2021). 3 ways virtual reality is transforming medical care. [Online].

Ava il ab le a t: https://www.nbcnews.com/mach/science/3-ways-virtual-reality-transforming-

medical-care-ncna794871

10. Tripathy, H. K., Mallick, P. K., & Mishra, S. (2021). Application and evaluation of classiﬁcation

model to detect autistic spectrum disorders in children. International Journal of Computer

Applications in Technology, 65(4), 368–377.

11. Institute of Imagination. (2021). Five surprising facts about virtual reality. Institute of Imag-

ination. [Online]. Available at: https://ioi.london/latest/ﬁve-surprising-facts-about-virtual-rea

lity/

12. Program-Ace. (2021). Virtual reality in healthcare: A new solution for rehabilitation?

[Online]. Available at: https://program-ace.com/blog/virtual-reality-in-healthcare-a-new-sol

ution-for-rehabilitation

13. Imaginovation.net. (2021). [Online]. Available at: https://www.imaginovation.net/blog/10-inc

redible-use-cases-of-vr-healthcare-explained-with-examples/

14. A Magazine Singapore. (2021). Treating dementia with virtual reality. [Online]. Available at:

https://read-a.com/treating-dementia-with-virtual-reality/

15. Kim, O., Pang, Y., & Kim, J. H. (2019). The effectiveness of virtual reality for people with mild

cognitive impairment or dementia: A meta-analysis. BMC Psychiatry, 19, 219. https://doi.org/

10.1186/s12888-019-2180-x

16. Wu, J., Ma, Y., & Ren, Z. (2020). Rehabilitative effects of virtual reality technology for mild

cognitive impairment: A systematic review with meta-analysis. Frontiers in Psychology, 11.

https://doi.org/10.3389/fpsyg.2020.01811. ISSN 1664-1078

17. Pears, M., Yiasemidou, M., Ismail, M. A., Veneziano, D., & Biyani, C. S. (2020). Role of immer-

sive technologies in healthcare education during the COVID-19 epidemic. SAGE Journals.

[Online]. Available at: https://doi.org/10.1177/0036933020956317

18. Nature.com. (2021). [Online]. Available at: https://www.nature.com/articles/s41398-020-

0697-x.pdf

19. Hitl.washington.edu. (2021). HITLab projects: VR therapy for spider phobia. [Online].

Available at: http://www.hitl.washington.edu/projects/exposure/

20. Juan, M.-C., & Calatrava, J. (2011). An augmented reality system for the treatment of phobia

to small animals viewed via an optical see-through HMD: Comparison with a similar system

viewed via a video see-through HMD. International Journal of Human-Computer Interaction,

27, 436–449. https://doi.org/10.1080/10447318.2011.552059

21. Wrzesien, M., Alcañiz Raya, M., Botella, C., Burkhardt, J.-M., Bretón-López, J., Ortega, M., &

Brotons, D. (2013). The therapeutic lamp: Treating small-animal phobias. IEEE Computer

Graphics and Applications, 33.https://doi.org/10.1109/MCG.2013.12

22. Accenture.com. (2021). [Online]. Available at: https://www.accenture.com/_acnmedia/PDF-

102/Accenture-Digital-Health-Tech-Vision-2019.pdf#zoom=100

23. Nalla. (2021). From digital to DARQ: How technology is changing healthcare. Nalla. [Online].

Available at: https://nalla.co.uk/from-digital-to-darq-how-technology-is-changing-healthcare/

24. Hhs.gov. (2021). [Online]. Available at: https://www.hhs.gov/sites/default/ﬁles/blockchain-

for-healthcare-tlpwhite.pdf

25. BMC Blogs. (2021). The human impact of ﬁve emerging technologies in the healthcare sector.

[Online]. Available at: https://www.bmc.com/blogs/human-impact-of-emerging-technologies-

in-healthcare/

26. Montanaro, A. (2016). Quantum algorithms: An overview. npj Quantum Information, 2(1).

27. Hodges, A. (2021). [Online]. arXiv.org. Available at: https://arxiv.org/ftp/quant-ph/papers/

0512/0512248.pdf

28. Miliard, M. (2021). Accenture has a DARQ vision of healthcare’s ‘post-digital’ era. [Online].

Healthcare IT News. Available at: https://www.healthcareitnews.com/news/accenture-has-

darq-vision-healthcares-post-digital-era

Immersive Technologies in the Healthcare Space 451

29. Yiasemidou, M., Glassman, D., Mushtaq, F., Athanasiou, C., Williams, M.-M., Jayne, D., &

Miskovic, D. (2017). Mental practice with interactive 3D visual aids enhances surgical

performance. Surgical Endoscopy, 31, 4111–4117.

30. Stoica, R.-A., & Abreu, G. (2019). 6G: The wireless communications network for collaborative

and AI applications.

31. MIT Technology Review. (2021). Ready for 6G? How AI will shape the network of the future.

[Online]. Available at: https://www.technologyreview.com/2019/04/19/136036/ready-for-6g-

how-ai-will-shape-the-network-of-the-future/

32. Shahraki, A., Piran, M., & Taherkordi, A. (2021). A comprehensive survey on 6G networks:

Applications, core services, enabling technologies, and future challenges. [Online]. arXiv.org.

Available at: https://arxiv.org/pdf/2101.12475.pdf

33. Elmeadawy, S., & Shubair, R. (2021). Enabling technologies for 6G future wireless communi-

cations: Opportunities and challenges. [Online]. arXiv.org. Available at: https://arxiv.org/pdf/

2002.06068.pdf

34. Giordani, G., Mezzavilla, M., Rangan, S., & Zorzi, M. (2021). Towards 6G networks: Use

cases and technologies. [Online]. arXiv.org. Available at: https://arxiv.org/pdf/1903.12216.pdf

35. Gollakota, S., Al Hassanieh, H., Ransford, B., Katabi, D., & Fu, K. (2021). IMD shield: Securing

implantable medical devices. [Online]. Groups.csail.mit.edu. Available at: https://groups.csail.

mit.edu/netmit/IMDShield/

36. Van Den Berg, D., Glans, R., De Koning, D., Kuipers, F. A., Lugtenburg, J., Polachan,

K., Venkata, P. T., Singh, C., Turkovic, B., & Van Wijk, B. (2017). Challenges in haptic

communications over the tactile internet. IEEE Access, 5, 23502–23518.

37. Zhang, Z., Xiao, Y., Ma, Z., Xiao, M., Ding, Z., Lei, X., Karagiannidis, G. K., & Fan, P.

(2019). 6G wireless networks: Vision, requirements, architecture, and key technologies. IEEE

Vehicular Technology Magazine, 14(3), 28–41.

38. Bracq, M. S., Michinov, E., & Jannin, P. (2019). Virtual reality simulation in nontechnical

skills training for healthcare professionals: A systematic review. Simulation in Healthcare, 14,

188–194.

Artiﬁcial Intelligence in Telemedicine:

A Brief Survey

Sibanjan Debeeprasad Das and Pradip Kumar Bala

Abstract Nowadays, telemedicine services based on Artiﬁcial Intelligence are not

conﬁned to research labs rather they have become a part of human efforts to

improve Healthcare services. To coordinate distant medical operations in clinical

centers, telemedicine used digital information and broadcast inter-communicative

approaches. The overall management of medical norms and patient well-being frame-

work is disrupted by machine intelligence in telemedicine by providing advanced

methods of coordination. This scenario can be seen in regions of telehealth appli-

cations where Artiﬁcial Intelligence use cases are utilized to inﬂuence or build new

rare medical approaches. This study discusses the use of AI in telehealth. Some vital

applications are discussed here. A brief literature survey highlighting some contri-

butions of AI in telehealth is presented. Major challenges and solutions are also

highlighted.

Keywords Telemedicine ·Artiﬁcial Intelligence ·Disease diagnosis ·Mobile

health ·Healthcare

1 Introduction to Telemedicine

Telemedicine is the usage of technology by a medical professional to diagnose and

carry out treatment of patients in a remote location. Telehealth is a growing sector

of the healthcare industry that has been gaining attraction and forming a proﬁtable

sector, on the basis of Transparency Market Research. The implementation of these

practices and technologies is showing increasing trend within healthcare providers

and institutions. Examples of technologies used in delivering services are video

conferencing and the Internet of Things (IoT). This progressing development in

technology conﬁdes the path to the expansions of connections along with the internet

S. D. Das (B)·P. K. Bala

Indian Institute of Management, Ranchi, India

e-mail: sibanjandas@gmail.com

P. K. Bala

e-mail: pradip.bala@iimranchi.ac.in

S. Mishra et al. (eds.), Augmented Intelligence in Healthcare: A Pragmatic and Integrated

Analysis, Studies in Computational Intelligence 1024,

https://doi.org/10.1007/978-981-19- 1076-0_23

453

454 S. D. Das and P. K. Bala

and the increasing data that has created greater opportunities in the excellence of

health and welfare globally. Sharing of data, information, and analytics watch or

tracking wearables, cloud tech, and robotics are rising as they are creating innovation.

There has to be a patient wait to enjoy the outcomes. This is leading us to a better

future for telemedicine, starting with at basics at onset. This can be done by bettering

the facilities available for patient monitoring and adding extra facilities could be

beneﬁcial for the patients [1]. Hence, this is very useful in transferring the information

via internet and involves less spread of germs [2,3]. The main aim of telemedicine is

cope with accessing and communicating to lessen the cost of the technology. They,

thus help us in minimizing the cost of the overall procedures [4]. This method is

getting popular, but gradually as people need to adapt to it [5,6]. They can help in

subjugating the cost of it by spending less on other unimportant structures, hence

making this is a priority project [7]. This enables to provide information for image

processing in a better way. What here in this paper we mainly focus on is to better

the scope and possibilities of telemedicine in the arena for more people to know it

and get comfortable with. Figure 1shows us the realms of telemedicine and its most

effective branches in which it can beneﬁt us in tomorrow and the days to come.

In this paper, we will discuss how AI can be implemented in telemedicine to better

the present and increase the scopes for future. This can majorly be used to monitor

patients, treat them from the comfort of their home, and seam in an effortless drive

between the patient and the doctor thus giving more comfort and privacy. Here,

Fig. 1 Telemedicine in a nutshell

Artiﬁcial Intelligence in Telemedicine: A Brief Survey 455

according to our research, we can say that telemedicine can be majorly divided into

two categories:

Virtual Consultancies: This is when organizations create platforms consultancies

and discussion seminars between the patients and doctors, talk more about their

issues, wherein, they take the aid of AI and ML to state and ﬁgure out the patient’s

electronic health or medical record (EHR/EMR) for providing the accurate solutions.

Diagnosis Aid: This is practically when organizations create and design chatbots to

interact freely with their patients and diagnose their issues regardless of any hustle

and also at the comfort of their homes.

2 Applications of Telemedicine Using AI

First, we will explore the longer term of AI within remote patient monitoring and how

intelligent assistance can help medical professionals form a diagnosis and treatment

plan. We then have to examine how advances in data retrieval and analysis will

impact healthcare information technology as well as improve collaboration between

medical experts around the world.

Remote Patient Examining

This is one of the best and till date convenient of this technology. This technology

supports and enhances real-time interaction between patients and doctors. Owing to

the rise of uses in AI technology, the possibilities of remotely monitoring a patient’s

health seem to be almost inﬁnite and beyond.

In near future, we may get to see less face-to-face interactions as it is more

comfortable and spares the risk of germs, owing to that in Corona, people are starting

to realize more about social distancing. We can monitor the health of patients via AI

and treat them.

Use of Intelligent Assistance for Diagnosis

The current technology grants the patients to monitor their breathing, heart rate, and

the blood pressure. But in the new technology being developed by start-up companies,

we use AI for screening and evaluating patients so that doctors can quite efﬁciently

form a diagnosis, diagnose all medications from even a remote location.

Combining ML into this equation could prove very efﬁcient in emergency situ-

ations even. Let’s take an example where AI helps in collection of records from

patients who are on route to an urgent need thereby providing prevalent analysis

rather than waiting to collect much-needed information, once the patient arrives.

Enhances in Healthcare Information Technology

According to the Annals of Internal Medicine, around half of a clinical professional’s

time is wasted on analysis of digital medical records. But, the latest advancements in

big data processing and artiﬁcial immune systems make the retrieval of these digital

456 S. D. Das and P. K. Bala

records very effective. A system model that is used presently is determined using a

communication platform to gather data of patients and store it in a manner that it can

be instantly retrieved.

Another study is focused on how smartphones linked via smart bands or devices

can help us track down our pressure, heart rate, and a lot more.

Improved Collaboration Among Healthcare Professionals

Telemedicine can only grow forward, if it is openly embraced by all the health

ofﬁcials in a proper collaboration. Doctors can treat their patients with the help and

advice from other senior doctors as well leading to a better treatment. Thus, we will

have a chance of better medical at even low costs.

Information Analysis and Collaboration

This technology helps to reach out to medical experts who not only encourage but

also collaborate with some new data in the ﬁeld of medicine. This will also renew

research in the ﬁeld of medicine with beneﬁts on the usage of big data analytics

in producing the results effectively as well as the use of genetic neural networks to

understand data and how they are to be dealt with. This part mainly focuses on how

AI can be implemented in this medical world to boost impacts. Here, by this, we can

get the information to be analyzed, except just storing it. Here, in this program, we

will strictly depend on results.

Table 1down here explains how telemedicine is being properly utilized since it

is showing promising futuristic effects. Some examples are stated down below.

3 Considerations and Issues

As we see AI offers us all the resources to boost healthcare delivery through tele-

health tools, it is high time that we consider the social and ethical aspects of using it.

Like other technological advances in healthcare, AI too will cause imbalance in many

sectors of healthcare delivery including access to services, work-ﬂows, communica-

tion, and the operational procedures between different providers and patients [13–15].

With all the new advancements, AI-enabled telemedicine can reach through a wave

of excitement and potential, but can also have to tread failure, disappointment, and

fear before establishing a stability. We should aim to make the stable static, the sooner

we can. The three steps to these issues are:

Ensuring equity—While AI boosts access to the delivery of services, they also make

sure the do’s and do not. We make sure that the technology uses healthcare delivery

enabling services delivery for those who need it most, like in rural areas as well as

in undeveloped sectors.

Monitoring the technological divide—In a scattered set, some are proﬁcient users

of technology, whereas the rest cannot. People expect to gain from advancements in

telemedicine. There can be aged people not very much ﬂuent with the technology.

Artiﬁcial Intelligence in Telemedicine: A Brief Survey 457

Tabl e 1 Summary of telemedicine and its applications

Telemedicine topic Category Details

A predictive model for assistive

technology adoption for people

with dementia [5]

Information analysis and

collaboration

It uses a k-Nearest-Neighbor to

understand the behavior of

people affected with dementia

and predict how doctors can

successfully diagnose

A telerehabilitation application

with set of consultation classes

[6]

Healthcare information

technology

It addresses the type of issue

under low bandwidth network

conditions. They hold

customized consultations

illustrating rehabilitation process

for the individuals adaptively

A patient monitoring system for

dengue via wireless connection:

Wi-Mon [7]

Patient monitoring This is a monitoring network

offering wireless systems as on

Wireless Body Area Network

(WBAN) concept

An effective telemedicine

security using wavelet-based

watermarking [8]

Healthcare information

technology

This provides an algorithm that

interwinds digital and wavelet

watermarks on medical images

to assure privacy by embedding

them

COMPASS: which is an

interoperable personal health

system to monitor and compress

signals in COPD Thomas [9]

Patient monitoring This paper allows all android

devices to implore our health.

This demonstrates the sensing of

data and converting it into way

to be packed usable

The detection of fetal

electrocardiogram through

OFDM which uses neuro-fuzzy

logic and wavelets systems for

telemetry [10]

Patient monitoring This paper detects and monitors

exact electrocardiogram and the

positioning and doing of a fetus

inside an abdomen using fuzzy

logic

Ubiquitous health monitoring for

logic-centered architecture [11]

Patient monitoring This paper presents a

logic-oriented software which

helps us in monitoring our

health using a logic using a

special software

Functions, ambient ontology and

e-Diagnostics of mobile

cyber-physical systems for

health care: [12]

Patient monitoring This paper tells that if the patient

is wearing a certain wearable,

then doctors can monitor it and

receive updates. This, an

application of fuzzy systems

were used to identify the

procedural actions for the time

being

458 S. D. Das and P. K. Bala

Fig. 2 Telemedicine growth in past and future years

We have to make sure that AI enhances our ability to provide quality patient-centered

care rather than increasing the digital divisions.

AI is only a means—HIT implementation in healthcare has been described as a

journey and as the digitization of healthcare. If it proceeds at record speeds, this

will become more intriguing than others. We must try to enhance the AI by using

pragmatic tools in its development and also enhance the delivery and design, thus

creating something static.

Figure 2shows us the static rising growth of telemedicine in the years to come

and hence popularizing the concept even more.

Some of the positive telehealth outcomes we have seen over the years include:

•Providing access to patient from underdeveloped areas.

•Improving the medications before the patient is caught up with something grave.

•Enhancing the health and checking up using AI.

•Less number of physical contact, helping less spreading of diseases.

•Almost minimal travel costs with ambiance facility.

•Provide a secure environment to elders.

•Supply and aid any injured labor in a construction site.

•Betterment of clinical facilities and paramedics.

•Overall costs of hospital are also being reduced.

Artiﬁcial Intelligence in Telemedicine: A Brief Survey 459

4 Challenges Faced in Implementation

While the way telehealth was used was diverse, there were a lot of challenges in the

way to implement, but we also found solutions to it. The challenges we faced are:

•Security and interoperations threw challenges because numerous healthcare

organizations were needed to interact with the sourcing. Grantees arranged secu-

rity technologies inbrid to tackle these issues, by building ﬁrewalls and encryption,

through sharing and building protocols. This coordination between the consumer

and client can pave way for success.

•Image resolution and video quality require speciﬁc band length. The clients

complained that the video quality was extremely low, which is inadequate for

medical purposes. The client introduces telemedicine in major projects in order

to solve these issues. But, in the meantime for a stretch of 15 years or more, band

with has developed greatly.

•Technical support is very essential and cost-effectiveness is a factor for these

programs. In rural areas, there are very mere resources, thus leading to third-

party hindrance and it obstructs the adequate technical scope, thus leading to

increased costs and a lot more expenses [16,17]. Thus, we can assume how much

overbearing it can be.

•Organizational culture alters the telemedicine and its realms. The more the better

stake, the more is the organizational spirit. But, then people were not very inclined

to the telemedicine sector as it mainly is still lacking in major sectors due to

viable resources [18]. The main challenge of telemedicine is that it has to meet

the geographic distances and come closer to us virtually so that we can embrace

it fully without any hesitations.

•Provider retention in rural areas may be quite taxing at times. In rural areas,

doctors are not well paid and do not get a lavish life, unlike what we seek. Thus,

they may also lack. Before the participation, people have to travel a lot to seek

these [19–21]. Hence it directly affects costs. Now, in remote regions, easily

access enabled medical centers are not in abundance. Thus the distance coverage

and time wasted during travel are few primary challenges encountered by patients.

Besides this, the waiting period after reaching the center is also too much since the

ratio between patients attended by medical staff is quite taxing. After the work,

the turnover is quite taxing as it does not serve enough to the doctors [22].

At present some innovative software applications are in practice in context of

telemedicine.

•AccuRx is one recent innovation that facilitates multimedia information sharing

among usual practitioners and patients in many foreign nations.

•Mobile Telemed acts as a virtual clinical care model utilized in homes of patients,

nursing centers, and educational centers.

•eVisit is one telehealth interface enabling medical staff in providing virtual screens

for pandemics and help in curing non-emergent patients.

460 S. D. Das and P. K. Bala

•Vivify is one recent patient tracking platform helping medical staff to monitor

overall health status of patients and evaluate health metrics such as pulse rate,

breathing trends, and body temperature among them.

5 Conclusion

Concluding ﬁnally, we know telemedicine can exceed and enhance greatly if given

the proper exposure and reaches, thus enhancing the technology. This is a very

simple technology that can be always be used by anyone to use aggravate the topics

of telemedicine in a lot better way. This is a vast process where anyone can just

discover a wide unending scope of possibilities that can help a lot more patients and

save a lot more lives. We can say that telemedicine has done a lot more progress

in every ﬁeld, but there are a lot more challenges that have to be implemented and

corrected. These applications will greatly enhance the process using AI, that is why

we have to ensure how to make these more efﬁcient and cost-effective so that they

can be implemented in undeveloped rural areas without any hassles.

References

1. Rath, M., & Mishra, S. (2019). Advanced-level security in network and real-time applications

using machine learning approaches. In Machine learning and cognitive science applications

in cyber security (pp. 84–104). IGI Global.

2. Dinya, E., & Tóth, T. (2013). Health informatics: eHEALTH and TELEMEDICINE. Semmel-

weis University Institute of Health Informatics.

3. World Health Organization. (2010). 2010 opportunities and developments report on the second

global survey on eHealth global observatory for eHealth series—Volume 2 TELEMEDICINE

in member states WHO library cataloguing-in publication data (Report). WHO Press.

4. Ray, C., Tripathy, H. K., & Mishra, S. (2019, June). Assessment of autistic disorder

using machine learning approach. In International Conference on Intelligent Computing and

Communication (pp. 209–219). Springer Singapore.

5. Zhang, S., McClean, S. I., Nugent, C. D., Donnelly, M. P., Galway,L., Scotney, B. W., & Cleland,

I. (2014). A predictive model for assistive technology adoption for people with dementia. IEEE

Journal of Biomedical and Health Informatics, 18(1), 375–383.

6. Kiong, T. K., & Narayanan, A. S. (2014). A telerehabilitation application with pre-deﬁned

consultation classes. In 2014, International Conference on Circuits, Power and Computing

Technologies [ICCPCT-2014] (pp. 1238–1244).

7. Nubenthan, S., & Shalomy, C. (2017). A wireless continuous patient monitoring system for

dengue: Wi-Mon. In 2017 6th National Conference on Technology and Management (NCTM),

Nagercoil (pp. 23–27). IEEE.

8. Epstein, R. M., Fiscella, K., Lesser, C. S., & Stange, K. C. (2010). Why the nation needs a

policy push on patient-centered health care. Health Affairs (Millwood). 29(08), 1489–1495.

9. Hofer, M. S., & Bromuri, S. (2015). COMPASS: An interoperable personal health system

to monitor and compress signals in chronic obstructive pulmonary disease. In Proceedings

of the 9th International Conference on Pervasive Computing Technologies for Healthcare

(pp. 304–311), Istanbul. IEEE.

Artiﬁcial Intelligence in Telemedicine: A Brief Survey 461

10. Kumar, P., Sharma, S. K., & Prasad, S. (2016). Detection of fetal electrocardiogram through

OFDM, neuro-fuzzy logic and wavelets systems for telemetry. In 2016 10th International

Conference on Intelligent Systems and Control (ISCO) (pp. 1–4), Coimbatore. IEEE.

11. Lewandowski, J., Arochena, H. E., Naguib, R. N. G., Chao, K.-M., & Garcia-Perez, A. (2014).

Logic-centered architecture for ubiquitous health monitoring. IEEE Journal of Biomedical and

Health Informatics, 18(5), 1525–1532.

12. Costanzo, A., Faro, A., Giordano, D., & Pino, C. (2016). Mobile cyber physical systems

for health care: Functions, ambient ontology and e-diagnostics. In 2016 13th IEEE Annual

Consumer Communications & Networking Conference (CCNC), Las Vegas, NV (pp. 972–975).

13. Mishra, S., Sahoo, S., & Mishra, B. K. (2019). Addressing security issues and standards in

Internet of things. In Emerging trends and applications in cognitive computing (pp. 224–257).

IGI Global.

14. Mishra, S., Mahanty, C., Dash, S., & Mishra, B. K. (2019). Implementation of BFS-NB hybrid

model in intrusion detection system. In Recent developments in machine learning and data

analytics (pp. 167–175). Springer Singapore.

15. Sahoo, S., Mishra, S., Mishra, B. K. K., & Mishra, M. (2018). Analysis and implementation

of artiﬁcial bee colony optimization in constrained optimization problems. In Handbook of

research on modeling, analysis, and application of nature-inspired metaheuristic algorithms

(pp. 413–432). IGI Global.

16. Jena, L., Kamila, N. K., & Mishra, S. (2014). Privacy preserving distributed data mining

with evolutionary computing. In Proceedings of the International Conference on Frontiers of

Intelligent Computing: Theory and Applications (FICTA) 2013 (pp. 259–267). Springer.

17. Tripathy, H. K., Mishra, S., Thakkar, H. K., & Rai, D. (2021). CARE: A collision-aware

mobile robot navigation in grid environment using improved breadth ﬁrst search. Computers &

Electrical Engineering, 94, 107327.

18. Drake, C., Zhang, Y., Chaiyachati, K. H., & Polsky, D. (2019). The limitations of poor broad-

band internet access for telemedicine use in rural America: An observational study. Annals of

Internal Medicine, 171, 382–384. https://doi.org/10.7326/M19-0283

19. Jena, L., Patra, B., Nayak, S., Mishra, S., & Tripathy, S. (2021). Risk prediction of kidney

disease using machine learning strategies. In Intelligent and cloud computing (pp. 485–494).

Springer Singapore.

20. Ash, J. S., Berg, M., & Coiera, E. (2004). Some unintended consequences of information

technology in health care: The nature of patient care information system-related errors. Journal

of the American Medical Informatics Association, 11, 104–112. https://doi.org/10.1197/jamia.

M1471

21. Mishra, S., Raj, A., Kayal, A., Choudhary, V., Verma, P., & Biswal, L. (2012). Study of cluster

based routing protocols in wireless sensor networks. International Journal of Scientiﬁc and

Engineering Research, 3(7).

22. Subramanian, S., Pamplin, J. C., Hravnak, M., Hielsberg, C., Riker, R., Rincon, F., Laudanski,

K., Adzhigirey, L. A., Moughrabieh, M. A., Winterbottom, F. A., & Herasevich, V. (2020).

Tele-critical care: An update from the society of critical care medicine tele-ICU committee.

Critical Care Medicine, 48, 553–561.

Infectious Diseases Reporting System

Using Naïve Bayes Classiﬁcation

Algorithm

Ishola D. Muraina and Abdullahi Umar Farouk

Abstract Many approaches have been used in the past to address issues of spreading

infectious diseases in the society with inclusion of reporting system. However, there

is less or no study on the use of machine learning techniques in designing the reporting

system for infectious diseases. Thus, the main objective of this study is to design an

infectious disease reporting system that makes use of Naïve Bayes classiﬁcation algo-

rithm. The issue was addressed through a quantitative research approach to design

a system that uses Naïve Bayes classiﬁcation algorithm while reporting infectious

diseases with respect to their associated symptoms. A model was developed together

with system Pseudocode and the result revealed that the use of machine learning tech-

nique in the study eases the recommendation aspect of the system toward creating

awareness of infectious diseases in the society. Hence, the study may serve as assis-

tance to healthcare industry toward mitigation or eradication of infectious diseases

in the societies.

Keywords Naïve Bayes classiﬁcation algorithm ·Reporting system ·Infectious

disease ·Disease mitigation ·Disease awareness

1 Introduction

A reporting system has been described as a tool used to documents the incidents

and their causes toward investigating and gaining more understanding of the matter

at hand [5]. This helps in sourcing or mining information from the pool of store

data for onwards execution of decision-making. Studies by Scheving et al. [25]

and Katz et al. [14] stressed that reporting system in the clinical domain has been

assisted in addressing the patients’ health histories and behavioral changes or patterns

that may occur in the trend of their recovery processes. Besides that, electronic

form of reporting systems had been argued to ease transfer and sharing of hospital

specializations and status of patients to risk or prone to infections [7,8]. In other

I. D. Muraina (B)·A. U. Farouk

Computer Science Department, Faculty of Science, Yusuf Maitama Sule University (Formerly,

Northwest University), Kano, Nigeria

e-mail: ishod200@gmail.com

S. Mishra et al. (eds.), Augmented Intelligence in Healthcare: A Pragmatic and Integrated

Analysis, Studies in Computational Intelligence 1024,

https://doi.org/10.1007/978-981-19-1076-0_24

463

464 I. D. Muraina and A. U. Farouk

words, a clinical reporting system ensures accelerated access to historical data of

patients.

The effect of infectious diseases is believed to have been felt around the globe

[17,22] as a result of havoc being caused, especially in the era of covid-19 pandemic.

Infectious diseases are believed to be caused by pathogenic microorganisms such as

viruses, bacteria, parasites, and fungi, thus capable of transmitting from one person

to another [28]. There have been efforts toward mitigating the spread of infectious

diseases, while support has been given by the multinational companies and healthcare

industries [18,19]. This has led to development and design of different methods to

challenge the spread of infectious diseases through evolvement of some predictive

models and reporting systems, thus intensifying its awareness. However, there has

been less or no study on applying machine learning algorithm that takes classiﬁcation

technique into consideration while designing reporting system for infectious diseases.

Therefore, this study applies Naïve Bayes (NB) classiﬁcation algorithm in designing

infectious diseases reporting system toward its mitigation.

2 Naïve Bayes Classiﬁcation Algorithm

The NB has widely been used as data mining algorithm for classiﬁcation of attributes

that work independently of one another [2,3]. This shows that NB is highly accurate

classiﬁer and competent in terms of taking care of some activities that have to do

with assumption of selection and classiﬁcation of trained and examined data, known

as supervised data. The study of Chen et al. [3] stressed that one of merits of the NB

is its classiﬁcation accuracy which is always obtained when some attributes are being

independently selected as depicted in Fig. 1, thus ﬁts to be used in real applications.

Researchers have argued that its simplicity characteristic has made it to be regarded as

a powerful machine learning technique and ﬁts to be used in different areas of studies

toward addressing issues in real-time prediction, weather forecast, spam ﬁltering,

recommender systems, and medical diagnosis [15,26]. This implies that NB algo-

rithm ﬁts in computerization of applications for medical purposes, speciﬁcally in

reporting different diseases using the same procedural approaches.

Fig. 1 ANaïveBayes

classiﬁer [15]

Infectious Diseases Reporting System Using Naïve Bayes … 465

2.1 Conceptual Development of Naive Bayes Algorithm

Studies of Kaviani and Dhotre [15] and Kökver et al. [16] stressed that the NB

classiﬁer is based on the Bayes theorem or Bayesian network as shown in Fig. 1and

Eq. 1.

Aand Bare subjected to be random numbers as shown below:

P(A|B)=P(B|A)P(A)/P(B)(1)

where

P(A) is the independent probability of event Aat primitive level.

P(B) is independent probability of event B.

P(A|B) is Probability of Bevent (conditional probability) when it is known that

Aevent occurs.

P(B|A) is Probability of event A(aftershock probability) when event Bis known.

In other words, the concept of the NB can be better explained in such a way that,

Xis believed to be instance of unknown class membership X={x1, x2, …, xn} with

attribute values. Thus, it is assumed to be n class and C1, C2, …, Cn class values

are accepted. According to Erol et al. [6], the possibilities are calculated as:

P(X|Ci)=P(X|Ci)P(Ci)

P(X)(2)

Moreover, demystiﬁcation of probability P(X|Ci) toward reducing the load in the

calculation and assuming that Xivalues of the class are independent of each other,

we obtain

P(X|Ci)=



k=1

P(Xk|Ci)(3)

To classify the unknown class X, the values in the numerator is ﬁt to be considered

since denominators in P(Ci|X)inEq.1are equal to each other. Hence, the class of

unknown instance is believed to be the same as the class of the largest of the values:

arg min

Ci{P(X|Ci)P(Ci)}(4)

Equation 4is known as post probabilities and can be considered as post-

classiﬁcation method and represented as:

CMAP =arg min

Cin



k=1

P(Xk|Ci)(5)

Furthermore, the NB classiﬁcation algorithm in real sense that depicts and eases

individual understanding is represented in ﬂowchart in Fig. 2.

466 I. D. Muraina and A. U. Farouk

Fig. 2 Flowchart representation of Naïve Bayes classiﬁcation algorithm

Figure 2represents behavior of underlying probabilistic model and how it could be

used to deal with uncertainty about the model in a principle way toward determining

probabilities of the outcomes.

3 Related Works

The reporting system especially in healthcare domain has been used to address

management of clinical, self-management, care planning, and setting goals for both

patients and hospital workers [29]. This shows that reporting system has been posi-

tively impacted through the introduction of Information and Communication Tech-

nology (ICT) in the operation and accessibility of healthcare to all and sundry. Studies

of Cohen et al. [4] and Weldring and Smith [30] have stressed that application of

the ICT in healthcare sector has moved beyond patient-centered modeling to directly

Infectious Diseases Reporting System Using Naïve Bayes … 467

gather data from the patients. This has led to development of self-reported system

where patients report their health status and the pace trend of improvement of their

health with respect to the medication given to them. The patient self-reported system

does allow to directly interact with health ofﬁcials without intermediary that could

compromise the information which can jeopardize the result of assessment status of

patient to the medication in use.

One of the applications of healthcare self-reporting argued by the study of Judge-

Golden et al. [13] focuses on fostering and simplifying gaining of access and explo-

ration of different methods for contraceptives for women. The self-reported system

has helped in determining the best approaches for women to choose together with

other factors, especially in the western world. On the other hand, some healthcare

reporting system fails to address the ultimate needs of both patients and health ofﬁ-

cials toward achieving homogeneous outcome of the system [10,11]. Thus, the use

of self-reporting system has not been used to optimal level to achieve the set objective

for its development.

Moreover, the use of uniﬁed hospital data surveillance system has been helpful

in handling the issue related to COVID-19 in the USA through the collection of data

from the hospital that earlier registered with the Centers for Medicare and Medicaid

Services (CMS) in December of 2020 [24]. This assists in assessing the coverage

of vaccines among the hospitals and the healthcare workers who may likely be the

carriers of infections if proper cautions are not taken. The study reveals how vulner-

able the healthcare personnel is to the infectious diseases through its recommendation

function of the reporting system that is used to collate the data which are related to

Covid-19. A related study by Tokalic et al. [27] has revealed the strength brought

by the health reporting system in terms of minimizing the fragility and stress being

faced by the healthcare professionals in familiarizing with immediate and real-time

information of patients. This implies that healthcare reporting systems are always

designed with stress management attributes, showing it is resilient to any effects that

may evolve in their operations. Hence, it could be deduced that reporting systems in

healthcare have great impact on the healthcare operational delivery to the patients

and the rest of the society, therefore, some of the supported healthcare report systems

in healthcare research are summarized in Table 1.

Table 1has exploratory represents the involvement of report system in healthcare

operations and activities. Thus, the use of Naïve Bayes classiﬁcation algorithm is

scarce to ﬁnd in reporting systems in the healthcare research.

4 Materials and Methods

The main objective of this study is to design an infectious diseases reporting system

using the NB classiﬁcation algorithm. Therefore, proper reporting of the identiﬁed

infectious diseases in context of this study is guided by the NB classiﬁcation.

468 I. D. Muraina and A. U. Farouk

Tabl e 1 Prominent studies on report system in healthcare research

Authors Title Objective/main activities

Faveri and Roessler [9]Clowning during

COVID-19—a survey of

European Healthcare Clowning

Organisations highlights the

role of humour and art in the

healthcare system

Investigation of impact of

Covid-19 on healthcare clowning

organizations through the use of

healthcare system

Moons et al. [20] Patient-reported outcomes in

adults with congenital heart

disease: inter-country variation,

standard of living and

healthcare system factors

Geographical differences in

patient-reported outcomes of

adults with congenital heart

disease

Manderscheid et al. [21] Reporting system for critical

incidents in cross-sectoral

healthcare (CIRS-CS): pre-test

of a reporting sheet and

optimization of a reporting

system

Testing of the content validity of

the reporting sheet and to optimize

the cross-sectoral critical incident

reporting system

Alhashem et al. [1]The Bethesda System for

Reporting Thyroid

Cytopathology: a retrospective

review of its diagnostic utility

at Johns Hopkins Aramco

Healthcare, Saudi Arabia

Evaluation of diagnostic utility of

the Bethesda System for

Reporting Thyroid Cytopathology

for Saudi population, by

comparing the malignancy risk

based on histopathology

Ruiz-Correa et al. [23]Health sentinel: a mobile

crowdsourcing platform for

self-reported surveys provides

early detection of COVID-19

clusters in San Luis Potosí,

Mexico

Deployment of self-assessment

and medical reporting for contact

tracing activities

De Kam et al. [5]How incident reporting systems

can stimulate social and

participative learning: a

mixed-methods study

Investigation of serious incidents

to understand what causes patient

harm in the context of Dutch

incident reporting systems

Jermacane et al. [12] An evaluation of the electronic

reporting system for the

enhanced surveillance of

carbapenemase-producing

Gram-negative bacteria in

England

Evaluation of assessment uptake,

timeliness, and completeness of

data provided and explore

potential barriers and facilitators

to adopting the electronic system

Katz et al. [14] An improved patient safety

reporting system increases

reports of disruptive behavior

in the perioperative setting

Demonstrate that a brief

educational intervention in

addition to an upgraded reporting

system with positive event options

resulted in an improved system for

feedback on the culture of safety

(continued)

Infectious Diseases Reporting System Using Naïve Bayes … 469

Tabl e 1 (continued)

Authors Title Objective/main activities

Scheving et al. [25]Implementation of a pilot

electronic stroke outcome

reporting system for emergency

care providers

Pilot implementation of the

electronic reporting system leads

to discovery of omission of

protected health information

enhanced transmission of outcome

report which allows ﬂexibility in

the mode of transmission

4.1 Model and Computation for Infectious Diseases

Reporting System

Prior to design of infectious diseases reporting system, a model was ﬁrst devel-

oped and guided by NB classiﬁcation algorithm as shown in Fig. 3. This is strictly

used while different diseases are being reported by the system vis-à-vis the selected

symptoms in the system.

Moreover, the intended reporting system for infectious diseases is computed to

recognize three different diseases such as SARS, Flu, and Tuberculosis, while recom-

mendation of attributed diseases is in line with the NB classiﬁcation algorithm, thus

represented by the following Pseudocode:

INITIALIZE Variable aName as Array of 3

INITIALIZE Variable iIndex as Integer

INITIALIZE Variable Name State, Address, Relationship As String

OUTPUT “Enter patients Name”

ACCEPT patients Name and ASSIGN to aName

OUTPUT “Enter Address”

ACCEPT patients Address and ASSIGN to Address

OUTPUT “Enter Relationship”

ACCEPT Relationship and ASSIGN to Relationship

OUTPUT “Select three symptoms that matches patient’s Symptoms from below

list”

DISPLAY Symptoms

Start ﬁnite for loop

INITIALIZE Variable iIndex =0, iIndex<4, Increment iIndex.

Display Enter symptoms

ACCEPT Symptoms and ASSIGN to aName

IF (aName is equal to muscle pain AND chest pain AND sore

throat AND abdominal pain)

THEN

Likely Disease is SARS

IF (aName is equal to facial swelling AND bleeding from mouth

AND Weight loss AND High temperature)

470 I. D. Muraina and A. U. Farouk

THEN

Likely Disease is Tuberculosis

IF (aName is equal to night sweats AND dry cough AND persistent

cough AND sore throat)

Likely Disease is Flu

IF NOT Disease is not identiﬁed

Fig. 3 A model represents infectious diseases reporting system

Infectious Diseases Reporting System Using Naïve Bayes … 471

The system is trained to recognize and link the associated symptoms of three

different infectious diseases (SARS, Flu, and Tuberculosis) after comparison and

thus, report the likely infectious diseases as shown in the above Pseudocode.

4.2 System Veriﬁcation

Infectious diseases reporting system was designed based on Fig. 3and veriﬁed

through the prototyping development. The system is capable of displaying the likely

disease based on the available symptoms and the overall reports of infectious diseases

can be generated as shown in Figs. 4and 5, respectively.

Fig. 4 Statistic of infectious

diseases

472 I. D. Muraina and A. U. Farouk

Fig. 5 Infectious disease

report generation

5 Discussion and Conclusion

As infectious disease is known as critical issue that requires critical approach, there is

need for a reporting system that takes machine learning techniques into consideration

while reporting its existence or prevalence in the immediate environments. This

study has critically discussed merits and applications of NB classiﬁcation algorithm

in designing the infectious diseases reporting system. This has been shown in the

system modeling and computation where data were trained to recommend speciﬁc

infectious diseases based on the attributed symptoms. Thus, the designed infectious

diseases reporting system using NB classiﬁcation algorithm was tested and veriﬁed

as shown in Figs. 4and 5.

Conclusively, the objective of the study which is to apply the NB classiﬁca-

tion algorithm in designing infectious diseases reporting system toward its mitiga-

tion has been achieved through the presented mode and computational processes to

test the algorithm. Hence, the study will be extended in future by simulating the

responsiveness of the reporting system while using the NB classiﬁcation algorithm.

Infectious Diseases Reporting System Using Naïve Bayes … 473

References

1. Alhashem, M. H., Alabidi, A., & Aly, M. G. (2021). The Bethesda System for Reporting

Thyroid Cytopathology: A retrospective review of its diagnostic utility at Johns Hopkins

Aramco Healthcare, Saudi Arabia. American Journal of Otolaryngology—Head and Neck

Medicine and Surgery, 42, 1–5.

2. Arthur, M. (2021). Review the performance of the Bernoulli Naïve Bayes classiﬁer in the

intrusion detection systems using recursive feature elimination with cross-validate selection of

the best number of features. Procedia Computer Science, 190, 564–570.

3. Chen, S., Webb, G. I., Liu, L. & Ma, X. (2019). A novel selective Naïve Bayes algorithm.

Knowledge-Based Systems, 1–12.

4. Cohen, D. J., Keller, S. R., Hayes, G. R., Dorr, D. A., Ash, J. S., & Sittig, D. F. (2016).

Integrating patient-generated health data into clinical care settings or clinical decision-making:

Lessons learned from project HealthDesign. JMIR Human Factors, 3(2), e26.

5. De Kam, D., Kok, J., Grit, K., Leistikow, I., Vlemminx, M., & Bal, R. (2020). How incident

reporting systems can stimulate social and participative learning: A mixed-methods study.

Health Policy, 124, 834–841.

6. Erol, H., Tyoden, B. M., & Erol, R. (2018). Classiﬁcation performances of data mining clus-

tering algorithms for remotely sensed multispectral image data. In Innovations in Intelligent

Systems and Applications (pp. 1–4).

7. Freeman, R., Ironmonger, D., Puleston, R., Hopkins, K. L., Welfare, W., & Hope, R. (2016).

Enhanced surveillance of carbapenemase producing Gram-negative bacteria to support national

and international prevention and control efforts. Clinical Microbiology and Infection, 22, 896–

897.

8. Freeman, R., Moore, L. S. P., Charlett, A., Donaldson, H., & Holmes, A. H. (2015).

Exploring the epidemiology of carbapenem-resistant Gram-negative bacteria in west London

and the utility of routinely collected hospital microbiology data. Journal of Antimicrobial

Chemotherapy, 70, 1212–1218.

9. Faveri, S. D., & Roessler, M. (2021). Clowning during COVID-19—A survey of European

Healthcare Clowning Organisations highlights the role of humour and art in the healthcare

system. Public Health, 196, 82–84.

10. Jensen, R. E., Rothrock, N. E., & DeWitt, E. M. (2015). The role of technical advances in the

adoption and integration of patient-reported outcomes in clinical care. Medical Care, 53(2),

153–159.

11. Jensen, R. E., Potosky, A. L., & Reeve, B. B. (2015). Validation of the PROMIS physical

function measures in a diverse U.S. population-based cohort of cancer patients. Quality of Life

Research, 24(10), 2333–2344.

12. Jermacane, D., Coope, C. M., Ironmonger, D., Cleary, P., Muller-Pebody, B., Hope, R., Hopkins,

S., Puleston, R., Freeman, R., Hopkins, K. L., Johnson, A. P., Woodford, N., & Oliver, I.

(2019). An evaluation of the electronic reporting system for the enhanced surveillance of

carbapenemase-producing Gram-negative bacteria in England. Journal of Hospital Infection,

102, 17–24.

13. Judge-Golden, C. P., Wolgemuth, T. E., Zhao, X., Mor, M. K., & Borrero, S. (2020). Agreement

between self-reported “ideal” and currently used contraceptive methods among women veteran

using the veteran affairs healthcare system. Women’s Health Issues, 1–9.

14. Katz, M. G., Rockne, W. Y., Braga, R., & McKellar, S. (2019). An improved patient safety

reporting system increases reports of disruptive behaviourin the perioperative setting. American

Journal of Surgery, 1–6.

15. Kaviani, P., & Dhotre, S. (2017). Short survey on Naïve Bayes algorithm. International Journal

of Advance Engineering and Research Development, 4(11), 607–611.

16. Kökver, Y., Bari ¸sçi, N., Çiftçi, A., & Ekmekçi, Y. (2014). Hipertansiyona Etki Eden Faktörlerin

Veri Madencili˘gi Yöntemleriyle ˙

Incelenmesi. Engineering Sciences, 9(2), 15–25.

474 I. D. Muraina and A. U. Farouk

17. Li, L., Xu, W., Wagner, A. L., Dong, X., Yin, J., Zhang, Y., & Boulton, M. L. (2019). Evalu-

ation of health education interventions on Chinese factory workers knowledge, practices and

behaviors related to infectious diseases. Journal of Infectious and Public Health, 12, 70–76.

18. Muraina, I. D., Lawal, I. A., & Mu’azu, A. A. (2021). A smart cloud-based architecture for

predicting infectious diseases using key weather elements. In Proceedings of 1st International

Conference on Data Science and Engineering (pp. 20–25).

19. Muraina, I. D., Madaki, R. R., & Suleiman, A. U. (2021). Data analytic approach for assessment

status of awareness of tuberculosis in Nigeria (Chap. 13). In Handbook of machine learning for

computational optimization: Applications and case studies (pp. 243–250). Taylor & Francis

Group.

20. Moons, P., Kovacs, A. H., Luyckx, K., Thomet, C., Budts, W., Enomoto, J., Sluman, M. A.,

Yang, H. L., Jackson, J. L., Khairy, P., & Cook, S. C. (2017). Patient-reported outcomes in

adults with congenital heart disease: Inter-country variation, standard of living and healthcare

system factors. International Journal of Cardiology, 1–8.

21. Manderscheid, L., Meyer, S., Kuypers, M., & Koberlein-Neu, J. (2019). Reporting system

for critical incidents in cross-sectoral healthcare (CIRS-CS): Pre-test of a reporting sheet

and optimization of a reporting system. Zeitschrift für Evidenz, Fortbildung und Qualität im

Gesundheitswesen, 147–148, 58–66.

22. Omodior, O., Luetke, M. C., & Nelson, E. J. (2018). Mosquito-borne infectious disease, risk-

perceptions, and personal protective behaviour among U.S international travellers. Journal of

Preventive Medicine Reports, 12, 336–342.

23. Ruiz-Correa, S., López-Revilla, R., Díaz-Barriga, F., Marmolejo-Cossío, F., del Carmen

Robledo-Valero, V., Hernández-Huérfano, E. E., Álvarez-Rivera, L., Rangel-Martínez, M. L.,

Lutzow-Steiner, M. Á., Ortiz-Vázquez, L. A., Mendoza-Lara, A. R., Olivo-Rodríguez, M.,

Galván-Ramírez, M. S., Morales-Neri, Á. E., Martínez-Donjuan, V. U., Cervantes-Irurzo, M. I.,

Comas-García, A., Hernández-Maldonado, F., & Aguilar-Acosta, C. (2021). Health Sentinel: A

mobile crowdsourcing platform for self-reported surveys provides early detection of COVID-

19 clusters in San Luis Potosí, Mexico. International Journal of Medical Informatics, 153,

1–12.

24. Reses, H. E., Jones, E. S., Richardson, D. B., Cate, K. M., Walker, D. W., & Shapiro, C. N.

(2021). Covid-19 vaccination coverage among hospital-based healthcare personnel reported

through the Department of Health and Human Services Uniﬁed Hospital Data Surveillance

System, United States, January 20, 2021–September 15, 2021. American Journal of Infectious

Control, 49, 1554–1557.

25. Scheving, W. L., Ebersole, J. M., Froehler, M., Moore, D., Brown-Espaillat, K., & Closser, J.

(2019). Implementation of a pilot electronic stroke outcome reporting system for emergency

care providers. American Journal of Emergency Medicine, 1–4.

26. Shaban, W. M., Rabie, A. H., Saleh, A. I., & Abo-Elsoud, M. A. (2021). Accurate detection

of COVID-19 patients based on distance biased Naïve Bayes (DBNB) classiﬁcation strategy.

Pattern Recognition, 119, 1–15.

27. Tokalic, R., Vidak, M., Kaknjo, M. M., & Marusic, A. (2021). Antifragility of healthcare

systems in Croatia and Bosnia and Herzegovinia: Learning from man-made and natural crises.

The Lancet Regional Health-Europe, 9, 1–6.

28. WHO. (2019, October 5). Infectious diseases. Retrieved from www.who.int/topics/infectious_d

iseases/en/

29. Wesley, D. B., Schubel, L., Hsiao, C., Burn, S., Howe, J., Kellogg, K., Lincoln, A., Kim,

B., & Ratwani, R. (2019). A socio-technical systems approach to the use of health it for

patient reported outcomes: Patient and healthcare provider perspectives. Journal of Biomedical

Informatics, 1–8.

30. Weldring, T., & Smith, S. M. S. (2013). Patient-reported outcomes (PROs) and patient-reported

outcome measures (PROMs). Health Services Insights, 6, 61–68.

A Comprehensive Study of Explainable

Artiﬁcial Intelligence in Healthcare

Aryan Mohanty and Sushruta Mishra

Abstract The recent development of Artiﬁcial intelligence and Machine learning,

in general, has exhibited impressive results in a variety of ﬁelds, especially through

the introduction of deep learning (DL). Even though they show an extraordinary

performance in a substantial number of jobs and have tremendous potential. This

surge in performance is usually pulled off through the increase in model complexity,

giving rise to the black-box model and creating confusion about how they work

and, ultimately, how they make judgments. This uncertainty has made it difﬁcult for

machine-learning programs to be used in more sensitive but essential areas, such as

health care, where their beneﬁts can be enormous, Thus giving birth to the need for

Explainable AI. Explainable Artiﬁcial Intelligence (XAI) is a new machine-learning

research subject aiming at decoding how AI systems make black-box decisions. This

chapter focuses on the need for Explainable AI in the ﬁeld of healthcare and some

techniques like LIME, SHAP, PDPs, and a few others, through which complex models

can be explained. We will see the use of explainable methods by analyzing two case

studies. Through the use of this article, clinicians, theorists, and practitioners can get

a better insight into how these models work and can help to bring a high level of

accountability and transparency.

Keywords Deep learning ·Blackbox ·Explainable AI ·Healthcare ·Transparency

1 Introduction

In recent years, there has been an increase in the number of applications based on

artiﬁcial intelligence that has been used in a variety of ﬁelds. Starting from healthcare,

defense, law, governance, ﬁnance, etc. all have beneﬁted from AI-based algorithms.

In the present, Machine learning (ML) has emerged as one of the main sub-branch of

Artiﬁcial Intelligence, especially with the growing popularity of Deep Learning and

Neural Networks. As a relatively new method of machine learning, deep learning has

A. Mohanty ·S. Mishra (B)

Kalinga Institute of Industrial Technology, Bhubaneswar, India

e-mail: sushruta.mishrafcs@kiit.ac.in

S. Mishra et al. (eds.), Augmented Intelligence in Healthcare: A Pragmatic and Integrated

Analysis, Studies in Computational Intelligence 1024,

https://doi.org/10.1007/978-981-19- 1076-0_25

475

476 A. Mohanty and S. Mishra

revolutionized machine learning. Deep learning is not a single method but rather a

collection of techniques that build neural networks with several layers of information.

Contemporary computing technologies, like graphics processing units (GPUs), are

required for training since these deep neural networks are very sophisticated. Because

of such notable developments, Machine Learning has signiﬁcantly impacted both

research and industrial uses that its inﬂuence and possible results cannot be ignored

anymore. For example, even a momentarily malfunctioning of an algorithm in a

system can lead to accidents in some cases, and in a critical domain like healthcare,

where human lives are on the line, things can get sensitive if we are not careful

while using these algorithms. However, Medical tasks can be signiﬁcantly improved

using machine learning, artiﬁcial neural networks (ANNs), Deep learning, and other

subﬁelds. These methods can aid us in enhancing our medical functionality, such as

detecting a disease in its early stages, which can help people heal faster or prevent

the problem from worsening in the future [1–4].

Nevertheless, these technologies have yet to reach their full potential in this

regard. Still, it is possible to generate very reliable results using any one of the

deep learning technologies available today. Typically, these methods are usually

opaque, if not wholly undetectable, making it tough to comprehend their actions.

Even competent specialists may ﬁnd it hard to grasp the working of these models

fully. All these methods have a very complex functioning process that makes it difﬁ-

cult to explain how deep learning algorithms arrive at a speciﬁc answer, leading us

to the BlackBox problem. Nowadays, BlackBox machine-learning models are fast

being deployed with a tag of AI-enabled technology for numerous crucial ﬁelds of

human existence. These AI-powered models struggle to gain the trust of ordinary

people because they are less transparent and accountable. As a result, the explain-

ability of machine-learning algorithms has become a severe concern, leading to the

new area of AI research known as Explainable AI (XAI). XAI has attracted much

attention from vital areas such as healthcare, where understanding how an answer

was achieved is just as crucial as getting the answer. Because of this explainability

and interpretability property of XAI has become one of the hottest topics in the

machine-learning community.

2 Explainable Artiﬁcial Intelligence

Recently, a variety of explainable artiﬁcial intelligence (XAI) methods have been

created and tested as a way to address the opaqueness and black-box nature

of machine-learning methodologies. New terminology and ideas were created to

differentiate XAI approaches as a result of the introduction of newer methods.

The following section will brieﬂy explain essential words and ideas in order to

provide the reader with a broad overview of the area, who may be unfamiliar with

XAI. This section is not attempting to provide a full summary and instead points to

refer to current surveys.

A Comprehensive Study of Explainable Artiﬁcial Intelligence … 477

The notion of XAI has been around for a long time. However, it was not until

2017 that the Defense Advanced Research Projects Agency (DARPA) introduced

the term to the research community in response to the growing need for reliable and

transparent machine learning [5].

XAI initiatives, according to DARPA [6], are directed at two primary aims.

I. Maintain strong learning performance while producing more explainable

models; and

II. To increase public conﬁdence in learned models and create a more efﬁ-

cient collaboration between humans and artiﬁcial agents as a result of this

development.

There are three ways DARPA recommends in order to achieve the ﬁrst aim: Deep

explanation, interpretable models, and model induction.

Deep explanation is the process of integrating deep learning and certain addi-

tional techniques to build hybrid systems that can learn about more explainable

characteristics or representations, as well as capabilities for generating explanations.

Interpretable models are approaches that allow for the learning of more structured

representations or tracking of causal connections.

Model induction strategy encapsulates methods for inferring an approximate

explainable mode from any ML model, for example, a black box.

A multidisciplinary approach is needed to achieve the second aim, i.e., improving

the user experience, which requires a better grasp of psychological factors that deter-

mine whether or not an explanation is adequate, along with techniques for collecting

more explainable models.

As per researchers, the ability to recognize connections and contextual information

is a crucial foundation for explanations.

The term “explicability” describes the capability of a model’s attributes to be

transparent. The goal of understandability is taken a step further by “explainability,”

which indicates that the logic, model, or evidence for a ﬁnding can be communi-

cated in a way that a human can understand the context. Understanding and explain-

ability can both be viewed as required prerequisites for achieving interpretability.

Interpretability may be achieved at two levels: globally and locally [7].

Whereas the global method describes the model and its reasoning overall, local

methods explain speciﬁc judgments or predictions, regardless of the model’s inner

structure. The ultimate objective of such systems would be to achieve a machine-

learning model that is highly robust, which would assist people in improving their

performance in their tasks.

This chapter will talk about the potential of artiﬁcial intelligence (AI) in the

medical area and the necessity for XAI in healthcare. We will also review explain-

ability techniques of machine learning (ML) in healthcare, classify them with

examples, and showcase their uses with the help of case studies.

As a result, doctors and practitioners will better understand the usage of inter-

pretable algorithms available in various forms, which may also pave the way for

478 A. Mohanty and S. Mishra

specialized medical education to realize the potential of these technologies. From

this article, both technically minded and non-technically oriented readers may ﬁnd

helpful references for popular terminology that may pique their curiosity which

may develop an interest in this nascent academic topic. The proposed explainability

approaches may be valuable to clinicians and medical practitioners who are already

familiar with these words.

3 Potential of AI for Healthcare

Due to increased accessibility to healthcare data and rapid technological advance-

ments, artiﬁcial intelligence (AI) has the capability to revolutionize healthcare.

The effects of artiﬁcial intelligence (AI) have been hotly disputed in the medical

literature [8–10]. AI can be used to build high-level capabilities that can “study”

features from a wide range of health care data and utilize the learned knowledge

to aid practice in the clinic. To improve its correctness depending on input, AI can

be provided with learning and self-preparation skills. Making clinical decisions will

be supported by an AI gadget that displays journal articles with up-to-date medical

information, manuals, and professional practices to recommend good patient care

[11]. An AI system may also help eliminate medical and therapeutic errors, which

might be unavoidable in a human clinical trial [12,13]. Artiﬁcial Intelligence (AI) in

healthcare has made vital breakthroughs as AI has lately re-surfaced in scientiﬁc and

public awareness. New AI-powered technologies are being introduced into clinical

contexts at a rapid pace. Nonetheless, healthcare has been described as one of the

most intriguing AI application domains [14]. Researchers have developed numerous

clinical decision-making programs since the middle of the twentieth century [15,16].

From the 1970s, legal-based approaches have been used to understand ECGs [17],

Identify illnesses [18], select alternative therapies [19], provide scientiﬁc, logical

explanations [20], and assist physicians in formulating thoughts and ideas in difﬁcult

patient situations [21]. However, it is expensive to build rule-based systems, and

they might be unreliable, as their decision rules must be clearly expressed. On top

of all that, it is hard to illustrate interactions at a higher level between various bits

of erudition provided by various professionals since the efﬁcacy of structures is

hampered by medical information that is too broad in scope.

Unlike the ﬁrst generation of AI, which was more concerned with the collection

of expert medical data and the implementation of sturdy decision-making processes,

Recent AI research has used machine-learning methods, which can look at tricky

relationships to detect clinical patterns of previously unknown results. By analyzing

patterns extracted from all medical records, machine-learning algorithms learn to

produce the best input effect given in future situations [22]. Artiﬁcial intelligence

applications can be built using machine-learning techniques that investigate previ-

ously unknown data patterns without creating rules for making decisions for each

given work or accounting for complicated interactions between entry features. AI

tools have tended to be developed using machine-learning techniques [22,23]. AI

A Comprehensive Study of Explainable Artiﬁcial Intelligence … 479

has seen a resurgence since deep learning—which entails training artiﬁcial neural

networks with several layers (i.e., deep neural networks) upon large sets of data—has

been actively applied to major amounts of labeled data [24]. Current neural networks

are growing increasingly sophisticated, with several having more than 100 layers.

As a result of multi-layer deep neural networks, it is feasible to represent complex

input–output interactions; however, greater performance may need more data and

processing time or sophisticated architecture approaches. Today’s typical neural

networks contain tens of millions or billions of parameters, and training requires

considerable power to calculate. Thanks to recent advances in computer science,

in-depth learning now has the ability to use computers that require Deep-learning

algorithms, on the other hand, “very hungry data” in some cases [25]. The rapid

expansion of AI has opened the way for the use of collected health data to develop

robust algorithms that can make it easier to diagnose and allow for a more straight-

forward approach to health care by making the drugs themselves and implementing

better targeting solutions in a faster and more efﬁcient way. Even if artiﬁcial intel-

ligence (AI) can reform the medical ﬁeld, considerable hurdles need to be over-

come. It is essential to acquire data that represent the target patient group because

deep learning relies mainly on the obtainability of vast quantities of a high-quality

dataset. Due to differences in learning and distortion in data from multiple healthcare

settings, a model trained on the data from one facility may not generalize well. Still,

with the use of advanced deep algorithmic techniques that can deal with the unique

characteristics of distinct data sets. As a result, prediction can become more efﬁcient

and safe, which can save many lives.

4 Explainable Artiﬁcial Intelligence for Healthcare

Even though deep learning-based artiﬁcial intelligence technologies are set to change

healthcare, there will be obstacles. The rise of AI-based diagnosis systems has raised

severe concerns about the reliability of AI-based results. In the medical and health-

care section of the digital industry, trust is a critical aspect of AI’s continuation. There

are a lot of advanced deep learning models that give results that are incomprehensible

to untrained humans because they are so complex. Although these models can outper-

form humans in terms of efﬁciency, this gain in performance has been achieved by

increasing the complexity of the models, turning them into “black box” approaches. It

creates confusion in conveying logical interpretations that can validate model results,

how they work, and, ultimately, how they make judgments. Understanding what the

deep learning algorithm perceives in the clinical data can be challenging because

there are possibly millions of parameters in the model. This black-box design makes

it difﬁcult to validate the AI algorithms that have been developed. An effective deep-

learning model must be shown to identify the correct portion of the data without

over-emphasizing irrelevant discoveries. In the last several years, new approaches

have been created to deﬁne AI models, including imaging techniques. In addition to

480 A. Mohanty and S. Mishra

occlusion maps [26] and salience maps [27]—which are all extensively utilized—

class activation maps [28] and attention maps [29] are also commonly employed

levers. When the result is a picture, localization and segmentation may be easily

understood. Understanding deep neural network models that have been trained using

non-imaging data is far more complex as there has been a burst of curiosity in artiﬁcial

intelligence (AI) based on deep learning in the medical industry. Advanced learning

approaches have been applied in a wide variety of health care systems, involving

diagnosis, prognosis, treatment planning, patient management, and other complex

operations.

Theoretically, interpretability in medicine, on the other hand, has not been

explored to its full potential yet. It is also considered that there are several factors

such as risk and responsibilities in the clinical sector that cannot be considered in

other ﬁelds. Human life could be put in danger if medical choices are left to AI

algorithms that lack transparency and accountability, which would be irresponsible

[30]. Several researchers have used XAI to make their predictive models more inter-

pretable. By reﬁning and optimizing less complex AI models while improving their

efﬁcacy, these techniques focus on preserving the interpretability of less sophisti-

cated AI models. However, model optimization is not always straightforward and is

not usually a simple operation. In this chapter, we categorized XAI in medicine and

healthcare with the help of six techniques, which include (1) LIME, (2) SHAP, (3)

PDPs, (4) ICE, (5) ALE, and (6) Permutation Feature Importance.

5 Some Methods for Achieving Explainability

Let us have a look at some fascinating explainability techniques that can help us

understand the foundation on which a model makes a prediction, making it more

transparent and dependable.

5.1 Local Interpretable Model-Agnostic Explanations

(LIME)

LIME method is a method for ﬁtting local, interpretable models for explaining a

single prediction in any black-box machine-learning model. The main idea of this

method is that it is simpler to estimate a black-box model by using a basic model

[31] (e.g., a linear model) locally (in the vicinity of the single prediction that has to

be explained) instead of attempting a global approximation of a model.

In simpler term LIME examines what happens to the predictions when a machine-

learning model is fed with a skewed set of data. LIME generates a brand-new dataset

from the altered samples and the black-box model’s predictions. LIME then runs

an interpretable model on the new set of data, measured by the distance between

A Comprehensive Study of Explainable Artiﬁcial Intelligence … 481

the sampled occurrences and the instance of interest. However, the learned model

does not have to be an excellent global approximation of machine-learning model

predictions. Local ﬁdelity is another term for this degree of precision.

Follow these steps to take advantage of the LIME algorithm [32]:

•Simply choose an instance where you would want an explanation of the black-box

forecast. Choose an instance that piques attention.

•Consider making some modiﬁcations to your dataset and seeing what the black

box says.

•Samples are given a relative weight based on their proximity to the target instance.

•Use the dataset and variations to build an interpretable weighted model.

•If we examine the regional model, we may be able to provide an explanation for

the forecast.

Figure 1shows the LIME results of the top ten biomarker markers using Random

Forest classiﬁer for normal breast cancer, primary breast cancer, and metastatic breast

cancer. (A) Interpretation of LIME for normal breast cancer. The negative event is

described in blue, while the positive example is described in orange. The ﬁrst column

shows the possibilities for guessing the negative and positive results obtained from

classiﬁers. Feature contributions in probability are shown in the second column. The

Fig. 1 The LIME predictions on the RF model for the categorization of normal, primary, and

metastatic breast cancer samples [33]

482 A. Mohanty and S. Mishra

original data values are shown in the third column. (B) Interpretation of LIME for

primary breast cancer. (C) Interpretation of LIME for metastatic breast cancer.

5.1.1 Advantages

•LIME is one of the few methods capable of working with both tabular and graphic

data simultaneously.

•LIME is an easy-to-use programming language that is accessible in Python (lime

library) and R (lime package and iml package).

•Local Surrogate models trained locally can offer explanations with more (inter-

pretable) characteristics than the original model. Data instances are, of course,

required to provide these interpretable characteristics.

5.1.2 Disadvantages

•The main disadvantage of the LIME method is that it cannot fully explain the

decision-making process of a black-box model.

•Another problem is its Instability; Instability here indicates that it is hard to believe

the explanations. In a simulated situation, the explanations for two extremely near

locations varied considerably.

This is the main reason why we believe it should be used with caution in the

ﬁeld of healthcare, as in many cases, a clinician is legally required to explain a

prediction fully.

5.2 Shapley Additive Explanations (SHAP)

Before we go to SHAP, let us ﬁrst look at what Shapley Values are

Cooperative Game Theory is the source of Shapley Values approach. It describes

how to share the full beneﬁts of the game equally among the participants according

to their individual contributions, assuming that everyone is cooperative. Assume that

each element x(j)of a given example x(i)participates in a game in which prediction

is beneﬁcial and expands the theory to explain good predictions.

Which triggers a local approach to show us how much each element contributes to

the example x(i)predictions related to the average forecast. This can be achieved by

examining the median contribution of the feature to the prediction of a single piece

of data, where the presented order of the coherence of the symbols is important.

The purpose of SHAP is to calculate the contribution of each element to a condi-

tional forecast xto deﬁne it. Shapley values are calculated using the SHAP deﬁnition

process, which is built on the idea of a cohesive game. Data model feature values

serve as component members. Shapley prices give us a way to spread the “pay-

out” (=prediction) between features on the fairway. For example, pixels can be

A Comprehensive Study of Explainable Artiﬁcial Intelligence … 483

clumped into larger pixels, and predictions are spread out to interpret the image. The

Shapley Value description is presented as an additional process of the input element,

a straightforward model, which is one of the new features that SHAP brings to the

table.

5.2.1 Advantages

•Game theory provides a solid theoretical framework for SHAP’s operations

research and development efforts.

•SHAP ties together the Lime and Shapley data using a mathematical formula. For

a deeper understanding of both approaches, this information is quite beneﬁcial. It

also aids in the consolidation of interpretable machine learning.

•For tree-based models, SHAP offers a fast implementation. This was crucial to

SHAP’s success because the slow computation is the most signiﬁcant obstacle to

Shapley value acceptance.

5.2.2 Disadvantages

•Its primary disadvantage is that the computation time of calculating Shapley values

is very high; however, the SHAP framework provides different optimizations to

calculate the Shapley values in real-world scenarios efﬁciently.

•SHAP can be used to provide purposefully deceptive readings that mask biases.

Figure 2shows the importance of SHAP features as mean absolute Shapley values.

The essential component was the number of years on hormonal contraception, which

changed the excepted absolute cancer risk by 2.4 percentage points on average (0.024

on the x-axis).

Although the feature importance plot is helpful, it does not provide any additional

information. Next, we will look at the summary plot to see if it can provide us with

more information.

Figure 3shows that a couple of years on hormonal contraception is directly

proportional to the predicted cancer risk.

The link between the value of a feature and its effect on the prediction can be seen

in the summary plot. However, to fully comprehend the fundamental nature of the

link, we must examine SHAP-dependent graphs.

Figure 4illustrates that compared to 0 years, a reduction in the cancer likelihood

is found in the few years, whereas in the case of a large number of years, there is

a rise in the predicted cancer probability. In the event of interactions, the SHAP

dependence plot will be substantially more spread along the y-axis. To enhance the

dependent plot, these feature interactions might be highlighted.

484 A. Mohanty and S. Mishra

Fig. 2 The signiﬁcance of SHAP features importance in predicting cervical cancer using a trained

random forest [31]

Fig. 3 SHAP summary plot [31]

A Comprehensive Study of Explainable Artiﬁcial Intelligence … 485

Fig. 4 SHAP features dependence characterized by the number of years of reliance on hormonal

contraception [31]

5.3 Partial Dependence Plots (PDPs)

PDPs are a type of visualization that aids in interpreting any kind of black-box

prediction model by demonstrating how certain characteristics or groups of features

affect the model’s predictions. PDPs, in particular, highlight how a collection of

characteristics inﬂuences the mean projected value by leaving the remainder of the

features (its supplementary feature set) out [34].

Figure 5shows, in terms of age, the PDP suggests that the risk is minimal until

the age of 40, after which it increases. The longer you take hormonal contraception,

the greater your chance of developing cancer, especially after ten years. Because

there were few high-value data points for both parameters, the partial dependence

approximate in some places is less dependable.

We can also see how two traits are somewhat interdependent at the same time.

The plot in Fig. 6depicts the rise in cancer likelihood at 45. Women under the age

of 25 who had one or two pregnancies had a decreased predicted cancer risk than

women who had no pregnancies or more than two pregnancies. But proceed with

caution when concluding: this might be a correlation rather than a causative link!

486 A. Mohanty and S. Mishra

Fig. 5 PDPs of cancer risk depending on age and years of use of hormonal contraception [31]

Fig. 6 PDP on the possibility of cancer risk and the interplay between age and number of

pregnancies [31]

A Comprehensive Study of Explainable Artiﬁcial Intelligence … 487

5.3.1 Advantages

•Partially dependent charts are easy to compute. All data points in the model are

assumed to have the same feature value. Thus the partial dependency function

reﬂects the average forecast.

•Partial dependency charts are simple to create.

•The partial dependency charts calculation has a logical meaning. We make modi-

ﬁcations to a feature and watch how the predictions change as a result. As a result,

we can examine the causal link between the characters and the prediction.

5.3.2 Disadvantages

•Some PD charts do not display the distribution of features. Leaving out the distri-

bution might lead to erroneous conclusions because regions with little or no data

may be misinterpreted.

•The most signiﬁcant problem with PD plots is the assumption of independence.

All other features are considered to be uncorrelated with those for which the

partial dependency is being calculated (solved by ALE).

•Because PD plots only display average marginal effects, heterogeneous effects

may be missed (solved by ICE).

5.4 Individual Conditional Expectation (ICE)

Interpretability is improved by using ICE plots, a model-independent interpretability

method that improves the PDP notion. With an ICE plot, you can see how the predic-

tion changes concerning a particular characteristic for each instance, but you only

see one line with a partial dependence plot. The average of the ICE plot’s lines is

what we call a PDP. The values for a line (and one instance) may be measured by

holding all other traits constant, producing variations of this instance by changing

the feature’s value with values from a grid, and making predictions for these recently

produced instances using the black-box model. The end outcome is a network of

points with the feature value from the grid and the corresponding predictions.

What are the beneﬁts of focusing on individual expectations rather than partial

dependencies? Plots that show just partial dependency may hide a complex rela-

tionship resulting from several different factors interacting. An average association

between a feature and its prediction may be seen using PDPs [34]. This works only

if the interactions between the PDP-calculated qualities and the other attributes are

weak. If interactions exist, the ICE plot will reveal a wealth of information.

In Fig. 7, each line represents each woman. For most women, the likelihood of

developing cancer increases as they become older. For some women with a cancer

risk of more than 0.4, the risk does not alter signiﬁcantly as they get older.

488 A. Mohanty and S. Mishra

Fig. 7 Cervical cancer risk by age is depicted using an ICE plot [31]

There are difﬁculties with ICE sites that it may be challenging to determine if

ICE curves are different for everyone as they start with different predictions. A

straightforward option is to place curves at a certain point in the feature and then

simply display the difference between the forecast and the actual value. The resulting

ICE structure is known as the centered ICE plot (c-ICE).

In Fig. 8, at the age of 14, lines are set to zero. Until the age of 45, when the

projected likelihood increases, most women’s forecasts stay constant from when

they were 14 years old.

5.4.1 Advantages

•ICE curves are more accessible to grasp than partial dependency graphs. If the

feature of interest is changed, the predictions will be represented by a single line.

•ICE curves, as opposed to partial dependency graphs, can reveal connections with

heterogeneity.

5.4.2 Disadvantages

•Two characteristics would need the creation of several overlapping surfaces, which

ICE curves cannot do.

A Comprehensive Study of Explainable Artiﬁcial Intelligence … 489

Fig. 8 Predicted cancer probability by age in a centered ICE plot [31]

•PDPs and ICE curves have an identical drawback: If the feature of interest is

linked to the other features, speciﬁc spots in the lines may be incorrect data points

based on the combined feature distribution.

•If multiple ICE curves are plotted, the plot will get cluttered, and nothing will be

visible.

5.5 Accumulated Local Effects (ALE)

The ALE structures include conditional ones instead of a few distributions, trying

to overcome the biggest mistake of the PDPs, their independent thinking between

the elements. Instead of measuring the predictive value, ALE sites incorporate the

difference between the prediction to be more predictable than other factors, thus

minimizing the impact of the corresponding data [34].

Using a random forest model to predict the likelihood of cervical cancer depending

on risk variables. As a result, we can see the local impacts that have accumulated for

two of the features:

The age feature of the ALE plot in Fig. 9shows that the projected cancer likelihood

is low on average until age 40, then increases. The number of years on contraceptives

after eight years is linked to higher anticipated cancer risk.

490 A. Mohanty and S. Mishra

Fig. 9 ALE graphs show the inﬂuence of age and years of hormonal contraception on the anticipated

risk of cervical cancer [31]

5.5.1 Advantages

•Because ALE sites are not selective, they can be used with compatible features.

In this case, the partial dependence graphs fail because they exclude the possible

or perhaps improbable physical combination of feature value values.

•PDPs take longer to calculate and measure with O(n), but ALE charts are faster

and measured with O(n).

•ALE sites have a speciﬁc meaning: Depending on the value, the ALE structure

shows the related effect of changing a feature in the forecast. ALE data sites have

a zero-focused axis.

5.5.2 Disadvantages

•With many intervals, ALE sites may have less stability (with more ups and downs).

Decreasing the number of intervals makes predictions ﬁrmer and smoother, and

marks are part of the complexity of the prediction model. Setting the number of

intervals is a challenge because there is no suitable solution.

•Compared to partially learning graphs, ALE sites are much harder to create and

understand.

•Even though ALE plots are not biased when features are linked, interpretation is

challenging when features are highly correlated.

A Comprehensive Study of Explainable Artiﬁcial Intelligence … 491

5.6 Permutation Feature Importance

As a result of the PFI (Permutation Feature Signiﬁcance) technique, we can estimate

the relative importance of different features by examining how much their permuta-

tion affects our prediction error. First, the dataset is divided in half, with 80% going

to training and 20% going to testing. Stratiﬁed tenfold cross-validation is used to

determine the optimum hyperparameters using the training set. Next, we compute

the baseline accuracy as the average prediction score in the provided test set using

the trained model with all features. After that, the values of different features are

repeatedly mixed on the test set, and the average prediction score of the previously

trained model on the updated dataset is computed. As a ﬁnal step, the signiﬁcance

of each characteristic is assessed by comparing its score to the accuracy threshold.

5.6.1 Advantages

•If the feature details are missing, the model error increases.

•Model behavior can be traced to the importance of its feature in a highly pressured

and global way.

•Using error rate rather than error variation has the advantage of making the feature

value test relevant to all problems.

•All interactions with other factors are automatically considered in value metrics

when calculating the value of a feature. You remove the effects of merging with

other features when you enable the feature.

•There is no need to re-teach the model when considering the value of the

permission feature.

5.6.2 Disadvantages

•When determining the feature signiﬁcance, it is not obvious if one should utilize

training or testing data.

•The model’s inaccuracy is correlated with the permutation feature’s signiﬁcance.

This is not always a negative thing, but it is not always what you want.

•Whether training or testing data should be used to assess the feature’s importance

is debatable.

•Permutation feature importance is inversely proportional to the model’s accuracy.

Even though this is not always a bad thing, it is something to keep in mind.

•By dividing the signiﬁcance between the two characteristics, adding a linked

feature might reduce the importance of the associated feature.

To predict cervical cancer, we used a random forest model. 1-AUC is used to

calculate error increases (1 subtract the area below the ROC curve). Factors related

to an increase in single-error error (=no change) had little predictive value for

cervical cancer.

492 A. Mohanty and S. Mishra

Fig. 10 The signiﬁcance of each characteristic in anticipating cervical cancer using a random forest

[31]

In Fig. 10, the most important feature was Hormonal.Contraceptives..years..

Permuting Hormonal.Contraceptives..years. resulted in an increase in 1-AUC by

a factor of 6.13.

Hormonal.Contraceptives..years. was an essential characteristic. After permuta-

tion, it was linked to an increase in the inaccuracy of 6.13.

6 Case Study Analysis of Explainable AI

In this section, we are going to analyze two cases from two papers. One case from

An Explainable Machine-Learning Model for Early Detection of Parkinson’s Disease

using LIME on DaTSCAN Imagery [35] and another one from Lung cancer survival

period prediction and understanding: Deep learning approaches [36].

We will not dive into the prediction results proposed in the papers, which use

different machine-learning approaches to get the result. This article is not aiming

to present a comprehensive summary of these papers. Those who wish to know the

details of the result may go through the original paper mentioned above.

A Comprehensive Study of Explainable Artiﬁcial Intelligence … 493

6.1 Analysis-1

Let us see how the LIME method helped explain a model’s prediction by using paper

[35] as reference. However, before analyzing, here are some information related to

the paper for better intuition.

Parkinson’s disease (PD) is a type of neurological illness that inﬂuences the

brain and nervous system. Most of the time, it affects dopaminergic neurons in the

substantia nigra, a portion of the brain that produces dopamine [37]. There are various

medical scans available, including MRI, fMRI, PET, and others. However, the Single-

photon Emission Computed Tomography (SPECT) method is probably widely used

in hospitals for the early detection of Parkinson’s disease [38]. A substance called

Ioﬂupane (123I) often referred to as FP-CIT, is used in SPECT imaging. This ligand

has a high afﬁnity and high efﬁciency for the dopamine transporters in the putamen

and caudate areas of the brain (hence the name DaTSCAN) [39]. Patients with

Parkinson’s disease have a smaller, more rounder striatum on scans. As shown in

Fig. 11, scans from healthy control participants maintain the C-bright shape illumi-

nation. So Parkinson’s disease diagnosis based on neuro-images may be suitable since

symptomatic therapy may be applied too late and not be a time-conscious option.

Additionally, SPECT scan pictures are manually evaluated in hospitals, putting the

diagnosis at risk of human mistakes. Nowadays, Deep learning has been routinely

utilized to diagnose a wide range of medical problems, with results often exceeding

industry standards [40].

We can rapidly and correctly categorize individuals as to whether or not a patient

has Parkinson’s disease (PD) by recognizing patterns in their SPECT scans using

deep learning, particularly around the putamen and caudate areas, which are smaller

than non-PD samples.

This section will look at the explainability solution provided as per the paper,

which uses the LIME (Local Interpretable Model-Agnostic Explanation) method

Fig. 11 SPECT DaTSCAN with putamen and caudate areas indicated by strong contrast [35]

494 A. Mohanty and S. Mishra

that helped get the binary classiﬁcation result (PD or non-PD) of the developed

black-box neural network. This aims to provide an insight to the medical experts

as to why the machine thinks in such a manner, providing a crucial view for the

decision-making process.

6.1.1 Explainability with LIME

LIME examines what happens to the model’s predictions when the user changes the

data instances in the model. LIME twists the feature values in a single data sample

and then examines the impact on the output as a consequence. In this way, LIME

generates a new set of data comprising permuted samples and their accompanying

black-box model predictions.

6.1.2 Interpretation of DaTSCANs

The putamen and caudate zones of the brain are the focus of our research. As a result,

the superpixels associated with these regions are identiﬁed by the LIME explanation

instance as the parts of the picture having the most important weights or impacts

on the prediction’s outcome. Since the ROI (region of interest) can be seen on the

samples, non-experts in the ﬁeld may more easily identify the patient’s diagnosis

when using LIME (Fig. 12).

The LIME explanation highlighted the healthy putamen and caudate areas of a

non-PD patient as the inﬂuential area in categorizing the data as healthy control, as

seen in the two instances from Fig. 13. The SPECT scans are shown in Fig. 13a,

c after preprocessing, and the equivalent output after applying LIME is shown in

Fig. 13b, d.

LIME said that the aberrant or decreased characteristics of a non-PD patient’s

putamen and caudate areas were the inﬂuential area in categorizing the data as PD

(as seen in the two cases from Fig. 14). Figure 14a, c show the SPECT scans (after

preprocessing), whereas Fig. 14b, d show the equivalent result after using LIME. We

Fig. 12 Samples PD classiﬁed interpretations [35]

A Comprehensive Study of Explainable Artiﬁcial Intelligence … 495

Fig. 13 Healthy control [35]

can notice that the emphasized superpixels in Fig. 14c, d are more distorted in the

explanations.

This might be due to an unusually high level of dopamine activity in the ROI’s

vicinity, which is a sign of the latest-onset PD. Smaller ROIs probably encour-

aged the model to learn PD-related particularities other than putamen and caudate

regions, which is why we see a heterogeneous superpixel distribution across all such

deﬁnitions of PD categories.

6.1.3 Summary of Analysis-1

From the above explanation, we can see how LIME helped explain the model used.

As a result of the vast and diversiﬁed sample size, we were able to justify the model

with high reliability. Health care personnel will save time and money by utilizing

this method. It will assist in the early detection of Parkinson’s disease by providing

clariﬁcations and building trust in the medical community in the use of computer

diagnostics.

496 A. Mohanty and S. Mishra

Fig. 14 Parkinson’s disease [35]

6.2 Analysis-2

In this analysis, we are not going to look at what results were given by deep learning

methods; rather, we will focus on the features on which these models depended to

obtain the outcome.

This work makes use of deep learning to accurately forecast the precise survival

duration while also increasing the model’s interpretability by investigating an eval-

uation of feature importance, highlighting the characteristics that have the most

signiﬁcant bearing on projections based on a time-series survival model [36].

We will analyze the value of the tested feature that will assist in investigating the

interpretation of the model, gaining more understanding of survival analysis models

and factors in the prediction of cancer time.

6.2.1 Feature Importance

The explainability of Machine-Learning models has risen to the top of the priority list

due to the rising worry over bias in AI systems. Explainability acquires paramount

A Comprehensive Study of Explainable Artiﬁcial Intelligence … 497

signiﬁcance in the healthcare area since healthcare practitioners must test, assess,

and often approve machine-learning models for use.

When looking at the model early on, it is helpful to know which features inﬂu-

ence the turnout most. This assessment gives model explainability by analyzing the

patterns the model discovered in the data. Inclusion, neutrality, and inspection would

all improve as a result of this. Therefore, all attempts to develop health machine-

learning models should include the importance of the feature as part of the process

of evaluating the performance of the model.

We evaluated feature importance for the identiﬁcation of the characteristics that

had the most inﬂuence on the deep learning models for this study in order to make

them interpretable. For this, we see Shapley Addictive exPlanations (SHAP) [41]

values at the local prediction level and Partial Dependence Plots (PDP) [42]atthe

global dataset level to examine the impact of each feature on the set of predictions

generated by the training set.

As previously stated, the interpretability of the deep learning models has been

given by the evaluation of feature signiﬁcance both at the local prediction level

and at the global level. Figure 15 shows the feature relevance at a local level for a

sample of predictions, whereas Fig. 16 shows the global-level effect of a few key

characteristics.

Locally, we know how each feature affects the forecast’s output thus, we can pick

the most critical characteristics for this speciﬁc prediction. On a global scale, we

look at how each characteristic impacts the ﬁnal output prediction and assess the

ﬁnal output as a function of that feature.

Fig. 15 SHAP explanation diagrams at the local level

498 A. Mohanty and S. Mishra

Fig. 16 The values of features are shown against global-level explanations of feature importance

(PDPs)

6.2.2 Use of SHAP

Using SHAP values, features that have a substantial inﬂuence on the sample of

predicted outcomes were obtained. Applying SHAP, the output results will show us

how important each feature is in terms of how it affects the ﬁnal result. The most

important features have a base value of 100%, while all have values that indicate

their suitability as a proportion of the starting point.

6.2.3 Use of PDPs

PDPs help us to see alterations in the ﬁnal result due to modiﬁcations in the value

of features. In addition, the distribution of features is designed to differentiate the

A Comprehensive Study of Explainable Artiﬁcial Intelligence … 499

apparent impact of output due to distribution inequality. This plot helps determine

whether the visual effect is signiﬁcant. The partial dependence structure works on the

basic assumption of the binary taxonomy problem, and a one-versus-all approach

is used to adapt the multiclass problem into a multiclass taxonomy problem that

transforms it into three independent taxonomy problems. Slightly dependent sites

are created independently in each of the three cases.

In Fig. 15 [36]: A typical set of SHAP local (prediction) level explanation graphs

quantifying the impact of each attribute on the creation of that speciﬁc prediction.

The top 15 characteristics that impact that speciﬁc forecast are displayed in the

SHAP explanation graphics. Groups “=6 months,” “0.5–2 years,” and “> 2 years”

are designated with the symbols “0.00,” “1.00,” and “2.00,” respectively. In the above

and below graph, the Y-axis is a measure of the response and the number of cases. At

the same time, X-axis represents feature importance, with a base value of 1 (100%),

and each element has its value ratio in the local level deﬁnition.

From Fig. 16 [36]: We can realize the inﬂuence of feature value on the response

variable; global-level explanations of feature importance are plotted against the

values of the feature. Feature values are also introduced to determine the magnitude

of that range in the database. Response variables were divided into three hypothetical

sections, with low survival intervals, intermediate, and upper extremities representing

the “=6 months,” “0.5–2 years,” and “2 years” stages of the classiﬁcation problem.

This is done to give a more objective approach to global feature importance for the

model.

7 Conclusion

Explainable AI is a critical tool for medical professionals to understand better and

evaluate AI-created solutions in the healthcare industry. It also presents other beneﬁts,

such as improved reliance on solutions used by medical professionals and increased

exposure to how the keys work.

In this chapter, we discussed the potential of artiﬁcial intelligence in the healthcare

ﬁeld, how AI has improved the practices used in the medical domain, and the scope

it has with the rise of deep learning and neural networks. The rapid growth of AI

has made it possible to use the acquired health data to create robust algorithms

that can simplify diagnoses and allow for a more objective approach to healthcare

by customizing drugs and targeting solutions for optimal performance in a timely

and effective manner. Then we discuss the role of Explainable AI in the medical,

as these methods like neural networks get quite complicated for everyday people

to understand, like how they get to the results and the means these methods use to

result. Then we look at some techniques through which we can achieve explainability,

we look the way how these techniques work, their advantage and some disadvantage

while using these, and we went through examples of the methods related to healthcare.

Next, we use two cases where these explainable techniques were used to get insights

into how the model used gets their results based on what factor. We have gone through

500 A. Mohanty and S. Mishra

two multi-factor-based strategies, LIME and SHAP, which have similar objectives but

very different approaches. Unlike LIME, which can only provide local deﬁnitions,

SHAP can provide global and local deﬁnitions. Many types of databases that deﬁne

a global database are constructed using a database, providing information about the

interplay between features and their compatibility.

We conclude that omitting explainability in clinical decision support systems

jeopardizes essential medical ethical norms and may have negative consequences for

both individual and public health.

The goal is to educate staff on the understanding and interpretation of AI descrip-

tive systems through various methods that can greatly help the health care envi-

ronment. The medical diagnostic model holds a person’s life, and we should be

certain enough to treat the patient as briefed by the black-box model but with better

understanding.

References

1. Kelly, C. J., Karthikesalingam, A., Suleyman, M., Corrado, G., & King, D. (2019). Key

challenges for delivering clinical impact with artiﬁcial intelligence. BMC Medicine, 17(1),

1–9.

2. Tripathy, H. K., Mallick, P. K., & Mishra, S. (2021). Application and evaluation of classiﬁcation

model to detect autistic spectrum disorders in children. International Journal of Computer

Applications in Technology, 65(4), 368–377.

3. Chen, L., Bentley, P., & Rueckert, D. (2017). Fully automatic acute ischemic lesion

segmentation in DWI using convolutional neural networks. NeuroImage: Clinical, 15,633–643.

4. Mishra, S., Dash, A., Ranjan, P.,& Jena, A. K. (2021). Enhancing heart disorders prediction with

attribute optimization. In Advances in Electronics, Communication and Computing (pp. 139–

145). Springer Singapore.

5. Adadi, A., & Berrada, M. (2018). Peeking inside the black-box: A survey on explainable

artiﬁcial intelligence (XAI). IEEE Access, 6, 52138–52160.

6. Gunning, D., & Aha, D. (2019). DARPA’s explainable artiﬁcial intelligence (XAI) program.

AI Magazine, 40(2), 44–58.

7. Schwalbe, G., & Finzel, B. (2021). XAI method properties: A (meta-) study. arXiv preprint

arXiv:2105.07190

8. Dilsizian, S. E., & Siegel, E. L. (2014). Artiﬁcial intelligence in medicine and cardiac imaging:

Harnessing big data and advanced computing to provide personalized medical diagnosis and

treatment. Current Cardiology Reports, 16(1), 441.

9. Patel, V. L., Shortliffe, E. H., Stefanelli, M., Szolovits, P., Berthold, M. R., Bellazzi, R., &

Abu-Hanna, A. (2009). The coming of age of artiﬁcial intelligence in medicine. Artiﬁcial

Intelligence in Medicine, 46(1), 5–17.

10. Jha, S., & Topol, E. J. (2016). Adapting to artiﬁcial intelligence: Radiologists and pathologists

as information specialists. JAMA, 316(22), 2353–2354.

11. Strickland, E. (2019). IBM Watson, heal thyself: How IBM overpromised and underdelivered

on AI health care. IEEE Spectrum, 56(4), 24–31.

12. Weingart, N. S., Wilson, R. M., Gibberd, R. W., & Harrison, B. (2000). Epidemiology of

medical error. BMJ, 320(7237), 774–777.

13. Ker, J., Wang, L., Rao, J., & Lim, T. (2017). Deep learning applications in medical image

analysis. IEEE Access, 6, 9375–9389.

A Comprehensive Study of Explainable Artiﬁcial Intelligence … 501

14. Yang, G., Ye, Q., & Xia, J. (2021). Unbox the black-box for the medical explainable AI via

multi-modal and multi-centre data fusion: A mini-review, two showcases and beyond. arXiv

preprint arXiv:2102.01998

15. Miller, R. A. (1994). Medical diagnostic decision support systems—Past, present, and future:

A threaded bibliography and brief commentary. Journal of the American Medical Informatics

Association, 1(1), 8–27.

16. Musen, M. A., Middleton, B., & Greenes, R. A. (2021). Clinical decision-support systems. In

Biomedical informatics (pp. 795–840). Springer.

17. Kundu, M., Nasipuri, M., & Basu, D. K. (2000). Knowledge-based ECG interpretation: A

critical review. Pattern Recognition, 33(3), 351–373.

18. De Dombal, F. T., Leaper, D. J., Staniland, J. R., McCann, A. P., & Horrocks, J. C. (1972).

Computer-aided diagnosis of acute abdominal pain. British Medical Journal, 2(5804), 9–13.

19. Shortliffe, E. H., Davis, R., Axline, S. G., Buchanan, B. G., Green, C. C., & Cohen, S. N.

(1975). Computer-based consultations in clinical therapeutics: Explanation and rule acquisition

capabilities of the MYCIN system. Computers and Biomedical Research, 8(4), 303–320.

20. Barnett, G. O., Cimino, J. J., Hupp, J. A., & Hoffer, E. P. (1987). DXplain: An evolving

diagnostic decision-support system. JAMA, 258(1), 67–74.

21. Miller, R. A., McNeil, M. A., Challinor, S. M., Masarie, F. E., Jr., & Myers, J. D. (1986).

The INTERNIST-1/quick medical REFERENCE project—Status report. Western Journal of

Medicine, 145(6), 816.

22. Yu, K. H., & Snyder, M. (2016). Omics proﬁling in precision oncology. Molecular & Cellular

Proteomics, 15(8), 2525–2536.

23. Deo, R. C. (2015). Machine learning in medicine. Circulation, 132(20), 1920–1930.

24. Mishra, S., Mohapatra, S. K., Mishra, B. K., & Sahoo, S. (2018). Analysis of mobile cloud

computing: Architecture, applications, challenges, and future perspectives. In Applications of

security, mobile, analytic, and cloud (SMAC) technologies for effective information processing

and management (pp. 81–104). IGI Global.

25. Yu, K. H., Beam, A. L., & Kohane, I. S. (2018). Artiﬁcial intelligence in healthcare. Nature

Biomedical Engineering, 2(10), 719–731.

26. Zeiler, M. D., & Fergus, R. (2014, September). Visualizing and understanding convolutional

networks. In European Conference on Computer Vision (pp. 818–833). Springer.

27. Simonyan, K., Vedaldi, A., & Zisserman, A. (2014). Deep inside convolutional networks: Visu-

alising image classiﬁcation models and saliency maps. In Workshop at International Conference

on Learning Representations.

28. Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2017). Grad-

CAM: Visual explanations from deep networks via gradient-based localization. In Proceedings

of the IEEE International Conference on Computer Vision (pp. 618–626).

29. Zhang, Z., Xie, Y., Xing, F., McGough, M., & Yang, L. (2017). MDNet: A semantically and

visually interpretable medical image diagnosis network. In Proceedings of the IEEE Conference

on Computer Vision and Pattern Recognition (pp. 6428–6436).

30. Quinn, T. P., Jacobs, S., Senadeera, M., Le, V., & Coghlan, S. (2021). The three ghosts of

medical AI: Can the black-box present deliver? Artiﬁcial Intelligence in Medicine, 102158.

31. Mishra, S., Panda, A., & Tripathy, K. H. (2018). Implementation of re-sampling technique

to handle skewed data in tumor prediction. Journal of Advanced Research in Dynamical and

Control Systems, 10, 526–530.

32. Chen, H., Michalopoulos, G., Subendran, S., Yang, R., Quinn, R., Oliver, M., Butt, Z., & Wong,

A. (2019). Interpretability of ML models for health data—A case study.

33. Modhukur, V., Sharma, S., Mondal, M., Lawarde, A., Kask, K., Sharma, R., & Salumets, A.

(2021). Machine learning approaches to classify primary and metastatic cancers using tissue

of origin-based DNA methylation proﬁles. Cancers, 13(15), 3768.

34. Linardatos, P., Papastefanopoulos, V., & Kotsiantis, S. (2021). Explainable AI: A review of

machine learning interpretability methods. Entropy, 23(1), 18.

35. Magesh, P. R., Myloth, R. D., & Tom, R. J. (2020). An explainable machine learning model

for early detection of Parkinson’s disease using LIME on DaTSCAN imagery. Computers in

Biology and Medicine, 126, 104041.

502 A. Mohanty and S. Mishra

36. Doppalapudi, S., Qiu, R. G., & Badr, Y. (2021). Lung cancer survival period prediction and

understanding: Deep learning approaches. International Journal of Medical Informatics, 148,

104371.

37. Poewe, W., Seppi, K., Tanner, C. M., Halliday, G. M., Brundin, P., Volkmann, J., Schrag,

A.-E., & Lang, A. E. (2017). Parkinson disease. Nature Reviews Disease Primers, 3(1).

38. Booth, T. C., Nathan, M., Waldman, A. D., Quigley, A. M., Schapira, A. H., & Buscombe,

J. (2015). The role of functional dopamine-transporter SPECT imaging in Parkinsonian

syndromes, part 1. American Journal of Neuroradiology, 36(2), 229–235.

39. Tripathy, H. K., Mishra, S., Thakkar, H. K., & Rai, D. (2021). CARE: A collision-aware

mobile robot navigation in grid environment using improved breadth ﬁrst search. Computers &

Electrical Engineering, 94, 107327.

40. Lundervold, A. S., & Lundervold, A. (2019). An overview of deep learning in medical imaging

focusing on MRI. Zeitschrift für Medizinische Physik, 29(2), 102–127.

41. Lundberg, S. M., & Lee, S. I. (2017, December). A uniﬁed approach to interpreting model

predictions. In Proceedings of the 31st International Conference on Neural Information

Processing Systems (pp. 4768–4777).

42. Friedman, J. H. (2001). Greedy function approximation: A gradient boosting machine. Annals

of Statistics, 1189–1232.

Deep Learning Based Stroke Disease Prediction using Multi Modal Biosignals

Conference Paper

Dec 2023

Augmented Intelligence enabled Deep Neural Networking (AuDNN) Framework for Skin Cancer Classification and Prediction Using Multi-Dimensional datasets on Industrial IoT Standards

Article

Mar 2023
MICROPROCESS MICROSY

Smart COVID-shield: an IoT driven reliable and automated prototype model for COVID-19 symptoms tracking

Article

Full-text available

Jun 2022
COMPUTING

IoT technology is revolutionizing healthcare and is transforming it into more personalized healthcare. In the context of COVID-19 pandemic, IoT`s intervention can help to detect its spread. This research proposes an effective “Smart COVID-Shield” that is capable of automatically detecting prevalent symptoms like fever and coughing along with ensuring social distancing norms are properly followed. It comprises three modules which include Cough Detect Module (CDM) for dry cough detection, Temperature Detect module (TDM) for high-temperature monitoring, and Distance Compute Module (DCM) to track social distancing norm violator. The device comprises a combination of a lightweight fabric suspender worn around shoulders and a flexible belt wrapped around the waist. The suspender is equipped with a passive infrared (PIR) sensor and temperature sensor to monitor persistent coughing patterns and high body temperature and the ultra-sonic sensor verify 6 feet distance for tracking an individual's social distancing norms. The developed model is implemented in an aluminum factory to verify its effectiveness. Results obtained were promising and reliable when compared to conventional manual procedures. The model accurately reported when body temperature rises. It outperformed thermal gun as it accurately recorded a mean of only 4.65 candidates with higher body temperature as compared to 8.59% with the thermal gun. A significant reduction of 3.61% on social distance violators was observed. Besides this, the latency delay of 10.32 s was manageable with the participant count of over 800 which makes it scalable.

A Smart Cloud-Based Architecture for Predicting Infectious Diseases Using Key Weather Elements

Conference Paper

Full-text available

Dec 2021

Infectious diseases have been characterized as capable of spreading directly or indirectly from one person to another, thus affect human growth socially and economically. Efforts to mitigate its prevalence have been applied in different ways like data mining and artificial intelligent with some successes. However, elements of weather which are found in every part of the world is yet to be considered in an intelligent way to alert people of likely outbreak of the infectious diseases in their immediate environments. Therefore, the objective of this study is to design an architecture that predicts the likely occurrence of infectious diseases by considering some weather elements like individual temperature and relative humidity of their immediate environment. The issue was addressed through a quantitative research approach by designing a smart cloud-based architecture that works with weather elements to predict infectious diseases. The designed architecture was validated and the results showed its capacity to smartly predict likely occurrence of infectious diseases in an environment. Hence, the study echoes the impact of informatics in healthcare industry towards eradicating the outbreak of infectious diseases.

Machine Learning and Radiomic Features to Predict Overall Survival Time for Glioblastoma Patients

Article

Full-text available

Dec 2021

Glioblastoma is an aggressive brain tumor with a low survival rate. Understanding tumor behavior by predicting prognosis outcomes is a crucial factor in deciding a proper treatment plan. In this paper, an automatic overall survival time prediction system (OST) for glioblastoma patients is developed on the basis of radiomic features and machine learning (ML). This system is designed to predict prognosis outcomes by classifying a glioblastoma patient into one of three survival groups: short-term, mid-term, and long-term. To develop the prediction system, a medical dataset based on imaging information from magnetic resonance imaging (MRI) and non-imaging information is used. A novel radiomic feature extraction method is proposed and developed on the basis of volumetric and location information of brain tumor subregions extracted from MRI scans. This method is based on calculating the volumetric features from two brain sub-volumes obtained from the whole brain volume in MRI images using brain sectional planes (sagittal, coronal, and horizontal). Many experiments are conducted on the basis of various ML methods and combinations of feature extraction methods to develop the best OST system. In addition, the feature fusions of both radiomic and non-imaging features are examined to improve the accuracy of the prediction system. The best performance was achieved by the neural network and feature fusions.

COVID-19 vaccination coverage among hospital-based healthcare personnel reported through the Department of Health and Human Services Unified Hospital Data Surveillance System, United States, January 20, 2021-September 15, 2021

Article

Full-text available

Nov 2021
AM J INFECT CONTROL

To protect both patients and staff, healthcare personnel (HCP) were among the first groups in the United States recommended to receive the COVID-19 vaccine. We analyzed data reported to the U.S. Department of Health and Human Services (HHS) Unified Hospital Data Surveillance System on COVID-19 vaccination coverage among hospital-based HCP. After vaccine introduction in December 2020, COVID-19 vaccine coverage rose steadily through April 2021, but the rate of uptake has since slowed; as of September 15, 2021, among 3,357,348 HCP in 2,086 hospitals included in this analysis, 70.0% were fully vaccinated. Additional efforts are needed to improve COVID-19 vaccine coverage among HCP.

Antifragility of healthcare systems in Croatia and Bosnia and Herzegovina: Learning from man-made and natural crises

Article

Full-text available

Oct 2021

Acute crises, such as a war or a pandemic, are the ultimate tests for health care systems’ resilience (temporary response to stress with change and adaptation) and antifragility (permanent benefit from change in response to stress). In this Health Policy paper, we analyse and discuss how the healthcare systems of two European countries – Bosnia and Herzegovina and Croatia – adapted to war as a man-made disaster, and how they adapted to COVID-19 pandemic twenty-five years later. These countries experienced full scale wars in recent history, which significantly changed their political and healthcare systems. This experience prepared the countries for the response to the pandemic, which coincided with two earthquakes in Croatia. We argue that healthcare systems in Croatia and Bosnia and Herzegovina are not only resilient but antifragile, and that they benefited from stressors they were exposed to. The antifragility of the two systems were primarily based on human effort – the strength, adaptability and resilience of health care professionals. We will look at lessons from the wars that were applied to the pandemic and discuss newly recognized opportunities and improvements.

Analysis and Implementation of Artificial Bee Colony Optimization in Constrained Optimization Problems

Chapter

Jan 2018

The growing complexity of real-world problems has motivated computer scientists to search for efficient problem-solving methods. Evolutionary computation and swarm intelligence meta-heuristics are outstanding examples that nature has been an unending source of inspiration. The behaviour of bees, bacteria, glow-worms, fireflies, slime moulds, cockroaches, mosquitoes and other organisms have inspired swarm intelligence researchers to devise new optimisation algorithms. Swarm Intelligence appears in biological swarms of certain insect species. It gives rise to complex and often intelligent behavior through complex interaction of thousands of autonomous swarm members. In this chapter, the ABC algorithm has been extended for solving constrained optimization problems and applied to a set of constrained problems.

Evolution of cloud computing and related security concerns

Conference Paper

Mar 2016

The Impact of First and Second Wave of the COVID-19 Pandemic: Comparative Analysis to Design Control Measures to Cope with Negative Effects of Future Infectious Diseases in Society

Article

Jan 2021

Mario Coccia

A Deep Neural Network Model for Effective Diagnosis of Melanoma Disorder

Chapter

Jul 2021

Skin cancer is one of the most critical diseases, and melanoma is the most widely observed skin cancer occurring in major population. Hence, early detection and treatment of melanoma disease are very important. This research study presents the classification of melanoma images using deep neural network. Principal component analysis (PCA) was used as a feature extraction algorithm in this study. Wavelet transform is also applied to remove noise and inconsistencies in the dataset. Performance was compared with various other classifiers like SVM, RBS, random forest and Naive Bayes. Deep neural network gave an optimal performance with performance indicators. Classification accuracy, precision, recall and F-score value recorded with deep neural network are 98.4%, 97.8%, 98.5% and 98.1%, respectively. Hence, our proposed work can be an effective classification framework in categorization and diagnosis of melanoma disease.

Twitter Sentiment Analysis using Machine Learning Techniques

Article

Feb 2020

Nowadays people share their views and opinions in twitter and other social media platforms, the way of recognizing sentiments and speculation in tweets is Twitter Sentiment Analysis. Determining the contradiction or sentiment of the tweets and then listing them into positive, negative and neutral tweets is the main classifying step in this process. The issue related to sentiment analysis is the naming of the correct congruous sentiment classifier algorithm to list the tweets. The foundation classifier techniques like Logistic regression, Naive Bayes classifier, Random Forest and SVMs are normally used. In this paper, the Naïve Bayes classifier and Logistic Regression has been used to perform sentiment analysis and classify based on the better accuracy of catagorizing Technique. The outcome shows that Naive Bayes classifier works better for this approach. Data pre-processing and feature extraction is realized as a portion of task.

Augmented Intelligence: Deep Learning Models for Healthcare

Abstract

Recommended publications

Artificial Intelligence in Healthcare: Algorithms and Decision Support Systems

A Comprehensive Study of Explainable Artificial Intelligence in Healthcare

Machine Learning Techniques in Medical Image: A Short Review

Relevance of Blockchain in Revolutionizing Health Records