ChapterPDF Available

Twenty Years Survey of Big Data: Definition, Concepts, and Applications in Engineering

Authors:
Massaki de Oliveira Igarashi at Universidade Presbiteriana Mackenzie
Massaki de Oliveira Igarashi
Paulo E. Sartorelli
Paulo E. Sartorelli
Paulo E. Sartorelli
  • This person is not on ResearchGate, or hasn't claimed this research yet.
Mariana Zuliani at Universidade Presbiteriana Mackenzie
Mariana Zuliani
Download full-text PDF
Read full-text
Download citation

Abstract

In the last decade, there was an exponential growth in data generation from different sources especially due to advances in information and communication technology. Thus, organizations have seen the potential to gain competitive edges from the analyses of this data, changing it in the information that, without Big Data tools, could not be obtained. In this context, this work brings a survey about Big Data and explains this concept has changed during the years. Moreover, this paper aims to elucidate the last twenty years of Big Data and its applications in different areas of engineering: civil, electrical, manufacturing, mechanical, materials, chemical, and software engineering.
Content uploaded by Massaki de Oliveira Igarashi
Author content
All content in this area was uploaded by Massaki de Oliveira Igarashi on Mar 10, 2021
Content may be subject to copyright.
86
396
249
458
228
288
826
0
200
400
600
800
1000
Civil
Engineering
Electrical
Engineering
Produc on
Engineering
Materials
Engineering
Mechanical
Engineering
Chemical
Engineering
So ware
Engineering
Number of Publica ons (x1000)/ Engineering Area
... The importance of data analytics in influencing decisionmaking [29] [30] Anyone is always eager to mine and maximise the useful data. Because it's has a direct impact on company earnings, it is one of the most crucial variables in BD. ...
Domain and Challenges of Big Data and Archaeological Photogrammetry With Blockchain
Article
Full-text available
  • Jan 2022
With gigantic growth of data volume that is moved across the web links today, there has been a gigantic measure of perplexing information produced. Extremely huge sets of data including universities, organizations framework, institution gas, petroleum sector, healthcare, that have so enormous thus complex information with more differed structure. The major challenge is how to handle this significant volume of data, also in archaeological photogrammetry which alluded to as Big Data. Although big data has to be securely flying and conveyed through the internet. It cannot be controlled with regular conventional methods that fail to handle it, so there is a need for more up-to-date developed tools. The big data have frequently divided into V’s characteristics beginning from three V’s: volume, velocity and variety. The initial three V’s have been stretched out during time through researches to arrive 56 V’s till now. Among them are three newfound by the author that implies it multiplied near twenty times. Researcher had to dive to search for all of these characteristics in many researches to detect and build comparisons to answer the old, current, and restored essential inquiry, “how many V’s aspects (characteristics) in big data with archaeological photogrammetry and blockchain”. This paper provides a comprehensive overview of all secured big data V’s (characteristics) as well as their strength and limitations with archaeological photogrammetry and blockchain.
View
Data analysis and feature selection for predictive maintenance: A case-study in the metallurgic industry
Article
Full-text available
  • Oct 2018
  • INT J INFORM MANAGE
Proactive Maintenance practices are becoming more standard in industrial environments, with a direct and profound impact on the competitivity within the sector. These practices demand the continuous monitorization of industrial equipment, which generates extensive amounts of data. This information can be processed into useful knowledge with the use of machine learning algorithms. However, before the algorithms can effectively be applied, the data must go through an exploratory phase: assessing the meaning of the features and to which degree they are redundant. In this paper, we present the findings of the analysis conducted on a real-world dataset from a metallurgic company. A number of data analysis and feature selection methods are employed, uncovering several relationships, which are systematized in a rule-based model, and reducing the feature space from an initial 47-feature dataset to a 32-feature dataset.
View
Big Data and software engineering: prospects for mutual enrichment
Article
Full-text available
  • Apr 2018
Software engineering has evolved over the last 50 years, initially as a response to the so-called software crisis (the problems that organizations had producing quality software systems on time and on budget) of the 1960s and 1970s. Software engineering (SE) has been defined as “the application of a systematic, disciplined, quantifiable approach to the development, operation, and maintenance of software; that is, the application of engineering to software”. Software engineering has developed a number of approaches to areas such as software requirements, software design, software testing, and software maintenance. Software development processes such as the waterfall model, incremental development, and the spiral model have been successfully applied to produce high-quality software on time and under budget. More recently, agile software development has gained popularity as an alternative to the more traditional development methods for development of complex systems. Within the last decade or so, advances in technologies such as mobile computing, social networks, cloud computing, and the Internet of things have given rise to massive datasets which have been given the name Big Data (BD). Big Data has been defined as data with 3Vs—high volume, velocity, and variety. Big Data contains so much data that low probability events are captured in the data. These events can be discovered using analytics methods and turned into actionable intelligence which can be used by businesses to gain a competitive advantage. Unfortunately, the very scale of BD often renders inadequate SQL-based relational database systems which have formed the backbone of data intensive systems for the last 30 years, requiring new NoSQL technologies to be effective. In this paper, we will explore how well-established SE technology can be adapted to support successful development of BD projects, as well as how BD techniques can be used to increase the utility of SE processes and techniques. Thus, BD and SE may mutually support and enrich each other.
View
Data mining-aided materials discovery and optimization
Article
Full-text available
  • Aug 2017
Recent developments in data mining-aided materials discovery and optimization are reviewed in this paper, and an introduction to the materials data mining (MDM) process is provided using case studies. Both qualitative and quantitative methods in machine learning can be adopted in the MDM process to accomplish different tasks in materials discovery, design, and optimization. State-of-the-art techniques in data mining-aided materials discovery and optimization are demonstrated by reviewing the controllable synthesis of dendritic Co3O4 superstructures, materials design of layered double hydroxide, battery materials discovery, and thermoelectric materials design. The results of the case studies indicate that MDM is a powerful approach for use in materials discovery and innovation, and will play an important role in the development of the Materials Genome Initiative and Materials Informatics.
View
Big Data Analytics in Chemical Engineering
Article
Full-text available
  • Feb 2017
Big data analytics is the journey to turn data into insights for more informed business and operational decisions. As the chemical engineering community is collecting more data (volume) from different sources (variety), this journey becomes more challenging in terms of using the right data and the right tools (analytics) to make the right decisions in real time (velocity). This article highlights recent big data advancements in five industries, including chemicals, energy, semiconductors, pharmaceuticals, and food, and then discusses technical, platform, and culture challenges. To reach the next milestone in multiplying successes to the enterprise level, government, academia, and industry need to collaboratively focus on workforce development and innovation. Expected final online publication date for the Annual Review of Chemical and Biomolecular Engineering Volume 8 is June 7, 2017. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.
View
Current and Future Challenges of Software Engineering for Services and Applications
Article
Full-text available
  • Dec 2016
ICT (Information and Communication Technology) and, in particular, software is more and more pervasive and it cannot be considered anymore as a minor element of a complex systems. In domains like cloud, big data, IoT (Internet of Things), CPS (Cyber-Physical Systems) it is the core element. We need to consolidate the software engineering discipline, which, despite the impressive achievements in the area of software technology, is probably one of the youngest scientific and technological disciplines with about 60 years of history. This paper summarizes the challenges that the Software Engineering for Services and Applications (SE4SA) cluster is considering as relevant.
View
Big Data for Internet of Things: A Survey
Article
  • May 2018
  • FUTURE GENER COMP SY
With the rapid development of the Internet of Things (IoT), Big Data technologies have emerged as a critical data analytics tool to bring the knowledge within IoT infrastructures to better meet the purpose of the IoT systems and support critical decision making. Although the topic of Big Data analytics itself is extensively researched, the disparity between IoT domains (such as healthcare, energy, transportation and others) has isolated the evolution of Big Data approaches in each IoT domain. Thus, the mutual understanding across IoT domains can possibly advance the evolution of Big Data research in IoT. In this work, we therefore conduct a survey on Big Data technologies in different IoT domains to facilitate and stimulate knowledge sharing across the IoT domains. Based on our review, this paper discusses the similarities and differences among Big Data technologies used in different IoT domains, suggests how certain Big Data technology used in one IoT domain can be re-used in another IoT domain, and develops a conceptual framework to outline the critical Big Data technologies across all the reviewed IoT domains.
View
Smart Maintenance Decision Support Systems (SMDSS) based on Corporate Big Data Analytics
Article
  • Aug 2017
  • EXPERT SYST APPL
The purpose of this article is to outline the architectural design and the conceptual framework for a Smart Maintenance Decision Support System (SMDSS) based on corporate data from a Fortune 500 company. Motivated by the rapidly transforming landscape for big data analytics and predictive maintenance decision making, we have created a system capable of providing end users with recommendations to improve asset lifecycles. Methodologically, a cost minimization algorithm is used to analyze a large industry service and warranty data sets and two analytical decision models were developed and applied to a case study for an electrical circuit breaker maintenance problem. Some of these techniques can be applied to other industries, such as jet engine maintenance, and can be expanded to others with implications for robust decision analysis. The SMDSS provides a predictive analytical model that can be applied in manufacturing and service based industries. Our findings and results show that existing solution algorithms and optimization models can be applied to large data sets to lay out executable decisions for managers.
View
Big data analysis for an electric vehicle charging infrastructure using open data and software
Conference Paper
  • May 2017
This paper describes a big data analysis strategy for electric vehicle charging infrastructure, mainly built upon open data sets and open software components. The data acquisition module periodically retrieves the real-time status information of each charger from the public data portal, while the downloaded XML files are parsed to extract fields of interest. At this stage, we present the distribution of charging facilities in Jeju City based on our own map viewer implementation, the city-wide dynamics of the number of chargers in operation based on MySQL queries, and the visualization of regional occupancy rates based on the R GISTools library. After combining a variety of statistical and machine learning techniques to understand the demand pattern of electric vehicle charging, we will integrate renewable energy to charging-intensive power grids as much as possible.
View
View
View