Bigdata Platform Design and Implementation Model

作者：Kyoo-Sung Noh, Lee Doo-sik · 发表于：Indian Journal of Science and Technology · 年份：2015 · DOI：10.17485/ijst/2015/v8i18/75864 · 被引用次数：17 · 研究领域：Cloud Computing and Resource Management、Advanced Data Storage Technologies、Scientific Computing and Data Management

Bigdata software platform technology using Hadoop Ecosystem is the essential element and also underlying technology for application software or service implementation of bigdata analysis. It is required to have bigdata platform technology that can ensure the scalability, reliability and high performance of system for processing and analyzing a variety of bigdata related tasks including structured data, unstructured data, semi-structured data, etc. Bigdata platform can process large amounts of data in parallel unlike those conventional application software solutions and it is an easily scalable system. Its technical components include collection (Flume and Sqoop), storage (Hadoop and NoSQL), search (Solr), analysis (R Analysis), visualization (Node.js), scheduler (Oozie), etc. The purpose of this study is to propose an optimized bigdata platform implementation model through S/W configuration based on open source.Keywords: Bigdata, Bigdata Visualization, Flume, Hadoop, MAP/REDUCE, Oozie, Platform, R Analysis, Sqoop, SQL on Hadoop