Amazing technological breakthrough possible @S-Logix pro@slogix.in

Office Address

  • #5, First Floor, 4th Street Dr. Subbarayan Nagar Kodambakkam, Chennai-600 024 Landmark : Samiyar Madam
  • pro@slogix.in
  • +91- 81240 01111

Social List

i2MapReduce: Incremental mapreduce for mining evolving big data - 2016

i2MapReduce: Incremental mapreduce for mining evolving big data

Research Area:  Big Data

Abstract:

As new data and updates are constantly arriving, the results of data mining applications become stale and obsolete over time. Incremental processing is a promising approach to refresh mining results. It utilizes previously saved states to avoid the expense of re-computation from scratch. In this paper, we propose i 2 MapReduce, a novel incremental processing extension to MapReduce. Compared with the state-of-the-art work on Incoop, i 2 MapReduce (i) performs key-value pair level incremental processing rather than task level re-computation, (ii) supports not only one-step computation but also more sophisticated iterative computation, and (iii) incorporates a set of novel techniques to reduce I/O overhead for accessing preserved fine-grain computation states. Experimental results on Amazon EC2 show significant performance improvements of i 2 MapReduce compared to both plain and iterative MapReduce performing re-computation.

Keywords:  

Author(s) Name:  Yanfeng Zhang; Shimin Chen; Qiang Wang and Ge Yu

Journal name:  

Conferrence name:  IEEE 32nd International Conference on Data Engineering (ICDE)

Publisher name:  IEEE

DOI:  10.1109/ICDE.2016.7498385

Volume Information: