Description

• Role:
  The MapReduce programming model consists of three main stages: Map, Shuffle and Sort, and Reduce. These stages help in processing large datasets in parallel across distributed computing clusters

• Role:
  The Map stage is the first phase of MapReduce where the input data is processed and transformed into key-value pairs.
• Function:
  In this stage, the input data is divided into smaller chunks (splits), and each chunk is processed by a separate Mapper task. Each Mapper reads a portion of the data, applies a user-defined Map function to transform the data into key-value pairs.
• Output:
  The output of the Map phase is a set of intermediate key-value pairs, which will later be grouped and processed further.
• Example:
  If the input is a large text file, the Map function might split the data into words (keys) and count their occurrences (values).
• Example Input: ["apple", "banana", "apple"]
• Mapper Output: [("apple", 1), ("banana", 1), ("apple", 1)]

• Role:
  The Shuffle and Sort stage is the intermediate phase that comes between the Map and Reduce stages. This stage organizes the data output from the Map phase and groups the key-value pairs by their keys.
• Function:
  After the Map tasks complete, the data is shuffled (i.e., the keys are grouped), and the values corresponding to each key are sorted. The shuffling process ensures that all occurrences of the same key are brought together.
• Output:
  The output of the Shuffle and Sort stage is a set of grouped key-value pairs, where all values associated with the same key are gathered together.
• Example:
  If the Map phase produces multiple key-value pairs with the same key (e.g., "apple"), the Shuffle stage groups all the values for that key together.
• Example Input from Map: [("apple", 1), ("banana", 1), ("apple", 1)]
• After Shuffle and Sort: [("apple", [1, 1]), ("banana", [1])]

• Role:
  The Reduce stage is the final phase in MapReduce where the grouped data is processed to produce the final output.
• Function:
  The Reducer takes the output of the Shuffle and Sort phase (which consists of key-value pairs where each key has a list of associated values). The Reduce function is applied to each group of values, and it aggregates or processes the data to generate the final result. This could involve summing values, averaging, finding the maximum, etc.
• Output:
  The final output is a set of key-value pairs where each key is associated with the processed result.
• Example:
  In a word count problem, the Reduce function would sum up the values (word counts) for each key (word).
• Example Input from Shuffle and Sort: [("apple", [1, 1]), ("banana", [1])]
• Reducer Output: [("apple", 2), ("banana", 1)]