Wednesday, November 27, 2013

Apache Hadoop 2.2.0

Apache Hadoop 2.2.0 consists of significant improvements over the previous stable release (hadoop-1.x).

Here is a short overview of the improvments to both HDFS and MapReduce.
  • HDFS Federation
    In order to scale the name service horizontally, federation uses multiple independent Namenodes/Namespaces. The Namenodes are federated, that is, the Namenodes are independent and don't require coordination with each other. The datanodes are used as common storage for blocks by all the Namenodes. Each datanode registers with all the Namenodes in the cluster. Datanodes send periodic heartbeats and block reports and handles commands from the Namenodes.
    More details are available in the HDFS Federation document.
  • MapReduce NextGen aka YARN aka MRv2
    The new architecture introduced in hadoop-0.23, divides the two major functions of the JobTracker: resource management and job life-cycle management into separate components.
    The new ResourceManager manages the global assignment of compute resources to applications and the per-application ApplicationMaster manages the application’s scheduling and coordination.
    An application is either a single job in the sense of classic MapReduce jobs or a DAG of such jobs.
    The ResourceManager and per-machine NodeManager daemon, which manages the user processes on that machine, form the computation fabric.
    The per-application ApplicationMaster is, in effect, a framework specific library and is tasked with negotiating resources from the ResourceManager and working with the NodeManager(s) to execute and monitor the tasks.
    More details are available in the YARN document.

Getting Started

The Hadoop documentation includes the information you need to get started using Hadoop. Begin with the Single Node Setup which shows you how to set up a single-node Hadoop installation. Then move on to the Cluster Setup to learn how to set up a multi-node Hadoop installation.

8 comments:

  1. Enormous Data is the sea of information now accessible to organizations by means of the social web, machine information and value-based information. machine learning certification

    ReplyDelete
  2. The following are not covered by the Lowe's Sweepstakes Entities: technical difficulties, data corruption, unauthorized access or modification, injuries, incomplete, inaccurate, garbled, or incomprehensible entries or communications.
    Lowe’s Survey

    ReplyDelete
  3. The sponsor disclaims all liability for entries that are misdirected, lost, damaged, illegible, garbled, incomplete, or arrive late. Evidence of transmission or submission shall not be considered evidence of Sponsor reception. Online participants are subject to the notices published on Burlington, including its privacy policy. Burlingtonfeedback.com

    ReplyDelete
  4. In conclusion, consumers have a great chance to offer their input and assist the firm in bettering its offerings through the Whataburger Survey. Completing the survey is simple and quick.
    WhataburgerVisit.com

    ReplyDelete
  5. You probably already know that every business wants to lead its industry and that satisfied customers are crucial to its expansion and survival.
    PublixSurvey.Com

    ReplyDelete
  6. Every predetermined session's winner receives a gift card good for fifty-two Chipotle Box Combo meals.
    ChipotleFeedback.com

    ReplyDelete
  7. Stated differently, the aim of the Publix customer survey is to ascertain whether you, as a customer, have experienced a favourable shopping experience at their superstores.
    PublixSurvey.Com

    ReplyDelete

The Open Source Web Hosting company