Posts

Showing posts from March, 2015

MapReduce in HADOOP 1

Image
                                                                                                                                                                                                   Submission Step 1 in figure: The submit() method on Job creates an internal Jobsubmitter instance and calls submitJobInternal() on it.The job submission process implemented by Jobsubmitter does the following. Step 2: Asks the jobtracker for a new job ID (by calling getNewJobId() on JobTracker) Step 3...

MapReduce in HADOOP 2

Image
Submission Step 1 in figure: The submit() method on Job creates an internal Jobsubmitter instance and calls submitJobInternal() on it.The job submission process implemented by Jobsubmitter does the following. Step 2: Asks the resource manager a new job Id. Step 3: Checks the output specification of the job,Computes input splits,Copies job resources (job JAR ,configuration,and split information)to HDFS. Step 4: Finally, the job is submitted by calling submitApplication() on the resource manager. Job Intitialization Step 5a and 5b:  When the resource manager receives a call to its submitApplication(), it hands off the request to the scheduler.The scheduler allocates a container,and the resource manager then launches the application master's process there , under the node manager's management. Step 6: The application master initializes job by creating a number of book keeping objects to keep track of the job's progress,as it will receive progr...