Hadoop interview questions part 2

Hadoop Interview Questions Part-2 5) Describe the significant distinction between HDFS prevent and InputSplit .

In easy conditions, prevent is the physical reflection of information while divided is the sensible reflection of information found in the prevent. Split functions a s a middleman between prevent and mapper.

Suppose we have two blocks: Block 1: ii nntteell Block 2: Ii ppaatt Now, considering the map, it will study first prevent from ii until ll, but does not know how to process the second prevent simultaneously. Here comes Separated into play, which will type a sensible team of Block1 and Block 2 as a individual prevent.

It then types key-value couple using inputformat and information audience and delivers map for further handling With inputsplit, if you have restricted sources, you can boost the divided size to restrict the number of charts. For example, if there are 10 prevents of 640MB (64MB each) and there are restricted sources, you can allocate ‘split size’ as 128MB. This will type a sensible team of 128MB, with only 5 charts performing simultaneously.

However, if the ‘split size’ property is set to incorrect, whole information file will type one inputsplit and is prepared by individual map, taking a longer period when the information file is larger.

6) What is shipped storage cache and what are its benefits?

Distributed Cache, in Hadoop, is a service by MapReduce structure to storage cache information files when required. Learn more in this MapReduce Guide now. Once a information file is cached for a particular job, hadoop will make it available on each information node both in system and in storage, where map and decrease jobs are performing.Later, you can easily accessibility and study the storage cache information file and fill any selection (like range, hashmap) in your rule.

Benefits of using allocated storage cache are: It markets easy, study only text/data information files and/or complicated types like jugs, information and others. These information are then un-archived at the servant node.

Distributed storage cache paths the advance timestamps of storage cache information files, which informs that the information files should not be customized until a job is performing currently.

7) Describe the distinction between NameNode, Gate NameNode and BackupNode.

NameNode is the primary of HDFS that controls the meta-data – the information of what information file charts to what prevent places and what prevents are saved on what datanode.

Simply, it’s the information about the information being saved. NameNode facilitates a listing tree-like framework composed of all the information files found in HDFS on a Hadoop group. It uses following information files for namespace: fsimage file- It monitors the newest checkpoint of the namespace. edits file-It is a log of changes that have been made to the namespace since checkpoint.

Checkpoint NameNode has the same listing framework as NameNode, and helps to make check points for namespace at frequent durations by installing the fsimage and modifications information file and margining them within a nearby listing. The new picture after consolidating is then submitted to NameNode.

There is the same node like Gate, generally known as Additional Node, but it does not assistance the ‘upload to NameNode’ performance.

Backup Node provides identical performance as Gate, implementing synchronization with NameNode. It preserves an up-to-date in-memory duplicate of information file system namespace and doesn’t need finding changes after frequent durations. The back-up node needs to preserve the existing state in-memory to an picture information file to make a new checkpoint.

You can join the DBA course to make your career as a certified DBA professional in this field.

Stay connected to CRB Tech for more technical optimization and other updates and information.