Virtual Machine for Single Node Hadoop Cluster

Virtual Machine for Single Node Hadoop ClusterThis article explains how to “create Virtual Machine on local machine and install a single node Hadoop cluster on it“.
This article is giving step-by-step process to install single node Hadoop cluster on local machine, which would be helpful in executing jobs on local machine. Single node cluster means only one DataNode is running and setting up all NameNode, DataNode, ResourceManager and NodeManager on a single machine. This is used for studying and testing purposes.

Steps to create instance

1. Download Oracle Virtual-Box on the local machine https://www.virtualbox.org/wiki/Downloads
2. Select the correct distribution. (OS X, Windows or Linux)
3. Install Virtual-Box on the System.
4. Download Cloudera Quickstart VM for Virtual-Box. https://www.cloudera.com/downloads/quickstart_vms/5-12.html
5. Unzip its content. (if it is in 7Zip format, use appropriate tools, eg 7Zip for windows)
6. Start Virtual-Box
7. Click on New
8. Give a name for the virtual machine
9. Select type as Linux
10. Select version as Other Linux 64 bit

 

 

 

 

 

 

 

 

 

 

 

11. Give 8GB of memory.
12. Select Use an existing virtual hard disk file.
13. Browse and select the .vmdk file in the un-zipped cloudera folder(refer step 5)
14. Create the virtual machine.
15. After creating the VM, go to settings and general
Shared Clipboard : bidirectional
Drag’n’Drop : host to guest.

In System settings go to CPU and provide maximum number of CPU Cores as possible. But the number should not exceed the number of available physical cores. This is required for better performance.

About: Avinash