Hadoop: Instal di Ubuntu 14.04 - Revision history

Onnowpurbo: /* ~/.bashrc: */

2015-11-10T01:10:39Z

~/.bashrc:

← Older revision		Revision as of 01:10, 10 November 2015
Line 147:		Line 147:

	update-alternatives --config java		update-alternatives --config java

	There is only one alternative in link group java (providing /usr/bin/java): /usr/lib/jvm/java-7-openjdk-amd64/jre/bin/java		There is only one alternative in link group java (providing /usr/bin/java): /usr/lib/jvm/java-7-openjdk-amd64/jre/bin/java
	Nothing to configure.		Nothing to configure.

Onnowpurbo: /* Menjalankan Hadoop */

2015-11-09T23:07:03Z

Menjalankan Hadoop

← Older revision		Revision as of 23:07, 9 November 2015
Line 431:		Line 431:
	tcp 0 0 0.0.0.0:8040 0.0.0.0:* LISTEN 1001 22344 7895/java		tcp 0 0 0.0.0.0:8040 0.0.0.0:* LISTEN 1001 22344 7895/java
	tcp 0 0 0.0.0.0:8042 0.0.0.0:* LISTEN 1001 22348 7895/java		tcp 0 0 0.0.0.0:8042 0.0.0.0:* LISTEN 1001 22348 7895/java
	tcp 0 0 0.0.0.0:50090 0.0.0.0:* LISTEN 1001 19321 7607/java		tcp 0 0 0.0.0.0:50090 0.0.0.0:* LISTEN 1001 19321 7607/java
	~~Stopping Hadoop~~

	==Monitor Job & Task==		==Monitor Job & Task==

Onnowpurbo: /* Stop Hadoop */

2015-11-09T23:06:41Z

Stop Hadoop

← Older revision		Revision as of 23:06, 9 November 2015
Line 433:		Line 433:
	tcp 0 0 0.0.0.0:50090 0.0.0.0:* LISTEN 1001 19321 7607/java		tcp 0 0 0.0.0.0:50090 0.0.0.0:* LISTEN 1001 19321 7607/java
	Stopping Hadoop		Stopping Hadoop

			==Monitor Job & Task==


			* NameNode daemon: http://hdnode01:50070
			* JobTracker daemon: http://hdnode01:50030
			* TaskTracker daemon: http://hdnode01:50060


	==Stop Hadoop==		==Stop Hadoop==
Line 447:		Line 455:
	cd /usr/local/hadoop/sbin		cd /usr/local/hadoop/sbin
	stop-all.sh		stop-all.sh



	==Hadoop Web Interface==		==Hadoop Web Interface==

Onnowpurbo: /* ~/.bashrc: */

2015-11-09T09:58:52Z

~/.bashrc:

← Older revision		Revision as of 09:58, 9 November 2015
Line 157:		Line 157:
	export JAVA_HOME=/usr/lib/jvm/java-7-openjdk-amd64/		export JAVA_HOME=/usr/lib/jvm/java-7-openjdk-amd64/
	export HADOOP_INSTALL=/usr/local/hadoop		export HADOOP_INSTALL=/usr/local/hadoop
			export HADOOP_HOME=/usr/local/hadoop/share/hadoop/common
			export HADOOP_VERSION=2.7.1
	export PATH=$PATH:$HADOOP_INSTALL/bin		export PATH=$PATH:$HADOOP_INSTALL/bin
	export PATH=$PATH:$HADOOP_INSTALL/sbin		export PATH=$PATH:$HADOOP_INSTALL/sbin
Line 185:		Line 187:

	/usr/lib/jvm/java-7-openjdk-amd64/		/usr/lib/jvm/java-7-openjdk-amd64/



	===/usr/local/hadoop/etc/hadoop/hadoop-env.sh===		===/usr/local/hadoop/etc/hadoop/hadoop-env.sh===

Onnowpurbo: /* Menjalankan Hadoop */

2015-11-09T04:48:11Z

Menjalankan Hadoop

@@ Line 414: / Line 414: @@
   netstat -plten | grep java
   (Not all processes could be identified, non-owned process info
@@ Line 435: / Line 434: @@
 Stopping Hadoop
-$ pwd
+==Stop Hadoop==
-$ ls
+Kita dapat menjalankan
-We run stop-all.sh or (stop-dfs.sh and stop-yarn.sh) to stop all the daemons running on our machine:
+ stop-all.sh
-Hadoop Web Interfaces
+==Hadoop Web Interface==
 Let's start the Hadoop again and see its Web UI:
-hduser@laptop:/usr/local/hadoop/sbin$ start-all.sh
+ cd /usr/local/hadoop/sbin
+ start-all.sh
-Using Hadoop
+Akses ke
-If we have an application that is set up to use Hadoop, we can fire that up and start using it with our Hadoop installation!
+ http://localhost:50070
 ==Referensi==
 * http://www.bogotobogo.com/Hadoop/BigData_hadoop_Install_on_ubuntu_single_node_cluster.php

Onnowpurbo: /* Menjalankan Hadoop */

2015-11-09T04:40:29Z

Menjalankan Hadoop

Show changes

Onnowpurbo: /* /usr/local/hadoop/etc/hadoop/hdfs-site.xml */

2015-11-09T04:29:17Z

/usr/local/hadoop/etc/hadoop/hdfs-site.xml

Show changes

Onnowpurbo: /* /usr/local/hadoop/etc/hadoop/hdfs-site.xml */

2015-11-09T04:18:34Z

/usr/local/hadoop/etc/hadoop/hdfs-site.xml

← Older revision		Revision as of 04:18, 9 November 2015
Line 257:		Line 257:
	===/usr/local/hadoop/etc/hadoop/hdfs-site.xml===		===/usr/local/hadoop/etc/hadoop/hdfs-site.xml===

	~~The~~ /usr/local/hadoop/etc/hadoop/hdfs-site.xml ~~file needs to be configured for each~~ host ~~in the~~ cluster ~~that is being used~~.		File /usr/local/hadoop/etc/hadoop/hdfs-site.xml perlu di konfigurasi di setiap host di cluster yang kita gunakan. File tersebut berisi directory yang digunakan sebagai node dan datanode di host tersebut.
	~~It is used to specify the directories which will be used as the namenode and the~~ datanode ~~on that~~ host.

	~~Before editing this~~ file, ~~we need to create two directories which will contain the~~ namenode ~~and the~~ datanode ~~for this~~ Hadoop ~~installation~~.		Sebelum mengedit file, kita perlu membuat dua directory yang berisi namenode dan datanode untuk instalasi Hadoop ini. Ini dapat dilakukan menggunakan perintah:
	~~This can be done using the following commands~~:

	~~hduser@laptop:~$~~ sudo mkdir -p /usr/local/hadoop_store/hdfs/namenode		sudo mkdir -p /usr/local/hadoop_store/hdfs/namenode
	~~hduser@laptop:~$~~ sudo mkdir -p /usr/local/hadoop_store/hdfs/datanode		sudo mkdir -p /usr/local/hadoop_store/hdfs/datanode
	~~hduser@laptop:~$~~ sudo chown -R hduser:hadoop /usr/local/hadoop_store		sudo chown -R hduser:hadoop /usr/local/hadoop_store

	~~Open the~~ file ~~and enter the following content in between the~~ <configuration></configuration> ~~tag~~:		Buka file, dam masukan konfigurasi antara tag <configuration></configuration>:

	~~hduser@laptop:~$~~ vi /usr/local/hadoop/etc/hadoop/hdfs-site.xml		vi /usr/local/hadoop/etc/hadoop/hdfs-site.xml

	<configuration>		<configuration>
	<property>		<property>
	<name>dfs.replication</name>		<name>dfs.replication</name>
	<value>1</value>		<value>1</value>
	<description>Default block replication.		<description>Default block replication.
	The actual number of replications can be specified when the file is created.		The actual number of replications can be specified when the file is created.
	The default is used if replication is not specified in create time.		The default is used if replication is not specified in create time.
	</description>		</description>
	</property>		</property>
	<property>		<property>
	<name>dfs.namenode.name.dir</name>		<name>dfs.namenode.name.dir</name>
	<value>file:/usr/local/hadoop_store/hdfs/namenode</value>		<value>file:/usr/local/hadoop_store/hdfs/namenode</value>
	</property>		</property>
	<property>		<property>
	<name>dfs.datanode.data.dir</name>		<name>dfs.datanode.data.dir</name>
	<value>file:/usr/local/hadoop_store/hdfs/datanode</value>		<value>file:/usr/local/hadoop_store/hdfs/datanode</value>
	</property>		</property>
	</configuration>		</configuration>

Onnowpurbo: /* /usr/local/hadoop/etc/hadoop/mapred-site.xml */

2015-11-09T04:14:43Z

/usr/local/hadoop/etc/hadoop/mapred-site.xml

← Older revision		Revision as of 04:14, 9 November 2015
Line 254:		Line 254:
	</property>		</property>
	</configuration>		</configuration>




	===/usr/local/hadoop/etc/hadoop/hdfs-site.xml===		===/usr/local/hadoop/etc/hadoop/hdfs-site.xml===

Onnowpurbo: /* /usr/local/hadoop/etc/hadoop/core-site.xml */

2015-11-09T04:14:00Z

/usr/local/hadoop/etc/hadoop/core-site.xml

← Older revision		Revision as of 04:14, 9 November 2015
Line 230:		Line 230:


	4. /usr/local/hadoop/etc/hadoop/mapred-site.xml		===/usr/local/hadoop/etc/hadoop/mapred-site.xml===

	~~By default~~, ~~the /usr/local/hadoop/etc/hadoop/~~ folder ~~contains~~		Default, folder /usr/local/hadoop/etc/hadoop/ berisi
	/usr/local/hadoop/etc/hadoop/~~mapred-site.xml.template~~
	~~file which has to be renamed/copied with the name mapred-site.xml:~~

	~~hduser@laptop:~$ cp~~ /usr/local/hadoop/etc/hadoop/mapred-site.xml.template ~~/usr/local/hadoop/etc/hadoop/mapred-site.xml~~		/usr/local/hadoop/etc/hadoop/mapred-site.xml.template

	~~The~~ mapred-site.xml ~~file is used to specify which framework is being used for MapReduce.~~		yang harus di rename / di copy dengan nama mapred-site.xml:
	~~We need to enter the following content in between the <configuration></configuration> tag~~:

	~~<configuration>~~		cp /usr/local/hadoop/etc/hadoop/mapred-site.xml.template /usr/local/hadoop/etc/hadoop/mapred-site.xml
	~~<property>~~
	~~<name>~~mapred.~~job~~.~~tracker<~~/~~name>~~
	~~<value>localhost:54311<~~/~~value>~~
	~~<description>The host and port that the MapReduce job tracker runs~~
	~~at. If "~~local~~", then jobs are run in-process as a single map~~
	~~and reduce task.~~
	</~~description>~~
	</~~property>~~
	</~~configuration>~~

			File mapred-site.xml digunakan untuk menentukan framework yang digunakan untuk MapReduce. Kita perlu memasukan informasi berikut antara tag <configuration></configuration> :

			vi /usr/local/hadoop/etc/hadoop/mapred-site.xml

			<configuration>
			<property>
			<name>mapred.job.tracker</name>
			<value>localhost:54311</value>
			<description>The host and port that the MapReduce job tracker runs
			at. If "local", then jobs are run in-process as a single map
			and reduce task.
			</description>
			</property>
			</configuration>

	5. /usr/local/hadoop/etc/hadoop/hdfs-site.xml


			===/usr/local/hadoop/etc/hadoop/hdfs-site.xml===

	The /usr/local/hadoop/etc/hadoop/hdfs-site.xml file needs to be configured for each host in the cluster that is being used.		The /usr/local/hadoop/etc/hadoop/hdfs-site.xml file needs to be configured for each host in the cluster that is being used.