Hadoop: Hive Instalasi - Revision history

Onnowpurbo: /* SETTING Hive environment variable */

2015-11-14T01:16:49Z

SETTING Hive environment variable

← Older revision		Revision as of 01:16, 14 November 2015
Line 29:		Line 29:
	export PATH		export PATH

	==SETTING HADOOP_PATH di HIVE config.sh		==SETTING HADOOP_PATH di HIVE config.sh==


	cd /usr/lib/hive/apache-hive-1.2.1-bin/bin		cd /usr/lib/hive/apache-hive-1.2.1-bin/bin
	sudo gedit hive-config.sh		sudo gedit hive-config.sh

	~~Go to the line where the following statements are written~~		Cari kalimat di bawah ini

	# Allow alternate conf dir location.		# Allow alternate conf dir location.
Line 42:		Line 41:
	export HIVE_AUX_JARS_PATH=$HIVE_AUX_JARS_PATH		export HIVE_AUX_JARS_PATH=$HIVE_AUX_JARS_PATH

	~~Below this write the following~~		Selanjutnya, tambahkan di bawah-nya

	export HADOOP_HOME=/usr/local/hadoop ~~(write the path where hadoop file is there)~~		export HADOOP_HOME=/usr/local/hadoop

	Create Hive directories within HDFS		Create Hive directories within HDFS
Line 58:		Line 57:
	user@ubuntu:~$ hadoop fs -chmod g+w /usr/hive/warehouse		user@ubuntu:~$ hadoop fs -chmod g+w /usr/hive/warehouse

	HIVE ~~launch~~		==Menjalankan HIVE==

	~~Command~~

	~~user@ubuntu:~$ hive~~

	~~Hive shell will prompt:~~		Ketik
	~~OUTPUT~~

	~~Shell will look like~~		hive

	~~Logging initialized using configuration in jar:file:/usr/lib/hive/apache-hive-0.13.0-bin/lib/hive- common-0.13.0.jar!/hive-log4j.properties~~		Akan keluar
	~~hive>~~

	~~Creating a database~~		Logging initialized using configuration in jar:file:/usr/lib/hive/apache-hive-0.13.0-bin/lib/hive- common-0.13.0.jar!/hive-log4j.properties
			hive>

	~~Command~~		==Membuat Database==

	hive> create database mydb;		hive> create database mydb;

	~~OUTPUT~~		Akan keluar

	OK		OK
	Time taken: 0.369 seconds		Time taken: 0.369 seconds
	hive>		hive>

	~~Configuring~~ hive-site.xml:		==Konfigurasi hive-site.xml==

	Open with text-editor and change the following property		Open with text-editor and change the following property
Line 112:		Line 106:
	</property>		</property>

	~~Writing a~~ Script		==Tulis Script==

	~~Open a new terminal (CTRL+ALT+T)~~

	~~user@ubuntu:~$ sudo gedit sample.sql~~

	~~create database sample;~~
	~~use sample;~~
	~~create table product(product int, productname string, price float)[row format delimited fields terminated by ',';]~~
	~~describe product;~~

	~~load data local inpath ‘/home/hduser/input_to_product.txt’ into table product~~

	~~select * from product;~~

	~~SAVE and CLOSE~~

	~~user@ubuntu:~$ sudo gedit input_to_product.txt~~
	~~user@ubuntu:~$ cd /usr/lib/hive/apache-hive-0.13.0-bin/ $ bin/hive -f /home/hduser/sample.sql~~


			Edit

			sudo gedit sample.sql

			Masukan

			create database sample;
			use sample;
			create table product(product int, productname string, price float)[row format delimited fields terminated by ',';]
			describe product;

			load data local inpath ‘/home/hduser/input_to_product.txt’ into table product

			select * from product;

			Save

			sudo gedit input_to_product.txt
			cd /usr/lib/hive/apache-hive-0.13.0-bin/
			bin/hive -f /home/hduser/sample.sql

	==Referensi==		==Referensi==

	* http://doctuts.readthedocs.org/en/latest/hive.html		* http://doctuts.readthedocs.org/en/latest/hive.html

Onnowpurbo at 01:10, 14 November 2015

2015-11-14T01:10:08Z

← Older revision		Revision as of 01:10, 14 November 2015
Line 2:		Line 2:


			==Sumber==

	~~HIVE INSTALLATION~~		URL

	~~This section refers to the installation settings of Hive on a standalone system as well as on a system existing as a node in a cluster~~.		http://apache.claz.org/hive/stable/
	~~INTRODUCTION~~

	Apache Hive is a data warehouse infrastructure built on top of Hadoop for providing data summarization, query, and analysis. Apache Hive supports analysis of large datasets stored in Hadoop’s HDFS and compatible file systems such as Amazon S3 filesystem. It provides an SQL-like language called HiveQL(Hive Query Language) while maintaining full support for map/reduce.		Sebagai user biasa.
			Download & Extract

	~~Hive Installation~~		wget http://apache.claz.org/hive/stable/apache-hive-1.2.1-bin.tar.gz
	~~Installing HIVE:~~		tar zxvf apache-hive-1.2.1-bin.tar.gz

	~~Browse to the link: http:~~//apache.~~claz~~.~~org~~/~~hive~~/~~stable~~/		sudo mkdir /usr/lib/hive
			sudo mv apache-hive-1.2.1-bin /usr/lib/hive

	~~Click the apache-hive-0.13.0-bin.tar.gz~~		==SETTING Hive environment variable==

	~~Save and Extract it~~		cd
			sudo gedit ~/.bashrc

	~~Commands~~		Copy paste kalimat ini di bagian bawah file .bashrc

	~~user@ubuntu:~$~~ cd /usr/lib/		# Set HIVE_HOME
	~~user@ubuntu:~$ sudo mkdir~~ hive		export HIVE_HOME="/usr/lib/hive/apache-hive-1.2.1-bin"
	~~user@ubuntu:~$ cd Downloads~~		PATH=$PATH:$HIVE_HOME/bin
	~~user@ubuntu:~$ sudo mv~~ apache-hive-0.13.0-bin /~~usr/lib/hive~~		export PATH

	~~Setting Hive environment variable:~~		==SETTING HADOOP_PATH di HIVE config.sh

	~~Commands~~

	~~user@ubuntu:~$~~ cd		cd /usr/lib/hive/apache-hive-1.2.1-bin/bin
	~~user@ubuntu:~$~~ ~~sudo gedit ~/.bashrc~~		sudo gedit hive-config.sh

	~~Copy and paste the following lines at end of the file~~

	~~# Set HIVE_HOME~~
	~~export HIVE_HOME="~~/usr/lib/hive/apache-hive-0.13.0-bin"
	~~PATH=$PATH:$HIVE_HOME~~/bin
	~~export PATH~~

	~~Setting HADOOP_PATH in HIVE config.sh~~

	~~Commands~~

	~~user@ubuntu:~$ cd~~ ~~/usr/lib/hive/apache-hive-0.13.0-bin/bin~~
	~~user@ubuntu:~$~~ sudo gedit hive-config.sh

	Go to the line where the following statements are written		Go to the line where the following statements are written

	# Allow alternate conf dir location.		# Allow alternate conf dir location.
	HIVE_CONF_DIR="${HIVE_CONF_DIR:-$HIVE_HOME/conf"		HIVE_CONF_DIR="${HIVE_CONF_DIR:-$HIVE_HOME/conf"
	export HIVE_CONF_DIR=$HIVE_CONF_DIR		export HIVE_CONF_DIR=$HIVE_CONF_DIR
	export HIVE_AUX_JARS_PATH=$HIVE_AUX_JARS_PATH		export HIVE_AUX_JARS_PATH=$HIVE_AUX_JARS_PATH

	Below this write the following		Below this write the following

	export HADOOP_HOME=/usr/local/hadoop (write the path where hadoop file is there)		export HADOOP_HOME=/usr/local/hadoop (write the path where hadoop file is there)

	Create Hive directories within HDFS		Create Hive directories within HDFS
Line 100:		Line 88:
	Open with text-editor and change the following property		Open with text-editor and change the following property

	<property>		<property>
	<name>hive.metastore.local</name>		<name>hive.metastore.local</name>
	<value>TRUE</value>		<value>TRUE</value>
	<description>controls whether to connect to remove metastore server or open a new metastore server in Hive Client JVM</description>		<description>controls whether to connect to remove metastore server or open a new metastore server in Hive Client JVM</description>
	</property>		</property>

	<property>		<property>
	<name>javax.jdo.option.ConnectionURL</name>		<name>javax.jdo.option.ConnectionURL</name>
	<value>jdbc:mysql://usr/lib/hive/apache-hive-0.13.0-bin/metastore_db? createDatabaseIfNotExist=true</value>		<value>jdbc:mysql://usr/lib/hive/apache-hive-0.13.0-bin/metastore_db? createDatabaseIfNotExist=true</value>
	<description>JDBC connect string for a JDBC metastore</description>		<description>JDBC connect string for a JDBC metastore</description>
	</property>		</property>

	<property>		<property>
	<name>javax.jdo.option.ConnectionDriverName</name>		<name>javax.jdo.option.ConnectionDriverName</name>
	<value>com.mysql.jdbc.Driver</value>		<value>com.mysql.jdbc.Driver</value>
	<description>Driver class name for a JDBC metastore~~</description>~~		<description>Driver class name for a JDBC metastore</description>
	~~</property>~~

	~~<property>~~
	~~<name>hive.metastore.warehouse.dir</name>~~
	~~<value>/usr/hive/warehouse</value>~~
	~~<description>location of default database for the warehouse~~</description>
	</property>		</property>

			<property>
			<name>hive.metastore.warehouse.dir</name>
			<value>/usr/hive/warehouse</value>
			<description>location of default database for the warehouse</description>
			</property>

	Writing a Script		Writing a Script

Onnowpurbo: New page: Sumber: http://doctuts.readthedocs.org/en/latest/hive.html HIVE INSTALLATION This section refers to the installation settings of Hive on a standalone system as well as on a system exis...

2015-11-13T22:31:48Z

New page: Sumber: http://doctuts.readthedocs.org/en/latest/hive.html HIVE INSTALLATION This section refers to the installation settings of Hive on a standalone system as well as on a system exis...

New page

Sumber: http://doctuts.readthedocs.org/en/latest/hive.html

HIVE INSTALLATION

This section refers to the installation settings of Hive on a standalone system as well as on a system existing as a node in a cluster.
INTRODUCTION

Apache Hive is a data warehouse infrastructure built on top of Hadoop for providing data summarization, query, and analysis. Apache Hive supports analysis of large datasets stored in Hadoop’s HDFS and compatible file systems such as Amazon S3 filesystem. It provides an SQL-like language called HiveQL(Hive Query Language) while maintaining full support for map/reduce.

Hive Installation
Installing HIVE:

Browse to the link: http://apache.claz.org/hive/stable/

Click the apache-hive-0.13.0-bin.tar.gz

Save and Extract it

Commands

user@ubuntu:~$ cd /usr/lib/
user@ubuntu:~$ sudo mkdir hive
user@ubuntu:~$ cd Downloads
user@ubuntu:~$ sudo mv apache-hive-0.13.0-bin /usr/lib/hive

Setting Hive environment variable:

Commands

user@ubuntu:~$ cd
user@ubuntu:~$ sudo gedit ~/.bashrc

Copy and paste the following lines at end of the file

# Set HIVE_HOME
export HIVE_HOME="/usr/lib/hive/apache-hive-0.13.0-bin"
PATH=$PATH:$HIVE_HOME/bin
export PATH

Setting HADOOP_PATH in HIVE config.sh

Commands

user@ubuntu:~$ cd /usr/lib/hive/apache-hive-0.13.0-bin/bin
user@ubuntu:~$ sudo gedit hive-config.sh

Go to the line where the following statements are written

# Allow alternate conf dir location.
HIVE_CONF_DIR="${HIVE_CONF_DIR:-$HIVE_HOME/conf"
export HIVE_CONF_DIR=$HIVE_CONF_DIR
export HIVE_AUX_JARS_PATH=$HIVE_AUX_JARS_PATH

Below this write the following

export HADOOP_HOME=/usr/local/hadoop (write the path where hadoop file is there)

Create Hive directories within HDFS

Command

user@ubuntu:~$ hadoop fs -mkdir /usr/hive/warehouse

Setting READ/WRITE permission for table

Command

user@ubuntu:~$ hadoop fs -chmod g+w /usr/hive/warehouse

HIVE launch

Command

user@ubuntu:~$ hive

Hive shell will prompt:
OUTPUT

Shell will look like

Logging initialized using configuration in jar:file:/usr/lib/hive/apache-hive-0.13.0-bin/lib/hive- common-0.13.0.jar!/hive-log4j.properties
hive>

Creating a database

Command

hive> create database mydb;

OUTPUT

OK
Time taken: 0.369 seconds
hive>

Configuring hive-site.xml:

Open with text-editor and change the following property

<property>
<name>hive.metastore.local</name>
<value>TRUE</value>
<description>controls whether to connect to remove metastore server or open a new metastore server in Hive Client JVM</description>
</property>

<property>
<name>javax.jdo.option.ConnectionURL</name>
<value>jdbc:mysql://usr/lib/hive/apache-hive-0.13.0-bin/metastore_db? createDatabaseIfNotExist=true</value>
<description>JDBC connect string for a JDBC metastore</description>
</property>

<property>
<name>javax.jdo.option.ConnectionDriverName</name>
<value>com.mysql.jdbc.Driver</value>
<description>Driver class name for a JDBC metastore</description>
</property>

<property>
<name>hive.metastore.warehouse.dir</name>
<value>/usr/hive/warehouse</value>
<description>location of default database for the warehouse</description>
</property>

Writing a Script

Open a new terminal (CTRL+ALT+T)

user@ubuntu:~$ sudo gedit sample.sql

create database sample;
use sample;
create table product(product int, productname string, price float)[row format delimited fields terminated by ',';]
describe product;

load data local inpath ‘/home/hduser/input_to_product.txt’ into table product

select * from product;

SAVE and CLOSE

user@ubuntu:~$ sudo gedit input_to_product.txt
user@ubuntu:~$ cd /usr/lib/hive/apache-hive-0.13.0-bin/ $ bin/hive -f /home/hduser/sample.sql

==Referensi==

* http://doctuts.readthedocs.org/en/latest/hive.html