Hadoop: Hive Instalasi: Difference between revisions

From OnnoCenterWiki
Jump to navigationJump to search
Onnowpurbo (talk | contribs)
69,477 edits
Onnowpurbo (talk | contribs)
69,477 edits
New page: Sumber: http://doctuts.readthedocs.org/en/latest/hive.html HIVE INSTALLATION This section refers to the installation settings of Hive on a standalone system as well as on a system exis...
 
Onnowpurbo (talk | contribs)
69,477 edits
 
(One intermediate revision by the same user not shown)
Line 2: Line 2:




==Sumber==


HIVE INSTALLATION
URL


This section refers to the installation settings of Hive on a standalone system as well as on a system existing as a node in a cluster.
http://apache.claz.org/hive/stable/
INTRODUCTION


    Apache Hive is a data warehouse infrastructure built on top of Hadoop for providing data summarization, query, and analysis. Apache Hive supports analysis of large datasets stored in Hadoop’s HDFS and compatible file systems such as Amazon S3 filesystem. It provides an SQL-like language called HiveQL(Hive Query Language) while maintaining full support for map/reduce.
Sebagai user biasa.
Download & Extract


Hive Installation
wget http://apache.claz.org/hive/stable/apache-hive-1.2.1-bin.tar.gz
Installing HIVE:
tar zxvf apache-hive-1.2.1-bin.tar.gz


    Browse to the link: http://apache.claz.org/hive/stable/
sudo mkdir /usr/lib/hive
sudo mv apache-hive-1.2.1-bin /usr/lib/hive


    Click the apache-hive-0.13.0-bin.tar.gz
==SETTING Hive environment variable==


    Save and Extract it
cd
sudo gedit  ~/.bashrc


        Commands
Copy paste kalimat ini di bagian bawah file .bashrc


        user@ubuntu:~$ cd /usr/lib/
  # Set HIVE_HOME
        user@ubuntu:~$  sudo mkdir hive
  export HIVE_HOME="/usr/lib/hive/apache-hive-1.2.1-bin"
        user@ubuntu:~$  cd Downloads
PATH=$PATH:$HIVE_HOME/bin
        user@ubuntu:~$  sudo mv apache-hive-0.13.0-bin /usr/lib/hive
export PATH


Setting Hive environment variable:
==SETTING HADOOP_PATH di HIVE config.sh==


Commands
cd  /usr/lib/hive/apache-hive-1.2.1-bin/bin
sudo gedit hive-config.sh


user@ubuntu:~$  cd
Cari kalimat di bawah ini
user@ubuntu:~$  sudo gedit  ~/.bashrc


Copy and paste the following lines at end of the file
# Allow alternate conf dir location.
HIVE_CONF_DIR="${HIVE_CONF_DIR:-$HIVE_HOME/conf"
export HIVE_CONF_DIR=$HIVE_CONF_DIR
export HIVE_AUX_JARS_PATH=$HIVE_AUX_JARS_PATH


# Set HIVE_HOME
Selanjutnya, tambahkan di bawah-nya
export HIVE_HOME="/usr/lib/hive/apache-hive-0.13.0-bin"
PATH=$PATH:$HIVE_HOME/bin
export PATH


Setting HADOOP_PATH in HIVE config.sh
  export HADOOP_HOME=/usr/local/hadoop
 
Commands
 
user@ubuntu:~$ cd /usr/lib/hive/apache-hive-0.13.0-bin/bin
user@ubuntu:~$ sudo gedit hive-config.sh
 
Go to the line where the following statements are written
 
# Allow alternate conf dir location.
HIVE_CONF_DIR="${HIVE_CONF_DIR:-$HIVE_HOME/conf"
export HIVE_CONF_DIR=$HIVE_CONF_DIR
export HIVE_AUX_JARS_PATH=$HIVE_AUX_JARS_PATH
 
Below this write the following
 
export HADOOP_HOME=/usr/local/hadoop   (write the path where hadoop file is there)


Create Hive directories within HDFS
Create Hive directories within HDFS
Line 70: Line 57:
user@ubuntu:~$  hadoop fs -chmod g+w /usr/hive/warehouse
user@ubuntu:~$  hadoop fs -chmod g+w /usr/hive/warehouse


HIVE launch
==Menjalankan HIVE==
 
Command
 
user@ubuntu:~$  hive


Hive shell will prompt:
Ketik
OUTPUT


Shell will look like
hive


Logging initialized using configuration in jar:file:/usr/lib/hive/apache-hive-0.13.0-bin/lib/hive- common-0.13.0.jar!/hive-log4j.properties
Akan keluar
hive>


Creating a database
Logging initialized using configuration in jar:file:/usr/lib/hive/apache-hive-0.13.0-bin/lib/hive- common-0.13.0.jar!/hive-log4j.properties
hive>


Command
==Membuat Database==


hive> create database mydb;
hive> create database mydb;


OUTPUT
Akan keluar


OK
OK
Time taken: 0.369 seconds
Time taken: 0.369 seconds
hive>
hive>


Configuring hive-site.xml:
==Konfigurasi hive-site.xml==


Open with text-editor and change the following property
Open with text-editor and change the following property


<property>
<property>
    <name>hive.metastore.local</name>
    <name>hive.metastore.local</name>
    <value>TRUE</value>
    <value>TRUE</value>
    <description>controls whether to connect to remove metastore server or open a new metastore server in Hive Client JVM</description>
    <description>controls whether to connect to remove metastore server or open a new metastore server in Hive Client JVM</description>
</property>
</property>
 
<property>
<property>
    <name>javax.jdo.option.ConnectionURL</name>
    <name>javax.jdo.option.ConnectionURL</name>
    <value>jdbc:mysql://usr/lib/hive/apache-hive-0.13.0-bin/metastore_db? createDatabaseIfNotExist=true</value>
    <value>jdbc:mysql://usr/lib/hive/apache-hive-0.13.0-bin/metastore_db? createDatabaseIfNotExist=true</value>
    <description>JDBC connect string for a JDBC metastore</description>
    <description>JDBC connect string for a JDBC metastore</description>
</property>
</property>
 
<property>
<property>
    <name>javax.jdo.option.ConnectionDriverName</name>
    <name>javax.jdo.option.ConnectionDriverName</name>
    <value>com.mysql.jdbc.Driver</value>
    <value>com.mysql.jdbc.Driver</value>
    <description>Driver class name for a JDBC metastore</description>
    <description>Driver class name for a JDBC metastore</description>
</property>
 
<property>
    <name>hive.metastore.warehouse.dir</name>
    <value>/usr/hive/warehouse</value>
    <description>location of default database for the warehouse</description>
  </property>
  </property>
<property>
    <name>hive.metastore.warehouse.dir</name>
    <value>/usr/hive/warehouse</value>
    <description>location of default database for the warehouse</description>
  </property>


Writing a Script
==Tulis Script==
 
Open a new terminal (CTRL+ALT+T)
 
user@ubuntu:~$      sudo gedit sample.sql
 
create database sample;
use sample;
create table product(product int, productname string, price float)[row format delimited fields terminated by ',';]
describe product;
 
load data local inpath ‘/home/hduser/input_to_product.txt’ into table product
 
select * from product;
 
SAVE and CLOSE
 
user@ubuntu:~$ sudo gedit input_to_product.txt
user@ubuntu:~$ cd /usr/lib/hive/apache-hive-0.13.0-bin/ $ bin/hive -f /home/hduser/sample.sql
 


Edit


sudo gedit sample.sql


Masukan


create database sample;
use sample;
create table product(product int, productname string, price float)[row format delimited fields terminated by ',';]
describe product;
load data local inpath ‘/home/hduser/input_to_product.txt’ into table product
select * from product;


Save


sudo gedit input_to_product.txt
cd /usr/lib/hive/apache-hive-0.13.0-bin/
bin/hive -f /home/hduser/sample.sql


==Referensi==
==Referensi==


* http://doctuts.readthedocs.org/en/latest/hive.html
* http://doctuts.readthedocs.org/en/latest/hive.html

Latest revision as of 01:16, 14 November 2015

Sumber: http://doctuts.readthedocs.org/en/latest/hive.html


Sumber

URL 

http://apache.claz.org/hive/stable/

Sebagai user biasa. Download & Extract 

wget http://apache.claz.org/hive/stable/apache-hive-1.2.1-bin.tar.gz
tar zxvf apache-hive-1.2.1-bin.tar.gz

sudo mkdir /usr/lib/hive
sudo mv apache-hive-1.2.1-bin /usr/lib/hive

SETTING Hive environment variable

cd
sudo gedit  ~/.bashrc

Copy paste kalimat ini di bagian bawah file .bashrc 

# Set HIVE_HOME
export HIVE_HOME="/usr/lib/hive/apache-hive-1.2.1-bin"
PATH=$PATH:$HIVE_HOME/bin
export PATH

SETTING HADOOP_PATH di HIVE config.sh

cd  /usr/lib/hive/apache-hive-1.2.1-bin/bin
sudo gedit hive-config.sh

Cari kalimat di bawah ini 

# Allow alternate conf dir location.
HIVE_CONF_DIR="${HIVE_CONF_DIR:-$HIVE_HOME/conf"
export HIVE_CONF_DIR=$HIVE_CONF_DIR
export HIVE_AUX_JARS_PATH=$HIVE_AUX_JARS_PATH

Selanjutnya, tambahkan di bawah-nya 

export HADOOP_HOME=/usr/local/hadoop

Create Hive directories within HDFS

Command

user@ubuntu:~$ hadoop fs -mkdir /usr/hive/warehouse

Setting READ/WRITE permission for table

Command

user@ubuntu:~$ hadoop fs -chmod g+w /usr/hive/warehouse

Menjalankan HIVE

Ketik 

hive

Akan keluar 

Logging initialized using configuration in jar:file:/usr/lib/hive/apache-hive-0.13.0-bin/lib/hive- common-0.13.0.jar!/hive-log4j.properties
hive>

Membuat Database

hive> create database mydb;

Akan keluar 

OK
Time taken: 0.369 seconds
hive>

Konfigurasi hive-site.xml

Open with text-editor and change the following property 

<property>
    <name>hive.metastore.local</name>
    <value>TRUE</value>
    <description>controls whether to connect to remove metastore server or open a new metastore server in Hive Client JVM</description>
</property>

<property>
    <name>javax.jdo.option.ConnectionURL</name>
    <value>jdbc:mysql://usr/lib/hive/apache-hive-0.13.0-bin/metastore_db? createDatabaseIfNotExist=true</value>
    <description>JDBC connect string for a JDBC metastore</description>
</property>

<property>
    <name>javax.jdo.option.ConnectionDriverName</name>
    <value>com.mysql.jdbc.Driver</value>
    <description>Driver class name for a JDBC metastore</description>
</property>

<property>
    <name>hive.metastore.warehouse.dir</name>
    <value>/usr/hive/warehouse</value>
    <description>location of default database for the warehouse</description>
 </property>

Tulis Script

Edit 

sudo gedit sample.sql

Masukan 

create database sample;
use sample;
create table product(product int, productname string, price float)[row format delimited fields terminated by ',';]
describe product;

load data local inpath ‘/home/hduser/input_to_product.txt’ into table product

select * from product;

Save 

sudo gedit input_to_product.txt
cd /usr/lib/hive/apache-hive-0.13.0-bin/
bin/hive -f /home/hduser/sample.sql

Referensi

Retrieved from "https://lms.onnocenter.or.id/wiki/index.php?title=Hadoop:_Hive_Instalasi&oldid=44980"