Difference between revisions of "Hadoop: Hive Instalasi"

From OnnoWiki
Jump to navigation Jump to search
(New page: Sumber: http://doctuts.readthedocs.org/en/latest/hive.html HIVE INSTALLATION This section refers to the installation settings of Hive on a standalone system as well as on a system exis...)
 
 
(One intermediate revision by the same user not shown)
Line 2: Line 2:
  
  
 +
==Sumber==
  
−
HIVE INSTALLATION
+
URL
  
−
This section refers to the installation settings of Hive on a standalone system as well as on a system existing as a node in a cluster.
+
http://apache.claz.org/hive/stable/
−
INTRODUCTION
 
  
−
    Apache Hive is a data warehouse infrastructure built on top of Hadoop for providing data summarization, query, and analysis. Apache Hive supports analysis of large datasets stored in Hadoop’s HDFS and compatible file systems such as Amazon S3 filesystem. It provides an SQL-like language called HiveQL(Hive Query Language) while maintaining full support for map/reduce.
+
Sebagai user biasa.
 +
Download & Extract
  
−
Hive Installation
+
wget http://apache.claz.org/hive/stable/apache-hive-1.2.1-bin.tar.gz
−
Installing HIVE:
+
tar zxvf apache-hive-1.2.1-bin.tar.gz
  
−
    Browse to the link: http://apache.claz.org/hive/stable/
+
sudo mkdir /usr/lib/hive
 +
sudo mv apache-hive-1.2.1-bin /usr/lib/hive
  
−
    Click the apache-hive-0.13.0-bin.tar.gz
+
==SETTING Hive environment variable==
  
−
    Save and Extract it
+
cd
 +
sudo gedit  ~/.bashrc
  
−
        Commands
+
Copy paste kalimat ini di bagian bawah file .bashrc
  
−
        user@ubuntu:~$ cd /usr/lib/
+
  # Set HIVE_HOME
−
        user@ubuntu:~$  sudo mkdir hive
+
  export HIVE_HOME="/usr/lib/hive/apache-hive-1.2.1-bin"
−
        user@ubuntu:~$  cd Downloads
+
PATH=$PATH:$HIVE_HOME/bin
−
        user@ubuntu:~$  sudo mv apache-hive-0.13.0-bin /usr/lib/hive
+
export PATH
  
−
Setting Hive environment variable:
+
==SETTING HADOOP_PATH di HIVE config.sh==
  
−
Commands
+
cd  /usr/lib/hive/apache-hive-1.2.1-bin/bin
 +
sudo gedit hive-config.sh
  
−
user@ubuntu:~$  cd
+
Cari kalimat di bawah ini
−
user@ubuntu:~$  sudo gedit  ~/.bashrc
 
  
−
Copy and paste the following lines at end of the file
+
# Allow alternate conf dir location.
 +
HIVE_CONF_DIR="${HIVE_CONF_DIR:-$HIVE_HOME/conf"
 +
export HIVE_CONF_DIR=$HIVE_CONF_DIR
 +
export HIVE_AUX_JARS_PATH=$HIVE_AUX_JARS_PATH
  
−
# Set HIVE_HOME
+
Selanjutnya, tambahkan di bawah-nya
−
export HIVE_HOME="/usr/lib/hive/apache-hive-0.13.0-bin"
 
−
PATH=$PATH:$HIVE_HOME/bin
 
−
export PATH
 
  
−
Setting HADOOP_PATH in HIVE config.sh
+
  export HADOOP_HOME=/usr/local/hadoop
−
 
 
−
Commands
 
−
 
 
−
user@ubuntu:~$ cd /usr/lib/hive/apache-hive-0.13.0-bin/bin
 
−
user@ubuntu:~$ sudo gedit hive-config.sh
 
−
 
 
−
Go to the line where the following statements are written
 
−
 
 
−
# Allow alternate conf dir location.
 
−
HIVE_CONF_DIR="${HIVE_CONF_DIR:-$HIVE_HOME/conf"
 
−
export HIVE_CONF_DIR=$HIVE_CONF_DIR
 
−
export HIVE_AUX_JARS_PATH=$HIVE_AUX_JARS_PATH
 
−
 
 
−
Below this write the following
 
−
 
 
−
export HADOOP_HOME=/usr/local/hadoop   (write the path where hadoop file is there)
 
  
 
Create Hive directories within HDFS
 
Create Hive directories within HDFS
Line 70: Line 57:
 
user@ubuntu:~$  hadoop fs -chmod g+w /usr/hive/warehouse
 
user@ubuntu:~$  hadoop fs -chmod g+w /usr/hive/warehouse
  
−
HIVE launch
+
==Menjalankan HIVE==
−
 
 
−
Command
 
−
 
 
−
user@ubuntu:~$  hive
 
  
−
Hive shell will prompt:
+
Ketik
−
OUTPUT
 
  
−
Shell will look like
+
hive
  
−
Logging initialized using configuration in jar:file:/usr/lib/hive/apache-hive-0.13.0-bin/lib/hive- common-0.13.0.jar!/hive-log4j.properties
+
Akan keluar
−
hive>
 
  
−
Creating a database
+
Logging initialized using configuration in jar:file:/usr/lib/hive/apache-hive-0.13.0-bin/lib/hive- common-0.13.0.jar!/hive-log4j.properties
 +
hive>
  
−
Command
+
==Membuat Database==
  
−
hive> create database mydb;
+
hive> create database mydb;
  
−
OUTPUT
+
Akan keluar
  
−
OK
+
OK
−
Time taken: 0.369 seconds
+
Time taken: 0.369 seconds
−
hive>
+
hive>
  
−
Configuring hive-site.xml:
+
==Konfigurasi hive-site.xml==
  
 
Open with text-editor and change the following property
 
Open with text-editor and change the following property
  
−
<property>
+
<property>
−
    <name>hive.metastore.local</name>
+
    <name>hive.metastore.local</name>
−
    <value>TRUE</value>
+
    <value>TRUE</value>
−
    <description>controls whether to connect to remove metastore server or open a new metastore server in Hive Client JVM</description>
+
    <description>controls whether to connect to remove metastore server or open a new metastore server in Hive Client JVM</description>
−
</property>
+
</property>
−
 
+
−
<property>
+
<property>
−
    <name>javax.jdo.option.ConnectionURL</name>
+
    <name>javax.jdo.option.ConnectionURL</name>
−
    <value>jdbc:mysql://usr/lib/hive/apache-hive-0.13.0-bin/metastore_db? createDatabaseIfNotExist=true</value>
+
    <value>jdbc:mysql://usr/lib/hive/apache-hive-0.13.0-bin/metastore_db? createDatabaseIfNotExist=true</value>
−
    <description>JDBC connect string for a JDBC metastore</description>
+
    <description>JDBC connect string for a JDBC metastore</description>
−
</property>
+
</property>
−
 
+
−
<property>
+
<property>
−
    <name>javax.jdo.option.ConnectionDriverName</name>
+
    <name>javax.jdo.option.ConnectionDriverName</name>
−
    <value>com.mysql.jdbc.Driver</value>
+
    <value>com.mysql.jdbc.Driver</value>
−
    <description>Driver class name for a JDBC metastore</description>
+
    <description>Driver class name for a JDBC metastore</description>
−
</property>
 
−
 
 
−
<property>
 
−
    <name>hive.metastore.warehouse.dir</name>
 
−
    <value>/usr/hive/warehouse</value>
 
−
    <description>location of default database for the warehouse</description>
 
 
  </property>
 
  </property>
 +
 +
<property>
 +
    <name>hive.metastore.warehouse.dir</name>
 +
    <value>/usr/hive/warehouse</value>
 +
    <description>location of default database for the warehouse</description>
 +
  </property>
  
−
Writing a Script
+
==Tulis Script==
−
 
 
−
Open a new terminal (CTRL+ALT+T)
 
−
 
 
−
user@ubuntu:~$      sudo gedit sample.sql
 
−
 
 
−
create database sample;
 
−
use sample;
 
−
create table product(product int, productname string, price float)[row format delimited fields terminated by ',';]
 
−
describe product;
 
−
 
 
−
load data local inpath ‘/home/hduser/input_to_product.txt’ into table product
 
−
 
 
−
select * from product;
 
−
 
 
−
SAVE and CLOSE
 
−
 
 
−
user@ubuntu:~$ sudo gedit input_to_product.txt
 
−
user@ubuntu:~$ cd /usr/lib/hive/apache-hive-0.13.0-bin/ $ bin/hive -f /home/hduser/sample.sql
 
−
 
 
  
 +
Edit
  
 +
sudo gedit sample.sql
  
 +
Masukan
  
 +
create database sample;
 +
use sample;
 +
create table product(product int, productname string, price float)[row format delimited fields terminated by ',';]
 +
describe product;
 +
 +
load data local inpath ‘/home/hduser/input_to_product.txt’ into table product
 +
 +
select * from product;
  
 +
Save
  
 +
sudo gedit input_to_product.txt
 +
cd /usr/lib/hive/apache-hive-0.13.0-bin/
 +
bin/hive -f /home/hduser/sample.sql
  
 
==Referensi==
 
==Referensi==
  
 
* http://doctuts.readthedocs.org/en/latest/hive.html
 
* http://doctuts.readthedocs.org/en/latest/hive.html

Latest revision as of 08:16, 14 November 2015

Sumber: http://doctuts.readthedocs.org/en/latest/hive.html


Sumber

URL 

http://apache.claz.org/hive/stable/

Sebagai user biasa. Download & Extract 

wget http://apache.claz.org/hive/stable/apache-hive-1.2.1-bin.tar.gz
tar zxvf apache-hive-1.2.1-bin.tar.gz

sudo mkdir /usr/lib/hive
sudo mv apache-hive-1.2.1-bin /usr/lib/hive

SETTING Hive environment variable

cd
sudo gedit  ~/.bashrc

Copy paste kalimat ini di bagian bawah file .bashrc 

# Set HIVE_HOME
export HIVE_HOME="/usr/lib/hive/apache-hive-1.2.1-bin"
PATH=$PATH:$HIVE_HOME/bin
export PATH

SETTING HADOOP_PATH di HIVE config.sh

cd  /usr/lib/hive/apache-hive-1.2.1-bin/bin
sudo gedit hive-config.sh

Cari kalimat di bawah ini 

# Allow alternate conf dir location.
HIVE_CONF_DIR="${HIVE_CONF_DIR:-$HIVE_HOME/conf"
export HIVE_CONF_DIR=$HIVE_CONF_DIR
export HIVE_AUX_JARS_PATH=$HIVE_AUX_JARS_PATH

Selanjutnya, tambahkan di bawah-nya 

export HADOOP_HOME=/usr/local/hadoop

Create Hive directories within HDFS

Command

user@ubuntu:~$ hadoop fs -mkdir /usr/hive/warehouse

Setting READ/WRITE permission for table

Command

user@ubuntu:~$ hadoop fs -chmod g+w /usr/hive/warehouse

Menjalankan HIVE

Ketik 

hive

Akan keluar 

Logging initialized using configuration in jar:file:/usr/lib/hive/apache-hive-0.13.0-bin/lib/hive- common-0.13.0.jar!/hive-log4j.properties
hive>

Membuat Database

hive> create database mydb;

Akan keluar 

OK
Time taken: 0.369 seconds
hive>

Konfigurasi hive-site.xml

Open with text-editor and change the following property 

<property>
    <name>hive.metastore.local</name>
    <value>TRUE</value>
    <description>controls whether to connect to remove metastore server or open a new metastore server in Hive Client JVM</description>
</property>

<property>
    <name>javax.jdo.option.ConnectionURL</name>
    <value>jdbc:mysql://usr/lib/hive/apache-hive-0.13.0-bin/metastore_db? createDatabaseIfNotExist=true</value>
    <description>JDBC connect string for a JDBC metastore</description>
</property>

<property>
    <name>javax.jdo.option.ConnectionDriverName</name>
    <value>com.mysql.jdbc.Driver</value>
    <description>Driver class name for a JDBC metastore</description>
</property>

<property>
    <name>hive.metastore.warehouse.dir</name>
    <value>/usr/hive/warehouse</value>
    <description>location of default database for the warehouse</description>
 </property>

Tulis Script

Edit 

sudo gedit sample.sql

Masukan 

create database sample;
use sample;
create table product(product int, productname string, price float)[row format delimited fields terminated by ',';]
describe product;

load data local inpath ‘/home/hduser/input_to_product.txt’ into table product

select * from product;

Save 

sudo gedit input_to_product.txt
cd /usr/lib/hive/apache-hive-0.13.0-bin/
bin/hive -f /home/hduser/sample.sql

Referensi

Retrieved from "https://onnocenter.or.id/wiki/index.php?title=Hadoop:_Hive_Instalasi&oldid=44980"