content top

Be a Extraordinary Man !!

“One machine can do the work of fifty ordinary men. No machine can do the work of one extraordinary man.”

-Elbert Hubbard, The Roycroft Dictionary and Book of Epigrams, 1923

Read More

Loading Data in Hive

Loading Data in Hive

Apache Hive supports analysis of large datasets stored in Hadoop-compatible file systems such as Amazon S3 filesystem. It provides an SQL-like language called HiveQL while maintaining full support for map/reduce.hive To accelerate queries, it provides indexes, including bitmap indexes.

For installing hadoop please follow our post here

Loading data from flat files to Hive Tables :

 hive> LOAD DATA LOCAL INPATH './examples/files/out_001223.txt' OVERWRITE INTO TABLE sample2;

The keyword ‘overwrite’ signifies that existing data in the table is deleted. If the ‘overwrite’ keyword is omitted, data files are appended to existing data sets.

 hive> LOAD DATA LOCAL INPATH './examples/files/out_23.txt' OVERWRITE INTO TABLE sample PARTITION (ds='2008-08-15');
 hive> LOAD DATA LOCAL INPATH './examples/files/out_122.txt' OVERWRITE INTO TABLE sample PARTITION (ds='2008-08-08');

The two LOAD statements above load data into two different partitions of the table sample. Table sample must be created as partitioned by the key ds for this to succeed.


Basics of Hive QL

Creating Hive tables :

 hive> CREATE TABLE sample ( col1 INT, col2 STRING);
 hive> CREATE TABLE sample2 ( col1 INT, col2 STRING) PARTITIONED BY (ds STRING);

Browsing through them :

hive> SHOW TABLES '.*e';
hive> DESCRIBE sample2;

Altering tables :

 hive> ALTER TABLE sample ADD COLUMNS (new_col INT);
 hive> ALTER TABLE sample2 ADD COLUMNS (new_col2 INT COMMENT 'a comment');
 hive> ALTER TABLE sample RENAME TO sample1;

Dropping tables :

 hive> DROP TABLE pokes;
Read More

Installing HIVE

Installing HIVE

To look at the overview of HIVE, please click here

STEP 1 : Downloadhive

Download stable release from apache hive

STEP 2 : Unzip

Copy hive tar file in required folder and run the following command to extract

tar -xzvf hive-x.y.z.tar.gz

STEP 3 : Setting hive path

  • export HIVE_HOME=/opt/hive
  • export PATH=$PATH:$HIVE_HOME/bin

STEP 4 : logout from the user and login with the same user again. check whether the settings are applied or not by using echo $HIVE_HOME

STEP 5 : Check whether hadoop is running or not, if not start the hadoop.

STEP 6 : Run following four commands

$HADOOP_HOME/bin/hadoop fs -mkdir /tmp
$HADOOP_HOME/bin/hadoop fs -mkdir /user/hive/warehouse
$HADOOP_HOME/bin/hadoop fs -chmod g+w /tmp
$HADOOP_HOME/bin/hadoop fs -chmod g+w /user/hive/warehouse

STEP 7 : Running HIVE


DONE : Now your play ground is ready.

NOTE : Hive runs on top of Hadoop. So install hadoop before you start installing HIVE. And also ensure all  required ports open.

Read More

Introducing Apache Hadoop: The Modern Data Operating System

free robux

Read More
content top
%d bloggers like this:
shared on