Sqoop Installation Guide

SQOOP INSTALLATIONGUIDE

Lecturer : Prof. Kyungbaek Kim

Presenter : Zubair Amjad

Sqoop

http://www.sas.com/content/dam/SAS/sv_se/image/logo2/hadoop_elephant.png

Requirements

•Hadoop

•Sqoop

•MySQL Server

•JDBC driver

Environment

•Ubuntu 12.04

•Hadoop 1.0.3

•Sqoop 1.4.4

Hadoop Fully Distributed Mode

Sqoop Installation

•Download the sqoop-1.4.4.bin_hadoop-1.0.0.tar.gz file

•ww.apache.org/dyn/closer.cgl/sqoop/1.4.4

•Unzip the tar file

•sudo tar -zxvf sqoop-1.4.4.bin_hadoop-1.0.0.tar.gz

•Move sqoop-1.4.4.bin_hadoop-1.0.0 to sqoop

•sudo mv sqoop 1.4.4.bin_hadoop-1.0.0 sqoop

•Create a directory sqoop in usr/lib

•sudo mkdir /usr/lib/sqoop

•Go to the folder sqoop and copy files to /usr/lib/sqoop

•sudo mv ./* /usr/lib/sqoop

•Go to root directory and open bashrc file

•sudo gedit ~/.bashrc

•Add SQOOP_HOME variable

•export SQOOP_HOME=/usr/lib/sqoop

•export PATH=$PATH:$SQOOP_HOME/bin

•Check sqoop installation

•sqoop version

MySQL Server

•Install mysql server

•sudo apt-get install mysql-server

•Give appropriate username and password

•Download mysql-connector-java-5.1.28-bin.jar and copyto /usr/lib/sqoop/lib

•cp mysql-connnectpr-java-5.1.28-bin.jar /usr/lib/sqoop/lib/

•Login to mysql server

•mysql -u root –p

Create Database

•Create a database

•create database temp_db;

•show databases;

•Add a table in the database

•use temp_db;

•CREATE TABLE employee (id INT, name VARCHAR(20), emailVARCHAR(20));

•Insert some rows using INSERT INTO

•Grant privileges to root user

•grant all privileges on *.* to root@IP IDENTIFIED BY ‘root’WITH GRANT OPTION;

Create Database

Create Table

Insert rows in table

Grant Permissions

Sqoop Import

•Import the table from temp_db to HDFS

•sqoop import -connect jdbc:mysql://localhost:3306/temp_db -username root -P -table employee –target-dir /sqoopOut –m 1

Table Import

HDFS

Sqoop Export

•Export data from HDFS to database table

•Create a table in data base

•CREATE TABLE stud1 (id INT, name VARCHAR(20));

•Export data in file stud to database

•sqoop export --connect jdbc:mysql://192.168.34.130/temp_db --username root -P --table stud1 -m 1 --export-dir /sqoopexport/stud

Export

Exported Table

Thank you