apache hcatalog 설치

홈페이지 : http://incubator.apache.org/hcatalog/
설치환경 : Ubuntu 64bit
hadoop-1.0.0, hive-0.8.1, mysql 5.1.63, zookeeper-3.3.4, hcatalog-0.4.0, hbase-0.92.1

hcatalog 설치

$ wget http://mirror.apache-kr.org/incubator/hcatalog/hcatalog-0.4.0-incubating/hcatalog-src-0.4.0-incubating.tar.gz
$ tar xvzf hcatalog-src-0.4.0-incubating.tar.gz
$ cd hcatalog-src-0.4.0-incubating
$ ant -Dhcatalog.version=0.4.0 tar
$ cp /home/hcatalog/hcatalog-src-0.4.0-incubating/build/hcatalog-0.4.0.tar.gz /home/hcatalog/
$ cd /home/hcatalog/
$ tar xvzf hcatalog-0.4.0.tar.gz
$ ln -s hcatalog-0.4.0/ hcatalog

hive는 이미 설치되어 있으므로 생략 한다.

hcatalog를 서버로 구동하기 위한 스크립트 실행했더니 아래와 같은 사용법이 나왔다.

$ ./share/hcatalog/scripts/hcat_server_install.sh
Usage: ./share/hcatalog/scripts/hcat_server_install.sh -r root -d dbroot -h hadoop_home -p server_port
root is the directory where you like to install the HCatalog server
/usr/local/hcat is suggested.
dbroot is the directory where your mysql connector jar is located.
hadoop_home is the directory of your Hadoop installation.
server_port is the listening port of the HCatalog server
All paths must be absolute

입력 파라미터는 아래와 같다.
dbroot : mysql connector jar 가 있는 위치
hadoop_home : Hadoop 이 설치된 위치
server_port : HCatalog server가 사용할 port

아래의 값을 입력하고 다시 구동..

$ ./share/hcatalog/scripts/hcat_server_install.sh -r /home/hcatalog/hcatalog/ -d /home/hcatalog/hcatalog/lib -h /home/hadoop/hadoop -p 10001
Your root directory must be readable, writable, and executable by its owner.

root 디렉토리에 rwx권한이 필요하다고 합니다.

$ sudo chmod 777 /home/hcatalog/hcatalog-0.4.0/

다시 시도!

$ ./share/hcatalog/scripts/hcat_server_install.sh -r /home/hcatalog/hcatalog/ -d /home/hcatalog/hcatalog/lib -h /home/hadoop/hadoop -p 10001
Installing into [/home/hcatalog/hcatalog/]
Installation successful

환경변수 설정

$ export HCAT_HOME=/home/hcatalog/hcatalog

hive설정 파일 세팅

$ sudo vi /home/hive/hive/conf/hive-site.xml
< ?xml version="1.0"?>
< ?xml-stylesheet type="text/xsl" href="configuration.xsl"?>





  hive.metastore.execute.setugi
  true
  In unsecure mode, setting this property to true will cause the metastore to execute DFS operations using the client's reported user and group permissions. Note that this property must be set on both the client and server sides. Further note that its best effort. If client sets its to true and server sets it to false, client setting will be ignored.



  hive.metastore.local
  false
  controls whether to connect to remove metastore server or open a new metastore server in Hive Client JVM



  javax.jdo.option.ConnectionURL
  jdbc:mysql://127.0.0.1:3306/hive?createDatabaseIfNotExist=true



  javax.jdo.option.ConnectionDriverName
  com.mysql.jdbc.Driver



  javax.jdo.option.ConnectionUserName
  user



  javax.jdo.option.ConnectionPassword
  password



  hive.metastore.warehouse.dir
  WAREHOUSE_DIR
  location of default database for the warehouse



  hive.metastore.sasl.enabled
    SASL_ENABLED
    If true, the metastore thrift interface will be secured with SASL. Clients must authenticate with Kerberos.



  hive.metastore.kerberos.keytab.file
  KEYTAB_PATH
  The path to the Kerberos Keytab file containing the metastore thrift server's service principal.



  hive.metastore.kerberos.principal
  KERBEROS_PRINCIPAL
  The service principal for the metastore thrift server. The special string _HOST will be replaced automatically with the correct host name.



  hive.metastore.cache.pinobjtypes
  Table,Database,Type,FieldSchema,Order
  List of comma separated metastore object types that should be pinned in the cache



  hive.metastore.uris
  thrift://127.0.0.1:10001
  URI for client to contact metastore server



  hive.semantic.analyzer.factory.impl
  org.apache.hcatalog.cli.HCatSemanticAnalyzerFactory
  controls which SemanticAnalyzerFactory implemenation class is used by CLI



  hadoop.clientside.fs.operations
  true
  FS operations are owned by client



  hive.metastore.client.socket.timeout
  60
  MetaStore Client socket timeout in seconds



  hive.security.authorization.enabled
  true
  enable or disable the hive client authorization



  hive.security.authorization.manager
  org.apache.hcatalog.security.StorageDelegationAuthorizationProvider
  the hive client authorization manager class name.
  The user defined authorization class should implement interface org.apache.hadoop.hive.ql.security.authorization.HiveAuthorizationProvider.
  HCatalog uses a model, where authorization checks are delegated to the storage layer (hdfs, hbase, ...).
  

편리한 사용을 위해 hive의 설정 파일과 실행파일을 hcatalog에 심볼릭링크.

$ sudo ln -s /home/hive/hive/conf/hive-site.xml /home/hcatalog/hcatalog/etc/hcatalog/hive-site.xml
$ sudo ln -s /home/hive/hive/bin/hive /home/hcatalog/hcatalog/bin/hive

hcatalog 서버 구동!

$ ./sbin/hcat_server.sh start
Hadoop not found.

흠.. Hadoop이 없데서 bashrc에 hadoop과 hive path추가

$ vi /home/hcatalog/.bashrc
...
export HIVE_HOME=/home/hive/hive
export HADOOP_HOME=/home/hadoop/hadoop

다시 구동!

$ ./sbin/hcat_server.sh start
Started metastore server init, testing if initialized correctly...
$ ./sbin/hcat_server.sh: 줄 89: /home/hcatalog/hcatalog/sbin/../var/log/hcat.out: 그런 파일이나 디렉터리가 없습니다

흠냐.. 이번엔 로그파일을 못만든다네.. 디렉토리 생성

$ mkdir /home/hcatalog/hcatalog/var
$ mkdir /home/hcatalog/hcatalog/var/log

다시 구동!

$ ./sbin/hcat_server.sh start
Started metastore server init, testing if initialized correctly...
Metastore initialized successfully on port[10001].

설치 성공!  우후훗! (도움을 주신 GeekDani 님 감사 ^^)

참고 : http://mixellaneous.tistory.com/1123