어제의 oozie 설치 삽질기 에 많은 분의 조언을 주셨다..
sstrato.kong 님께서 제일 먼저 3.3.0사용하시라고 조언을..
3.0.1(https://github.com/yahoo/oozie) 버전이 최신인줄 알고 받았더니..
알고보니 3.3.1(https://github.com/apache/oozie)이 최신..OTL
그루터의 김형준 수석님께서도 깨알같은 조언을..
oozie-site.xml 설정에서 조금 이상한게 mysql 드라이버 설정 및 서버 설정을 oozie.service.JPAService.jdbc.driver, oozie.service.JPAService.jdbc.url 이렇게 JPA proeprty를 이용해야 하는 걸로 알고 있습니다. 저도 그렇게 설정해서 잘 쓰고 있습니다. ㅋㅋㅋ mysql 접속해서 use oozie; 하신 다음에 테이블 잘 만들어 있는지 확인해보세요. 그리고 클라우몬은 oozie 서버 사용하지 않습니다. 클라우몬 자체가 oozie 서버 역할을 수행합니다.
그리고 Dani 님께서는 직접 포스팅까지
http://blog.geekple.com/2013/02/15/oozie-install/
너무너무 감사합니다.
덕분에 성공적으로 설치하였습니다.
1. 우선, 서버에 maven 이 설치안되어 있어서 설치.
]$ cd $OOZIE_HOME/util ]$ wget http://mirror.apache-kr.org/maven/maven-3/3.0.4/binaries/apache-maven-3.0.4-bin.tar.gz ]$ tar xvzf apache-maven-3.0.4-bin.tar.gz ]$ export PATH=$PATH:$OOZIE_HOME/bin:/home/oozie/util/apache-maven-3.0.4/bin
2. oozie 최신버전 다운로드
]$ wget http://apache.mirror.cdnetworks.com/oozie/3.3.1/oozie-3.3.1.tar.gz ]$ tar xvzf oozie-3.3.1.tar.gz
3. oozie를 maven 으로 build
]$ cd $OOZIE_HOME/util/oozie-3.3.1 ]$ ./bin/mkdistro.sh -DskipTests .... [INFO] ------------------------------------------------------------------------ [INFO] Reactor Summary: [INFO] [INFO] Apache Oozie Main ................................. SUCCESS [0.868s] [INFO] Apache Oozie Client ............................... SUCCESS [1:17.524s] [INFO] Apache Oozie Hadoop 1.0.1.oozie-3.3.1 ............. SUCCESS [11.251s] [INFO] Apache Oozie Hadoop Distcp 1.0.1.oozie-3.3.1 ...... SUCCESS [2.522s] [INFO] Apache Oozie Hadoop 1.0.1.oozie-3.3.1 Test ........ SUCCESS [14.492s] [INFO] Apache Oozie Hadoop 2.0.2-alpha.oozie-3.3.1 ....... SUCCESS [59.832s] [INFO] Apache Oozie Hadoop 2.0.2-alpha.oozie-3.3.1 Test .. SUCCESS [10.649s] [INFO] Apache Oozie Hadoop Distcp 2.0.2-alpha.oozie-3.3.1 SUCCESS [2.391s] [INFO] Apache Oozie Hadoop Libs .......................... SUCCESS [8.645s] [INFO] Apache Oozie Hbase 0.94.2.oozie-3.3.1 ............. SUCCESS [9.546s] [INFO] Apache Oozie Hbase Libs ........................... SUCCESS [0.845s] [INFO] Apache Oozie Share Lib Pig ........................ SUCCESS [1:00.229s] [INFO] Apache Oozie Share Lib Streaming .................. SUCCESS [0.664s] [INFO] Apache Oozie Share Lib Distcp ..................... SUCCESS [0.288s] [INFO] Apache Oozie Share Lib Hive ....................... SUCCESS [34.434s] [INFO] Apache Oozie Share Lib Sqoop ...................... SUCCESS [3.378s] [INFO] Apache Oozie Share Lib Oozie ...................... SUCCESS [0.236s] [INFO] Apache Oozie Core ................................. SUCCESS [1:03.470s] [INFO] Apache Oozie Docs ................................. SUCCESS [1:00.399s] [INFO] Apache Oozie WebApp ............................... SUCCESS [2:30.123s] [INFO] Apache Oozie Examples ............................. SUCCESS [14.067s] [INFO] Apache Oozie Share Lib ............................ SUCCESS [10.419s] [INFO] Apache Oozie Tools ................................ SUCCESS [17.553s] [INFO] Apache Oozie Distro ............................... SUCCESS [29.777s] [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 10:44.304s [INFO] Finished at: Thu Feb 15 08:47:17 KST 2013 [INFO] Final Memory: 112M/427M [INFO] ------------------------------------------------------------------------ Oozie distro created, DATE[2013.02.15-07:36:31GMT] VC-REV[unavailable], available at [/home/oozie/oozie-3.3.1/distro/target] ]$ cp $OOZIE_HOME/util/oozie-3.3.1/distro/target/oozie-3.3.1-distro.tar.gz $OOZIE_HOME ]$ cd $OOZIE_HOME ]$ tar xvzf oozie-3.3.1-distro.tar.gz ]$ ln -s oozie-3.3.1 oozie
4. extjs 를 war 에 포함
]$ cd $OOZIE_HOME/oozie ]$ wget http://dev.sencha.com/deploy/ext-2.2.1.zip && unzip ext-2.2.1.zip -d ext-2.2.1 ]$ ./bin/oozie-setup.sh -hadoop 0.20.2 $HADOOP_HOME -extjs ext-2.2.1 -jars "$HADOOP_HOME/lib/commons-configuration-1.6.jar:$HADOOP_HOME/lib/jackson-core-asl-1.0.1.jar:$HADOOP_HOME/lib/jackson-mapper-asl-1.0.1.jar:/home/oozie/util/mysql-connector-java-5.1.20.jar" setting CATALINA_OPTS="$CATALINA_OPTS -Xmx1024m" Injecting following Hadoop JARs /home/hadoop/hadoop/hadoop-core-1.0.0.jar New Oozie WAR file with added 'Hadoop JARs, ExtJS library, JARs' at /home/oozie/oozie/oozie-server/webapps/oozie.war INFO: Oozie is ready to be started ]$
여기서 -hadoop 옵션에서 우리가 사용하는 버전이 1.0.0 이라서 아래와 같이 실행하면 에러가 났는데..
./bin/oozie-setup.sh -hadoop 1.0.0 $HADOOP_HOME -extjs ext-2.2.1 -jars "$HADOOP_HOME/lib/commons-configuration-1.6.jar:$HADOOP_HOME/lib/jackson-core-asl-1.0.1.jar:$HADOOP_HOME/lib/jackson-mapper-asl-1.0.1.jar:/home/oozie/util/mysql-connector-java-5.1.20.jar" Exiting: Unsupported Hadoop version '', supported versions: 0.20.1, 0.20.2, 0.20.104, 0.20.200, 0.23.x and 2.x
Dani님의 포스팅을 보고 이유를 알았다..ㅠ
하둡버전이 1.0.4를 사용하지만 설치시에 버전은 0.20.2로 작성하시면 됩니다. 하둡 버전에 따라서 core 또는 common 등으로 나뉘기때문에 그걸 구분하기 위한 단순한 정보에 불과합니다.
5. “oozie-site.xml” 수정
...
<property>
<name>oozie.service.JPAService.create.db.schema</name>
<value>true</value>
<description>
Creates Oozie DB.
If set to true, it creates the DB schema if it does not exist. If the DB schema exists is a NOP.
If set to false, it does not create the DB schema. If the DB schema does not exist it fails start up.
</description>
</property>
<property>
<name>oozie.service.JPAService.jdbc.driver</name>
<value>com.mysql.jdbc.Driver</value>
<description>
JDBC driver class.
</description>
</property>
<property>
<name>oozie.service.JPAService.jdbc.url</name>
<value>jdbc:mysql://host:3306/oozie</value>
<description>
JDBC URL.
</description>
</property>
<property>
<name>oozie.service.JPAService.jdbc.username</name>
<value>oozie</value>
<description>
DB user name.
</description>
</property>
<property>
<name>oozie.service.JPAService.jdbc.password</name>
<value>passwd</value>
<description>
DB user password.
IMPORTANT: if password is emtpy leave a 1 space string, the service trims the value,
if empty Configuration assumes it is NULL.
</description>
</property>
...
6. mysql 에 oozie DB 생성은 지난 oozie 설치 삽질기때 생성해두었으므로 pass
7. oozie-start
]$ ./bin/oozie-start.sh Setting OOZIE_HOME: /home/oozie/oozie Setting OOZIE_CONFIG: /home/oozie/oozie/conf Sourcing: /home/oozie/oozie/conf/oozie-env.sh setting CATALINA_OPTS="$CATALINA_OPTS -Xmx1024m" Setting OOZIE_CONFIG_FILE: oozie-site.xml Setting OOZIE_DATA: /home/oozie/oozie/data Setting OOZIE_LOG: /home/oozie/oozie/logs Setting OOZIE_LOG4J_FILE: oozie-log4j.properties Setting OOZIE_LOG4J_RELOAD: 10 Setting OOZIE_HTTP_HOSTNAME: host Setting OOZIE_HTTP_PORT: 11000 Setting OOZIE_ADMIN_PORT: 11001 Setting OOZIE_BASE_URL: http://host:11000/oozie Setting CATALINA_BASE: /home/oozie/oozie/oozie-server Setting CATALINA_OUT: /home/oozie/oozie/logs/catalina.out Setting CATALINA_PID: /home/oozie/oozie/oozie-server/temp/oozie.pid Using CATALINA_OPTS: -Xmx1024m -Dderby.stream.error.file=/home/oozie/oozie/logs/derby.log Adding to CATALINA_OPTS: -Doozie.home.dir=/home/oozie/oozie -Doozie.config.dir=/home/oozie/oozie/conf -Doozie.log.dir=/home/oozie/oozie/logs -Doozie.data.dir=/home/oozie/oozie/data -Doozie.config.file=oozie-site.xml -Doozie.log4j.file=oozie-log4j.properties -Doozie.log4j.reload=10 -Doozie.http.hostname=host -Doozie.admin.port=11001 -Doozie.http.port=11000 -Doozie.base.url=http://host:11000/oozie Using CATALINA_BASE: /home/oozie/oozie/oozie-server Using CATALINA_HOME: /home/oozie/oozie/oozie-server Using CATALINA_TMPDIR: /home/oozie/oozie/oozie-server/temp Using JRE_HOME: /usr Using CLASSPATH: /home/oozie/oozie/oozie-server/bin/bootstrap.jar Using CATALINA_PID: /home/oozie/oozie/oozie-server/temp/oozie.pid ]$ ]$ ./bin/oozie admin -oozie http://host:11000/oozie -status System mode: NORMAL ]$
7. 웹 페이지 접속!
One Response to oozie 설치 성공!