一、将Oozie安装包上传到Master节点并解压
$ sudo tar -zxvf oozie-4.1.0-cdh5.14.0.tar.gz -C /opt/cludera/
$ cd /opt/cludera/
$ sudo mv oozie-4.1.0-cdh5.14.0/ oozie
$ sudo chown -R hadoop ./oozie/
二、为oozie分配hdfs的权限
为oozie分配hdfs的权限,编辑集群上所有机器上的 core-site.xml文件 ,增加如下配置:
<property>
<name>hadoop.proxyuser.hadoop.hosts</name>
<value>*</value>
</property>
<property>
<name>hadoop.proxyuser.hadoop.groups</name>
<value>*</value>
</property>
三、解压oozie目录下的hadooplibs.tar.gz,获取hadoop的jar包,在hadooplibs目录下
$ cd oozie/
$ sudo tar -zxvf oozie-hadooplibs-4.1.0-cdh5.14.0.tar.gz
### 在解压的目录下./oozie-4.1.0-cdh5.14.0/hadooplibs
$ sudo mkdir libext
$ sudo cp -r oozie-4.1.0-cdh5.14.0/hadooplibs/hadooplib-2.6.0-cdh5.14.0.oozie-4.1.0-cdh5.14.0/* libext/
三、复制 ExtJS 2.2]到/oozie/libext目录下
下载ExtJs的包:http://archive.cloudera.com/gplextras/misc/ext-2.2.zip
$ sudo cp ext-2.2.zip /opt/cloudera/oozie/libext/
四、设置Oozie使用的数据库(MySQL)
这里提到的数据库是关系型数据库,用来存储Oozie的数据。Oozie自带一个Derby,不过Derby只能拿来做实验,在实际应用中不适用。这里我选择mysql作为Oozie的数据库,接下来就是创建Oozie用的数据库
mysql> create database oozie; #这个oozie数据库与oozie-site.xml中localhost:3306/oozie的hive对应,用来保存oozie元数据
mysql> grant all on *.* to oozie@localhost identified by 'oozie'; #将所有数据库的所有表的所有权限赋给oozie用户,后面的oozie是配置oozie-site.xml中配置的连接密码
mysql> grant all on *.* to oozie@% identified by 'oozie'; #将所有数据库的所有表的所有权限赋给oozie用户,后面的oozie是配置oozie-site.xml中配置的连接密码
mysql> flush privileges; #刷新mysql系统权限关系表
oozie-site.xml 配置mysql的连接属性:
<property>
<name>oozie.service.JPAService.jdbc.driver</name>
<value>com.mysql.jdbc.Driver</value>
</property>
<property>
<name>oozie.service.JPAService.jdbc.url</name>
<value>jdbc:mysql://localhost:3306/oozie</value>
</property>
<property>
<name>oozie.service.JPAService.jdbc.username</name>
<value>oozie</value>
</property>
<property>
<name>oozie.service.JPAService.jdbc.password</name>
<value>oozie</value>
</property>
将mysql的jar包添加到oozie的lib目录下:
$ sudo cp mysql-connector-java-5.1.45-bin.jar /opt/cloudera/oozie/lib
五、将jar包打包到Oozie的war包中
使用oozie-setup.sh 脚本配置Oozie,将所有的组件添加到libext/ 目录下。
打包成jar包的命令如下:
$ bin/oozie-setup.sh prepare-war
结果展示:
...(此处省略若干行)
INFO: Adding extension: /opt/cloudera/oozie/libext/jetty-util-6.1.26.cloudera.4.jar
INFO: Adding extension: /opt/cloudera/oozie/libext/jsr305-3.0.0.jar
INFO: Adding extension: /opt/cloudera/oozie/libext/leveldbjni-all-1.8.jar
INFO: Adding extension: /opt/cloudera/oozie/libext/log4j-1.2.17.jar
INFO: Adding extension: /opt/cloudera/oozie/libext/netty-3.10.5.Final.jar
INFO: Adding extension: /opt/cloudera/oozie/libext/netty-all-4.0.23.Final.jar
INFO: Adding extension: /opt/cloudera/oozie/libext/paranamer-2.3.jar
INFO: Adding extension: /opt/cloudera/oozie/libext/protobuf-java-2.5.0.jar
INFO: Adding extension: /opt/cloudera/oozie/libext/servlet-api-2.5.jar
INFO: Adding extension: /opt/cloudera/oozie/libext/slf4j-api-1.7.5.jar
INFO: Adding extension: /opt/cloudera/oozie/libext/slf4j-log4j12-1.7.5.jar
INFO: Adding extension: /opt/cloudera/oozie/libext/snappy-java-1.0.4.1.jar
INFO: Adding extension: /opt/cloudera/oozie/libext/stax-api-1.0-2.jar
INFO: Adding extension: /opt/cloudera/oozie/libext/xercesImpl-2.10.0.jar
INFO: Adding extension: /opt/cloudera/oozie/libext/xml-apis-1.4.01.jar
INFO: Adding extension: /opt/cloudera/oozie/libext/xmlenc-0.52.jar
INFO: Adding extension: /opt/cloudera/oozie/libext/xz-1.0.jar
INFO: Adding extension: /opt/cloudera/oozie/libext/zookeeper-3.4.5-cdh5.14.0.jar
New Oozie WAR file with added 'ExtJS library, JARs' at /opt/cloudera/oozie/oozie-server/webapps/oozie.war
INFO: Oozie is ready to be started
查看生成的warv包,由上面的输出信息可知,war包在目录:/opt/cloudera/oozie/oozie-server/webapps/oozie.war:
$ cd oozie-server/webapps/
$ ls
oozie.war ROOT
六、bin/oozie-setup.sh 使用讲解
bin/oozie-setup.sh prepare-war [-d directory] [-secure]
sharelib create -fs <FS_URI> [-locallib <PATH>]
sharelib upgrade -fs <FS_URI> [-locallib <PATH>]
db create|upgrade|postupgrade -run [-sqlfile <FILE>]
参数说明:
1. sharelib create -fs <FS_URI> [-locallib <PATH>]
上传新的sharelib到hdfs上
第一个参数:hdfs 的URI
第二个参数:指定lib目录,通常要指定一下。oozie-sharelib-4.1.0-cdh5.14.0-yarn.tar.gz(maprecude2版本,我们需指定的),oozie-sharelib-4.1.0-cdh5.14.0.tar.gz(maprecude1版本,我们不需要)
具体执行命令如下:
$ bin/oozie-setup.sh sharelib create -fs hdfs://Master:9000 -locallib
oozie-sharelib-4.1.0-cdh5.14.0-yarn.tar.gz
运行结果显示:
setting CATALINA_OPTS="$CATALINA_OPTS -Xmx1024m"
log4j:WARN No appenders could be found for logger (org.apache.hadoop.util.Shell).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/cloudera/oozie/libtools/slf4j-simple-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/cloudera/oozie/libtools/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/cloudera/oozie/libext/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.SimpleLoggerFactory]
the destination path for sharelib is: /user/hadoop/share/lib/lib_20180815192828
由上述结果可直,在hdfs的路径:/user/hadoop/share/lib/lib_20180815192828生成了jar包
2. sharelib upgrade -fs <FS_URI> [-locallib <PATH>]
Upgrade 命令已经被抛弃了, 可以使用create 命令去创建新版本的sharelibone
3. db create|upgrade|postupgrade -run [-sqlfile <FILE>]
创建Oozie数据库,可使用如下命令进行创建:
./bin/ooziedb.sh create -sqlfile oozie.sql -run DB Connection
运行结果如下:
setting CATALINA_OPTS="$CATALINA_OPTS -Xmx1024m"
Validate DB Connection
...
Oozie DB has been created for Oozie version '4.1.0-cdh5.14.0'
The SQL commands have been written to: oozie.sql
在Oozie目录下,会生成一个oozie.sql文件。
六、启动Oozie
$ ./bin/oozied.sh start
其实就是启动一个tomcat,jps一下会发现多了一个进程BootStrap.
在浏览器查看Oozie,输入网址:Master的IP:11000
至此,Oozie已经成功安装!