搭建Hadoop伪分布式环境-作业001
搭建Linux环境
1 配置VM###
2 配置虚拟机磁盘
3 配置CenOS
4 配置磁盘，选择YES
5 配置主机名
6 配置网络
这里也可以添加DNS####
7 配置普通用户
useradd beifeng
passwd beifeng
2016-07-11_104420.jpg
sudo权限设置###
visudo
2016-07-11_141357.jpg
配置DNS
vim /etc/sysconfig/network-scripts/ifcfg-eth0
2016-07-11_104756.jpg
关闭selinux###
vim /etc/sysconfig/selinux
2016-07-11_104937.jpg
修改主机名###
vim /etc/hosts
192.168.159.111 bigdata-senior01.beifeng.com 192.168.159.112 bigdata-senior02.beifeng.com 192.168.159.113 bigdata-senior03.beifeng.com
2016-07-11_105224.jpg
关闭iptables###
service iptables stop
chkconfig iptables off
2016-07-11_105447.jpg
配置目录，上传对应的包###
2016-07-11_105703.jpg
chown修改权限###
chown -R beifeng:beifeng /opt/*
2016-07-11_110004.jpg
卸载openJDK，安装JDK和MAVEN###
rpm -qa | grep jdk
rpm -e --nodeps java-1.6.0-openjdk-1.6.0.0-1.50.1.11.5.el6_3.x86_64 java-1.7.0-openjdk-1.7.0.9-2.3.4.1.el6_3.x86_64
2016-07-11_110716.jpg
tar -zxf jdk-7u67-linux-x64.tar.gz -C /opt/modules/tar -zxf apache-maven-3.0.5-bin.tar.gz -C /opt/modules/tar -zxf hadoop-2.5.0.tar.gz -C /opt/modules/
删除掉： /opt/modules/hadoop-2.5.0/lib/native的包并替换####
删除掉：/opt/modules/hadoop-2.5.0/share/doc####
2016-07-11_111251.jpg
配置JDK和MAVEN环境变量###
vim /etc/profile
#JAVA_HOME
export JAVA_HOME=/opt/modules/jdk1.7.0_67
export PATH=$PATH:$JAVA_HOME/bin
#MAVEN_HOME
export MAVEN_HOME=/opt/modules/apache-maven-3.0.5
export PATH=$PATH:$MAVEN_HOME/bin
2016-07-11_111950.jpg
source /etc/profile
2016-07-11_112111.jpg
配置hadoop相关xml文件##
配置*-evn.sh文件export JAVA_HOME###
2016-07-11_112341.jpg
Hadoop-env.sh，yarn-env.sh，mapred-env.sh 这3个文件配置export JAVA_HOME
hadoop-env.sh:
2016-07-11_112715.jpg
yarn-env.sh:
2016-07-11_112834.jpg
mapred-env.sh:
2016-07-11_112929.jpg
配置自定义文件*-site.xml###
配置yarn-site.xml
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
<property>
<name>yarn.resourcemanager.hostname</name>
<value>bigdata-senior01.beifeng.com</value>
</property>
<property>
<name>yarn.log-aggregation-enable</name>
<value>true</value>
</property>
<property>
<name>yarn.log-aggregation.retain-seconds</name>
<value>106800</value>
</property>
配置salves：
`bigdata-senior01.beifeng.com`
配置：mapred-site.xml
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
配置：hdfs-site.xml
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
<property>
<name>dfs.permissions.enabled</name>
<value>false</value>
</property>
<property>
<name>dfs.namenode.secondary.http-address</name>
<value>bigdata-senior01.beifeng.com:50090</value>
</property>
配置core-site.xml
<property>
<name>fs.defaultFS</name>
<value>hdfs://bigdata-senior01.beifeng.com:8020</value>
</property>
<property>
<name>hadoop.tmp.dir</name>
<value>/opt/modules/hadoop-2.5.0/data/tmp</value>
</property>
<property>
<name>hadoop.http.staticuser.user</name>
<value>beifeng</value>
</property>