六、mapred配置与yarn配置

6.1 Hadoop三大核心组件
–分布式文件系统
–HDFS 已经部署完毕
–分布式计算框架
–mapreduce
–集群资源管理
–yarn

系统规划

6.2 •分布式计算框架mapred-site.xml
–改名
–FROM: mapred-site.xml.template
–To: mapred-site.xml
[root@nn01 hadoop]# cd /usr/local/hadoop/etc/hadoop
[root@nn01 hadoop]# mv mapred-site.xml.template mapred-site.xml
[root@nn01 hadoop]# vim mapred-site.xml
<configuration>
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
</configuration>
注：分布式计算框架mapred-site.xml，只支持local 和yarn 两种：
–单机使用local
–集群使用yarn
6.3 资源管理yarn-site.xml
–resourcemanager地址
–yarn.resourcemanager.hostname
–nodemanager使用哪个计算框架
–yarn.nodemanager.aux-services
–mapreduce_shuffle是我们使用计算框架的名称
[root@nn01 hadoop]# vim yarn-site.xml
<configuration>
<property>
<name>yarn.resourcemanager.hostname</name>
<value>nn01</value>
</property>
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
</configuration>
注：配置文件更新到node1，node2 ，node3，更新完成后执行下面操作
6.4 启动服务并验证
6.4.1 启动服务
[root@nn01 hadoop]# ./sbin/start-yarn.sh
starting yarn daemons
starting resourcemanager, logging to /usr/local/hadoop/logs/yarn-root-resourcemanager-nn01.out
node1: starting nodemanager, logging to /usr/local/hadoop/logs/yarn-root-nodemanager-node1.out
node3: starting nodemanager, logging to /usr/local/hadoop/logs/yarn-root-nodemanager-node3.out
node2: starting nodemanager, logging to /usr/local/hadoop/logs/yarn-root-nodemanager-node2.out
6.4.2 验证节点状态
[root@nn01 hadoop]# ./bin/yarn node -list
21/09/01 17:28:34 INFO client.RMProxy: Connecting to ResourceManager at nn01/192.168.0.10:8032
Total Nodes:3
Node-Id Node-State Node-Http-Address Number-of-Running-Containers
node2:41692 RUNNING node2:8042 0
node1:46589 RUNNING node1:8042 0
node3:34864 RUNNING node3:8042 0
6.4.3 验证角色 jps
[root@nn01 hadoop]# for i in node{1..3} ; do ssh $i jps ;done
1328 Jps
929 DataNode
1218 NodeManager
930 DataNode
1231 NodeManager
1343 Jps
924 DataNode
1196 NodeManager
1308 Jps

 6.5 使用web访问hadoop

–namenode  web页面(nn01)
–http://192.168.1.10:50070/
–secondory  name   node   web 页面(nn01)
–http://192.168.1.10:50090/
–datanode    web 页面(node1,node2,node3)
–http://192.168.1.11:50075/

–resourcemanager  web页面(nn01)
–http://192.168.1.10:8088/
–nodemanager   web 页面(node1,node2,node3)
–http://192.168.1.11:8042/

标签：nn01,mapred,配置,hadoop,yarn,192.168,nodemanager,root
来源： https://www.cnblogs.com/momo6656/p/15215758.html