首页 > 其他分享> > |NO.Z.00068|——————————|^^ 部署 ^^|——|Hadoop&PB级数仓.V02|---------------------------------------|PB数仓.v0

|NO.Z.00068|——————————|^^ 部署 ^^|——|Hadoop&PB级数仓.V02|---------------------------------------|PB数仓.v0

2022-04-11 19:34:27 作者：互联网

[BigDataHadoop：Hadoop&PB级数仓.V02] [BigDataHadoop.PB级企业电商离线数仓][|章节三|Hadoop|元数据管理工具Atlas：Atlas安装配置&账户密码|]

一、安装环境准备

### --- 安装概述说明

~~~     重点讲解Atlas，不对Atlas的依赖组件做讲解，组件均采用单机模式安装。
~~~     编译才能安装。

### --- 安装依赖

~~~     Maven 3.6.3（完成）
~~~     HBase 1.1.2（不需要安装，需要软件包）
~~~     Solr 5.5.1（不需要安装，需要软件包）
~~~     atlas 1.2.0（需要编译）
~~~     官方只提供了源码，没有提供二进制的安装版本，因此Atlas需要编译。

### --- 准备需要的资源包

[root@hadoop02 ~]# ll /opt/yanqi/software/
apache-atlas-1.2.0-sources.tar.gz
apache-maven-3.6.3-bin.tar.gz
hbase-1.1.2.tar.gz
solr-5.5.1.tgz

二、部署maven依赖版本包

### --- 下载maven版本包：apache-maven-3.6.3-bin.tar.gz

[root@hadoop02 ~]# ll /opt/yanqi/software/
apache-maven-3.6.3-bin.tar.gz

### --- 安装maven版本包
~~~     安装maven版本包

[root@hadoop02 ~]# tar -zxvf /opt/yanqi/software/apache-maven-3.6.3-bin.tar.gz \
-C /opt/yanqi/servers/

~~~     # 配置maven环境变量

[root@hadoop02 ~]# vim /etc/profile
##MAVEN_HOME
export MAVEN_HOME=/opt/yanqi/servers/apache-maven-3.6.3
export PATH=$PATH:$MAVEN_HOME/bin

~~~     # 使环境变量生效

[root@hadoop02 ~]# source /etc/profile

### --- 验证maven版本包是否部署成功
~~~     验证安装

[root@hadoop02 ~]# mvn --version
Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)

### --- maven设置阿里镜像
~~~     重要，否则非常慢

~~~     # 添加阿里云源地址
[root@hadoop02 ~]# vim $MAVEN_HOME/conf/settings.xml
# 加在 158 行后
    <mirror>
      <id>alimaven</id>
      <name>aliyun maven</name>
      <url>http://maven.aliyun.com/nexus/content/groups/public/</url>
      <mirrorOf>central</mirrorOf>
    </mirror>

三、安装步骤

### --- 准备软件包

[root@hadoop02 ~]# ll /opt/yanqi/software/
apache-atlas-1.2.0-sources.tar.gz
solr-5.5.1.tgz
hbase-1.1.2.tar.gz

### --- 解压缩源码，修改配置
~~~     # 解压缩

[root@hadoop02 ~]# cd /opt/yanqi/software/
[root@hadoop02 software]# tar -zxvf apache-atlas-1.2.0-sources.tar.gz
[root@hadoop02 software]# cd apache-atlas-sources-1.2.0/

~~~     # 修改配置

[root@hadoop02 apache-atlas-sources-1.2.0]# vim pom.xml
~~~ 修改配置参数
<npm-for-v2.version>3.10.8</npm-for-v2.version>     # 第645行
<hadoop.version>2.9.2</hadoop.version>              # 第652行

### --- 将HBase、Solr的包拷贝到对应的目录中
~~~     如果不拷贝这些包，就需要下载，下载 HBase 和 Solr 时速度很慢。
~~~     这里提前下载完所需的这两个组件，拷贝到对应目录中。

~~~     # 创建文件目录
[root@hadoop02 ~]# cd /opt/yanqi/software/apache-atlas-sources-1.2.0/
[root@hadoop02 apache-atlas-sources-1.2.0]# cd distro/

[root@hadoop02 distro]# mkdir solr
[root@hadoop02 distro]# mkdir hbase

~~~     # 拷贝软件包

[root@hadoop02 distro]# cp /opt/yanqi/software/solr-5.5.1.tgz ./solr/
[root@hadoop02 distro]# cp /opt/yanqi/software/hbase-1.1.2.tar.gz ./hbase/

### --- Atlas编译
~~~     下载依赖包地址
~~~     安装之前设置maven的下载镜像地址为阿里云镜像地址

[root@hadoop02 ~]# ls /root/.m2/repository/
antlr                     bouncycastle  commons-beanutils      commons-daemon      commons-io         concurrent  it              log4j     regexp     xerces    xpp3
aopalliance               cglib         commons-chain          commons-dbcp        commons-lang       de          jakarta-regexp  logkit    ring-cors  xml-apis
asm                       ch            commons-cli            commons-digester    commons-logging    dom4j       javax           nekohtml  sslext     xmlenc
avalon-framework          classworlds   commons-codec          commons-el          commons-net        edu         jline           net       stax       xmlpull
backport-util-concurrent  colt          commons-collections    commons-fileupload  commons-pool       info        joda-time       org       tomcat     xmlunit
biz                       com           commons-configuration  commons-httpclient  commons-validator  io          junit           oro       xalan      xom

~~~     # 编译Atlas版本包

[root@hadoop02 ~]# cd /opt/yanqi/software/apache-atlas-sources-1.2.0/
[root@hadoop02 apache-atlas-sources-1.2.0]# export MAVEN_OPTS="-Xms2g -Xmx2g"
[root@hadoop02 apache-atlas-sources-1.2.0]# mvn clean -DskipTests package -Pdist,embedded-hbase-solr
~~~ 输出参数
~~~ 编译过程中大概要下载600M左右的jar，持续的时间比较长。
[INFO] Building zip: /opt/yanqi/software/apache-atlas-sources-1.2.0/distro/target/apache-atlas-1.2.0-migration-exporter.zip
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] 
[INFO] Apache Atlas Server Build Tools 1.0 ................ SUCCESS [ 45.135 s]
[INFO] apache-atlas 1.2.0 ................................. SUCCESS [02:17 min]
[INFO] Apache Atlas Test Utility Tools 1.2.0 .............. SUCCESS [02:09 min]
[INFO] Apache Atlas Integration 1.2.0 ..................... SUCCESS [ 44.858 s]
[INFO] Apache Atlas Common 1.2.0 .......................... SUCCESS [ 10.657 s]
[INFO] Apache Atlas Client 1.2.0 .......................... SUCCESS [  2.263 s]
[INFO] atlas-client-common 1.2.0 .......................... SUCCESS [  4.763 s]
[INFO] atlas-client-v1 1.2.0 .............................. SUCCESS [  4.657 s]
[INFO] Apache Atlas Server API 1.2.0 ...................... SUCCESS [  2.996 s]
[INFO] Apache Atlas Notification 1.2.0 .................... SUCCESS [ 39.452 s]
[INFO] atlas-client-v2 1.2.0 .............................. SUCCESS [  4.233 s]
[INFO] Apache Atlas Graph Database Projects 1.2.0 ......... SUCCESS [  0.257 s]
[INFO] Apache Atlas Graph Database API 1.2.0 .............. SUCCESS [ 10.832 s]
[INFO] Graph Database Common Code 1.2.0 ................... SUCCESS [  3.870 s]
[INFO] Apache Atlas JanusGraph DB Impl 1.2.0 .............. SUCCESS [02:11 min]
[INFO] Apache Atlas Graph Database Implementation Dependencies 1.2.0 SUCCESS [  8.056 s]
[INFO] Shaded version of Apache hbase client 1.2.0 ........ SUCCESS [ 30.243 s]
[INFO] Shaded version of Apache hbase server 1.2.0 ........ SUCCESS [01:15 min]
[INFO] Apache Atlas Authorization 1.2.0 ................... SUCCESS [  6.001 s]
[INFO] Apache Atlas Repository 1.2.0 ...................... SUCCESS [ 52.256 s]
[INFO] Apache Atlas UI 1.2.0 .............................. SUCCESS [07:17 min]
[INFO] Apache Atlas Web Application 1.2.0 ................. SUCCESS [04:01 min]
[INFO] Apache Atlas Documentation 1.2.0 ................... SUCCESS [01:00 min]
[INFO] Apache Atlas FileSystem Model 1.2.0 ................ SUCCESS [ 11.244 s]
[INFO] Apache Atlas Plugin Classloader 1.2.0 .............. SUCCESS [  3.209 s]
[INFO] Apache Atlas Hive Bridge Shim 1.2.0 ................ SUCCESS [ 50.307 s]
[INFO] Apache Atlas Hive Bridge 1.2.0 ..................... SUCCESS [ 31.274 s]
[INFO] Apache Atlas Falcon Bridge Shim 1.2.0 .............. SUCCESS [ 51.045 s]
[INFO] Apache Atlas Falcon Bridge 1.2.0 ................... SUCCESS [  8.699 s]
[INFO] Apache Atlas Sqoop Bridge Shim 1.2.0 ............... SUCCESS [  6.696 s]
[INFO] Apache Atlas Sqoop Bridge 1.2.0 .................... SUCCESS [  8.834 s]
[INFO] Apache Atlas Storm Bridge Shim 1.2.0 ............... SUCCESS [ 11.894 s]
[INFO] Apache Atlas Storm Bridge 1.2.0 .................... SUCCESS [ 10.539 s]
[INFO] Apache Atlas Hbase Bridge Shim 1.2.0 ............... SUCCESS [  4.298 s]
[INFO] Apache Atlas Hbase Bridge 1.2.0 .................... SUCCESS [ 37.611 s]
[INFO] Apache Atlas Kafka Bridge 1.2.0 .................... SUCCESS [  3.626 s]
[INFO] Apache Atlas Distribution 1.2.0 .................... SUCCESS [07:46 min]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  37:36 min
[INFO] Finished at: 2021-10-11T14:10:31+08:00
[INFO] ------------------------------------------------------------------------

~~~     # 编译完的软件位置：
[root@hadoop02 ~]# cd /opt/yanqi/software/apache-atlas-sources-1.2.0/distro/target/

~~~     # 编译完的软件：
[root@hadoop02 ~]# ls /opt/yanqi/software/apache-atlas-sources-1.2.0/distro/target/
apache-atlas-1.2.0-bin.tar.gz

### --- Atlas安装
~~~     进入编译完成后maven目录下

[root@hadoop02 ~]# cd /opt/yanqi/software/apache-atlas-sources-1.2.0/distro/target/
[root@hadoop02 target]# ls apache-atlas-1.2.0-bin.tar.gz 
apache-atlas-1.2.0-bin.tar.gz 
[root@hadoop02 target]# cp apache-atlas-1.2.0-bin.tar.gz /opt/yanqi/software/

~~~     # 解压缩并安装Atlas

[root@hadoop02 ~]# cd /opt/yanqi/software/
[root@hadoop02 software]# tar -zxvf apache-atlas-1.2.0-bin.tar.gz -C ../servers/
[root@hadoop02 software]# cd ../servers/
[root@hadoop02 servers]# mv apache-atlas-1.2.0/ atlas-1.2.0

~~~     # 配置Atlas环境变量
[root@hadoop02 ~]# vim /etc/profile
##ATLAS_HOME
export ATLAS_HOME=/opt/yanqi/servers/atlas-1.2.0
export PATH=$PATH:$ATLAS_HOME/bin
 
~~~     # 使环境变量生效
[root@hadoop02 ~]# source /etc/profile

### --- Atlas启动

~~~     # 启动服务(第一次启动服务的时间比较长)
[root@hadoop02 ~]# cd $ATLAS_HOME/bin
[root@hadoop02 bin]# ./atlas_start.py           # 启动服务
~~~ 输出参数
configured for local hbase.
hbase started.
configured for local solr.
solr started.
setting up solr collections...
starting atlas on host localhost
starting atlas on port 21000
..............................................................................................................................................................................................................................................................................................................
Apache Atlas Server started!!!

[root@hadoop02 bin]# ./atlas_stop.py            # 关闭服务
[root@hadoop02 bin]# ./solr status              # 查看服务状态

~~~     # 检查后台进程 (1个atlas、2个HBase、1个solr后台进程)

[root@hadoop02 ~]# ps -ef | grep atlas
 ~~~ 启动的进程
/opt/yanqi/servers/atlas-1.2.0/server/webapp/atlas
/opt/yanqi/servers/atlas-1.2.0/hbase/bin/hbase-daemon.sh
org.apache.hadoop.hbase.master.HMaster
/opt/yanqi/servers/atlas-1.2.0/solr/server/solr

~~~     # 检查 solr 的状态：

[root@hadoop02 ~]# cd /opt/yanqi/servers/atlas-1.2.0/solr/bin
[root@hadoop02 bin]# ./solr status
~~~ 输出参数
Found 1 Solr nodes: 

Solr process 88510 running on port 9838
{
  "solr_home":"/opt/yanqi/servers/atlas-1.2.0/solr/server/solr",
  "version":"5.5.1 c08f17bca0d9cbf516874d13d221ab100e5b7d58 - anshum - 2016-04-30 13:28:18",
  "startTime":"2021-10-11T06:31:58.698Z",
  "uptime":"0 days, 0 hours, 28 minutes, 56 seconds",
  "memory":"28.9 MB (%5.9) of 490.7 MB",
  "cloud":{
    "ZooKeeper":"localhost:2181",
    "liveNodes":"1",
    "collections":"3"}}

~~~     # 检查zookeper状态：

[root@hadoop02 ~]# echo stat| nc localhost 2181

四、Atlas.UI管理界面

### --- Web服务：http://hadoop02:21000/login.jsp

~~~     用户名 / 口令：admin / admin
~~~     后台管理UI

五、设置Atlas服务的用户名及口令

### --- 设置Atlas服务的用户名和口令

~~~     #  账号的信息存储在文件 conf/users-credentials.properties 中。
[root@hadoop02 ~]# cd /opt/yanqi/servers/atlas-1.2.0/
[root@hadoop02 atlas-1.2.0]# vim conf/users-credentials.properties
~~~ 用户名口令参数
#username=group::sha256-password
admin=ADMIN::8c6976e5b5410415bde908bd4dee15dfb167a9c873fc4bb8a81f6f2ab448a918
rangertagsync=RANGER_TAG_SYNC::e3f67240f5117d1753c940dae9eea772d36ed5fe9bd9c94a300e40413f1afb9d

~~~     # 其中 Password 通过如下方式产生sha256sum 摘要信息：

[root@hadoop02 ~]# echo -n  "admin" | sha256sum
8c6976e5b5410415bde908bd4dee15dfb167a9c873fc4bb8a81f6f2ab448a918  -
[root@hadoop02 ~]# echo -n  "rangertagsync" | sha256sum
e3f67240f5117d1753c940dae9eea772d36ed5fe9bd9c94a300e40413f1afb9d  -

===============================END===============================

Walter Savage Landor:strove with none,for none was worth my strife.Nature I loved and, next to Nature, Art:I warm'd both hands before the fire of life.It sinks, and I am ready to depart ——W.S.Landor

来自为知笔记(Wiz)

标签：INFO,---------------------------------------,1.2,部署,hadoop02,PB,atlas,Atlas,root
来源： https://www.cnblogs.com/yanqivip/p/16131531.html