在centos7.2上Flink1.9.1源码编译（hadoop2.6.0-cdh5.15.1）
作者：互联网
编译步骤参照官网：https://ci.apache.org/projects/flink/flink-docs-release-1.9/flinkDev/building.html
0.按照官网：In addition you need Maven 3 and a JDK (Java Development Kit). Flink requires at least Java 8 to build.配置好前置条件。
1.github获取flink 最新代码
git clone https://github.com/apache/flink
2.切换到flink 1.9分支
git checkout release-1.9
3.查看自己的maven版本
[hadoop@hadoop002 ~]$ mvn -v
Apache Maven 3.6.1 (d66c9c0b3152b2e69ee9bac180bb8fcc8e6af555; 2019-04-05T03:00:29+08:00)
Maven home: /home/hadoop/app/apache-maven-3.6.1
Java version: 1.8.0_161, vendor: Oracle Corporation, runtime: /usr/java/jdk1.8.0_161/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "3.10.0-327.el7.x86_64", arch: "amd64", family: "unix"
[hadoop@hadoop002 ~]$
如果版本是 3.1.x and 3.2.x 则直接 mvn clean install -DskipTests
如果版本是 3.3.x，则采用如下步骤
mvn clean install -DskipTests -Dfast
cd flink-dist
mvn clean install
4.如要在其他第三方厂家使用flink 打包，例如要在cdh ,Hortonworks 平台上使用，我们需要指定hadoop 版本进行打包。那么要先对flink 的前置依赖 flink-shaded-hadoop-2 进行打包，在进行编译。
新建一个目录，从git上获取flink-shaded 的源码
git clone https://github.com/apache/flink-shaded.git
根据自己缺少的版本切换对应的代码分支，这里我缺少的是7.0版本的flink-shaded-hadoop-2
git checkout release-7.0
修改flink-shaded项目中的pom.xml 这里修改是为了加入cdh 等中央仓库，否则编译对应版本可能找不到cdh 相关的包。
在pom.xml文件中的<profiles></profiles>添加
<profile>
<id>vendor-repos</id>
<activation>
<property>
<name>vendor-repos</name>
</property>
</activation>
<!-- Add vendor maven repositories -->
<repositories>
<!-- Cloudera -->
<repository>
<id>cloudera-releases</id>
<url>https://repository.cloudera.com/artifactory/cloudera-repos</url>
<releases>
<enabled>true</enabled>
</releases>
<snapshots>
<enabled>false</enabled>
</snapshots>
</repository>
<!-- Hortonworks -->
<repository>
<id>HDPReleases</id>
<name>HDP Releases</name>
<url>https://repo.hortonworks.com/content/repositories/releases/</url>
<snapshots><enabled>false</enabled></snapshots>
<releases><enabled>true</enabled></releases>
</repository>
<repository>
<id>HortonworksJettyHadoop</id>
<name>HDP Jetty</name>
<url>https://repo.hortonworks.com/content/repositories/jetty-hadoop</url>
<snapshots><enabled>false</enabled></snapshots>
<releases><enabled>true</enabled></releases>
</repository>
<!-- MapR -->
<repository>
<id>mapr-releases</id>
<url>https://repository.mapr.com/maven/</url>
<snapshots><enabled>false</enabled></snapshots>
<releases><enabled>true</enabled></releases>
</repository>
</repositories>
</profile>
然后选择自己需要的CDH版本进行编译 flink-shaded-hadoop-2
mvn clean install -DskipTests -Drat.skip=true -Pvendor-repos -Dhadoop.version=2.6.0-cdh5.15.1
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] flink-shaded 7.0 ................................... SUCCESS [ 1.619 s]
[INFO] flink-shaded-force-shading 7.0 ..................... SUCCESS [ 0.446 s]
[INFO] flink-shaded-asm-6 6.2.1-7.0 ....................... SUCCESS [ 0.563 s]
[INFO] flink-shaded-guava-18 18.0-7.0 ..................... SUCCESS [ 1.176 s]
[INFO] flink-shaded-netty-4 4.1.32.Final-7.0 .............. SUCCESS [ 2.720 s]
[INFO] flink-shaded-netty-tcnative-dynamic 2.0.25.Final-7.0 SUCCESS [ 0.625 s]
[INFO] flink-shaded-jackson-parent 2.9.8-7.0 .............. SUCCESS [ 0.014 s]
[INFO] flink-shaded-jackson-2 2.9.8-7.0 ................... SUCCESS [ 1.097 s]
[INFO] flink-shaded-jackson-module-jsonSchema-2 2.9.8-7.0 . SUCCESS [ 0.770 s]
[INFO] flink-shaded-hadoop-2 2.6.0-cdh5.15.1-7.0 .......... SUCCESS [ 10.969 s]
[INFO] flink-shaded-hadoop-2-uber 2.6.0-cdh5.15.1-7.0 ..... SUCCESS [ 21.086 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 41.375 s
[INFO] Finished at: 2020-02-02T06:45:24+08:00
[INFO] ------------------------------------------------------------------------
flink-shaded-hadoop-2编译成功之后，在切换到flink的源码目录，因之前已经在根目录编译过了，切换到flink-dist，进行flink的编译：
mvn clean install -DskipTests -Pvendor-repos -Drat.skip=true -Pinclude-hadoop -Dhadoop.version=2.6.0-cdh5.15.1
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 23.090 s
[INFO] Finished at: 2020-02-02T06:45:56+08:00
[INFO] ------------------------------------------------------------------------
在..flink/flink-dist/target/flink-1.9-SNAPSHOT-bin/flink-1.9-SNAPSHOT中，找到自己需要的flink bin包
[hadoop@hadoop002 flink-1.9-SNAPSHOT-bin]$ ll
total 0
drwxrwxr-x 9 hadoop hadoop 117 Feb 2 06:45 flink-1.9-SNAPSHOT
muyingmiao
发布了134 篇原创文章 · 获赞 4 · 访问量 1万+
私信
关注
标签：INFO,SUCCESS,flink,Flink1.9,hadoop,7.0,源码,centos7.2,shaded 来源： https://blog.csdn.net/muyingmiao/article/details/104141399