其他分享
首页 > 其他分享> > Flink读取Iceberg数据

Flink读取Iceberg数据

作者:互联网

1.依赖

参考:https://iceberg.apache.org/docs/latest/flink/#reading-with-datastream

<dependency>
      <groupId>org.apache.flink</groupId>
      <artifactId>flink-java</artifactId>
      <version>1.12.0</version>
      <!--      <scope>provided</scope>-->
    </dependency>
    <dependency>
      <groupId>org.apache.flink</groupId>
      <artifactId>flink-streaming-java_2.12</artifactId>
      <version>1.12.0</version>
      <!--      <scope>provided</scope>-->
    </dependency>
    <dependency>
      <groupId>org.apache.flink</groupId>
      <artifactId>flink-clients_2.12</artifactId>
      <version>1.12.0</version>
      <!--      <scope>provided</scope>-->
    </dependency>
    <dependency>
      <groupId>org.apache.iceberg</groupId>
      <artifactId>iceberg-core</artifactId>
      <version>0.13.1</version>
    </dependency>

    <dependency>
      <groupId>org.apache.iceberg</groupId>
      <artifactId>iceberg-flink-runtime</artifactId>
      <version>0.12.1</version>
    </dependency>

2.代码

public class ReadIceberg {
    public static void main(String[] args) throws Exception {
        //创建flink执行环境
        StreamExecutionEnvironment env = StreamExecutionEnvironment.createLocalEnvironmentWithWebUI(new Configuration());
        TableLoader tableLoader = TableLoader.fromHadoopTable("hdfs://hadoop01:9000/warehouse/iceberg/ods_db/test");
        DataStream<RowData> batch = FlinkSource.forRowData()
                .env(env)
                .tableLoader(tableLoader)
                .streaming(false)
                .build();
        //打印
        batch.print();
        //提交任务
        env.execute("Test Iceberg Batch Read");
    }
}

标签:iceberg,provided,Flink,Iceberg,读取,flink,env,apache,org
来源: https://blog.csdn.net/qq_40105563/article/details/123611009