Retrying task after failure: Waiting for lock. org.apache.iceberg.hive.HiveTableOperations$WaitingFo

执行spark任务居然碰到了hive中表的被锁了，导致写数据时失败，这个还真是头一次碰到过这回事。

错误信息如下：

22/09/04 21:02:28 WARN Tasks: Retrying task after failure: Waiting for lock.
org.apache.iceberg.hive.HiveTableOperations$WaitingForLockException: Waiting for lock.
        at org.apache.iceberg.hive.HiveTableOperations.lambda$acquireLock$9(HiveTableOperations.java:460)
        at org.apache.iceberg.util.Tasks$Builder.runTaskWithRetry(Tasks.java:404)
        at org.apache.iceberg.util.Tasks$Builder.runSingleThreaded(Tasks.java:214)
        at org.apache.iceberg.util.Tasks$Builder.run(Tasks.java:198)
        at org.apache.iceberg.hive.HiveTableOperations.acquireLock(HiveTableOperations.java:454)
        at org.apache.iceberg.hive.HiveTableOperations.doCommit(HiveTableOperations.java:226)
        at org.apache.iceberg.BaseMetastoreTableOperations.commit(BaseMetastoreTableOperations.java:127)
        at org.apache.iceberg.SnapshotProducer.lambda$commit$2(SnapshotProducer.java:304)
        at org.apache.iceberg.util.Tasks$Builder.runTaskWithRetry(Tasks.java:404)
        at org.apache.iceberg.util.Tasks$Builder.runSingleThreaded(Tasks.java:214)
        at org.apache.iceberg.util.Tasks$Builder.run(Tasks.java:198)
        at org.apache.iceberg.util.Tasks$Builder.run(Tasks.java:190)
        at org.apache.iceberg.SnapshotProducer.commit(SnapshotProducer.java:282)
        at org.apache.iceberg.BaseReplacePartitions.commit(BaseReplacePartitions.java:26)
        at org.apache.iceberg.spark.source.Writer.commitOperation(Writer.java:144)
        at org.apache.iceberg.spark.source.Writer.replacePartitions(Writer.java:181)
        at org.apache.iceberg.spark.source.Writer.commit(Writer.java:119)
        at org.apache.spark.sql.execution.datasources.v2.WriteToDataSourceV2Exec.doExecute(WriteToDataSourceV2Exec.scala:76)
 

跑到hive的元数据库中查询时，在表HIVE_LOCKS中找到了一些信息。

根据出错信息，根据hl_table找到相关的表，查看最近的心跳时间字段，hl_last_heartbeat。
删除对应的记录即可。

delete  from HIVE_LOCKS where hl_table = '出错的表名';

标签：task,iceberg,java,lock,HiveTableOperations,Waiting,Tasks,apache,org
来源： https://www.cnblogs.com/30go/p/16656135.html