遇到问题---spark on k8s--挂载了efs的pod运行spark session 60后报java致命错误
作者:互联网
情况
我们在前面的文章中有提到过,目前spark on k8s运行某些类型的任务时,需要共享/tmp目录,我们通过efs来实现了 共享目录的挂载。
但是 可能遇到 挂载了efs的pod运行spark session 60后报java致命错误的情况。
pod状态如下:
spark-shell-1605604907012-exec-2 0/1 Error 0 68s 10.33.1.46 ip-10-33-6-62.cn-northwest-1.compute.internal <none> <none>
spark session打印详细报错如下:
20/11/17 09:21:56 INFO Executor: Using REPL class URI: spark://10.30.9.139:45645/classes
#
# A fatal error has been detected by the Java Runtime Environment:
#
# SIGBUS (0x7) at pc=0x00007fd2beb5560e, pid=14, tid=0x00007fd29b17f700
#
# JRE version: OpenJDK Runtime Environment (8.0_242-b08) (build 1.8.0_242-b08)
# Java VM: OpenJDK 64-Bit Server VM (25.242-b08 mixed mode linux-amd64 compressed oops)
# Problematic frame:
# V [libjvm.so+0x8f960e] PerfLongVariant::sample()+0x1e
#
# Core dump written. Defa
标签:java,efs,---,session,挂载,b08,spark,pod 来源: https://blog.csdn.net/q383965374/article/details/110442284