java-Spark Streaming正常关闭

我看到有一个选项可以在两个地方配置正常关机：

>在定义spark conf时：

“spark.streaming.stopGracefullyOnShutdown”, “true”.

>停止流上下文时：

JavaStreamingContext scc;

scc.stop(true,true)

两种选择之间有什么区别？

谢谢

解决方法:

第一种方式

sparkConf.set(“spark.streaming.stopGracefullyOnShutdown","true") 

Setting this parameter to True in spark configuration ensures the
proper graceful shutdown in new Spark version (1.4 onwards)
applications. Also we should not use 1st explicit shutdown hook
approach or call the ssc.stop method in the driver along with this
parameter . We can just set this parameter, and then call methods
ssc.start() and ssc.awaitTermination() . No need to call ssc.stop
method. Otherwise application might hung during shutdown.

第二种方式
scc.stop(true,true)

The scc.stop() method’s 1st boolean argument is for stopping the
associated spark context while the 2nd boolean argument is for
graceful shutdown of streaming context

chandan prakash已清楚地提到here

希望这可以帮助！

标签：apache-spark,spark-streaming,java
来源： https://codeday.me/bug/20191025/1931173.html