java – Apache光束窗口：考虑延迟数据但只发出一个窗格

当水印到达窗口末端x分钟时,我想发出一个单一窗格.这让我确保我处理一些迟到的数据,但仍然只发出一个窗格.我目前在java工作.

目前我无法找到解决这个问题的正确方法.当水印到达窗口的末尾时,我可以发出单个窗格,但随后会丢弃任何后期数据.我可以在窗口的末尾发出窗格,然后在我收到延迟数据时再次发出,但在这种情况下,我不会发出单个窗格.

我目前的代码与此类似：

.triggering(
    // This is going to emit the pane, but I don't want emit the pane yet!                                  
    AfterWatermark.pastEndOfWindow()

    // This is going to emit panes each time I receive late data, however 
    // I would like to only emit one pane at the end of the allowedLateness
).withAllowedLateness(allowedLateness).accumulatingFiredPanes())

如果仍然存在混淆,我想在水印通过allowedLateness时只发出一个窗格.

解决方法:

谢谢Guillem,最后我用你的答案找到了这个有很多apache梁例子的very useful link.从这里我想出了以下解决方案：

 // We first specify to never emit any panes
 .triggering(Never.ever())

 // We then specify to fire always when closing the window. This will emit a
 // single final pane at the end of allowedLateness
 .withAllowedLateness(allowedLateness, Window.ClosingBehavior.FIRE_ALWAYS)
 .discardingFiredPanes())

标签：dataflow,java,apache-beam,windowing
来源： https://codeday.me/bug/20191003/1846861.html