导出TensorBoard中的所有数据并平滑处理

在写machine learning 作业的时候遇到一个题就是如果把TensorBoard文件中的数据导出并自己重新画图。查找资料以后发现有两种方式：

1. 在TensorBoard 页面右边有一个Show data download links 选项，勾选后就可以在图表下方看见下载链接
2. 使用代码的方式

from tensorboard.backend.event_processing import event_accumulator
 
#加载日志数据
ea=event_accumulator.EventAccumulator('events.out.tfevents.1550994567.vvd-Inspiron-7557') 
ea.Reload()
print(ea.scalars.Keys())
 
val_psnr=ea.scalars.Items('val_psnr')
print(len(val_psnr))
print([(i.step,i.value) for i in val_psnr])

Output

['val_loss', 'val_psnr', 'loss', 'psnr', 'lr']
29
[(0, 33.70820617675781), (1, 34.52505874633789), (2, 34.26629638671875), (3, 35.47195053100586), (4, 35.45940017700195), (5, 35.336708068847656), (6, 35.467647552490234), (7, 35.919857025146484), (8, 35.29727554321289), (9, 35.63655471801758), (10, 36.219871520996094), (11, 36.178646087646484), (12, 35.93777847290039), (13, 35.587406158447266), (14, 36.198944091796875), (15, 36.241966247558594), (16, 36.379913330078125), (17, 36.28306198120117), (18, 36.03053665161133), (19, 36.20806121826172), (20, 36.21710968017578), (21, 36.42262268066406), (22, 36.00306701660156), (23, 36.4374885559082), (24, 36.163787841796875), (25, 36.53673553466797), (26, 35.99557113647461), (27, 36.96220016479492), (28, 36.63676452636719)]

Reference:
https://blog.csdn.net/zywvvd/article/details/88865416

但是这两种方法都有一个问题，导出的数据只有10000条。但是我想要的是导出所有的数据。最后在stackoverflow中找到的方法

在启动TensorBoard的时候加上参数--samples_per_plugin scalars=0 如：

tensorboard --samples_per_plugin scalars=0

再用第一种方法从页面上下载数据的时候，就可以下载所有的数据了
Reference:
https://stackoverflow.com/questions/43702546/tensorboard-doesnt-show-all-data-points

现在要对所有数据进行平滑处理：
代码：

import pandas as pd
import numpy as np
import os
def smooth(csv_path,weight=0.85):
    data = pd.read_csv(filepath_or_buffer=csv_path,header=0,names=['Step','Value'],dtype={'Step':np.int,'Value':np.float})
    scalar = data['Value'].values
    last = scalar[0]
    smoothed = []
    for point in scalar:
        smoothed_val = last * weight + (1 - weight) * point
        smoothed.append(smoothed_val)
        last = smoothed_val


    save = pd.DataFrame({'Step':data['Step'].values,'Value':smoothed})
    save.to_csv('smooth_'+csv_path)


if __name__=='__main__':
    smooth('test.csv')

Reference:
https://blog.csdn.net/Charel_CHEN/article/details/80364841
https://dingguanglei.com/tensorboard-xia-smoothgong-neng-tan-jiu/

标签：__,psnr,val,平滑,导出,TensorBoard,csv,data,smoothed
来源： https://blog.csdn.net/Hothy/article/details/118962803