python-视频帧作为Tensorflow图的输入

更具体地说,如何创建自定义阅读器,以从视频中读取帧并将其输入到tensorflow模型图中.

其次,如果可能的话,如何使用opencv解码帧以创建自定义阅读器？

是否有任何代码可以更好地说明目标(在python中)？

我主要从事通过面部表情进行情感识别的工作,并且在我的数据库中输入了视频.

最后,我尝试将Queue和QueueRunner与协调器一起使用,以期解决当前的问题.根据https://www.tensorflow.org/programmers_guide/threading_and_queues中的文档,QueueRunner运行enqueue操作,该操作又需要执行一个操作来创建一个示例(我们可以在此操作中使用opencv创建一个示例,以将帧作为示例返回到队列中吗？)

请注意,我的目的是让入队和出队操作在不同线程上同时发生.

以下是我到目前为止的代码：

def deform_images(images):
    with tf.name_scope('current_image'):
        frames_resized = tf.image.resize_images(images, [90, 160])
        frame_gray = tf.image.rgb_to_grayscale(frames_resized, name='rgb_to_gray')
        frame_normalized = tf.divide(frame_gray, tf.constant(255.0), name='image_normalization')

        tf.summary.image('image_summmary', frame_gray, 1)
        return frame_normalized

def queue_input(video_path, coord):
    global frame_index
    with tf.device("/cpu:0"):
        # keep looping infinitely

        # source: https://stackoverflow.com/questions/33650974/opencv-python-read-specific-frame-using-videocapture
        cap = cv2.VideoCapture(video_path)
        cap.set(1, frame_index)

        # read the next frame from the file, Note that frame is returned as a Mat.
        # So we need to convert that into a tensor.
        (grabbed, frame) = cap.read()

        # if the `grabbed` boolean is `False`, then we have
        # reached the end of the video file
        if not grabbed:
            coord.request_stop()
            return

        img = np.asarray(frame)
        frame_index += 1
        to_retun = deform_images(img)
        print(to_retun.get_shape())
        return to_retun

frame_num = 1

with tf.Session() as sess:
    merged = tf.summary.merge_all()
    train_writer = tf.summary.FileWriter('C:\\Users\\temp_user\\Documents\\tensorboard_logs', sess.graph)
    tf.global_variables_initializer()

    coord = tf.train.Coordinator()
    queue = tf.FIFOQueue(capacity=128, dtypes=tf.float32, shapes=[90, 160, 1])
    enqueue_op = queue.enqueue(queue_input("RECOLA-Video-recordings\\P16.mp4", coord))

    # Create a queue runner that will run 1 threads in parallel to enqueue
    # examples. In general, the queue runner class is used to create a number of threads cooperating to enqueue
    # tensors in the same queue.
    qr = tf.train.QueueRunner(queue, [enqueue_op] * 1)

    # Create a coordinator, launch the queue runner threads.
    # Note that the coordinator class helps multiple threads stop together and report exceptions to programs that wait
    # for them to stop.
    enqueue_threads = qr.create_threads(sess, coord=coord, start=True)

    # Run the training loop, controlling termination with the coordinator.
    for step in range(8000):
        print(step)
        if coord.should_stop():
            break

        frames_tensor = queue.dequeue(name='dequeue')
        step += 1

    coord.join(enqueue_threads)

train_writer.close()
cv2.destroyAllWindows()

谢谢！！

解决方法:

tf.QueueRunner不是最适合您目的的机制.在您拥有的代码中,以下行

enqueue_op = queue.enqueue(queue_input("RECOLA-Video-recordings\\P16.mp4", coord))

创建enqueue_op,它将使一个常数张量入队,即每次运行时queue_input函数返回的第一帧.即使QueueRunner反复调用它,它总是排队相同的张量,即在操作创建过程中提供给它的张量.取而代之的是,您可以简单地使enqueue操作将tf.placeholder作为其参数,然后在循环中重复运行它,并向其提供通过OpenCV抓取的帧.这是一些指导您的代码.

frame_ph = tf.placeholder(tf.float32)
enqueue_op = queue.enqueue(frame_ph)

def enqueue():
  while not coord.should_stop():
    frame = queue_input(video_path, coord)
    sess.run(enqueue_op, feed_dict={frame_ph: frame})

threads = [threading.Thread(target=enqueue)]

for t in threads:
  t.start()

# Your dequeue and training code goes here
coord.join(threads)

标签：video-processing,tensorflow,opencv,video-streaming,python
来源： https://codeday.me/bug/20191026/1934145.html