其他分享
首页 > 其他分享> > 史上最详细的有关自解码和主成分分析的笔记(Autoencoder Vs PCA)

史上最详细的有关自解码和主成分分析的笔记(Autoencoder Vs PCA)

作者:互联网

文章目录

一、编码器

1. 什么是自编码器

1.1 自编码介绍

1986年Rumelhart提出自动编码器的概念,并将其应用到高维复杂数据处理中,促进了神经网络的发展。自编码器可以理解成一个师徒还原其原始输入的系统:
在这里插入图片描述

左边的佳作Encoder,俗称编码器,右边的叫做Decoder,俗称解码器。主要目的是将输入x经过encoder和decoder转变成x’,然后对比输入x和输出x’使得他们两个无限接近。自解码是神经网络的一种,该网络可以看成两部分组成:

1.2 为何要重构输出

自解码尝试学习一个函数,换句话说,他尝试逼近一个恒等函数,从而是的输出接近输入。自编码也就是为了是的h能够反馈出一些特征,使得x = x’。直观上我们会感觉,让输入值等于输出值,其实对我们数据处理并没有什么意义。但是,我们可以通过训练输出值等于输入值,让潜在的空间表征h具有价值属性。在理想情况下,根据要分配的数据复杂度,来准确选择编码器和解码器的编码维数和容量,就可以成功地训练出任何所需的自编码器结构。

1.3 自编码的用途

自编码器目前主要有两个用途:

2. 自编码器的种类

自编码有很多种,比如基础自编码器、欠完备自编码器、正则自编码器、稀疏自编码器、去噪自编码器、收缩自编码器、多层自编码器、卷积自编码器等等。这里我们主要讲解下面几个编码器:

2.1 PCA自编码器

如下图,这是一个简单的自编码器的示意图:

如果自编码器仅仅使用线性激活,并且损失函数使用均方差(MSE),则最后执行的就是主成分分析。下面我们来构建一个简单的线性自编码器,对于3D数据集执行PCA,将其投影到2D:
首先我们生成随机的数据,将在下面几个模型中使用这个模拟数据:

import numpy as np

np.random.seed(4)
def generate_3d_data(m, w1=0.1, w2=0.3, noise=0.1):
    angles = np.random.rand(m) * 3 * np.pi / 2 - 0.5
    data = np.empty((m, 3))
    data[:, 0] = np.cos(angles) + np.sin(angles)/2 + noise * np.random.randn(m) / 2
    data[:, 1] = np.sin(angles) * 0.7 + noise * np.random.randn(m) / 2
    data[:, 2] = data[:, 0] * w1 + data[:, 1] * w2 + noise * np.random.randn(m)
    return data

X_train1 = generate_3d_data(60)
X_train = X_train1 - X_train1.mean(axis=0, keepdims=0)

PCA Autoencoder代码:

import tensorflow as tf
from tensorflow.keras import Input
from tensorflow.keras.layers import Dense
from tensorflow.keras import Model

x = Input(shape = (3,))
encoder = Dense(2)(x)
decoder = Dense(3)(encoder)

autoencoder = Model(x, decoder)
autoencoder.compile(loss='mse', optimizer = tf.keras.optimizers.SGD(lr=1.5))

训练模型,并且使他投影到2D:

history = autoencoder.fit(X_train, X_train, epochs = 20)
codings = encoder.predict(X_train)

注意,这里有两个X_train,分别表示输入和目标。
图2.1.1图2.1.2
左图显示的是原始数据,有图显示的是线性Autoencoder得到的结果。自编码器找到了将数据投影到最佳2D平面,并尽可能保留了数据中的方差(就像PCA一样)。
PCA Autoendocer完整代码:

import numpy as np
import tensorflow as tf
import matplotlib.pyplot as plt
from tensorflow.keras import Input
from tensorflow.keras import Model
from tensorflow.keras.layers import Dense

np.random.seed(4)

def generate_3d_data(m, w1=0.1, w2=0.3, noise=0.1):
    angles = np.random.rand(m) * 3 * np.pi / 2 - 0.5
    data = np.empty((m, 3))
    data[:, 0] = np.cos(angles) + np.sin(angles)/2 + noise * np.random.randn(m) / 2
    data[:, 1] = np.sin(angles) * 0.7 + noise * np.random.randn(m) / 2
    data[:, 2] = data[:, 0] * w1 + data[:, 1] * w2 + noise * np.random.randn(m)
    return data

X_train1 = generate_3d_data(60)
X_train = X_train1 - X_train1.mean(axis=0, keepdims=0)


x = Input(shape = (3,))
encoder = Dense(2)(x)
decoder = Dense(3)(encoder)

autoencoder = Model(x, decoder)
autoencoder.compile(loss='mse', optimizer = tf.keras.optimizers.SGD(lr=1.5))

np.random.seed(42)
tf.random.set_seed(42)

history = autoencoder.fit(X_train, X_train, epochs=20)
codings = autoencoder.predict(X_train)

fig = plt.figure(figsize=(4,3))
plt.plot(codings[:,0], codings[:, 1], "b.")
plt.xlabel("$z_1$", fontsize=18)
plt.ylabel("$z_2$", fontsize=18, rotation=0)
plt.grid(True)
plt.show()

fig = plt.figure()
ax = fig.add_subplot(projection='3d')
ax.scatter(X_train1[:, 0], X_train1[:, 1], X_train1[:, 2])
ax.set_xlabel('$x_1$'); ax.set_xlabel('$x_2$'); ax.set_xlabel('$x_3$')
plt.show()
2.2 基础自编码器

这种自编码器也是相当简单的,相较于PCA编码器来说,它使用了激活函数,也就是变成了非线性的编码器。它的输入和输出也是相同的,可以通过各种优化器和损失函数来学习重构。如果潜在表征层小于输入维数,则这个编码器是有损的,也就是说它会放弃一些数据特征。通过这个约束,来是的编码器具有压缩的特征:

import numpy as np
import tensorflow as tf
from tensorflow.keras import Input
from tensorflow.keras.layers import Dense
from tensorflow.keras import Model

x = Input(shape = (3,))
encoder = Dense(2, activation='selu')(x)
decoder = Dense(3, activation='selu')(encoder)

autoencoder = Model(x, decoder)
autoencoder.compile(loss='mse', optimizer = tf.keras.optimizers.SGD(lr=1.5))

与上PCA自编码器区别就是使用了激活函数。训练模型,并且使他投影到2D:

history = autoencoder.fit(X_train, X_train, epochs = 20)
codings = encoder.predict(X_train)

注意,这里有两个X_train,分别表示输入和目标。
在这里插入图片描述在这里插入图片描述

基础自编码器完整代码:
完整代码:

import numpy as np
import tensorflow as tf
import matplotlib.pyplot as plt
from tensorflow.keras import Input
from tensorflow.keras import Model
from tensorflow.keras.layers import Dense

np.random.seed(4)

def generate_3d_data(m, w1=0.1, w2=0.3, noise=0.1):
    angles = np.random.rand(m) * 3 * np.pi / 2 - 0.5
    data = np.empty((m, 3))
    data[:, 0] = np.cos(angles) + np.sin(angles)/2 + noise * np.random.randn(m) / 2
    data[:, 1] = np.sin(angles) * 0.7 + noise * np.random.randn(m) / 2
    data[:, 2] = data[:, 0] * w1 + data[:, 1] * w2 + noise * np.random.randn(m)
    return data

X_train1 = generate_3d_data(60)
X_train = X_train1 - X_train1.mean(axis=0, keepdims=0)


x = Input(shape = (3,))
encoder = Dense(2, activation='selu')(x)
decoder = Dense(3, activation='selu')(encoder)

autoencoder = Model(x, decoder)
autoencoder.compile(loss='mse', optimizer = tf.keras.optimizers.SGD(lr=1.5))

np.random.seed(42)
tf.random.set_seed(42)

history = autoencoder.fit(X_train, X_train, epochs=20)
codings = autoencoder.predict(X_train)

fig = plt.figure(figsize=(4,3))
plt.plot(codings[:,0], codings[:, 1], "b.")
plt.xlabel("$z_1$", fontsize=18)
plt.ylabel("$z_2$", fontsize=18, rotation=0)
plt.grid(True)
plt.show()

fig = plt.figure()
ax = fig.add_subplot(projection='3d')
ax.scatter(X_train1[:, 0], X_train1[:, 1], X_train1[:, 2])
ax.set_xlabel('$x_1$'); ax.set_xlabel('$x_2$'); ax.set_xlabel('$x_3$')
plt.show()
2.3 多层自编码器

多层自编码器又叫堆叠式自编码器,与基础自编码器相比,它多了一层隐藏层,当然,这里可以不止一层。这个时候我们就叫它多层自编码器,或者深度自编码器。之所以添加更多的隐藏层,是为了有助于自编码器学习更复杂的编码。要注意不要使编码器过于强大,即不要过多的隐藏层,因为如果过多的隐藏层,甚至多余数据特征的维度,那么输出也是这样的结果,会导致编码器并没有学会数据特征,而只是将数据原封不动的传递给了解码器。这类似于PCA的维度的选择,如果选择过多的维度,将会导致并不能给出精确的结果。

2.3.1 多层编码器基础

下图是多层自编码器示意图:
在这里插入图片描述
我们使用MNIST的数据,然后用自编码器对其进行处理。MNIST数据有784个输入,然后是100个神经元的隐藏层,接着30个神经元的中间层,再然后是100个神经元的隐藏层,最后是784个神经元的输出层。
首先载入数据:

import numpy as np
import tensorflow as tf
(X_train_full, y_train_full), (X_test, y_test) = tf.datasets.fashion_mnist.load_data()
X_train_full = X_train_full.astype(np.float32) / 255
X_test = X_test.astype(np.float32) / 255
X_train, X_valid = X_train_full[:-5000], X_train_full[-5000:]
y_train, y_valid = y_train_full[:-5000], y_train_full[-5000:]

多层自编码网络:

tf.random.set_seed(42)
np.random.seed(42)

stacked_encoder = Sequential([
    Flatten(input_shape=[28, 28]),
    Dense(100, activation="selu"),
    Dense(30, activation="selu"),])
stacked_decoder = Sequential([
    Dense(100, activation="selu", input_shape=[30]),
    Dense(28 * 28, activation="sigmoid"),
    Reshape([28, 28]),])

stacked_ae = Sequential([stacked_encoder, stacked_decoder])
stacked_ae.compile(loss="binary_crossentropy",
                   optimizer=tf.keras.optimizers.SGD(lr=1.5), metrics=[rounded_accuracy])
history = stacked_ae.fit(X_train, X_train, epochs=20,
                         validation_data=(X_valid, X_valid))

输出结果:
在这里插入图片描述
对代码的解释:

多层自编码器全部代码:

import tensorflow as tf
from tensorflow.keras import Sequential, Model
from tensorflow.keras.layers import Flatten, Dense, Reshape

(X_train_full, y_train_full), (X_test, y_test) = tf.keras.datasets.fashion_mnist.load_data()
X_train_full = X_train_full.astype(np.float32) / 255
X_test = X_test.astype(np.float32) / 255
X_train, X_valid = X_train_full[:-5000], X_train_full[-5000:]
y_train, y_valid = y_train_full[:-5000], y_train_full[-5000:]

def rounded_accuracy(y_true, y_pred):
    return tf.keras.metrics.binary_accuracy(tf.round(y_true), tf.round(y_pred))

tf.random.set_seed(42)
np.random.seed(42)

stacked_encoder = Sequential([
    Flatten(input_shape=[28, 28]),
    Dense(100, activation="selu"),
    Dense(30, activation="selu"),])
    
stacked_decoder = Sequential([
    Dense(100, activation="selu", input_shape=[30]),
    Dense(28 * 28, activation="sigmoid"),
    Reshape([28, 28]),])
    
stacked_ae = Sequential([stacked_encoder, stacked_decoder])
stacked_ae.compile(loss="binary_crossentropy",
                   optimizer=tf.keras.optimizers.SGD(lr=1.5), metrics=[rounded_accuracy])
history = stacked_ae.fit(X_train, X_train, epochs=20,validation_data=(X_valid, X_valid))

def show_reconstructions(model, images=X_valid, n_images=5):
    reconstructions = model.predict(images[:n_images])
    fig = plt.figure(figsize=(n_images * 1.5, 3))
    for image_index in range(n_images):
        plt.subplot(2, n_images, 1 + image_index)
        plot_image(images[image_index])
        plt.subplot(2, n_images, 1 + n_images + image_index)
        plot_image(reconstructions[image_index])
show_reconstructions(stacked_ae)
plt.show()
2.3.2 多层编码器的优化

多层自编码器在训练的时候,会花费比较长的时间。主要因为每个神经元都会有一个权重,并且编码器和解码器都会有相应的权重。其实很多自编码器都是对称的,这时候我们就可以将编码器和解码器的权重进行绑定,这样权重就会减少一半,是的训练速度翻倍,并且降低过拟合的风险。例如,如果自编码器总共有N层(不计输入层)。为了绑定各层之间的权重,可以定义一个自定义层:

class DenseTranspose(Layer):
	def __init__(self, dense, activation=None, **kwargs):
		self.dense = dense
		self.activation = tf.keras.activations.get(activation)
		super().__init__(**kwargs)
	def build(self, batch_input_shape):
		self.biases = self.add_weight(name="bias", initializer="zeros", shape=[shelf.dense.input_shape[-1])
		super().build(batch_input_shape)
	def call(self, inputs):
		z = tf.matmul(inputs, self.dense.weights[0], transpose_b=True)
		return self.activation(z + self.biases)

这里定义的DenseTranspose相当于一般的Dense,用法也是一样的,但是它使用了另一个Dense层的权重,并进行了转置(其实这在TensorFlow2中经常常见)。那么就可以建立一个新的自编码器,即绑定权重多层自编码器:

keras.backend.clear_session()
tf.random.set_seed(42)
np.random.seed(42)

dense_1 = keras.layers.Dense(100, activation="selu")
dense_2 = keras.layers.Dense(30, activation="selu")

tied_encoder = keras.models.Sequential([
    keras.layers.Flatten(input_shape=[28, 28]),
    dense_1,
    dense_2
])

tied_decoder = keras.models.Sequential([
    DenseTranspose(dense_2, activation="selu"),
    DenseTranspose(dense_1, activation="sigmoid"),
    keras.layers.Reshape([28, 28])
])

tied_ae = keras.models.Sequential([tied_encoder, tied_decoder])

tied_ae.compile(loss="binary_crossentropy",
                optimizer=keras.optimizers.SGD(lr=1.5), metrics=[rounded_accuracy])
history = tied_ae.fit(X_train, X_train, epochs=10,
                      validation_data=(X_valid, X_valid))

得到结果:
在这里插入图片描述
绑定权重多层自编码器全部代码:

import numpy as np
import tensorflow as tf
from tensorflow.keras import Input
from tensorflow.keras import Model
from tensorflow.keras import Sequential
from tensorflow.keras.layers import Flatten, Dense, Reshape

def rounded_accuracy(y_true, y_pred):
    return keras.metrics.binary_accuracy(tf.round(y_true), tf.round(y_pred))
    
class DenseTranspose(tf.keras.layers.Layer):
    def __init__(self, dense, activation=None, **kwargs):
        self.dense = dense
        self.activation = tf.keras.activations.get(activation)
        super().__init__(**kwargs)
    def build(self, batch_input_shape):
        self.biases = self.add_weight(name="bias",
                                      shape=[self.dense.input_shape[-1]],
                                      initializer="zeros")
        super().build(batch_input_shape)
    def call(self, inputs):
        z = tf.matmul(inputs, self.dense.weights[0], transpose_b=True)
        return self.activation(z + self.biases)

tf.keras.backend.clear_session()
tf.random.set_seed(42)
np.random.seed(42)

dense_1 = Dense(100, activation="selu")
dense_2 = Dense(30, activation="selu")

tied_encoder = Sequential([
    Flatten(input_shape=[28, 28]),
    dense_1,
    dense_2
])

tied_decoder = Sequential([
    DenseTranspose(dense_2, activation="selu"),
    DenseTranspose(dense_1, activation="sigmoid"),
    Reshape([28, 28])
])

tied_ae = Sequential([tied_encoder, tied_decoder])

tied_ae.compile(loss="binary_crossentropy",
                optimizer=tf.keras.optimizers.SGD(lr=1.5), metrics=[rounded_accuracy])
history = tied_ae.fit(X_train, X_train, epochs=10,
                      validation_data=(X_valid, X_valid))

def plot_image(image):
    plt.imshow(image, cmap="binary")
    plt.axis("off")
def show_reconstructions(model, images=X_valid, n_images=5):
    reconstructions = model.predict(images[:n_images])
    fig = plt.figure(figsize=(n_images * 1.5, 3))
    for image_index in range(n_images):
        plt.subplot(2, n_images, 1 + image_index)
        plot_image(images[image_index])
        plt.subplot(2, n_images, 1 + n_images + image_index)
        plot_image(reconstructions[image_index])
show_reconstructions(tied_ae)
plt.show()
2.4 卷积自编码器

在神经网络中,卷积神经网络的用途非常广。那么在自编码器中是否也可以应用卷积层呢?答案是肯定的,自编码器里也可以加上卷积层。其实自编码器在处理图像上,并不能很好的处理,因为一般图像很大,自编码器需要会费很多时间来计算权重。而卷积神经网络中是非常善于处理图像的,那么我们让二者结合是否可以做到即可以处理图像,又可以达到无监督学习和降维的操作呢?答案也是肯定的!卷积自编码器同样包括编码器和解码器,编码器是卷积层和池化层组成的常规卷积神经网络,他通常会减小出入的空间尺寸(即高度和宽度),同时会增加深度(即特征图的数量)。解码器必须进行相反的操作(放大图像并减少其深度到原始尺寸),为此可以使用转置卷积层(逆卷积层)。其实后来演化的UNET网络等都是使用的这个思想,这里不再赘述。下面还是用MNIST的数据来验证卷积自编码器:

import tensorflow.keras.optimizers import SGD
from tensorflow.keras.model import Sequential
from tensorflow.keras.layers import Reshape, Conv2D, MaxPool2D, Conv2DTranspose
tf.random.set_seed(42)
np.random.seed(42)

conv_encoder = Sequential([
	Reshape([28, 28, 1], input_shape=[28, 28]),
    Conv2D(16, kernel_size=3, padding="SAME", activation="selu"),
    MaxPool2D(pool_size=2),
    Conv2D(32, kernel_size=3, padding="SAME", activation="selu"),
    MaxPool2D(pool_size=2),
    Conv2D(64, kernel_size=3, padding="SAME", activation="selu"),
    MaxPool2D(pool_size=2)
])
conv_decoder = Sequential([
    Conv2DTranspose(32, kernel_size=3, strides=2, padding="VALID", activation="selu",input_shape=[3, 3, 64]),
    Conv2DTranspose(16, kernel_size=3, strides=2, padding="SAME", activation="selu"),
    Conv2DTranspose(1, kernel_size=3, strides=2, padding="SAME", activation="sigmoid"),
    Reshape([28, 28])
])
conv_ae = Sequential([conv_encoder, conv_decoder])

conv_ae.compile(loss="binary_crossentropy", optimizer=SGD(lr=1.0),metrics=[rounded_accuracy])
history = conv_ae.fit(X_train, X_train, epochs=5,validation_data=(X_valid, X_valid))
show_reconstructions(conv_ae)
plt.show()

编码器模型:

_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
reshape_4 (Reshape)          (None, 28, 28, 1)         0         
_________________________________________________________________
conv2d_6 (Conv2D)            (None, 28, 28, 16)        160       
_________________________________________________________________
max_pooling2d_6 (MaxPooling2 (None, 14, 14, 16)        0         
_________________________________________________________________
conv2d_7 (Conv2D)            (None, 14, 14, 32)        4640      
_________________________________________________________________
max_pooling2d_7 (MaxPooling2 (None, 7, 7, 32)          0         
_________________________________________________________________
conv2d_8 (Conv2D)            (None, 7, 7, 64)          18496     
_________________________________________________________________
max_pooling2d_8 (MaxPooling2 (None, 3, 3, 64)          0         
=================================================================
Total params: 23,296
Trainable params: 23,296
Non-trainable params: 0
_________________________________________________________________

解码器模型:

_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
reshape_4 (Reshape)          (None, 28, 28, 1)         0         
_________________________________________________________________
conv2d_6 (Conv2D)            (None, 28, 28, 16)        160       
_________________________________________________________________
max_pooling2d_6 (MaxPooling2 (None, 14, 14, 16)        0         
_________________________________________________________________
conv2d_7 (Conv2D)            (None, 14, 14, 32)        4640      
_________________________________________________________________
max_pooling2d_7 (MaxPooling2 (None, 7, 7, 32)          0         
_________________________________________________________________
conv2d_8 (Conv2D)            (None, 7, 7, 64)          18496     
_________________________________________________________________
max_pooling2d_8 (MaxPooling2 (None, 3, 3, 64)          0         
=================================================================
Total params: 23,296
Trainable params: 23,296
Non-trainable params: 0
_________________________________________________________________

输出的结果:
在这里插入图片描述
卷积自编码器完整代码:

from tensorflow.keras.optimizers import SGD
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Reshape, Conv2D, MaxPool2D, Conv2DTranspose
tf.random.set_seed(42)
np.random.seed(42)

def rounded_accuracy(y_true, y_pred):
    return tf.keras.metrics.binary_accuracy(tf.round(y_true), tf.round(y_pred))
    
conv_encoder = Sequential([
	Reshape([28, 28, 1], input_shape=[28, 28]),
    Conv2D(16, kernel_size=3, padding="SAME", activation="selu"),
    MaxPool2D(pool_size=2),
    Conv2D(32, kernel_size=3, padding="SAME", activation="selu"),
    MaxPool2D(pool_size=2),
    Conv2D(64, kernel_size=3, padding="SAME", activation="selu"),
    MaxPool2D(pool_size=2)
])
conv_decoder = Sequential([
    Conv2DTranspose(32, kernel_size=3, strides=2, padding="VALID", activation="selu",input_shape=[3, 3, 64]),
    Conv2DTranspose(16, kernel_size=3, strides=2, padding="SAME", activation="selu"),
    Conv2DTranspose(1, kernel_size=3, strides=2, padding="SAME", activation="sigmoid"),
    Reshape([28, 28])
])
conv_ae = Sequential([conv_encoder, conv_decoder])

conv_ae.compile(loss="binary_crossentropy", optimizer=SGD(lr=1.0),metrics=[rounded_accuracy])
history = conv_ae.fit(X_train, X_train, epochs=5,validation_data=(X_valid, X_valid))

conv_encoder.summary()
conv_decoder.summary()

def plot_image(image):
    plt.imshow(image, cmap="binary")
    plt.axis("off")
    
def show_reconstructions(model, images=X_valid, n_images=5):
    reconstructions = model.predict(images[:n_images])
    fig = plt.figure(figsize=(n_images * 1.5, 3))
    for image_index in range(n_images):
        plt.subplot(2, n_images, 1 + image_index)
        plot_image(images[image_index])
        plt.subplot(2, n_images, 1 + n_images + image_index)
        plot_image(reconstructions[image_index])
        
show_reconstructions(conv_ae)
plt.show()
2.5 循环自编码器

如果我们的数据是时间序列或者文本的时候,例如实时观测数据、股票信息、文本翻译等,那么递归神经网络可能更适合处理这些数据。那么同样是对数据进行降维,即循环自编码器。循环自编码器依然有两个部分:解码器和编码器。解码器是RNN,解码器与之相反。由于其同卷积自编码器很类似,这里只给出完整代码:

from tensorflow.keras.optimizers import SGD
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import LSTM, RepeatVector, TimeDistributed
tf.random.set_seed(42)
np.random.seed(42)

def rounded_accuracy(y_true, y_pred):
    return tf.keras.metrics.binary_accuracy(tf.round(y_true), tf.round(y_pred))
recurrent_encoder = Sequential([
	LSTM(100, return_sequences=True, input_shape=[28, 28]),
    LSTM(30)])
    
recurrent_decoder = Sequential([
	RepeatVector(28, input_shape=[30]),
    LSTM(100, return_sequences=True),
    TimeDistributed(keras.layers.Dense(28, activation="sigmoid"))])
    
recurrent_ae = Sequential([recurrent_encoder, recurrent_decoder])
recurrent_ae.compile(loss="binary_crossentropy", optimizer=SGD(lr=0.1))

history = recurrent_ae.fit(X_train, X_train, epochs=10, validation_data=(X_valid, X_valid))

def plot_image(image):
    plt.imshow(image, cmap="binary")
    plt.axis("off")
    
def show_reconstructions(model, images=X_valid, n_images=5):
    reconstructions = model.predict(images[:n_images])
    fig = plt.figure(figsize=(n_images * 1.5, 3))
    for image_index in range(n_images):
        plt.subplot(2, n_images, 1 + image_index)
        plot_image(images[image_index])
        plt.subplot(2, n_images, 1 + n_images + image_index)
        plot_image(reconstructions[image_index])
        
show_reconstructions(conv_ae)
plt.show()

结果:
在这里插入图片描述

2.5 去噪自编码器

去噪自编码器是正则编码器的一种。去噪自编码器,顾名思义是具有去除噪声的特性。即在输入数据中加入白噪声,只要不破坏原始数据,去噪声自编码器都比较好的将噪声去除,而恢复出原始数据。我们还是使用MNIST数据,由于原始数据没有噪声,这里我们需要给他加一个白噪声,只需要在输入层中附加一个GaussianNoise层即可,而这个高斯噪声层在神经网络中相当于Dropout层。我们直到,Dropout层只在训练的时候会被激活,同样GaussianNoise也是一样的:

import numpy as np
import tensorflow as tf
from tensorflow.keras.optimizers import SGD
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Flatten, GaussianNoise, Dense, Reshape

tf.random.set_seed(42)
np.random.seed(42)

def rounded_accuracy(y_true, y_pred):
    return tf.keras.metrics.binary_accuracy(tf.round(y_true), tf.round(y_pred))

denoising_encoder = Sequential([
	Flatten(input_shape=[28, 28]),
	GaussianNoise(0.2),
	Dense(100, activation="selu"),
	Dense(30, activation="selu")])
	
denoising_decoder = Sequential([
	Dense(100, activation="selu", input_shape=[30]),
    Dense(28 * 28, activation="sigmoid"),
    Reshape([28, 28])])
    
denoising_ae = Sequential([denoising_encoder, denoising_decoder])
denoising_ae.compile(loss="binary_crossentropy", optimizer=SGD(lr=1.0),metrics=[rounded_accuracy])
history = denoising_ae.fit(X_train, X_train, epochs=10, validation_data=(X_valid, X_valid))

def plot_image(image):
    plt.imshow(image, cmap="binary")
    plt.axis("off")
    
def show_reconstructions(model, images=X_valid, n_images=5):
    reconstructions = model.predict(images[:n_images])
    fig = plt.figure(figsize=(n_images * 1.5, 3))
    for image_index in range(n_images):
        plt.subplot(2, n_images, 1 + image_index)
        plot_image(images[image_index])
        plt.subplot(2, n_images, 1 + n_images + image_index)
        plot_image(reconstructions[image_index])
        
noise = tf.keras.layers.GaussianNoise(0.2)
show_reconstructions(denoising_ae, noise(X_valid, training=True))
plt.show()

结果:
在这里插入图片描述

同样,如果将上述程序中的GaussianNoise换成Dropout也是同样的,这里不再赘述。

2.6 稀疏自编码器

稀疏自编码器也是正则编码器的一种。稀疏正则化的特性将有利于数据的特征提取,通过损失函数中添加适当的函数项,强迫自编码器减少编码器中活动神经元的数量。稀疏正则化的自编码器必须反映训练数据集的独特统计特征,并不是简单的充当恒等函数。以这种方式训练,执行附带稀疏惩罚的复现任务能得到有用特征的模型。例如,强迫编码层中平均有5%的显著活动神经元,这迫使自编码器将每个输入表示为少量活动神经元的组合。结果会导致编码层中的每个神经元最终会代表一个有用的特征。

还有一种方法用来约束自编码器重构的方法,是对损失函数施加约束。比如,对损失函数添加一个正则化约束,这样会使自编码器学习到数据的系数表征。

一种简单的方法是在编码器中使用sigmoid激活函数(将编码限制在0到1之间的值),使用较大的编码层,并向编码层添加L1正则化:

import numpy as np
import tensorflow as tf
from tensorflow.keras.optimizers import SGD
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Flatten, Dense, ActivityRegularization

tf.random.set_seed(42)
np.random.seed(42)

def rounded_accuracy(y_true, y_pred):
    return tf.keras.metrics.binary_accuracy(tf.round(y_true), tf.round(y_pred))

sparse_l1_encoder = Sequential([
	Flatten(input_shape=[28, 28]),
    Dense(100, activation="selu"),
    Dense(300, activation="sigmoid"),
    ActivityRegularization(l1=1e-3)])
    
sparse_l1_decoder = Sequential([
    Dense(100, activation="selu", input_shape=[300]),
    Dense(28 * 28, activation="sigmoid"),
    Reshape([28, 28])
])
sparse_l1_ae = Sequential([sparse_l1_encoder, sparse_l1_decoder])
sparse_l1_ae.compile(loss="binary_crossentropy", optimizer=SGD(lr=1.0),metrics=[rounded_accuracy])
history = sparse_l1_ae.fit(X_train, X_train, epochs=10,validation_data=(X_valid, X_valid))

def plot_image(image):
    plt.imshow(image, cmap="binary")
    plt.axis("off")
    
def show_reconstructions(model, images=X_valid, n_images=5):
    reconstructions = model.predict(images[:n_images])
    fig = plt.figure(figsize=(n_images * 1.5, 3))
    for image_index in range(n_images):
        plt.subplot(2, n_images, 1 + image_index)
        plot_image(images[image_index])
        plt.subplot(2, n_images, 1 + n_images + image_index)
        plot_image(reconstructions[image_index])

show_reconstructions(sparse_l1_ae)
plt.show()

结果:
在这里插入图片描述

二、 主成分分析(PCA)

待续。。。
如果有问题,请留言,多谢!

标签:编码器,Autoencoder,keras,train,image,28,Vs,images,PCA
来源: https://blog.csdn.net/qq_28531269/article/details/120976178