Updated 19/Nov/2021 by Yoshihisa Nitta 
Variational Auto Encoder Training for MNIST dataset with Tensorflow 2 on Google Colab¶
Train Variational Auto Encoder on MNIST dataset. Variational Auto Encoder is a modification of the encoder and loss function for Auto Encoder.
MNIST データセットに対して Variational Auto Encoder をGoogle Colab 上の Tensorflow 2 で学習する¶
MNIST データセットに対して変分オートエンコーダを学習させる。 Variational Auto Encoder とは、AutoEncoder に対して encoder と losss function (損失関数) に変更を加えたものである。
#! pip install tensorflow==2.7.0
%tensorflow_version 2.x
import tensorflow as tf
print(tf.__version__)
About Encoder of VariationalAutoEncoder¶
With AutoEncoder, eath image is directly mapped to one point in the latent space. With VariationalAutoEncoder, each image is mapped to a multivariate normal distribution around a point in latent space.
The covariance matrix is a diagonal matrix because VariationalAutoEncoder considers that there is no correlation between any dimensions of the latent space. As a result, the encoder only needs to map each input to the mean vector and the variance vector, and does not have to worry about the correlation between the dimensions. Furthermore, when mapped to the logarithm of the variance, any real number in the range($-\infty$, $\infty$) can be assigned.
VariationalAutoEncoder の Encoder について¶
AutoEncoder では、各画像は潜在空間の1点に直接写像される。 VariationalAutoEncoder では、各画像は潜在空間のある点の周りの多変量正規分布に写像される。
変分オートエンコーダでは、潜在空間のどの次元間にも相関がないとみなすので、共分散行列は対角行列になる。 これにより、エンコーダは各入力を平均ベクトルと分散ベクトルに写像すればよく、次元間の相関を気にする必要はない。 さらに、分散の対数に写像すると (−∞,∞) の範囲のどのような実数でもとれる。
Covariance Matrix¶
The variance-covariance matrix is a matrix obtained by extending the concept of variance (an index showing the degree of distribution) to multidimensional random varaible.
Difinition for 2 random variables¶
For the random variables $X_1$ and $X_2$, the variance-covariance matrix is define as follows.
$\Sigma = \left( \begin{array}{cc} \sigma_1^2 & \sigma_{12} \\ \sigma_{12} & \sigma_2^2 \\ \end{array} \right )$
where
$\sigma_1^2 = \mbox{variance of } X_1 $,
$\sigma_2^2 = \mbox{variance of } X_2 $,
$\sigma_{12} = \mbox{covariance of } X_1 \mbox{ and } X_2$
$\Sigma$ is called a variance-covariance matrix because the variances are lined up on the diagonal components and the covariances are lined up on the off-diagonal components.
It is defined in the same way when there are $n$ random variables.
For the random variables $X_1$, $\cdots$, $X_n$, An $n\times n$ matrix is called a variance-covariance matris where the $ii$ component is $\sigma_i^2$, and the $ij$ component ($i\neq j$) is $\sigma_{ij}$.
Example: Find the variance and covariance of 5 data in 2 variables.¶
Suppose that $(x_i, y_i) = (4,5), (5, 7), (6,6), (7,9), (8,8)$ are given as data.
Mean of $x$ : $\mu_x = \displaystyle \frac{1}{5} (4+5+6+7+8)=6$,
Mean of $y$ : $\mu_y = \displaystyle \frac{1}{5}(5 + 7+6+9+8)=7$
Variance of $x$:
$\sigma_x^2 = \displaystyle\frac{1}{5}\sum_{k=1}^5 (x_i - \mu_x)^2 \\
\quad = \displaystyle \frac{1}{5} ((4-6)^2+(5-6)^2+(6-6)^2 + (7-6)^2 +(8-6)^2 ) \\
\quad = \displaystyle \frac{1}{5} (2^2 + 1^2 + 0^2 + 1^2 + 2^2) = 2$
Variance of $y$
$\sigma_y^2 = \displaystyle\frac{1}{5} \sum_{k=1}^{5} (y_i - \mu_y)^2 \\
\quad = \displaystyle{1}{5}((5-7)^2 +(7-7)^2 +(6-7)^2 +(9-7)^2 +(8-7)^2 ) \\
\quad = \displaystyle \frac{1}{5} (2^2 + 0^2 + 1^2 + 2^2 + 1^2) = 2$
Covariance of $x$ and $y$:
$\sigma_{xy} = \displaystyle \frac{1}{5} \sum_{k=1}^5 (x_i - \mu_x) (y_i - \mu_y) \\
\quad = \displaystyle \frac{1}{5} ((4-6)(5-7)+(5-6)(7-7)+(6-6)(6-7)+(7-6)(9-7)+(8-6)(8-7)) \\
\quad = \displaystyle \frac{1}{5}((-2)(-2) + (-1)\cdot 0 + 0 \cdot (-1) + 1 \cdot 2 + 2 \cdot 1)=\frac{8}{5}=1.6
$
Variance and covariance matrix of $x$ and $y$:
$\Sigma = \left(
\begin{array}{cc}
\sigma_x^2 & \sigma_{xy} \\
\sigma_{xy} & \sigma_y^2 \\
\end{array}
\right) = \left(
\begin{array}{cc}
2 & 1.6 \\
1.6 & 2 \\
\end{array}
\right)$
分散共分散行列¶
分散共分散行列とは、分散(散らばり具合を表す指標)の概念を多次元確率変数に拡張して行列としたもの。単に共分散行列と呼ぶこともある。
確率変数が2つの場合の定義¶
確率変数 $X_1$, $X_2$ に対して、分散共分散行列 を
$\Sigma =
\left( \begin{array}{cc}
\sigma_1^2 & \sigma_{12} \\
\sigma_{12} & \sigma_2^2 \\
\end{array} \right )$
と定義する。
ただし、
$\sigma_1^2 = X_1 \mbox{の分散}$,
$\sigma_2^2 = X_2 \mbox{の分散}$,
$\sigma_{12} = X_1 \mbox{と} X_2 \mbox{の共分散}$
を表す。対角成分に分散が並び、非対角成分には共分散が並ぶため、分散共分散行列と呼ばれる。
確率変数が $n$ 個の場合も同様に定義される。
確率変数 $X_1$, $\cdots$, $X_n$ に対して、
第 $ii$ 成分が $\sigma_i^2$,
第 $ij$ 成分 ($i \neq j$)が $\sigma_{ij}$
であるような $n\times n$行列 $\Sigma$ を
分散共分散行列と呼ぶ。
例題: 2変数の5個のデータの分散と共分散を求めよ。¶
データとして $(x_i, y_i) = (4, 5), (5, 7), (6, 6), (7,9), (8, 8)$ が与えられたとする。
$x$の平均
$\mu_x = \displaystyle\frac{1}{5} (4 + 5 + 6 + 7 + 8) = 6$,
$y$ の平均
$\mu_y = \displaystyle\frac{1}{5} (5 + 7 + 6 + 9 + 8) = 7$
$x$ の分散は
$\sigma_x^2 = \displaystyle\frac{1}{5}\sum_{k=1}^5 (x_i - \mu_x)^2 \\
\quad = \displaystyle \frac{1}{5} ((4-6)^2+(5-6)^2+(6-6)^2 + (7-6)^2 +(8-6)^2 ) \\
\quad = \displaystyle \frac{1}{5} (2^2 + 1^2 + 0^2 + 1^2 + 2^2) = 2$
$y$の分散は
$\sigma_y^2 = \displaystyle\frac{1}{5} \sum_{k=1}^{5} (y_i - \mu_y)^2 \\
\quad = \displaystyle{1}{5}((5-7)^2 +(7-7)^2 +(6-7)^2 +(9-7)^2 +(8-7)^2 ) \\
\quad = \displaystyle \frac{1}{5} (2^2 + 0^2 + 1^2 + 2^2 + 1^2) = 2$
$x$ と $y$の共分散は
$\sigma_{xy} = \displaystyle \frac{1}{5} \sum_{k=1}^5 (x_i - \mu_x) (y_i - \mu_y) \\
\quad = \displaystyle \frac{1}{5} ((4-6)(5-7)+(5-6)(7-7)+(6-6)(6-7)+(7-6)(9-7)+(8-6)(8-7)) \\
\quad = \displaystyle \frac{1}{5}((-2)(-2) + (-1)\cdot 0 + 0 \cdot (-1) + 1 \cdot 2 + 2 \cdot 1)=\frac{8}{5}=1.6
$
$x$ と $y$ の分散共分散行列は
$\Sigma = \left(
\begin{array}{cc}
\sigma_x^2 & \sigma_{xy} \\
\sigma_{xy} & \sigma_y^2 \\
\end{array}
\right) = \left(
\begin{array}{cc}
2 & 1.6 \\
1.6 & 2 \\
\end{array}
\right)$
Normal Distribution¶
Probability density function of one-dimensional normal distribution
$\displaystyle f(x | \mu, \sigma^2) = \frac{1}{\sqrt{2\pi\sigma^2}} e^{-\frac{(x-\mu)^2}{2\sigma^2}}$
where mean $\mu$, variance $\sigma^2$, standard deviation $\sigma$.
正規分布¶
平均(mean) $\mu$, 分散(variance) $\sigma^2$, 標準偏差(standard deviatioin) $\sigma$ として1次元の正規分布の確率密度関数
$\displaystyle f(x | \mu, \sigma^2) = \frac{1}{\sqrt{2\pi\sigma^2}} e^{-\frac{(x-\mu)^2}{2\sigma^2}}$
%matplotlib inline
import matplotlib.pyplot as plt
import numpy as np
x = np.linspace(-5, 5, 200)
def f(x, m, v):
d = x-m
return np.exp(- d*d / (2.0 * v)) / np.sqrt(2 * np.pi * v)
fig, ax = plt.subplots(1,1,figsize=(8,6))
ax.plot(x, f(x, 0.0, 0.2),label='str mean=0.0, variance=0.2',color='blue')
ax.plot(x, f(x, 0.0, 1.0),label='str mean=0.0, variance=1.0',color='red')
ax.plot(x, f(x, 0.0, 5.0),label='str mean=0.0, variance=5.0',color='orange')
ax.plot(x, f(x, -2.0, 0.5),label='str mean=-2.0, variance=0.5',color='green')
plt.legend()
plt.show()
VariationalAutoEncoder¶
Sample the point $z$ as follows.
$ z = \mu + \sigma \epsilon$
$\mu$ represents where to place the marker, $\sigma$ is its certainty, and $\epsilon$ is a randomly selected value according to the probability distribution. The point is tried to place around the $\mu$, so it is expected that the latent space will be continuous.
Since the relationship
$x = \displaystyle e^{\log x}$ holds, the following formula holds.
$\sigma = \displaystyle e^{\log \sigma} = \displaystyle e^{\frac{2 \log \sigma}{2}} = \displaystyle e^{\frac{\log \sigma^2}{2}}$
Therefore, using the calculated $\log$ of variance $\sigma^2$, calculate the following equation for each dimension.
$\mbox{sigma} = \sigma = \mbox{exp(log_var/2)}$
The features of the new variational encoder are as follows.
- Instead of connecting the
Flattenlayer directly to the layer in the latent space, connect it to the layers ofmuandlog_var. - The
Samplinglayer samples points in the latent space from the normal distribution defined bymuandlog_var. - There are 3 types of encoder model outputs:
mu,log_varandz.
変分オートエンコーダ¶
次の式を使って点 $z$ をサンプリングする。
$ z = \mu + \sigma \epsilon$
$\mu$ は目印をどこに置くか表し、$\sigma$ はその確信度、$\epsilon$ はどのぐらい離れておくかを確率分布に従ってランダムに選んだ値となる。 $\mu$ を目標としてその周囲に置こうとするので、 潜在空間が連続となることが期待される。
$x = \displaystyle e^{\log x}$ という関係が成り立つので $\sigma = \displaystyle e^{\log \sigma} = \displaystyle e^{\frac{2 \log \sigma}{2}} = \displaystyle e^{\frac{\log \sigma^2}{2}}$ が言える。 したがって、 分散 $\sigma^2$ の $\log$ を計算したもの $\mbox{log_var} = \log \sigma^2$ を使って、 $\mbox{sigma} = \sigma = \mbox{exp(log_var/2)}$ を各次元について計算する。
新しい変分エンコーダの特徴は次の通り。
- Flatten 層を直接潜在空間の層に接続するのではなく mu と log_var の層に接続する。
- Sampling 層は、mu と log_var で定義される正規分布から潜在空間内の点をサンプリングする。
- encoderのモデルの出力は mu, log_var, z の3種類となる。
! nvidia-smi
! cat /proc/cpuinfo
! cat /etc/issue
! free -h
from google.colab import drive
drive.mount('/content/drive')
! ls /content/drive
Download source file from Google Drive or nw.tsuda.ac.jp¶
Basically, gdown from Google Drive. Download from nw.tsuda.ac.jp above only if the specifications of Google Drive change and you cannot download from Google Drive.
Google Drive または nw.tsuda.ac.jp からファイルをダウンロードする¶
基本的に Google Drive から gdown してください。 Google Drive の仕様が変わってダウンロードができない場合にのみ、nw.tsuda.ac.jp からダウンロードしてください。
# Download source file
nw_path = './nw'
! rm -rf {nw_path}
! mkdir -p {nw_path}
if True: # from Google Drive
url_model = 'https://drive.google.com/uc?id=1ZCihR7JkMOity4wCr66ZCp-3ZOlfwwo3'
! (cd {nw_path}; gdown {url_model})
else: # from nw.tsuda.ac.jp
URL_NW = 'https://nw.tsuda.ac.jp/lec/GoogleColab/pub'
url_model = f'{URL_NW}/models/VariationalAutoEncoder.py'
! wget -nd {url_model} -P {nw_path}
! cat {nw_path}/VariationalAutoEncoder.py
%tensorflow_version 2.x
import tensorflow as tf
import numpy as np
print(tf.__version__)
# prepare data
(x_train_raw, y_train_raw), (x_test_raw, y_test_raw) = tf.keras.datasets.mnist.load_data()
print(x_train_raw.shape)
print(y_train_raw.shape)
print(x_test_raw.shape)
print(y_test_raw.shape)
x_train = x_train_raw.reshape(x_train_raw.shape+(1,)).astype('float32') / 255.0
x_test = x_test_raw.reshape(x_test_raw.shape+(1,)).astype('float32') / 255.0
print(x_train.shape)
print(x_test.shape)
%matplotlib inline
import matplotlib.pyplot as plt
import numpy as np
N = 10
selected_indices = np.random.choice(x_train_raw.shape[0], N)
fig, ax = plt.subplots(1, N, figsize=(2.8 * N, 2.8))
for i in range(N):
ax[i].imshow(x_train_raw[selected_indices[i]],cmap='gray')
ax[i].axis('off')
plt.show()
from nw.VariationalAutoEncoder import VariationalAutoEncoder
vae = VariationalAutoEncoder(
input_dim = (28, 28, 1),
encoder_conv_filters = [32, 64, 64, 64],
encoder_conv_kernel_size = [3, 3, 3, 3],
encoder_conv_strides = [1, 2, 2, 1],
decoder_conv_t_filters = [64, 64, 32, 1],
decoder_conv_t_kernel_size = [3, 3, 3, 3],
decoder_conv_t_strides = [1, 2, 2, 1],
z_dim = 2,
r_loss_factor = 1000
)
vae.encoder.summary()
vae.decoder.summary()
Training¶
Train in 3 ways.
[Caution]
Note that if you call the save_image() function in the Training (2), (3) below, encoder.predict() and decoder.predict() will work and the execution will be slow.
学習する¶
3通りで学習する。
[注意]
以下の学習 (2), (3) の途中で save_images()関数を呼び出すと、 encoder.predict()と decoder.predict() が動作して、実行が非常に遅くなるので注意すること。
MAX_EPOCHS = 200
learning_rate = 0.0005
(1) Training: Use vae.model.fit()¶
Note that the loss function is not specified at the call of vae.model.compile() function.
Since it cannot be calculated simply using y_true and y_pred, the train_step() function of the VAEModel class called from fit() is used to find loss and gradients and train them.
The self.optimizer of the VAEModel class referenced in the train_step() function is the optimizer given by the compile() function.
(1) 学習する: vae.model.fit() を使う¶
vae.model.compile() 関数の呼び出しにおいて、loss関数を指定しないことに注意が必要である。
y_true と y_pred を使って単純に計算できないので、fit() から呼び出される
VAEModel クラスの
train_step() 関数でlossとgradientsを求めて、trainingする。
train_step() 関数の中で参照される VAEModel クラスの self.optimizer は compile() 関数で与えられた optimizer である。
import os
save_path1 = '/content/drive/MyDrive/ColabRun/VAE01'
optimizer = tf.keras.optimizers.Adam(learning_rate = learning_rate)
vae.model.compile(optimizer=optimizer)
# まず、少ない回数 training してみる
history = vae.train_with_fit(
x_train,
batch_size = 32,
epochs = 3,
run_folder = save_path1
)
print(history.history)
print(history.history.keys())
#tmp = history.history['loss']
#print(len(tmp))
#print(len(tmp[0]))
loss1_1 = history.history['loss']
rloss1_1 = history.history['reconstruction_loss']
kloss1_1 = history.history['kl_loss']
# Load the saved parameters and weights.
# 保存してある学習結果をロードする。
vae_work = VariationalAutoEncoder.load(save_path1)
# Display the epoch count of the model.
# training のepoch回数を表示する。
print(vae_work.epoch)
# Training in addition
# 追加で training する。
vae_work.model.compile(optimizer)
history2 = vae_work.train_with_fit(
x_train,
batch_size = 32,
epochs = MAX_EPOCHS,
run_folder = save_path1
)
print(len(history2.history))
loss1_2 = history2.history['loss']
rloss1_2 = history2.history['reconstruction_loss']
kloss1_2 = history2.history['kl_loss']
loss1 = np.concatenate([loss1_1, loss1_2], axis=0)
rloss1 = np.concatenate([rloss1_1, rloss1_2], axis=0)
kloss1 = np.concatenate([kloss1_1, kloss1_2], axis=0)
VariationalAutoEncoder.plot_history([loss1, rloss1, kloss1], ['total_loss', 'reconstruct_loss', 'kl_loss'])
selected_indices = np.random.choice(range(len(x_test)), 10)
selected_images = x_test[selected_indices]
z_mean, z_log_var, z = vae_work.encoder(selected_images)
reconst_images = vae_work.decoder(z).numpy() # Convert Tensor to numpy array.
txts = [f'{p[0]:.3f}, {p[1]:.3f}' for p in z ]
%matplotlib inline
VariationalAutoEncoder.showImages(selected_images, reconst_images, txts, 1.4, 1.4)
(2) Training with tf.GradientTape() function.¶
Instead of using fit(), calculate the loss in your own train() function, find the gradients, and apply them to the variables.
The train_tf() function is speeding up by declaring <code>@tf.function</code> the compute_loss_and_grads() function.
(2) tf.GradientTape() 関数を使った学習¶
fit() 関数を使わずに、自分で記述した train() 関数内で loss を計算し、gradients を求めて、変数に適用する。
train_tf() 関数では、lossとgradientsの計算を行う compute_loss_and_grads() 関数を <code>@tf.function</code> 宣言することで高速化を図っている。
save_path2 = '/content/drive/MyDrive/ColabRun/VAE02/'
from nw.VariationalAutoEncoder import VariationalAutoEncoder
vae2 = VariationalAutoEncoder(
input_dim = (28, 28, 1),
encoder_conv_filters = [32, 64, 64, 64],
encoder_conv_kernel_size = [3, 3, 3, 3],
encoder_conv_strides = [1, 2, 2, 1],
decoder_conv_t_filters = [64, 64, 32, 1],
decoder_conv_t_kernel_size = [3, 3, 3, 3],
decoder_conv_t_strides = [1, 2, 2, 1],
z_dim = 2,
r_loss_factor = 1000
)
optimizer2 = tf.keras.optimizers.Adam(learning_rate=learning_rate)
log2_1 = vae2.train_tf(
x_train,
batch_size = 32,
epochs = 3,
shuffle=True,
run_folder = save_path2,
optimizer = optimizer2,
save_epoch_interval=50,
validation_data=x_test
)
print(log2_1.keys())
loss2_1 = log2_1['loss']
rloss2_1 = log2_1['reconstruction_loss']
kloss2_1 = log2_1['kl_loss']
val_loss2_1 = log2_1['val_loss']
val_rloss2_1 = log2_1['val_reconstruction_loss']
val_kloss2_1 = log2_1['val_kl_loss']
# Load the saved parameters and weights.
# 保存したパラメータと重みを読み込む
vae2_work = VariationalAutoEncoder.load(save_path2)
print(vae2_work.epoch)
# Train in addition
# 追加で training する。
log2_2 = vae2_work.train_tf(
x_train,
batch_size = 32,
epochs = MAX_EPOCHS,
shuffle=True,
run_folder = save_path2,
optimizer = optimizer2,
save_epoch_interval=50,
validation_data=x_test
)
loss2_2 = log2_2['loss']
rloss2_2 = log2_2['reconstruction_loss']
kloss2_2 = log2_2['kl_loss']
val_loss2_2 = log2_2['val_loss']
val_rloss2_2 = log2_2['val_reconstruction_loss']
val_kloss2_2 = log2_2['val_kl_loss']
loss2 = np.concatenate([loss2_1, loss2_2], axis=0)
rloss2 = np.concatenate([rloss2_1, rloss2_2], axis=0)
kloss2 = np.concatenate([kloss2_1, kloss2_2], axis=0)
val_loss2 = np.concatenate([val_loss2_1, val_loss2_2], axis=0)
val_rloss2 = np.concatenate([val_rloss2_1, val_rloss2_2], axis=0)
val_kloss2 = np.concatenate([val_kloss2_1, val_kloss2_2], axis=0)
VariationalAutoEncoder.plot_history(
[loss2, val_loss2],
['total_loss', 'val_total_loss']
)
VariationalAutoEncoder.plot_history(
[rloss2, val_rloss2],
['reconstruction_loss', 'val_reconstruction_loss']
)
VariationalAutoEncoder.plot_history(
[kloss2, val_kloss2],
['kl_loss', 'val_kl_loss']
)
z_mean2, z_log_var2, z2 = vae2_work.encoder(selected_images)
reconst_images2 = vae2_work.decoder(z2).numpy() # @tf.function 宣言のためdecoder() の返り値はTensorになっているのでnumpyの配列に変換する
txts2 = [f'{p[0]:.3f}, {p[1]:.3f}' for p in z2 ]
%matplotlib inline
VariationalAutoEncoder.showImages(selected_images, reconst_images2, txts2, 1.4, 1.4)
(3) Trainig with tf.GradientTape() function and Learning rate decay¶
Calculate the loss and gradients with the tf.GradientTape() function, and apply the gradients to the variables.
In addition, perform Learning rate decay in the optimizer.
(3) tf.GradientTape() 関数と学習率減数を使った学習¶
tf.GradientTape() 関数を使って loss と gradients を計算して、gradients を変数に適用する。
さらに、optimizer において Learning rate decay を行う。
save_path3 = '/content/drive/MyDrive/ColabRun/VAE03/'
from nw.VariationalAutoEncoder import VariationalAutoEncoder
vae3 = VariationalAutoEncoder(
input_dim = (28, 28, 1),
encoder_conv_filters = [32, 64, 64, 64],
encoder_conv_kernel_size = [3, 3, 3, 3],
encoder_conv_strides = [1, 2, 2, 1],
decoder_conv_t_filters = [64, 64, 32, 1],
decoder_conv_t_kernel_size = [3, 3, 3, 3],
decoder_conv_t_strides = [1, 2, 2, 1],
z_dim = 2,
r_loss_factor = 1000
)
# initial_learning_rate * decay_rate ^ (step // decay_steps)
lr_schedule = tf.keras.optimizers.schedules.ExponentialDecay(
initial_learning_rate = learning_rate,
decay_steps = 1000,
decay_rate=0.96
)
optimizer3 = tf.keras.optimizers.Adam(learning_rate=lr_schedule)
log3_1 = vae3.train_tf(
x_train,
batch_size = 32,
epochs = 3,
shuffle=True,
run_folder = save_path3,
optimizer = optimizer3,
save_epoch_interval=50,
validation_data=x_test
)
print(log3_1.keys())
loss3_1 = log3_1['loss']
rloss3_1 = log3_1['reconstruction_loss']
kloss3_1 = log3_1['kl_loss']
val_loss3_1 = log3_1['val_loss']
val_rloss3_1 = log3_1['val_reconstruction_loss']
val_kloss3_1 = log3_1['val_kl_loss']
# 保存したパラメータと重みを読み込む
vae3_work = VariationalAutoEncoder.load(save_path3)
print(vae3_work.epoch)
# 追加で training する。
log3_2 = vae3_work.train_tf(
x_train,
batch_size = 32,
epochs = MAX_EPOCHS,
shuffle=True,
run_folder = save_path3,
optimizer = optimizer3,
save_epoch_interval=50,
validation_data=x_test
)
loss3_2 = log3_2['loss']
rloss3_2 = log3_2['reconstruction_loss']
kloss3_2 = log3_2['kl_loss']
val_loss3_2 = log3_2['val_loss']
val_rloss3_2 = log3_2['val_reconstruction_loss']
val_kloss3_2 = log3_2['val_kl_loss']
loss3 = np.concatenate([loss3_1, loss3_2], axis=0)
rloss3 = np.concatenate([rloss3_1, rloss3_2], axis=0)
kloss3 = np.concatenate([kloss3_1, kloss3_2], axis=0)
val_loss3 = np.concatenate([val_loss3_1, val_loss3_2], axis=0)
val_rloss3 = np.concatenate([val_rloss3_1, val_rloss3_2], axis=0)
val_kloss3 = np.concatenate([val_kloss3_1, val_kloss3_2], axis=0)
VariationalAutoEncoder.plot_history(
[loss3, val_loss3],
['total_loss', 'val_total_loss']
)
VariationalAutoEncoder.plot_history(
[rloss3, val_rloss3],
['reconstruction_loss', 'val_reconstruction_loss']
)
VariationalAutoEncoder.plot_history(
[kloss3, val_kloss3],
['kl_loss', 'val_kl_loss']
)
z_mean3, z_log_var3, z3 = vae3_work.encoder(selected_images)
reconst_images3 = vae3_work.decoder(z3).numpy() # @tf.function 宣言のためdecoder() の返り値はTensorになっているのでnumpyの配列に変換する
txts3 = [f'{p[0]:.3f}, {p[1]:.3f}' for p in z3 ]
%matplotlib inline
VariationalAutoEncoder.showImages(selected_images, reconst_images3, txts3, 1.4, 1.4)
