diff --git a/README.md b/README.md index 30e2559..ec010d9 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,12 @@ # Spectral Amplitude Quantisation using NNs -Experiments with time/frequency sample rate conversion and VQ-VAE for quantising the spectrum of speech for vocoders. +Experiments with time/frequency sample rate conversion and VQ-VAE (Vector Quantised Variational Autoencoder) for quantising the speech spectrum for vocoders. + +This plot shows a VQ-VAE in action: + +![PCA of VQVAE encoder space](doc/vqvae_pca.png) + +The plot is a 2D histogram of the encoder space, white dots are the stage 1 VQ entries. The 16 dimensional data has been reduced to 2 dimensions using PCA. The plot was produced by `vq_vae_conv1d_2stage.py` ## Themes and Key Points @@ -26,7 +32,7 @@ Experiments with time/frequency sample rate conversion and VQ-VAE for quantising | vq_vae_demo.py | Simple demo of VQ, nice visualisation of training in action | Cool demo | | vq_vae_demo_2stage.py | vq_vae_demo.py extended to two stage VQ | Cool demo | | vq_vae_ratek.py | Single stage VQ-VAE with single Dense layer | No | -| vq_vae_ratek_conv1d.py | Two stage VQ-VAE with two conv1D layers | Yes, reasonable spectral distortion, cool plots | +| vq_vae_conv1d_2stage.py | Two stage VQ-VAE with two conv1D layers | Yes, reasonable spectral distortion, cool plots | ## Amplitude Sample Rate Conversion Using Neural Nets diff --git a/doc/vqvae_pca.png b/doc/vqvae_pca.png new file mode 100644 index 0000000..998eff2 Binary files /dev/null and b/doc/vqvae_pca.png differ diff --git a/vq_vae_conv1d_2stage.py b/vq_vae_conv1d_2stage.py index 84c5ed7..59ae4fb 100755 --- a/vq_vae_conv1d_2stage.py +++ b/vq_vae_conv1d_2stage.py @@ -341,7 +341,7 @@ while key != 'q': plt.title(t) plt.show(block=False) plt.pause(0.0001) - print("n-next b-back s-save_png q-quit", end='', flush=True); + print("n-next b-back s-save_png q-quit", end='\r', flush=True); key = getch.getch() if key == 'n': fs += nb_plots