vc

Functions

int main(int argc, char *argv[])

vc [ option ] gmmfile [ infile ]

• -l int

  • length of source vector $(1\le M_1+1)$

• -m int

  • order of source vector $(0\le M_1)$

• -L int

  • length of target vector $(1\le M_2+1)$

• -M int

  • order of target vector $(0\le M_2)$

• -k int

  • number of mixtures $(1\le K)$

• -f

  • use full or block covariance instead of diagonal one

• -d double+

  • delta coefficients

• -D str

  • filename of double-type delta coefficients

• -r int+

  • width of 1st (and 2nd) regression coefficients

• -magic double

  • magic number

• gmmfile str

  • double-type GMM parameters

• infile str

  • double-type source static+dynamic vector sequence

• stdout

  • double-type target static vector sequence

In the following example, the converted 4-th order vectors corresponding data.source are obtained using the trained 2-mixture GMM data.gmm.

delta -l 5 -d -0.5 0.0 0.5 data.source | \
  vc -k 2 -l 5 data.gmm > data.target

Parameters:

• argc – [in] Number of arguments.

• argv – [in] Argument vector.

Returns:

0 on success, 1 on failure.

See also

gmm gmmp

class GaussianMixtureModelBasedConversion

Perform GMM-based voice conversion.

The input is the $(D+1)(M_1+1)$-length source vectors:

$$\begin{array}{cccc}{\mathit{X}}_0,& {\mathit{X}}_1,& \dots ,& {\mathit{X}}_{T-1},\end{array}$$

where

$${\mathit{X}}_t={\left[\begin{array}{cccc}{\mathit{x}}_t^{\mathsf{T}}& {\mathrm{\Delta }}^{(1)}{\mathit{x}}_t^{\mathsf{T}}& \cdots & {\mathrm{\Delta }}^{(D)}{\mathit{x}}_t^{\mathsf{T}}\end{array}\right]}^{\mathsf{T}}.$$

The output is the $(M_2+1)$-length target vectors:

$$\begin{array}{cccc}{\mathit{y}}_0,& {\mathit{y}}_1,& \dots ,& {\mathit{y}}_{T-1}.\end{array}$$

The optimal target vectors can be drived in a maximum likelihood sense:

$$\begin{array}{r}\begin{array}{rcl}\hat{\mathit{y}}& =& \underset{\mathit{y}}{\mathrm{argmax}}p(\mathit{Y}|\mathit{X},\mathit{\lambda })\\ & =& \underset{\mathit{y}}{\mathrm{argmax}}\sum _{\mathit{m}}p(\mathit{m}|\mathit{X},\mathit{\lambda })p(\mathit{Y}|\mathit{X},\mathit{m},\mathit{\lambda }),\end{array}\end{array}$$

where $\mathit{\lambda }$ is the GMM that models the joint vectors. The mean vector and the covariance matrix of the $m$-th mixture component are written as

$$\begin{array}{r}{\mathit{\mu }}_m=\left[\begin{array}{c}{\mathit{\mu }}_m^{(X)}\\ {\mathit{\mu }}_m^{(Y)}\end{array}\right]\end{array}$$

and

$$\begin{array}{r}{\mathit{\Sigma }}_m=\left[\begin{array}{cc}{\mathit{\Sigma }}_m^{(XX)}& {\mathit{\Sigma }}_m^{(XY)}\\ {\mathit{\Sigma }}_m^{(YX)}& {\mathit{\Sigma }}_m^{(YY)}\end{array}\right].\end{array}$$

To easily compute the ML estimate, the maximum a posteriori approximation is applied:

$$\begin{array}{r}\begin{array}{rcl}\hat{\mathit{y}}& =& \underset{\mathit{y}}{\mathrm{argmax}}p(\hat{\mathit{m}}|\mathit{X},\mathit{\lambda })p(\mathit{Y}|\mathit{X},\hat{\mathit{m}},\mathit{\lambda })\\ & =& \underset{\mathit{y}}{\mathrm{argmax}}\prod _{t=0}^{T-1}p({\hat{m}}_t|{\mathit{X}}_t,\mathit{\lambda })p({\mathit{Y}}_t|{\mathit{X}}_t,{\hat{m}}_t,\mathit{\lambda }),\end{array}\end{array}$$

where

$$p({\hat{m}}_t|{\mathit{X}}_t,\mathit{\lambda })=\underset{m}{max}\frac{w_m\mathcal{N}({\mathit{X}}_t|{\mathit{\mu }}_m^{(X)},{\mathit{\Sigma }}_m^{(XX)})}{\sum _{n=1}^M{w}_n\mathcal{N}({\mathit{X}}_t|{\mathit{\mu }}_n^{(X)},{\mathit{\Sigma }}_n^{(XX)})},$$

and

$$\begin{array}{r}\begin{array}{rcl}p({\mathit{Y}}_t|{\mathit{X}}_t,m,\mathit{\lambda })& =& \mathcal{N}({\mathit{Y}}_t|{\mathit{E}}_{m,t}^{(Y)},{\mathit{D}}_{m,t}^{(Y)}),\\ {\mathit{E}}_{m,t}^{(Y)}& =& {\mathit{\mu }}_m^{(Y)}+{\mathit{\Sigma }}_m^{(YX)}{\mathit{\Sigma }}_m^{(XX{)}^{-1}}({\mathit{X}}_t-{\mathit{\mu }}_m^{(X)}),\\ {\mathit{D}}_{m,t}^{(Y)}& =& {\mathit{\Sigma }}_m^{(YY)}-{\mathit{\Sigma }}_m^{(YX)}{\mathit{\Sigma }}_m^{(XX{)}^{-1}}{\mathit{\Sigma }}_m^{(XY)}.\end{array}\end{array}$$

The converted static vector sequence $\hat{\mathit{y}}$ under the constraint between static and dynamic components is obtained by the maximum likelihood parameter generation algorithm.

[1] T. Toda, A. W. Black, and K. Tokuda, “Voice conversion based on maximum-likelihood estimation of spectral parameter trajectory,” IEEE Transactions on Audio, Speech, and Language Processing, vol. 15, no. 8, pp. 2222-2235, 2007.

Public Functions

GaussianMixtureModelBasedConversion(int num_source_order, int num_target_order, const std::vector<std::vector<double>> &window_coefficients, const std::vector<double> &weights, const std::vector<std::vector<double>> &mean_vectors, const std::vector<SymmetricMatrix> &covariance_matrices, bool use_magic_number, double magic_number = 0.0)

Parameters:

• num_source_order – [in] Order of source vector, $M_1$.

• num_target_order – [in] Order of target vector, $M_2$.

• window_coefficients – [in] Window coefficients. e.g.) { {-0.5, 0.0, 0.5}, {1.0, -2.0, 1.0} }

• weights – [in] $K$ mixture weights.

• mean_vectors – [in] $K$ mean vectors. The shape is $[K,(D+1)(M_1+M_2+2)]$.

• covariance_matrices – [in] $K$ covariance matrices. The shape is $[K,(D+1)(M_1+M_2+2),(D+1)(M_1+M_2+2)]$.

• use_magic_number – [in] Whether to use magic number.

• magic_number – [in] A magic number represents a discrete symbol.

inline int GetNumSourceOrder() const

Returns:

Order of source vector.

inline int GetNumTargetOrder() const

Returns:

Order of target vector.

inline bool IsValid() const

Returns:

True if this object is valid.

bool Run(const std::vector<std::vector<double>> &source_vectors, std::vector<std::vector<double>> *target_vectors) const

Parameters:

• source_vectors – [in] $M_1$-th order source vectors containing dynamic components. The shape is $[T,(D+1)(M_1+1)]$.

• target_vectors – [out] $M_2$-th order target vectors. The shape is $[T,(M_2+1)]$.

Returns:

True on success, false on failure.