biVSSKrSSKJrJr SSKrSSKJr SSKJr /r SS\R S\R S\R S\ S \ S \ S \ S \R 4S jjr "SS\RR5r"SS\RR5r"SS\RR5r"SS\RR5rg)N)OptionalUnion)Tensor) functionalpsd_spsd_nreference_vectorsolutiondiagonal_loadingdiag_epsepsreturncUS:Xa[R"XX$XV5nU$US:Xa[R"U5nO[R"XX$US9n[R"XX$XV5nU$)axCompute the MVDR beamforming weights with ``solution`` argument. Args: psd_s (torch.Tensor): The complex-valued power spectral density (PSD) matrix of target speech. Tensor with dimensions `(..., freq, channel, channel)`. psd_n (torch.Tensor): The complex-valued power spectral density (PSD) matrix of noise. Tensor with dimensions `(..., freq, channel, channel)`. reference_vector (torch.Tensor): one-hot reference channel matrix. solution (str, optional): Solution to compute the MVDR beamforming weights. Options: [``ref_channel``, ``stv_evd``, ``stv_power``]. (Default: ``ref_channel``) diagonal_loading (bool, optional): If ``True``, enables applying diagonal loading to ``psd_n``. (Default: ``True``) diag_eps (float, optional): The coefficient multiplied to the identity matrix for diagonal loading. It is only effective when ``diagonal_loading`` is set to ``True``. (Default: ``1e-7``) eps (float, optional): Value to add to the denominator in the beamforming weight formula. (Default: ``1e-8``) Returns: torch.Tensor: the mvdr beamforming weight matrix ref_channelstv_evd)r r )Fmvdr_weights_soudenrtf_evd rtf_powermvdr_weights_rtf) rrr r r r r beamform_vectorstvs ^/home/james-whalen/.local/lib/python3.13/site-packages/torchaudio/transforms/_multi_channel.py_get_mvdr_vectorrsr:= //>Nbjp  y ))E"C++e,[TU]5 XlX lX0lgN)super__init__rr r )selfrr r __class__s rr$ PSD.__init__Ds $"rspecgrammaskcUb UR(aURSS9n[R"XURUR 5nU$)a' Args: specgram (torch.Tensor): Multi-channel complex-valued spectrum. Tensor with dimensions `(..., channel, freq, time)`. mask (torch.Tensor or None, optional): Time-Frequency mask for normalization. Tensor with dimensions `(..., freq, time)` if multi_mask is ``False`` or with dimensions `(..., channel, freq, time)` if multi_mask is ``True``. (Default: ``None``) Returns: torch.Tensor: The complex-valued PSD matrix of the input spectrum. Tensor with dimensions `(..., freq, channel, channel)` dim)rmeanrpsdr r )r%r(r)r/s rforward PSD.forwardJs@  yyRy(eeHDNNDHH= r)r rr )FTgV瞯[T U]5 US;a[SRU55eXlX lX0lX@lXPlX`l [U5Ul [R"S5n[R"S5n[R"S5n [R"S5n URSU5 URSU5 URSU 5 URSU 5 g)N)rr stv_powerzK`solution` must be one of ["ref_channel", "stv_evd", "stv_power"]. Given {}rr mask_sum_s mask_sum_n)r#r$ ValueErrorformatrr rr@r rArr/r9zerosregister_buffer) r%rr rr@r rArrrErFr&s rr$ MVDR.__init__s    ]ddemn ' $(  z?#kk!n#kk!n#(;;q> #(;;q>  We, We, \:6 \:6rrrmask_smask_nr r r rc UR(aURSS9nURSS9nURRS:XaBXlX lUR SS9UlUR SS9Ul[XXVXxU 5$URX5nURX$5nXlX lUR UR SS9-UlURUR SS9-Ul[XXVXxU 5$)a>Recursively update the MVDR beamforming vector. Args: psd_s (torch.Tensor): The complex-valued power spectral density (PSD) matrix of target speech. Tensor with dimensions `(..., freq, channel, channel)`. psd_n (torch.Tensor): The complex-valued power spectral density (PSD) matrix of noise. Tensor with dimensions `(..., freq, channel, channel)`. mask_s (torch.Tensor): Time-Frequency mask of the target speech. Tensor with dimensions `(..., freq, time)` if multi_mask is ``False`` or with dimensions `(..., channel, freq, time)` if multi_mask is ``True``. mask_n (torch.Tensor or None, optional): Time-Frequency mask of the noise. Tensor with dimensions `(..., freq, time)` if multi_mask is ``False`` or with dimensions `(..., channel, freq, time)` if multi_mask is ``True``. reference_vector (torch.Tensor): One-hot reference channel matrix. solution (str, optional): Solution to compute the MVDR beamforming weights. Options: [``ref_channel``, ``stv_evd``, ``stv_power``]. (Default: ``ref_channel``) diagonal_loading (bool, optional): If ``True``, enables applying diagonal loading to ``psd_n``. (Default: ``True``) diag_eps (float, optional): The coefficient multiplied to the identity matrix for diagonal loading. It is only effective when ``diagonal_loading`` is set to ``True``. (Default: ``1e-7``) eps (float, optional): Value to add to the denominator in the beamforming weight formula. (Default: ``1e-8``) Returns: torch.Tensor: The MVDR beamforming weight matrix. r+r,rD) rr.rndimrsumrErFr_get_updated_psd_speech_get_updated_psd_noise) r%rrrLrMr r r r r s r_get_updated_mvdr_vectorMVDR._get_updated_mvdr_vectorsL ??[[R[(F[[R[(F ::??a JJ$jjRj0DO$jjRj0DO#E2BN^jmn n00?E//>EJJ"oo r 0BBDO"oo r 0BBDO#E2BN^jmn nrcURURURSS9-- nSURURSS9-- nURUS-XS--nU$)aUpdate psd of speech recursively. Args: psd_s (torch.Tensor): The complex-valued power spectral density (PSD) matrix of target speech. Tensor with dimensions `(..., freq, channel, channel)`. mask_s (torch.Tensor): Time-Frequency mask of the target speech. Tensor with dimensions `(..., freq, time)`. Returns: torch.Tensor: The updated PSD matrix of target speech. rOr,rD.NN)rErQr)r%rrL numerator denominators rrRMVDR._get_updated_psd_speechiOOt9K'KL 4??VZZBZ-??@  Y77%oB^:^^ rcURURURSS9-- nSURURSS9-- nURUS-XS--nU$)aUpdate psd of noise recursively. Args: psd_n (torch.Tensor): The complex-valued power spectral density (PSD) matrix of noise. Tensor with dimensions `(..., freq, channel, channel)`. mask_n (torch.Tensor or None, optional): Time-Frequency mask of the noise. Tensor with dimensions `(..., freq, time)`. Returns: torch.Tensor: The updated PSD matrix of noise. rOr,rDrW)rFrQr)r%rrMrXrYs rrSMVDR._get_updated_psd_noiser[rr(c URnURS:a[SUR35eUR 5(d[SUR35eUR[ R :XaUR5nUc[R"S5 SU- nURX5nURX5n[ R"UR5SSUR[ RS9nUS UR4RS5 UR (a4UR#XVX#XpR$UR&UR(5nO,[+XVXpR$UR&UR(5n[,R."X5n U R1U5$) aPerform MVDR beamforming. Args: specgram (torch.Tensor): Multi-channel complex-valued spectrum. Tensor with dimensions `(..., channel, freq, time)` mask_s (torch.Tensor): Time-Frequency mask of target speech. Tensor with dimensions `(..., freq, time)` if multi_mask is ``False`` or with dimensions `(..., channel, freq, time)` if multi_mask is ``True``. mask_n (torch.Tensor or None, optional): Time-Frequency mask of noise. Tensor with dimensions `(..., freq, time)` if multi_mask is ``False`` or with dimensions `(..., channel, freq, time)` if multi_mask is ``True``. (Default: None) Returns: torch.Tensor: Single-channel complex-valued enhanced spectrum with dimensions `(..., freq, time)`. z?Expected at least 3D tensor (..., channel, freq, time). Found: ziThe type of ``specgram`` tensor must be ``torch.cfloat`` or ``torch.cdouble``. Found: Nz=``mask_n`` is not provided, use ``1 - mask_s`` as ``mask_n``.rD)devicedtype.)rbrPrGshape is_complexr9cfloatcdoublewarningswarnr/rIsizerarfill_rArTr r@r rrapply_beamformingto) r%r(rLrMrbrruw_mvdrspecgram_enhanceds rr0 MVDR.forward#sx& ==1 ^_g_m_m^nop p""$$$NN+-  >>U\\ )'')H > MMY ZZF** KK ,X__EMM Z #t  &&q) ;;22fa@Q@QSWS`S`F&eA}}dFWFWY]YfYfgF//A ##E**r) r r@rFrErrAr/rrrr )rrFTHz>FrTrq:0yE>r")r2r3r4r5r6intstrr7r8r$r9rrTrRrSrr0r:r;r<s@rr>r>asAJ% !!7!7!7 !7  !7  !7!7!7T&!%7o||7o||7o 7o  7o  ,, 7o7o7o7o7o 7orU\\5<<TYT`T`"ELL%,,SXS_S_$^b1+ 1+.3ll1+DLU\\DZ1+ 1+1+rr>cV\rSrSrSrSS\S\S\S\\\4S\S\ S \ S \4S jjr S r g )RTFMVDRiWaMinimum Variance Distortionless Response (*MVDR* :cite:`capon1969high`) module based on the relative transfer function (RTF) and power spectral density (PSD) matrix of noise. .. devices:: CPU CUDA .. properties:: Autograd TorchScript Given the multi-channel complex-valued spectrum :math:`\textbf{Y}`, the relative transfer function (RTF) matrix or the steering vector of target speech :math:`\bm{v}`, the PSD matrix of noise :math:`\bf{\Phi}_{\textbf{NN}}`, and a one-hot vector that represents the reference channel :math:`\bf{u}`, the module computes the single-channel complex-valued spectrum of the enhanced speech :math:`\hat{\textbf{S}}`. The formula is defined as: .. math:: \hat{\textbf{S}}(f) = \textbf{w}_{\text{bf}}(f)^{\mathsf{H}} \textbf{Y}(f) where :math:`\textbf{w}_{\text{bf}}(f)` is the MVDR beamforming weight for the :math:`f`-th frequency bin, :math:`(.)^{\mathsf{H}}` denotes the Hermitian Conjugate operation. The beamforming weight is computed by: .. math:: \textbf{w}_{\text{MVDR}}(f) = \frac{{{\bf{\Phi}_{\textbf{NN}}^{-1}}(f){\bm{v}}(f)}} {{\bm{v}^{\mathsf{H}}}(f){\bf{\Phi}_{\textbf{NN}}^{-1}}(f){\bm{v}}(f)} r(rtfrreference_channelr r r rcb[R"X#XEXg5n[R"X5n U $)a Args: specgram (torch.Tensor): Multi-channel complex-valued spectrum. Tensor with dimensions `(..., channel, freq, time)` rtf (torch.Tensor): The complex-valued RTF vector of target speech. Tensor with dimensions `(..., freq, channel)`. psd_n (torch.Tensor): The complex-valued power spectral density (PSD) matrix of noise. Tensor with dimensions `(..., freq, channel, channel)`. reference_channel (int or torch.Tensor): Specifies the reference channel. If the dtype is ``int``, it represents the reference channel index. If the dtype is ``torch.Tensor``, its shape is `(..., channel)`, where the ``channel`` dimension is one-hot. diagonal_loading (bool, optional): If ``True``, enables applying diagonal loading to ``psd_n``. (Default: ``True``) diag_eps (float, optional): The coefficient multiplied to the identity matrix for diagonal loading. It is only effective when ``diagonal_loading`` is set to ``True``. (Default: ``1e-7``) eps (float, optional): Value to add to the denominator in the beamforming weight formula. (Default: ``1e-8``) Returns: torch.Tensor: Single-channel complex-valued enhanced spectrum with dimensions `(..., freq, time)`. )rrrk) r%r(rxrryr r r rnspectrum_enhanceds rr0RTFMVDR.forwardrs2@##C0AU]c//A  rNTrqrs) r2r3r4r5r6rrrtr7r8r0r:r}rrrwrwWst@"&"!"!"! "! !f- "!  "!"!"! "!"!rrwcj\rSrSrSrSS\S\S\S\\\4S\S\ S \ S \ R 4S jjr S r g ) SoudenMVDRiaGMinimum Variance Distortionless Response (*MVDR* :cite:`capon1969high`) module based on the method proposed by *Souden et, al.* :cite:`souden2009optimal`. .. devices:: CPU CUDA .. properties:: Autograd TorchScript Given the multi-channel complex-valued spectrum :math:`\textbf{Y}`, the power spectral density (PSD) matrix of target speech :math:`\bf{\Phi}_{\textbf{SS}}`, the PSD matrix of noise :math:`\bf{\Phi}_{\textbf{NN}}`, and a one-hot vector that represents the reference channel :math:`\bf{u}`, the module computes the single-channel complex-valued spectrum of the enhanced speech :math:`\hat{\textbf{S}}`. The formula is defined as: .. math:: \hat{\textbf{S}}(f) = \textbf{w}_{\text{bf}}(f)^{\mathsf{H}} \textbf{Y}(f) where :math:`\textbf{w}_{\text{bf}}(f)` is the MVDR beamforming weight for the :math:`f`-th frequency bin. The beamforming weight is computed by: .. math:: \textbf{w}_{\text{MVDR}}(f) = \frac{{{\bf{\Phi}_{\textbf{NN}}^{-1}}(f){\bf{\Phi}_{\textbf{SS}}}}(f)} {\text{Trace}({{{\bf{\Phi}_{\textbf{NN}}^{-1}}(f) \bf{\Phi}_{\textbf{SS}}}(f))}}\bm{u} r(rrryr r r rcb[R"X#XEXg5n[R"X5n U $)a Args: specgram (torch.Tensor): Multi-channel complex-valued spectrum. Tensor with dimensions `(..., channel, freq, time)`. psd_s (torch.Tensor): The complex-valued power spectral density (PSD) matrix of target speech. Tensor with dimensions `(..., freq, channel, channel)`. psd_n (torch.Tensor): The complex-valued power spectral density (PSD) matrix of noise. Tensor with dimensions `(..., freq, channel, channel)`. reference_channel (int or torch.Tensor): Specifies the reference channel. If the dtype is ``int``, it represents the reference channel index. If the dtype is ``torch.Tensor``, its shape is `(..., channel)`, where the ``channel`` dimension is one-hot. diagonal_loading (bool, optional): If ``True``, enables applying diagonal loading to ``psd_n``. (Default: ``True``) diag_eps (float, optional): The coefficient multiplied to the identity matrix for diagonal loading. It is only effective when ``diagonal_loading`` is set to ``True``. (Default: ``1e-7``) eps (float, optional): Value to add to the denominator in the beamforming weight formula. (Default: ``1e-8``) Returns: torch.Tensor: Single-channel complex-valued enhanced spectrum with dimensions `(..., freq, time)`. )rrrk) r%r(rrryr r r rnr{s rr0SoudenMVDR.forwards2@&&u5FZbh//A  rr}Nr~) r2r3r4r5r6rrrtr7r8r9r0r:r}rrrrsw>"&"!"!"! "! !f- "!  "!"!"! "!"!rrrr)rgtypingrrr9r torchaudiorr__all__rur7r8rnnModulerr>rwrr}rrrs" & "!& <<& <<&ll& &  &  & & \\&R'%((//'Ts+588??s+l=!ehhoo=!@<!<!r