-ji- SSKJr SSKrSSKrSSKrSSKrSSKJr \R"S\R-5r \R"\ 5r \R"S5\R- r\R"S5rSSjrSSjr\R$"S5SS j5rSS jr\R$"S5SS j5rSS jrSS jrSSjrSSjrSSjrSSSjjrSSSjjrg)) annotationsNerfc.[R"X5$N)np logaddexplog_plog_qs Y/home/james-whalen/.local/lib/python3.13/site-packages/optuna/samplers/_tpe/_truncnorm.py_log_sumr3s << %%cbU[R"[R"X- 5*5-$r )r log1pexpr s r _log_diffr7s$ 288RVVEM223 33ricUS- nUS:aS[R"U*5-nU$US:aSS[R"U5--nU$SS[R"U5-- nU$)N;f?g;f?g;f??)matherfcr)axys r _ndtr_singler;sr F A; $))QB-  H Z # # # H # ! $ $ Hrc*SS[US- 5--$)Nrrrrs r_ndtrr"Is s1v:& &&rcPUS:a [U*5*$US:a[R"[U55$SUS--[R"U*5- S[R"S[R-5-- nSnSnSnSnSUS-- nSnS n[ X#- 5[ R R:aNUS- nUnU*nXV-nUSU-S- -nX7U-U-- n[ X#- 5[ R R:aMNU[R"U5-$) Nirrgrr)rrlogpiabssys float_infoepsilon) rlog_LHS last_totalright_hand_side numerator denom_factor denom_conssignis r_log_ndtr_singler5Ns"1uaR   3wxx Q((QTkDHHaRL(3!dgg+1F+FFGJOILQTJ D A j* +cnn.D.D D Q$ u" QUQY )+l:: j* +cnn.D.D D TXXo. ..rcl[R"[SS5"U5R[5$)Nr&)r frompyfuncr5astypefloatr!s r _log_ndtrr:is& ==)1a 0 3 : :5 AArc"US-*S- [- $)Nrg@)_norm_pdf_logC)rs r _norm_logpdfr=ms T7S=> ))rc^ US:*nUS:nX#-)nSSjm SU 4SjjnSSjn[R"U[R[RS9nX=nR(a T "XU5Xr'X=n R(a U"XU5Xs'X=n R(a U"XU5Xt'[R "U5$)z3Log of Gaussian probability mass within an intervalrc>[[U5[U55$r )rr:rbs rmass_case_left'_log_gauss_mass..mass_case_leftzs1y|44rc>T"U*U*5$r )rrArBs rmass_case_right(_log_gauss_mass..mass_case_right}sqb1"%%rc\[R"[U5*[U*5- 5$r )r rr"r@s rmass_case_central*_log_gauss_mass..mass_case_centrals$xxq E1"I-..r) fill_valuedtyper np.ndarrayrArNreturnrN)r full_likenan complex128sizereal) rrA case_left case_right case_centralrFrIouta_lefta_right a_centralrBs @r_log_gauss_massr\qs QIQJ+,L5& / ,,qRVV2== AC,$$') == &&)'Z=A_$ **-i<I 773<rcnUS:nUR5n[R"[R"X5*5X!'[R"U5n[R "US:5=nSR (a%[R"SX$[--5*X4'[R "US:5=nSR (a7[*[R"[R"X%*55-X5'[S5Hn[U5nSUS--[- nXr- [R"Xx- 5-n X9-n[R"[R"U 5S[R"U5-:5(dM O X1==S -ss'U$) a Use the Newton method to efficiently find the root. `ndtri_exp(y)` returns `x` such that `y = log_ndtr(x)`, meaning that the Newton method is supposed to find the root of `f(x) := log_ndtr(x) - y = 0`. Since `df/dx = d log_ndtr(x)/dx = (ndtr(x))'/ndtr(x) = norm_pdf(x)/ndtr(x)`, the Newton update is x[n + 1] := x[n] - (log_ndtr(x) - y) * ndtr(x) / norm_pdf(x). As an initial guess, we use the Gaussian tail asymptotic approximation: 1 - ndtr(x) \simeq norm_pdf(x) / x --> log(norm_pdf(x) / x) = -1/2 * x**2 - 1/2 * log(2pi) - log(x) First recall that y is a non-positive value and y = log_ndtr(inf) = 0 and y = log_ndtr(-inf) = -inf. If abs(y) is very small, we first derive -x such that z = log_ndtr(-x) and then flip the sign. Please note that the following holds: z = log_ndtr(-x) --> z = log(1 - ndtr(x)) = log(1 - exp(y)) = log(-expm1(y)). Recall that as long as ndtr(x) = exp(y) > 0.5 --> y > -log(2) = -0.693..., x becomes positive. ndtr(x) = exp(y) \simeq 1 + y --> -y \simeq 1 - ndtr(x). From this, we can calculate: log(1 - ndtr(x)) \simeq log(-y) \simeq -1/2 * x**2 - 1/2 * log(2pi) - log(x). Because x**2 >> log(x), we can ignore the second and third terms, leading to: log(-y) \simeq -1/2 * x**2 --> x \simeq sqrt(-2 log(-y)). where we take the positive `x` as abs(y) becomes very small only if x >> 0. The second order approximation version is sqrt(-2 log(-y) - log(-2 log(-y))). If abs(y) is very large, we use log_ndtr(x) \simeq -1/2 * x**2 - 1/2 * log(2pi) - log(x). To solve this equation analytically, we ignore the log term, yielding: log_ndtr(x) = y \simeq -1/2 * x**2 - 1/2 * log(2pi) --> y + 1/2 * log(2pi) = -1/2 * x**2 --> x**2 = -2 * (y + 1/2 * log(2pi)) --> x = sqrt(-2 * (y + 1/2 * log(2pi)) For the moderate y, we use Eq. (13), i.e., standard logistic CDF, in the following paper: - `Approximating the Cumulative Distribution Function of the Normal Distribution `__ The standard logistic CDF approximates ndtr(x) with: exp(y) = ndtr(x) \simeq 1 / (1 + exp(-pi * x / sqrt(3))) --> exp(-y) \simeq 1 + exp(-pi * x / sqrt(3)) --> log(exp(-y) - 1) \simeq -pi * x / sqrt(3) --> x \simeq -sqrt(3) / pi * log(exp(-y) - 1). g{Gzrgdr%rg:0yE>)copyr r'expm1 empty_likenonzerorSsqrtr<_ndtri_exp_approx_Cranger:rallr)) rflippedzr small_inds moderate_inds_ log_ndtr_xlog_norm_pdf_xdxs r _ndtri_exprqsW^%iG A!*--.AJ aAjjR(( !,11)G!HII AG,, a055//"&&1CSBS9T2UU 3Zq\ 1~5nz'B C C  66"&&*tbffQi// 0 0 J"J Hrc[R"XU5upn[R"XU5upnUS:nU)n[X5nSSjnSSjn[R"U5nX=n R (aU"XUX#XS5X'X=n R (aU"XUX$XT5X'XS:HXS:H'X S:HXS:H'[ RXU:H'U$)a Compute the percent point function (inverse of cdf) at q of the given truncated Gaussian. Namely, this function returns the value `c` such that: q = \int_{a}^{c} f(x) dx where `f(x)` is the probability density function of the truncated normal distribution with the lower limit `a` and the upper limit `b`. More precisely, this function returns `c` such that: ndtr(c) = ndtr(a) + q * (ndtr(b) - ndtr(a)) for the case where `a < 0`, i.e., `case_left`. For `case_right`, we flip the sign for the better numerical stability. rcp[[U5[R"U5U-5n[ U5$r )rr:r r'rqqrrAlog_mass log_Phi_xs rppf_leftppf..ppf_lefts*Yq\266!9x+?@ )$$rcv[[U*5[R"U*5U-5n[ U5*$r )rr:r rrqrts r ppf_rightppf..ppf_rights3Yr]BHHaRL8,CD 9%%%rr&) rurNrrNrArNrvrNrOrN)r atleast_1dbroadcast_arraysr\rcrSrrQ) rurrArUrVrvrxr{rXq_leftq_rights rppfrsmmA!$GA!!!!*GA!AIJq$H%& -- C,$$!&I, hFYZ= &&#Gz]AM8K_`F)CQKF)CQK((CQK JrcU=(d [RR5n[R"XX#5RnUR SSUS9n[ X`U5U-U-$)z This function generates random variates from a truncated normal distribution defined between `a` and `b` with the mean of `loc` and the standard deviation of `scale`. rr&)lowhighrS)r random RandomState broadcastshapeuniformr)rrAlocscale random_staterS quantiless rrvsr s_ :299#8#8#:L <<c ) / /D$$$>I yQ % '# --rcTX- U- n[R"XU5upn[U5[X5- [R"U5- n[R "XU5upn[R "X:HX:X:-/[R[R*/US9$)N)default) r r}r=r\r'r~selectrQinf)rrrArrrXs rlogpdfrs EAmmA!$GA! q/OA1 1BFF5M AC!!!*GA! 99afqu/0266BFF72CS QQr)r rNrrNrOrN)rr9rOr9)rrNrOrN)rrNrOrNrM)rrNrOrN)rurNrnp.ndarray | floatrArrOrN)rr&N) rrNrArNrrrrrznp.random.RandomState | NonerOrN)rr&) rrNrrrArrrrrrOrN) __future__r functoolsrr*numpyr optuna.samplers._tpe._erfrrer( _norm_pdf_Cr'r<rf_log_2rr lru_cacherr"r5r:r=r\rqrrrrErrrscD# )iiDGG $ +&iilTWW, !&4 T    '  T//4B*$NE P*` !15 ... .  . / .  .* ! R R R R  R  R  Rr