!gZXddlmZddlmZddlZddlZddlZddlm Z ddl m Z ddl m Z ddlmZddlmZmZdd lmZdd lmZdd lmZmZmZmZgd Zd#d Zd$dZd%dZ d& d'dZ d(dZ!dZ"dZ#dZ$dZ%dZ&dZ'dZ(dZ)dZ*dZ+dZ,dZ-dZ.dZ/d Z0d!Z1d)d"Z2y)*) annotations)lrangeN) DataFrame)offsets) to_offset)Literal) _is_recarray_is_using_pandas) ValueWarning)NDArray) array_like bool_likeint_like string_like) lagmat lagmat2ds add_trendduplication_matrixelimination_matrixcommutation_matrixvecvechunvecunvechfreq_to_periodct|d}t|dd}t|dd}gd}|dk(r|jS|d k(r|d d }d }n#|d k(s|dk(r|d d}|dk(r|d d}d }n|dk(rd}t|rd dlm}t |t|d }|rAt|tjrtj|}n&|j}ntj|}t|}tjtj d |d ztj"d z} tj$| } |dk(r | d d d f} d |vr!|rd} |j'| d } n.safe_is_const~s6!66!9+@qCx0@@! s148axisr'zx is constant.z, z3x contains one or more constant columns. Column(s) z are constant.z Adding a constant with trend='z' is not allowed.r)indexcolumns)rrcopyr statsmodels.tools.sm_exceptionsr.NotImplementedErrorr isinstancepdSeriesrr1 asanyarraylenvanderarangefloat64fliplrapplyr2r3ndimshaper<joinstr ValueErrorr;concat column_stack)xrrr&r< trendorderr. is_pandasnobstrendarrr6 col_constptp0 col_is_constnz_constbase_errr const_colsmsgorders r5rr'sP+G w0L MEn.FL 2G |vvx #"1+ $%3,"1+ C<alG % AF!"455 D)I a # QAA MM!  q6Dyy !TAXRZZ0*q.Hyy"H |AqD> e|  !  q1I66"--*3D19L#qtqy1H I 66) w&66Q;/H ii 3I>G!!R\\2"#))!%G+DqCF+D!EJM%,n6" "A%HYZ o%'!!"+#AqrE?AbE<<I qM IIa%jq ) HqM OOAggJ ' H),Es?K4ct|d}t|d}t|dd}|d}|dkr|jd|z}|jdk(r |dddf}|dd|f}|d ur|dz}nk|d ur|jd}nW|dkr|jd|zdz}||jdkDr)|jd}t j d t|}t||d }t|}t||jd} |rE||vr!|j|j|n | j| j|tj||d|f|||d| ffS)a Returns an array with lags included given an array. Parameters ---------- x : array_like An array or NumPy ndarray subclass. Can be either a 1d or 2d array with observations in columns. col : int or None `col` can be an int of the zero-based column index. If it's a 1d array `col` can be None. lags : int The number of lags desired. drop : bool Whether to keep the contemporaneous variable for the data. insert : bool or int If True, inserts the lagged values after `col`. If False, appends the data. If int inserts the lags at int. Returns ------- array : ndarray Array with lags Examples -------- >>> import statsmodels.api as sm >>> data = sm.datasets.macrodata.load() >>> data = data.data[['year','quarter','realgdp','cpi']] >>> data = sm.tsa.add_lag(data, 'realgdp', lags=2) Notes ----- Trims the array both forward and backward, so that the array returned so that the length of the returned array is len(`X`) - lags. The lags are returned in increasing order, ie., t-1,t-2,...,t-lags lagsdroprQr-)rJNrr,TFz number of variables, inserting at the last positionBoth)trim)rrr rKrJwarningswarnr rrpopr;r1rP) rQcolr_r`insertcontempins_idxndlags first_cols last_colss r5add_lagrmsN D& !D T6 "D1c"A { Qwggaj3vv{ agJ3iG ~' 5''!* A:WWQZ&(1,F AGGAJ WWQZF MM!   GT /FJw +I *  NN:++C0 1 MM)//#. / ??AdeZ/0&!DE9 2 is not implemented until it is neededrr7)N)rrJintTr?rKmeanr1rErFfloatlinalgpinvdot)rQr]r8rTresidtrendsbetas r5detrendr{s, UG $E D& !Dvv{s4yA~ CC !! >   771:D zAFFFN"299U4[1UQY?yy~~f%))!,BFF64((vv{s4yA~ Lrnc t|d}t|d}t|ddd}t|dd }|}t|d d d }t |d xr|}|dn|}|j }|r|dvr t dd}|j\}} |dvr| }||k\r t dtj||z| |dzzf} tdt|dzD]%} || || z ||z| z | || z z| || z dzzf<'|dvrd} n|dvr|} n t d|dvr t| } n|} |r$|}t|trQ|jDcgc] }t!|}}tt#||jdk7r!t dt!|j$g}|Dcgc] }t!|}}t|D]>}t!|dz}|j'|Dcgc]}t!|dz|zc}@t| d | |j(|} | j*| d }|dvr4||}|j-|d}n| | | |d f}|dk(r | | | d |f}|dk(r|fS|Scc}wcc}wcc}w)a, Create 2d array of lags. Parameters ---------- x : array_like Data; if 2d, observation in rows and variables in columns. maxlag : int All lags from zero to maxlag are included. trim : {'forward', 'backward', 'both', 'none', None} The trimming method to use. * 'forward' : trim invalid observations in front. * 'backward' : trim invalid initial observations. * 'both' : trim invalid observations on both sides. * 'none', None : no trimming of observations. original : {'ex','sep','in'} How the original is treated. * 'ex' : drops the original array returning only the lagged values. * 'in' : returns the original array and the lagged values as a single array. * 'sep' : returns a tuple (original array, lagged values). The original array is truncated to have the same number of rows as the returned lagmat. use_pandas : bool If true, returns a DataFrame when the input is a pandas Series or DataFrame. If false, return numpy ndarrays. Returns ------- lagmat : ndarray The array with lagged observations. y : ndarray, optional Only returned if original == 'sep'. Notes ----- When using a pandas DataFrame or Series with use_pandas=True, trim can only be 'forward' or 'both' since it is not possible to consistently extend index values. Examples -------- >>> from statsmodels.tsa.tsatools import lagmat >>> import numpy as np >>> X = np.arange(1,7).reshape(-1,2) >>> lagmat(X, maxlag=2, trim="forward", original='in') array([[ 1., 2., 0., 0., 0., 0.], [ 3., 4., 1., 2., 0., 0.], [ 5., 6., 3., 4., 1., 2.]]) >>> lagmat(X, maxlag=2, trim="backward", original='in') array([[ 5., 6., 3., 4., 1., 2.], [ 0., 0., 5., 6., 3., 4.], [ 0., 0., 0., 0., 5., 6.]]) >>> lagmat(X, maxlag=2, trim="both", original='in') array([[ 5., 6., 3., 4., 1., 2.]]) >>> lagmat(X, maxlag=2, trim="none", original='in') array([[ 1., 2., 0., 0., 0., 0.], [ 3., 4., 1., 2., 0., 0.], [ 5., 6., 3., 4., 1., 2.], [ 0., 0., 5., 6., 3., 4.], [ 0., 0., 0., 0., 5., 6.]]) maxlag use_pandasrbTforwardbackwardbothnoneoptionalr%original)exsepinr$rQr-N)rJr/r)rrzEtrim cannot be 'none' or 'backward' when used on Series or DataFramesr)rrzmaxlag should be < nobsr,)rr)rrztrim option not validzSColumns names must be distinct after conversion to string (if not already strings).z.L.r:)rrr7r)rrrr r lowerrNrKr1zerosrangerqrDr@rr<rMsetnameextendr;ilocr`)rQr}rbrr~origrSdropidxrTnvarlmkstartobsstopobsr x_columnsrfr<laglag_strr_leadss r5rr)s Rfh 'F:|4J  7  D 8Z9LMH D1c.A t,;I\6tD ::"aggaj0 0 QVV I'013s811= NC#'lG NN)L3CHu,w6L M Nr(7|177G Dwwxy! } $OE99YQ9/D(7"GH,- u x''12E5U{ 342MsI=0J/J c t|d}t|dd}t|ddd}||}t||}t|d}|jd k(r"|rt j |}n3|dddf}n)|jd k(s|jd kDr td |j\}} |r|rt|jddd f||d d} | jddd|d zfg} td | D]J} t|jdd| f||d d} | j| jdd||d zfLt j| d S|rtj|}t|ddd f||d ddd|d zfg} td | D]3} | jt|dd| f||d dd||d zf5tj | S)a Generate lagmatrix for 2d array, columns arranged by variables. Parameters ---------- x : array_like Data, 2d. Observations in rows and variables in columns. maxlag0 : int The first variable all lags from zero to maxlag are included. maxlagex : {None, int} The max lag for all other variables all lags from zero to maxlag are included. dropex : int Exclude first dropex lags from other variables. For all variables, except the first, lags from dropex to maxlagex are included. trim : str The trimming method to use. * 'forward' : trim invalid observations in front. * 'backward' : trim invalid initial observations. * 'both' : trim invalid observations on both sides. * 'none' : no trimming of observations. use_pandas : bool If true, returns a DataFrame when the input is a pandas Series or DataFrame. If false, return numpy ndarrays. Returns ------- ndarray The array with lagged observations, columns ordered by variable. Notes ----- Inefficient implementation for unequal lags, implemented for convenience. maxlag0maxlagexT)rrbrrNr,rr-z'Only supports 1 and 2-dimensional data.r)rbrr~r7)rbr)rrmaxr rJrArrNrKrrrappendrOr1rCrP) rQrrdropexrbr~r}rSrTrr_lagslirs r5rrsLw *G*t  ? yya((  MM!  qAwTD9!]w{]:JKF1d^  1QT7F =6HqL((   ??6 ""rnc$|jdS)NF)ravelmats r5rrs 99S>rnc\|jjtt|SN)rrtake _triu_indicesrDrs r5rrs 55::mCH- ..rncBtj|\}}||z|zSr)r1 tril_indicesrrowscolss r5 _tril_indicesr#"#JD$ !8d?rncBtj|\}}||z|zSr)r1 triu_indicesrs r5rr(rrncBtj|\}}||z|zSr)r1 diag_indicesrs r5 _diag_indicesr-rrncttjt|}||zt|k(sJ|j ||fdS)Nrr])rqr1sqrtrDreshape)vrs r5rr2sC BGGCFOA q5CF?? 99aV39 ''rnc Rddtjddt|zzzz}ttj|}tj ||f}||tj |<||jz}|tj|xxdzcc<|S)Ng?r9r,r-) r1rrDrqroundrrrrr)rrresults r5rr8s "rwwq1s1v:~.. /D rxx~ D XXtTl #F$%F2??4 ! fhh F 2??4 !Q&! Mrnct|d}tj||dzzdz}tj|Dcgc]}t |j c}j Scc}w)z Create duplication matrix D_n which satisfies vec(S) = D_n vech(S) for symmetric matrix S Returns ------- D_n : ndarray rr,r-)rr1eyearrayrrrr)rtmprQs r5rrGsY CA &&a!e! "C 8841VAY__&4 5 7 774s A/ct|d}ttjtj||f}tj ||z|dk7S)z Create the elimination matrix L_n which satisfies vech(M) = L_n vec(M) for any matrix M Parameters ---------- Returns ------- rr)rrr1trilonesr)r vech_indicess r5rrUsK CArwwrww1v/0L 66!a%=* ++rnct|d}t|d}tj||z}tj||zj ||fd}|j |j dS)z Create the commutation matrix K_{p,q} satisfying vec(A') = K_{p,q} vec(A) Parameters ---------- p : int q : int Returns ------- K : ndarray (pq x pq) pqrrrr7)rr1rrFrrr)rrKindicess r5rresl CACA q1u AiiA&&1vS&9G 66'--/6 **rnc tj|dz }tj|dz }tdt|D]8}||}t|D]}||xx||||z dz zzcc<|d||d|:|S)z Transforms params to induce stationarity/invertability. Parameters ---------- params : array_like The AR coefficients Reference --------- Jones(1980) r-r,N)r1tanhrrD)params newparamsrjakiters r5_ar_transparamsrzs #I ''&1* C 1c&k "  aL1X 7E J!iE A 66 6J 7BQ "1  rnc,|j}|j}tt|dz ddD]?}||}t|D]"}||||||z dz zzd|dzz z ||<$|d||d|Adtj|z}|S)z Inverse of the Jones reparameterization Parameters ---------- params : array_like The transformed AR coefficients r,rr9r-N)r=rrDr1arctanh)rrrrr invarcoefss r5_ar_invtransparamsrs[[]F ++-C 3v;?Ar * 1I1X E -!fQY].C*CCAF CJ !Wr RZZ''J rnc dtj| z dtj| zz j}dtj| z dtj| zz j}tdt |D]8}||}t|D]}||xx||||z dz zz cc<|d||d|:|S)z Transforms params to induce stationarity/invertability. Parameters ---------- params : ndarray The ma coeffecients of an (AR)MA model. Reference --------- Jones(1980) r,N)r1expr=rrD)rrrrbrs r5_ma_transparamsrsbfffWo%!bfffWo*=>DDFI w Aw$7 8 > > @C1c&k "  aL1X 7E J!iE A 66 6J 7BQ "1  rnc|j}tt|dz ddD]?}||}t|D]"}||||||z dz zz d|dzz z ||<$|d||d|Atjd|z d|zz  }|S)z Inverse of the Jones reparameterization Parameters ---------- params : ndarray The transformed MA coefficients r,rr9r-N)r=rrDr1log)macoefsrrrr invmacoefss r5_ma_invtransparamsrs ,,.C 3w>> x = np.array([1, 3, 9., 19, 8.]) >>> levels = unintegrate_levels(x, 2) >>> levels array([ 1., 2.]) >>> unintegrate(np.diff(x, 2), levels) array([ 1., 3., 9., 19., 8.]) Nr,r9r)listrDre unintegrater1cumsumr_)rQlevelsx0s r5rrso4&\!_F 6{Q ZZ^299RUU2q5\2F;; B 99RUU2q5\ ""rnct|tjs t|}t|tjsJ|jj }d}|dvs|j |ry|dk(s|j dry|dk(s|j dry |d k(s|j d ry |d k(ry|dk(ry|dk(rytdj|)a$ Convert a pandas frequency to a periodicity Parameters ---------- freq : str or offset Frequency to convert Returns ------- int Periodicity of freq Notes ----- Annual maps to 1, quarterly maps to 4, monthly to 12, weekly to 52. )zA-zAS-zY-zYS-zYE-)AYr,Q)zQ-QSQEM)zM-MSME WzW-4DBHzDfreq {} not understood. Please report if you think this is in error.) r@r DateOffsetr rule_codeupper startswithrNformat)freq yearly_freqss r5rrs$ dG.. / dG.. // / >>   !D4L zT__\: (:; (:; -    &&,fTl  rn)r Fr))Nr,FT)r,r)rrF) r}rqrbz.Literal['forward', 'backward', 'both', 'none']rzLiteral['ex', 'sep', 'in']r~boolreturnzKNDArray | DataFrame | tuple[NDArray, NDArray] | tuple[DataFrame, DataFrame])NrrF)rzstr | offsets.DateOffsetr rq)3 __future__rstatsmodels.compat.pythonrrcnumpyr1pandasrArpandas.tseriesrpandas.tseries.frequenciesrtypingrstatsmodels.tools.datar r r>r statsmodels.tools.typingr statsmodels.tools.validationr rrr__all__rrmr{rrrrrrrrrrrrrrrrrrrrnr5rs","0A8, @ FMP`,bAJ04! TT?T0T T [ TpEJT#n/   (  8 , +*.,2*G4#D* rn