Estimation of parameters of autoregressive models with fractional differences in the presence of additive noise

Full Text

Introduction
To describe processes of various nature, equations with derivatives are increasingly used. and differences of
fractional order. Despite the lack of a simple interpretation, which give derivatives, integrals and differences
of integers, models described by fractional-order equations, make it possible to accurately simulate many
processes in physics and technology [1–4]. In connection with the active development and application of
equations with differences and fractional derivatives for modeling and forecasting problems, methods for
estimating systems have also begun to actively develop, describing fractional-order equations and differences.
Autoregressions with fractional differences are widely used in the analysis of time series with long
memory [5; 6]. There are a large number of different models with generalizations of fractional differences,
1The work was carried out as part of the development program of the Scientific and Educational Mathematical Center of
the Volga Federal District, agreement No. 075-02-2023-931.
94
Ivanov D.V. Estimation of parameters of autoregressive models with fractional differences in the presence of additive noise
Иванов Д.В. Оценивание параметров авторегрессии с разностями дробного порядка при наличии аддитивного шума
such as Gegenbauer autoregressive moving average (GARMA) [7; 8], fractional ARUMA [9], seasonal
autoregressive fractionally integrated moving average (SARFIMA) [10; 11], and autoregressive tempered
fractionally integrated moving average (ARTFIMA) [12; 13]. Various aspects of using fractional differences
for time series analysis have been considered [14; 15].
It is known that for autoregressive models of an integer order, autoregressive models with additive noise can
exceed the accuracy of ARMA models and autoregressive models [16]. An overview of methods for estimating
integer-order autoregressions in the presence of noise is presented in [17]. In the articles [8; 18; 19], the author
considered the estimation of autoregressions with fractional-order differences in the presence of noise with a
known noise ratio.
The article presents a new method for estimating parameters autoregressive models with fractional
differences in the presence of additive noise with an unknown variance of additive noise.
1. Basic results
Time series, is described by linear stochastic equations with fractional order differences:
zi =
Σr
m=1
b(m)Δ mzi????1 + ζi, yi = zi + ξi, (1.1)
where b(m) are constant coefficients; 0 < α1 . . . < αr; Γ(α) =
1∫
0
e????tt ????1dt ;
Δ mzi =
Σi
j=0
(−1)j
(
αm
j
)
zi????j is fractional difference;
(
αm
j
)
= ????( m+1)
????(j+1)????( m????j+1) is generalized binomial coefficients.
It is required to estimate the unknown coefficients of the dynamic system described by (1.1) from the
observed sequence {yi} with noise for the known orders r , αm.
If r and, αm are unknown, it is necessary to apply algorithms based on global optimization, such as
genetic algorithms [8].
The following assumptions are introduced:
A1. The dynamic system (1) is asymptotically stable.
A2. Noises {ξi} and {ζi} are statistically independent sequences with E{ξi} = 0, E{ζi} = 0, E
{
ξ2
i
}
=
= σ2
< ∞, E
{
ζ2
i
}
= σ2
< ∞ a.s., where E is the expectation operator.
A3. The output sequence {zi} , is independent of noise sequence {ξi}. The noise sequences {ξi} ,{ζi} are
mutually independent.
In [18], the following objective function was proposed for estimating the parameters:
min
b
∥Y − Cb∥2
2
1 + γ + bTHb
, (1.2)
where
H(mk)
= limi!1
1
N
ΣN????1
i=0
(
αm
j
)(
αk
j
)
N????j
N ,m = 1, r, k = 1, r,
C =
(
φT1
. . . φT
N
)
∈ RrN, Y = (y1 . . . yN) ∈ RN, b =
(
b(1) . . . b(r)
)T ∈ Rr,
φi = (Δ 1yi????1, ...,Δ ryi????1) ∈ R1r, γ = σ2
/σ2

Theorem 2.1. [18] Let the dynamic system described by Equation (1.1)with initial zero conditions and
assumptions A1–A3 be introduced. Then, the estimate of the coefficients determined by expression (1.2)
exists, is unique, and converges to the true value of the coefficients with probability 1, i.e.:
ˆb
(N) −a:→s:
N!1
b0 (1.3)
.
Proof. The proof of the theorem is similar to the proof given in [20].
The minimum of function (1.2) can be found as a solution to the biased normal system of equations
(
CTC − ˆσ2
H
)
ˆb
= CT Y. (1.4)
If the noise variance is unknown σ2
, then it is necessary to use the estimate of the additive noise variance
σ2
. The variance estimate ˆσ2
can be found as the minimal generalized singular value
ˆσ = σmin
(
¯ C,L
)
, (1.5)
Вестник Самарского университета. Естественнонаучная серия 2023. Том 29, № 3. С. 93–99
Vestnik of Samara University. Natural Science Series 2023, vol. 29, no. 3, pp. 93–99 95
where σmin
(
¯ C,L
)
is the minimal generalized singular number of matrices ¯ C and L,
¯ C =
(
Y C
)
.
¯H
= ¯L
T
¯L
, ¯H =
(
1 + γ 0
0 H
)
.
In [17] a review of methods for parametr estimation integer-order autoregressions with additive noise
is presented. One of the most accurate was the approach proposed in the article [21]. This article uses a
generalization of this approach to the case of autoregressions with fractional order differences. The maximum
value of the variance σ2
max is if the variance σ2
= 0 is defined as
σ2
max = σ2
min
(
¯ C,Lmax
)
where σmin
(
¯ C,Lmax
)
is minimal generalised singular values of matrices ¯ C and Lmax,
¯H
max = ¯LT
max
¯L
max, ¯Hmax =
(
1 0
0 H
)
.
The true value of the variance belongs to the interval σ2

∈
(
0 σ2
max
)
.
In [21], high-order Yule-Walker equations are used to determine the variance. However, this approach
cannot be applied directly, since it is impossible to obtain a vector of instrumental shifts for equation (1.1).
Minimization (1.2) can be written as an eigenvector problem:
(
¯ CT ¯ C − ˆσ2

¯H

)ˆ¯b = 0, (1.6)
where ¯b =
(
−1
b
)
.
Equation (1.6) requires knowing not only the variance of the additive noise σ2
, but also the variance σ2
.
In order to eliminate the need to evaluate σ2
and σ2
" simultaneously for fractional order autoregressions, we
use generalized instrumental variables [22], the application of generalized instrumental variables for fractional
order systems is considered in the article [23].
The vector of instrumental variables ψi satisfies the equality
lim
N!1
(
CT
 
¯ C − σ2

¯H
 
)
¯b
= 0, (1.7)
where
C  =
(
ψT
1 . . . ψTN
)
∈ RrN,
ψi = (Δ 1yi????2, ...,Δ ryi????2) ∈ R1r, ¯H  =
(
0 H 
)
,
H(mk)
  = limi!1
1
N
ΣN????1
i=0
(
αm
j − 1
)(
αk
j
)
N????j
N ,m = 1, r, k = 1, r,
For a finite sample, equality (1.7) will not be strict, the problem of determining the variance estimate
ˆσ2
can be described as a quadratic function minimization problem
min
2(0; max)
J (σ) , (1.8)
where
J (σ) =¯bT
(
CT
 
¯ C − σ2

¯H
 
)T (
CT
 
¯ C − σ2

¯H  )
¯b.
Based on equations (4), (6) and (8), an iterative algorithm is proposed for estimating the parameters ˆb
and the variance ˆσ2
.
Step 1. Determine the maximum value of the variance σ2
max is defined as
σ2
max = σ2
min
(
¯ C,Lmax
)
.
Step 2. Start from a generic value σ2

∈
(
0 σ2
max
)
.
Step 3. Compute the parameter vector from equation (1.4)
(
CTC − ˆσ2
H
)
ˆb
= CT Y,
Step 4. Compute the cost function (1.8)
J (σ) =ˆ¯b
T (
CT
 
¯ C − σ2

¯H
 
)T (
CT
 
¯ C − σ2

¯H
 
)ˆ¯ b.
Step 5. Choose a new value σ2
. The choice can be made using one of the methods of one-dimensional
optimization.
Step 6. Repeat steps 3–5 until the value associated with the minimum of is found.
96
Ivanov D.V. Estimation of parameters of autoregressive models with fractional differences in the presence of additive noise
Иванов Д.В. Оценивание параметров авторегрессии с разностями дробного порядка при наличии аддитивного шума
2. Simulation results
The proposed algorithm has been compared with ordinary least squares and the algorithm based on
objective function (1.2) with a known noise variance ratio. The minimum (1.2) of the objective function can
be found from the solution of the equation (1.4) or the augmented system of equations [24].
Test cases were compared by the following characteristics:the normalized root mean square error (NRMSE)
of parameter estimation, defined as
δb =
√

ˆb
− b0

2
/
∥b0∥2·1000/0,
and normalized root mean square error of modelling (NRMSEM), defined as
δz =
√
∥ˆz − z∥2
/
∥z∥2·1000/0.
The results were based on 50 independent Monte-Carlo simulations.
Example 1. The AR model is described by the equation
zi = 0.45Δ
????0:1zi????1 + ζi, yi = zi + ξi, (2.1)
Noise standard deviation ratio
σ
/
σz = 0.5, γ = 2.605
The number of data points N in each simulation was 10000.
Table 2.1 shows the mean values of tNRMSE and NRMSEM and their standard deviations.
Table 2.1
Mean values of NRMSE and NRMSEM and their standard deviations
Таблица 2.1
Средние значения NRMSE и NRMSEM и их стандартные отклонения
Ordinary least
squares, %
Algorithm with known
ratio, %
Proposed algorithm
with unknown ratio,
%
δb 8.95 ± 5.74 1.05 ± 1.20 1.44 ± 1.93
δz 43.50 ± 15.42 12.88 ± 9.88 13.55 ± 10.98
Example 2. The AR model is described by the equation
zi = 0.5Δ0:7zi????1 + ζi, yi = zi + ξi, (2.2)
Noise standard deviation ratio
σ
/
σz = 0.5, γ = 2.11
The number of data points N in each simulation was 2000.
Table 2.2 shows the mean values of tNRMSE and NRMSEM and their standard deviations.
Table 2.2
Mean values of NRMSE and NRMSEM and their standard deviation
Таблица 2.2
Средние значения NRMSE и NRMSEM и их стандартное отклонение
Ordinary least
squares, %
Algorithm with known
ratio, %
Proposed algorithm
with unknown ratio,
%
δb 15.73 ± 2.62 2.16 ± 1.46 3.00 ± 2.74
δz 26.70 ± 2.36 5.82 ± 4.47 7.13 ± 5.39
Conclusion
This paper proposed an estimation method of the parameters of fractional AR models with additive noise. The simulation results showed that the parameter estimates obtained using the proposed algorithm are highly accurate.
Further development of the proposed approach is the study of the best choice of instrumental variables and the choice of the weighting matrix.

About the authors

D. V. Ivanov

Samara National Research University; Samara State University of Transport

Author for correspondence.
Email: dvi85@list.ru
ORCID iD: 0000-0002-5021-5259

associate professor, Candidate of Physical and Mathematical Sciences, Department of Information Security; associate professor, Department of Information Technologies

34, Moskovskoye shosse, Samara, 443086, Russian Federation; 2B, Svobody Street, Samara, 443066, Russian Federation

References

Supplementary files