Appendix
Throughout Appendix,
M
denotes a generic constant which may take different values at different places. Let
t
i
∈
(
0,1
)
,
i
=
1,2
,
3,4
. Proofs of our theorems are based on the following lemmas. First of all, we introduce the following notations:
(A.1)
s
n
,
j
(
z
;
β
)
=
1
n
∑
t
=
1
n
K
h
(
β
τ
x
t
-
z
)
(
β
τ
x
t
-
z
h
)
j
y
t
,
222222222222222222
j
=
0,1
,
2
,
…
,
s
n
,
j
,
k
(
z
;
β
)
=
1
n
∑
t
=
1
n
K
h
(
β
τ
x
t
-
z
)
(
β
τ
x
t
-
z
h
)
j
y
t
-
k
2
,
2222222
k
=
1,2
,
…
,
p
,
j
=
0,1
,
2
,
…
,
g
(
z
;
β
)
=
1
n
∑
t
=
1
n
d
t
(
z
;
β
)
y
t
,
r
(
z
;
β
)
=
1
n
∑
t
=
1
n
d
t
(
z
;
β
)
,
g
k
(
z
;
β
)
=
1
n
∑
t
=
1
n
d
t
(
z
;
β
)
y
t
-
k
2
,
k
=
1,2
,
…
,
p
,
μ
j
=
∫
v
j
K
(
v
)
d
v
.
Lemma A.1.
Under the conditions of Theorem 3, one has
(A.2)
sup
θ
∈
Θ
|
S
n
,
j
(
β
τ
x
;
β
)
-
μ
j
f
(
β
τ
x
)
|
=
O
p
{
h
+
(
log
n
n
h
)
1
/
2
}
.
Proof.
Note that
(A.3)
E
[
S
n
,
j
(
β
τ
x
;
β
)
]
=
E
[
K
h
(
β
τ
x
t
-
β
τ
x
)
(
β
τ
x
t
-
β
τ
x
h
)
j
]
=
∫
1
h
K
(
z
t
-
β
τ
x
h
)
(
z
t
-
β
τ
x
h
)
j
f
(
z
t
)
d
z
t
=
∫
K
(
v
)
v
j
f
(
v
h
+
β
τ
x
)
d
v
=
μ
j
f
(
β
τ
x
)
+
R
1
(
β
τ
x
;
β
)
,
where
R
1
(
β
τ
x
;
β
)
=
∫
K
(
v
)
v
j
[
f
(
v
h
+
β
τ
x
)
-
f
(
β
τ
x
)
]
d
v
=
h
∫
K
(
v
)
v
j
+
1
f
′
(
t
1
v
h
+
β
τ
x
)
d
v
.
By conditions (A1) and (A8), as
h
→
0
, we have
(A.4)
sup
θ
∈
Θ
|
R
1
(
β
τ
x
;
β
)
|
=
O
(
h
)
.
Next, we show that
(A.5)
sup
θ
∈
Θ
|
S
n
,
j
(
β
τ
x
;
β
)
-
E
[
S
n
,
j
(
β
τ
x
;
β
)
]
|
=
O
p
{
(
log
n
n
h
)
1
/
2
}
.
By condition (A6) and the law of large numbers, we have
S
n
,
j
(
β
τ
x
;
β
)
→
p
E
[
S
n
,
j
(
β
τ
x
;
β
)
]
.
Note
(A.6)
|
S
n
,
j
(
β
1
τ
x
;
β
1
)
-
S
n
,
j
(
β
2
τ
x
;
β
2
)
|
≤
1
n
∑
t
=
1
n
K
h
(
β
1
τ
x
t
-
β
1
τ
x
)
×
|
[
β
1
τ
x
t
-
β
1
τ
x
h
]
j
-
[
β
2
τ
x
t
-
β
2
τ
x
h
]
j
|
+
1
n
∑
t
=
1
n
|
β
2
τ
x
t
-
β
2
τ
x
h
|
j
×
|
K
h
(
β
1
τ
x
t
-
β
1
τ
x
)
-
K
h
(
β
2
τ
x
t
-
β
2
τ
x
)
|
=
I
1
+
I
2
.
For
I
1
,
(A.7)
|
[
β
1
τ
x
t
-
β
1
τ
x
h
]
j
-
[
β
2
τ
x
t
-
β
2
τ
x
h
]
j
|
≤
∑
i
=
1
j
|
β
1
τ
x
t
-
β
1
τ
x
h
|
j
-
i
|
β
2
τ
x
t
-
β
2
τ
x
h
|
i
-
1
|
(
β
1
-
β
2
)
τ
(
x
t
-
x
)
h
|
≤
j
∥
x
t
-
x
h
∥
j
∥
β
1
-
β
2
∥
.
We have
(A.8)
I
1
≤
j
n
∑
t
=
1
n
[
sup
θ
∈
Θ
K
h
(
β
τ
x
t
-
β
τ
x
)
]
∥
x
t
-
x
h
∥
j
∥
β
1
-
β
2
∥
.
For
I
2
,
(A.9)
|
K
h
(
β
1
τ
x
t
-
β
1
τ
x
)
-
K
h
(
β
2
τ
x
t
-
β
2
τ
x
)
|
≤
[
sup
θ
∈
Θ
|
K
h
′
(
β
*
τ
x
t
-
β
*
τ
x
)
|
]
∥
x
t
-
x
h
∥
∥
β
1
-
β
2
∥
,
where
β
*
lies between
β
1
and
β
2
. Thus,
(A.10)
I
2
≤
1
n
∑
t
=
1
n
[
sup
θ
∈
Θ
|
K
h
′
(
β
*
τ
x
t
-
β
*
τ
x
)
|
]
∥
x
t
-
x
h
∥
j
+
1
∥
β
1
-
β
2
∥
.
Combining (A.8) with (A.10), we have
|
S
n
,
j
(
z
;
β
1
)
-
S
n
,
j
(
z
;
β
2
)
|
≤
B
n
∥
β
1
-
β
2
∥
, where
(A.11)
B
n
=
j
n
∑
t
=
1
n
[
sup
θ
∈
Θ
K
h
(
β
τ
x
t
-
β
τ
x
)
]
∥
x
t
-
x
h
∥
j
+
1
n
∑
t
=
1
n
[
sup
θ
∈
Θ
|
K
h
′
(
β
*
τ
x
t
-
β
*
τ
x
)
|
]
∥
x
t
-
x
h
∥
j
+
1
=
B
n
1
+
B
n
2
.
Note that
(A.12)
E
(
B
n
1
)
=
j
h
∫
[
sup
θ
∈
Θ
K
h
(
β
τ
x
t
-
β
τ
x
)
]
∥
x
t
-
x
h
∥
j
p
(
x
t
)
d
x
t
=
j
∫
[
sup
θ
∈
Θ
K
(
β
τ
v
)
]
∥
v
∥
j
p
(
v
h
+
x
)
d
v
=
j
p
(
x
)
∫
[
sup
θ
∈
Θ
K
(
β
τ
v
)
]
∥
v
∥
j
d
v
+
j
h
∫
[
sup
θ
∈
Θ
K
(
β
τ
v
)
]
∥
v
∥
j
∂
p
(
t
2
v
h
+
x
)
∂
x
τ
v
d
v
,
E
(
B
n
2
)
=
1
h
∫
[
sup
θ
∈
Θ
|
K
h
′
(
β
*
τ
x
t
-
β
*
τ
x
)
|
]
×
∥
x
t
-
x
h
∥
j
+
1
p
(
x
t
)
d
x
t
=
∫
[
sup
θ
∈
Θ
|
K
h
′
(
β
*
τ
v
)
|
]
∥
v
∥
j
+
1
p
(
v
h
+
x
)
d
v
=
p
(
x
)
∫
[
sup
θ
∈
Θ
|
K
h
′
(
β
*
τ
v
)
|
]
∥
v
∥
j
+
1
d
v
+
h
∫
[
sup
θ
∈
Θ
|
K
h
′
(
β
*
τ
v
)
|
]
∥
v
∥
j
+
1
∂
p
(
t
2
v
h
+
x
)
∂
x
τ
v
d
v
.
On the compact support set
S
, it is easy to obtain
E
(
B
n
)
<
∞
, implying
B
n
=
O
p
(
1
)
.
According to Lemma 1 and Theorem 1 of Andrews [11], together with Lemma 1 of Xue and Pang [6], we can obtain (A.5).
Combining (A.4) with (A.5), we complete the proof of this lemma.
Lemma A.2.
Under the conditions of Theorem 3, one has
(A.13)
sup
θ
∈
Θ
|
r
(
z
;
β
)
-
μ
2
f
2
(
z
)
|
=
O
p
{
h
+
(
log
n
n
h
)
1
/
2
}
.
Proof.
Note that
(A.14)
r
(
z
;
β
)
=
S
n
,
0
(
z
;
β
)
S
n
,
2
(
z
;
β
)
-
S
n
,
1
2
(
z
;
β
)
=
[
S
n
,
0
(
z
;
β
)
-
f
(
z
)
+
f
(
z
)
]
×
[
S
n
,
2
(
z
;
β
)
-
μ
2
f
(
z
)
+
μ
2
f
(
z
)
]
-
[
S
n
,
1
(
z
;
β
)
-
0
]
2
=
[
S
n
,
0
(
z
;
β
)
-
f
(
z
)
]
[
S
n
,
2
(
z
;
β
)
-
μ
2
f
(
z
)
]
+
μ
2
f
(
z
)
[
S
n
,
0
(
z
;
β
)
-
f
(
z
)
]
+
f
(
z
)
[
S
n
,
2
(
z
;
β
)
-
μ
2
f
(
z
)
]
+
μ
2
f
2
(
z
)
-
[
S
n
,
1
(
z
;
β
)
-
0
]
2
.
Then, we have
(A.15)
sup
θ
∈
Θ
|
r
(
z
;
β
)
-
μ
2
f
2
(
z
)
|
≤
sup
θ
∈
Θ
|
S
n
,
0
(
z
;
β
)
-
f
(
z
)
|
sup
θ
∈
Θ
|
S
n
,
2
(
z
;
β
)
-
μ
2
f
(
z
)
|
+
μ
2
f
(
z
)
sup
θ
∈
Θ
|
S
n
,
0
(
z
;
β
)
-
f
(
z
)
|
+
f
(
z
)
sup
θ
∈
Θ
|
S
n
,
2
(
z
;
β
)
-
μ
2
f
(
z
)
|
+
[
sup
θ
∈
Θ
|
S
n
,
1
(
z
;
β
)
-
0
|
]
2
.
According to Lemma A.1, it is easy to obtain the result of this lemma.
Lemma A.3.
Under the assumption of Theorem 3, one has
(A.16)
sup
θ
∈
Θ
|
s
n
,
j
(
β
τ
x
;
β
)
-
μ
j
f
(
β
τ
x
)
σ
(
β
τ
x
;
β
)
|
=
O
p
{
h
+
(
log
n
n
h
)
1
/
2
}
,
sup
θ
∈
Θ
|
s
n
,
j
,
k
(
β
τ
x
;
β
)
-
μ
j
f
(
β
τ
x
)
σ
k
(
β
τ
x
;
β
)
|
=
O
p
{
h
+
(
log
n
n
h
)
1
/
2
}
.
Proof.
Note that
(A.17)
E
[
s
n
,
j
(
β
τ
x
;
β
)
]
=
E
{
E
[
K
h
(
β
τ
x
t
-
β
τ
x
)
(
β
τ
x
t
-
β
τ
x
h
)
j
y
t
∣
β
τ
x
t
]
}
=
E
[
K
h
(
β
τ
x
t
-
β
τ
x
)
(
β
τ
x
t
-
β
τ
x
h
)
j
σ
(
β
τ
x
t
;
β
)
]
=
∫
1
h
K
(
z
t
-
β
τ
x
h
)
(
z
t
-
β
τ
x
h
)
j
σ
(
z
t
;
β
)
f
(
z
t
)
d
z
t
=
∫
K
(
v
)
v
j
σ
(
v
h
+
β
τ
x
;
β
)
f
(
v
h
+
β
τ
x
)
d
v
=
μ
j
f
(
β
τ
x
)
σ
(
β
τ
x
;
β
)
+
R
2
(
β
τ
x
;
β
)
,
where
(A.18)
R
2
(
z
;
β
)
=
f
(
β
τ
x
)
∫
K
(
v
)
v
j
[
σ
(
v
h
+
β
τ
x
;
β
)
-
σ
(
β
τ
x
;
β
)
]
d
v
+
σ
(
β
τ
x
;
β
)
∫
K
(
v
)
v
j
[
f
(
v
h
+
β
τ
x
)
-
f
(
β
τ
x
)
]
d
v
+
∫
K
(
v
)
v
j
[
σ
(
v
h
+
β
τ
x
;
β
)
-
σ
(
β
τ
x
;
β
)
]
2222
×
[
f
(
v
h
+
β
τ
x
)
-
f
(
β
τ
x
)
]
d
v
=
h
f
(
β
τ
x
)
∫
K
(
v
)
v
j
+
1
σ
′
(
t
3
v
h
+
β
τ
x
;
β
)
d
v
+
h
σ
(
β
τ
x
;
β
)
∫
K
(
v
)
v
j
+
1
f
′
(
t
4
v
h
+
β
τ
x
)
d
v
+
h
2
∫
K
(
v
)
v
j
+
2
σ
′
(
t
3
v
h
+
β
τ
x
;
β
)
f
′
(
t
4
v
h
+
β
τ
x
)
d
v
.
By conditions (A1) and (A3), as
h
→
0
, we have
(A.19)
sup
θ
∈
Θ
|
R
2
(
β
τ
x
;
β
)
|
=
O
(
h
)
.
Next, we show that
(A.20)
sup
θ
∈
Θ
|
s
n
,
j
(
β
τ
x
;
β
)
-
E
[
s
n
,
j
(
β
τ
x
;
β
)
]
|
=
O
p
{
(
log
n
n
h
)
1
/
2
}
.
By condition (A6) and the law of large numbers, we have
s
n
,
j
(
β
τ
x
;
β
)
→
p
E
[
s
n
,
j
(
β
τ
x
;
β
)
]
.
Note
(A.21)
|
s
n
,
j
(
β
1
τ
x
;
β
1
)
-
s
n
,
j
(
β
1
τ
x
;
β
2
)
|
≤
1
n
∑
t
=
1
n
K
h
(
β
1
τ
x
t
-
β
1
τ
x
)
|
[
β
1
τ
x
t
-
β
1
τ
x
h
]
j
222222222222222222222222
-
[
β
2
τ
x
t
-
β
2
τ
x
h
]
j
|
|
y
t
|
+
1
n
∑
t
=
1
n
[
β
2
τ
x
t
-
β
2
τ
x
h
]
j
|
K
h
(
β
1
τ
x
t
-
β
1
τ
x
)
222222222222222222222222
-
K
h
(
β
2
τ
x
t
-
β
2
τ
x
)
|
|
y
t
|
≤
M
j
n
∑
t
=
1
n
[
sup
θ
∈
Θ
K
h
(
β
τ
x
t
-
β
τ
x
)
]
[
e
t
2
+
h
t
(
θ
U
)
]
2222222222
×
∥
x
t
-
x
h
∥
j
∥
β
1
-
β
2
∥
+
M
n
∑
t
=
1
n
[
sup
θ
∈
Θ
|
K
h
′
(
β
*
τ
x
t
-
β
*
τ
x
)
|
]
222222222222
×
[
e
t
2
+
h
t
(
θ
U
)
]
∥
x
t
-
x
h
∥
j
+
1
∥
β
1
-
β
2
∥
=
C
n
∥
β
1
-
β
2
∥
,
where
(A.22)
C
n
=
M
j
n
∑
t
=
1
n
[
sup
θ
∈
Θ
K
h
(
β
τ
x
t
-
β
τ
x
)
]
[
e
t
2
+
h
t
(
θ
U
)
]
222222
×
∥
x
t
-
x
h
∥
j
+
M
n
∑
t
=
1
n
[
sup
θ
∈
Θ
|
K
h
′
(
β
*
τ
x
t
-
β
*
τ
x
)
|
]
22222222
×
[
e
t
2
+
h
t
(
θ
U
)
]
∥
x
t
-
x
h
∥
j
+
1
.
As discussing
B
n
in the proof of Lemma A.1, we can obtain
E
(
C
n
)
<
∞
, implying
C
n
=
O
p
(
1
)
.
According to Lemma 1 and Theorem 1 of Andrews [11], together with Lemma 1 of Xue and Pang [6], we can obtain (A.20).
Combining (A.19) with (A.20), we prove the first result of this lemma. By mimicking the above proof, we can show that the other result holds.
Lemma A.4.
Under the assumption of Theorem 3, one has
(A.23)
sup
θ
∈
Θ
|
g
(
z
;
β
)
-
μ
2
f
2
(
z
)
σ
(
z
;
β
)
|
=
O
p
{
h
+
(
log
n
n
h
)
1
/
2
}
,
sup
θ
∈
Θ
|
g
k
(
z
;
β
)
-
μ
2
f
2
(
z
)
σ
k
(
z
;
β
)
|
=
O
p
{
h
+
(
log
n
n
h
)
1
/
2
}
.
Proof.
The proof of Lemma A.4 is the same as that of Lemma A.2, so we omit it.
Lemma A.5.
Under the conditions of Theorem 3, one has
(A.24)
sup
θ
∈
Θ
|
1
r
(
z
;
β
)
-
1
μ
2
f
2
(
z
)
|
=
O
p
{
h
+
(
log
n
n
h
)
1
/
2
}
.
Proof.
By conditions (A1) and (A8), we know
f
(
z
)
≠
0
,
μ
2
≠
0
.
According to Lemma A.2, for all
θ
∈
Θ
, it is uniformly followed that
r
(
z
;
β
)
=
μ
2
f
2
(
z
)
+
o
p
(
1
)
. Thus, for all
θ
∈
Θ
, it is uniformly followed that
1
/
r
(
z
;
β
)
=
1
/
μ
2
f
2
(
z
)
+
o
p
(
1
)
.
Proof of Theorem 3.
Note that
(A.25)
|
σ
^
(
z
;
β
)
-
σ
(
z
;
β
)
|
=
|
g
(
z
;
β
)
r
(
z
;
β
)
-
μ
2
f
2
(
z
)
σ
(
z
;
β
)
μ
2
f
2
(
z
)
|
≤
|
1
r
(
z
;
β
)
|
|
g
(
z
;
β
)
-
μ
2
f
2
(
z
)
σ
(
z
;
β
)
|
+
μ
2
f
2
(
z
)
|
σ
(
z
;
β
)
|
|
1
r
(
z
;
β
)
-
1
μ
2
f
2
(
z
)
|
.
Then, we have
(A.26)
sup
θ
∈
Θ
|
σ
^
(
z
;
β
)
-
σ
(
z
;
β
)
|
≤
[
sup
θ
∈
Θ
|
1
r
(
z
;
β
)
-
1
μ
2
f
2
(
z
)
|
+
1
μ
2
f
2
(
z
)
]
×
[
sup
θ
∈
Θ
|
g
(
z
;
β
)
-
μ
2
f
2
(
z
)
σ
(
z
;
β
)
|
]
+
μ
2
f
2
(
z
)
sup
θ
∈
Θ
|
σ
(
z
;
β
)
|
sup
θ
∈
Θ
|
1
r
(
z
;
β
)
-
1
μ
2
f
2
(
z
)
|
=
O
p
{
h
2
+
(
log
n
n
h
)
1
/
2
}
.
With conditions (A1), (A3), and (A8), we can obtain the first result of this theorem by Lemmas A.4 and A.5. The other result can be similarly proved.
Proof of Theorem 4.
Note that
(A.27)
m
^
(
z
;
θ
)
-
m
(
z
;
θ
)
=
σ
^
(
z
;
β
)
ω
0
+
∑
k
=
1
p
ω
k
σ
^
k
(
z
;
β
)
-
σ
(
z
;
β
)
ω
0
+
∑
k
=
1
p
ω
k
σ
k
(
z
;
β
)
=
σ
^
(
z
;
β
)
-
σ
(
z
;
β
)
ω
0
+
∑
k
=
1
p
ω
k
σ
^
k
(
z
;
β
)
-
σ
(
z
;
β
)
[
1
ω
0
+
∑
k
=
1
p
ω
k
σ
k
(
z
;
β
)
2222222222222
-
1
ω
0
+
∑
k
=
1
p
ω
k
σ
^
k
(
z
;
β
)
]
=
σ
^
(
z
;
β
)
-
σ
(
z
;
β
)
ω
0
+
∑
k
=
1
p
ω
k
σ
^
k
(
z
;
β
)
-
σ
(
z
;
β
)
∑
k
=
1
p
ω
k
[
σ
^
k
(
z
;
β
)
-
σ
k
(
z
;
β
)
]
[
ω
0
+
∑
k
=
1
p
ω
k
σ
^
k
(
z
;
β
)
]
[
ω
0
+
∑
k
=
1
p
ω
k
σ
k
(
z
;
β
)
]
.
Then, we have
(A.28)
sup
θ
∈
Θ
|
m
^
(
z
;
θ
)
-
m
(
z
;
θ
)
|
≤
1
ω
0
sup
θ
∈
Θ
|
σ
^
(
z
;
β
)
-
σ
(
z
;
β
)
|
+
sup
θ
∈
Θ
|
σ
(
z
;
β
)
|
ω
0
2
∑
k
=
1
p
ω
k
sup
θ
∈
Θ
|
σ
^
k
(
z
;
β
)
-
σ
k
(
z
;
β
)
|
.
By condition (A3), applying Theorem 3, we can complete the proof of Theorem 4.
Proof of Theorem 5.
By the law of large numbers, we have
L
n
(
θ
)
→
p
L
(
θ
)
.
It is easy to get that
(A.29)
L
n
(
θ
1
)
-
L
n
(
θ
2
)
=
1
n
∑
t
=
1
n
{
[
ε
t
2
(
θ
1
)
h
t
(
θ
1
)
-
ε
t
2
(
θ
2
)
h
t
(
θ
2
)
]
[
log
h
t
(
θ
1
)
-
log
h
t
(
θ
2
)
]
222222222
+
[
ε
t
2
(
θ
1
)
h
t
(
θ
1
)
-
ε
t
2
(
θ
2
)
h
t
(
θ
2
)
]
}
π
(
x
t
)
=
1
n
∑
t
=
1
n
{
1
h
t
(
θ
2
)
[
log
h
t
(
θ
1
)
-
log
h
t
(
θ
2
)
]
+
ε
t
2
(
θ
1
)
[
1
h
t
(
θ
1
)
-
1
h
t
(
θ
2
)
]
+
1
h
t
(
θ
2
)
[
ε
t
2
(
θ
1
)
-
ε
t
2
(
θ
2
)
]
}
π
(
x
t
)
=
1
n
∑
t
=
1
n
[
∂
log
h
t
(
θ
1
*
)
∂
θ
τ
+
ε
t
2
(
θ
1
)
∂
∂
θ
τ
(
1
h
t
(
θ
2
*
)
)
]
222222222
×
π
(
x
t
)
(
θ
1
-
θ
2
)
+
1
n
∑
t
=
1
n
π
(
x
t
)
h
t
(
θ
2
)
[
2
y
t
-
m
(
β
τ
x
t
;
θ
1
)
h
t
(
θ
1
)
-
m
(
β
τ
x
t
;
θ
2
)
h
t
(
θ
2
)
]
×
{
m
(
β
τ
x
t
;
θ
1
)
[
h
t
(
θ
1
)
-
h
t
(
θ
2
)
]
+
h
t
(
θ
2
)
[
m
(
β
τ
x
t
;
θ
1
)
-
m
(
β
τ
x
t
;
θ
2
)
]
}
=
1
n
∑
t
=
1
n
[
1
h
t
(
θ
1
*
)
∂
h
t
(
θ
1
*
)
∂
θ
τ
-
ε
t
2
(
θ
1
)
h
t
2
(
θ
2
*
)
∂
h
t
(
θ
2
*
)
∂
θ
τ
]
×
π
(
x
t
)
(
θ
1
-
θ
2
)
+
1
n
∑
t
=
1
n
[
2
y
t
-
m
(
β
τ
x
t
;
θ
1
)
h
t
(
θ
1
)
-
m
(
β
τ
x
t
;
θ
2
)
h
t
(
θ
2
)
]
×
[
m
(
β
τ
x
t
;
θ
1
)
h
t
(
θ
2
)
∂
h
t
(
θ
3
*
)
∂
θ
τ
+
∂
m
(
β
τ
x
t
;
θ
4
*
)
∂
θ
τ
]
×
π
(
x
t
)
(
θ
1
-
θ
2
)
,
where
θ
i
*
(
i
=
1,2
,
3,4
)
lies between
θ
1
and
θ
2
.
Note that
(A.30)
1
h
t
(
θ
)
<
1
ω
0
,
y
t
-
i
2
h
t
(
θ
)
<
1
ω
i
,
i
=
1,2
,
…
,
p
.
1
h
t
(
θ
)
∂
h
t
(
θ
)
∂
θ
τ
=
(
1
h
t
(
θ
)
,
y
t
-
1
2
h
t
(
θ
)
,
…
,
y
t
-
p
2
h
t
(
θ
)
,
O
1
×
(
q
-
1
)
)
.
Then, we have
∥
(
1
/
h
t
(
θ
)
)
(
∂
h
t
(
θ
)
/
∂
θ
τ
)
∥
<
M
.
We can derive that
(A.31)
|
ε
t
(
θ
)
|
=
|
y
t
-
m
(
β
τ
x
t
;
θ
)
h
t
(
θ
)
|
=
|
m
(
β
τ
x
t
;
θ
0
)
h
t
+
e
t
h
t
-
m
(
β
τ
x
t
;
θ
)
h
t
(
θ
)
|
≤
M
h
t
(
θ
U
)
+
|
e
t
h
t
|
≤
M
h
t
(
θ
U
)
+
1
2
(
e
t
2
+
h
t
)
≤
M
[
e
t
2
+
h
t
(
θ
U
)
]
.
Similarly we have
(A.32)
|
ε
t
2
(
θ
)
|
≤
M
[
e
t
2
h
t
(
θ
U
)
+
h
t
2
(
θ
U
)
]
.
Following (A.29), we have
(A.33)
|
L
n
(
θ
1
)
-
L
n
(
θ
2
)
|
≤
D
n
∥
θ
1
-
θ
2
∥
,
where
(A.34)
D
n
=
M
n
∑
t
=
1
n
{
1
+
e
t
2
+
e
t
2
h
t
(
θ
U
)
+
h
t
2
(
θ
U
)
+
h
t
(
θ
U
)
}
π
(
x
t
)
.
By condition (A5), we can obtain
E
(
D
n
)
<
∞
, implying
D
n
=
O
p
(
1
)
.
According to Lemma 1 and Theorem 1 of Andrews [11], we can obtain
(A.35)
sup
θ
∈
Θ
|
L
n
(
θ
)
-
L
(
θ
)
|
=
o
p
(
1
)
.
Note that
ε
^
t
(
θ
)
=
ε
t
(
θ
)
+
[
m
(
β
τ
x
t
;
θ
)
-
m
^
(
β
τ
x
t
;
θ
)
]
h
t
(
θ
)
; we have
(A.36)
L
^
n
(
θ
)
-
L
n
(
θ
)
=
1
n
∑
t
=
1
n
π
(
x
t
)
h
t
(
θ
)
[
ε
^
t
2
(
θ
)
-
ε
t
2
(
θ
)
]
=
1
n
∑
t
=
1
n
[
m
^
(
β
τ
x
t
;
θ
)
-
m
(
β
τ
x
t
;
θ
)
]
2
h
t
(
θ
)
π
(
x
t
)
+
2
n
∑
t
=
1
n
[
m
(
β
τ
x
t
;
θ
)
-
m
^
(
β
τ
x
t
;
θ
)
]
22222222222
×
[
y
t
-
m
(
β
τ
x
t
;
θ
)
h
t
(
θ
)
]
π
(
x
t
)
.
Then, we can get that
(A.37)
sup
θ
∈
Θ
|
L
^
n
(
θ
)
-
L
n
(
θ
)
|
≤
1
n
∑
t
=
1
n
h
t
(
θ
)
π
(
x
t
)
[
sup
θ
∈
Θ
|
m
^
(
β
τ
x
t
;
θ
)
-
m
(
β
τ
x
t
;
θ
)
|
]
2
+
2
n
∑
t
=
1
n
[
e
t
2
+
h
t
(
θ
U
)
]
π
(
x
t
)
22222222222
×
sup
θ
∈
Θ
|
m
^
(
β
τ
x
t
;
θ
)
-
m
(
β
τ
x
t
;
θ
)
|
.
Note that
E
[
h
t
(
θ
)
π
(
x
t
)
]
=
E
{
π
(
x
t
)
E
[
h
t
(
θ
)
∣
x
t
]
}
and
E
[
e
t
2
π
(
x
t
)
]
=
E
[
π
(
x
t
)
]
. Invoking Theorem 3, condition (A5), and the law of large numbers for
(
1
/
n
)
∑
t
=
1
n
h
t
(
θ
)
π
(
x
t
)
and
(
1
/
n
)
∑
t
=
1
n
e
t
2
π
(
x
t
)
, we can show that
(A.38)
sup
θ
∈
Θ
|
L
^
n
(
θ
)
-
L
n
(
θ
)
|
=
o
p
(
1
)
.
With (A.35) and (A.38), we have
(A.39)
sup
θ
∈
Θ
|
L
^
n
(
θ
)
-
L
(
θ
)
|
=
o
p
(
1
)
.
According to Theorem 2.12 and Lemma 14.3 of Kosorok [12], we can obtain that
θ
^
converges in probability to the true parameter
θ
0
.
Next we continue to show the consistency of nonparametric estimate. Indeed, we have
(A.40)
m
^
(
z
;
θ
^
n
)
-
m
(
z
;
θ
0
)
=
m
^
(
z
;
θ
^
n
)
-
m
(
z
;
θ
^
n
)
+
m
(
z
;
θ
^
n
)
-
m
(
z
;
θ
0
)
=
O
p
{
h
2
+
(
log
n
n
h
)
1
/
2
}
+
∂
m
(
z
;
θ
)
∂
θ
τ
|
θ
=
θ
0
·
(
θ
^
n
-
θ
0
)
[
1
+
o
p
(
1
)
]
=
o
p
(
1
)
.
To prove Theorem 6, we make a Taylor expansion about
∂
L
^
n
(
θ
)
/
∂
θ
at
θ
0
,
(A.41)
0
=
∂
L
^
n
(
θ
)
∂
θ
|
θ
=
θ
^
n
=
∂
L
^
n
(
θ
)
∂
θ
|
θ
=
θ
0
+
∂
2
L
^
n
(
θ
)
∂
θ
∂
θ
τ
|
θ
=
θ
~
·
(
θ
^
n
-
θ
0
)
,
where
θ
~
lies between
θ
^
n
and
θ
0
.
Thus one has
(A.42)
n
(
θ
^
n
-
θ
0
)
=
{
-
∂
2
L
^
n
(
θ
)
∂
θ
∂
θ
τ
|
θ
=
θ
~
}
-
1
{
n
∂
L
^
n
(
θ
)
∂
θ
|
θ
=
θ
0
}
.
We need the following lemmas to deal with
n
∂
L
^
n
(
θ
)
/
∂
θ
|
θ
=
θ
0
and
-
∂
2
L
^
n
(
θ
)
/
∂
θ
∂
θ
τ
|
θ
=
θ
~
, respectively.
Lemma A.6.
Under the conditions of Theorem 3, it holds that
(A.43)
sup
θ
∈
Θ
|
D
n
,
j
(
β
τ
x
;
β
)
|
=
O
p
{
(
log
n
n
h
)
1
/
2
}
,
j
=
0,1
,
…
,
where
(A.44)
D
n
,
j
(
z
;
β
)
=
1
n
∑
t
=
1
n
K
h
(
β
τ
x
t
-
z
)
(
β
τ
x
t
-
z
h
)
j
[
y
t
-
σ
(
β
τ
x
t
;
β
)
]
.
Proof.
Note that
(A.45)
E
[
D
n
,
j
(
β
τ
x
;
β
)
]
=
E
{
E
[
D
n
,
j
(
β
τ
x
;
β
)
∣
β
τ
x
t
]
}
=
0
.
By the law of large numbers and condition that the process
{
y
t
,
x
t
}
is stationary and ergodic, we can obtain that
D
n
,
j
(
β
τ
x
;
β
)
converges in probability to 0. Then according to Lemma 1 and Theorem 1 of Andrews [11], together with Lemma 1 of Xue and Pang [6], we can prove that the result of this lemma holds as the proof of Lemma A.1.
Lemma A.7.
If conditions (A1)–(A8) are satisfied, then as
n
→
∞
and
h
→
0
such that
n
h
3
→
0
, it holds that
(A.46)
sup
θ
∈
Θ
|
σ
^
′
(
z
;
β
)
-
σ
′
(
z
;
β
)
|
=
O
p
{
h
+
(
log
n
n
h
3
)
1
/
2
}
,
sup
θ
∈
Θ
|
σ
^
k
′
(
z
;
β
)
-
σ
k
′
(
z
;
β
)
|
=
O
p
{
h
+
(
log
n
n
h
3
)
1
/
2
}
,
where
(A.47)
h
σ
^
′
(
z
;
β
)
=
∑
t
=
1
n
w
t
(
z
;
β
)
y
t
∑
t
=
1
n
d
t
(
z
;
β
)
,
h
σ
^
k
′
(
z
;
β
)
=
∑
t
=
1
n
w
t
(
z
;
β
)
y
t
-
k
2
∑
t
=
1
n
d
t
(
z
;
β
)
,
w
t
(
z
;
β
)
=
K
h
(
β
τ
x
t
-
z
)
{
(
β
τ
x
t
-
z
h
)
S
n
,
0
(
z
;
β
)
-
S
n
,
1
(
z
;
β
)
}
.
Proof.
Applying the fact that
(
1
/
n
)
∑
t
=
1
n
w
t
(
z
;
β
)
=
0
, we can get that
(A.48)
h
σ
^
′
(
z
;
β
)
-
h
σ
′
(
z
;
β
)
=
(
1
n
∑
t
=
1
n
w
t
(
z
;
β
)
[
σ
′
y
t
-
σ
(
β
τ
x
t
;
β
)
+
σ
(
β
τ
x
t
;
β
)
22222222222222222
-
σ
(
z
;
β
)
-
(
β
τ
x
t
-
z
)
σ
′
(
z
;
β
)
]
1
n
∑
t
=
1
n
)
×
(
1
n
∑
t
=
1
n
d
t
(
z
;
β
)
)
-
1
=
(
1
/
n
)
∑
t
=
1
n
w
t
(
z
;
β
)
[
y
t
-
σ
(
β
τ
x
t
;
β
)
]
(
1
/
n
)
∑
t
=
1
n
d
t
(
z
;
β
)
+
(
1
n
∑
t
=
1
n
w
t
(
z
;
β
)
(
β
τ
x
t
-
z
h
)
2
h
2
2
σ
′′
(
z
;
β
)
×
[
1
+
o
(
1
)
]
∑
t
=
1
n
)
(
1
n
∑
t
=
1
n
d
t
(
z
;
β
)
)
-
1
.
Note that
(A.49)
1
n
∑
t
=
1
n
d
t
(
z
;
β
)
=
S
n
,
0
(
z
;
β
)
S
n
,
2
(
z
;
β
)
-
S
n
,
1
2
(
z
;
β
)
,
1
n
∑
t
=
1
n
w
t
(
z
;
β
)
(
β
τ
x
t
-
z
h
)
2
=
S
n
,
0
(
z
;
β
)
S
n
,
3
(
z
;
β
)
-
S
n
,
1
(
z
;
β
)
S
n
,
2
(
z
;
β
)
,
1
n
∑
t
=
1
n
w
t
(
z
;
β
)
[
y
t
-
σ
(
β
τ
x
t
;
β
)
]
=
S
n
,
0
(
z
;
β
)
D
n
,
1
(
z
;
β
)
-
S
n
,
1
(
z
;
β
)
D
n
,
0
(
z
;
β
)
.
By Lemmas A.1 and A.6, we have
(A.50)
sup
θ
∈
Θ
|
1
n
∑
t
=
1
n
d
t
(
z
;
β
)
-
μ
2
f
2
(
z
)
|
=
O
p
{
h
+
(
log
n
n
h
)
1
/
2
}
,
sup
θ
∈
Θ
|
1
n
∑
t
=
1
n
w
t
(
z
;
β
)
(
β
τ
x
t
-
z
h
)
2
|
=
O
p
{
h
+
(
log
n
n
h
)
1
/
2
}
,
sup
θ
∈
Θ
|
1
n
∑
t
=
1
n
w
t
(
z
;
β
)
[
y
t
-
σ
(
β
τ
x
t
;
β
)
]
|
=
O
p
{
h
+
(
log
n
n
h
)
1
/
2
}
.
Then, we can obtain that
(A.51)
h
sup
θ
∈
Θ
|
σ
^
′
(
z
;
β
)
-
σ
′
(
z
;
β
)
|
≤
[
sup
θ
∈
Θ
|
1
n
∑
t
=
1
n
w
t
(
z
;
β
)
[
y
t
-
σ
(
β
τ
x
t
;
β
)
]
|
]
×
[
sup
θ
∈
Θ
|
1
n
-
1
∑
t
=
1
n
d
t
(
z
;
β
)
-
1
μ
2
f
2
(
z
)
|
+
1
μ
2
f
2
(
z
)
]
+
[
sup
θ
∈
Θ
|
1
n
∑
t
=
1
n
w
t
(
z
;
β
)
(
β
τ
x
t
-
z
h
)
2
|
]
×
{
h
2
2
[
1
+
o
(
1
)
]
sup
θ
∈
Θ
|
σ
′′
(
z
;
β
)
|
}
×
[
sup
θ
∈
Θ
|
1
n
-
1
∑
t
=
1
n
d
t
(
z
;
β
)
-
1
μ
2
f
2
(
z
)
|
+
1
μ
2
f
2
(
z
)
]
=
O
p
{
h
2
+
(
log
n
n
h
)
1
/
2
}
.
Thus, it can be shown that
(A.52)
sup
θ
∈
Θ
|
σ
^
′
(
z
;
β
)
-
σ
′
(
z
;
β
)
|
=
O
p
{
h
+
(
log
n
n
h
3
)
1
/
2
}
.
By mimicking the above proof, we can show that the other result of this lemma holds.
Lemma A.8.
Under the conditions of Theorem 3, it holds that
(A.53)
sup
θ
∈
Θ
|
∂
σ
^
(
β
τ
x
;
β
)
∂
β
(
1
)
-
σ
′
(
β
τ
x
;
β
)
J
τ
[
x
-
g
(
β
τ
x
)
]
|
=
O
p
{
h
2
+
(
log
n
n
h
)
1
/
2
}
,
sup
θ
∈
Θ
|
∂
σ
^
k
(
β
τ
x
;
β
)
∂
β
(
1
)
-
σ
k
′
(
β
τ
x
;
β
)
J
τ
[
x
-
g
(
β
τ
x
)
]
|
=
O
p
{
h
2
+
(
log
n
n
h
)
1
/
2
}
,
k
=
1,2
,
…
,
p
,
where
g
(
β
τ
x
)
=
E
(
x
t
∣
β
τ
x
t
=
β
τ
x
)
.
Proof.
Here we only prove the first result of this lemma. In the same token, we can obtain the other result.
For any given point
z
, we use local approximation as
σ
(
z
;
β
)
+
σ
′
(
z
;
β
)
(
β
τ
x
t
-
z
)
to approximate
σ
(
β
τ
x
t
;
β
)
. The estimators
σ
^
(
z
;
β
)
and
σ
^
′
(
z
;
β
)
are obtained by solving the kernel estimating functions with respect to
σ
(
z
;
β
)
,
σ
′
(
z
;
β
)
:
(A.54)
∑
t
=
1
n
[
y
t
-
σ
^
(
z
;
β
)
-
σ
^
′
(
z
;
β
)
(
β
τ
x
t
-
z
)
]
k
h
(
β
τ
x
t
-
z
)
=
0
,
∑
t
=
1
n
[
y
t
-
σ
^
(
z
;
β
)
-
σ
^
′
(
z
;
β
)
(
β
τ
x
t
-
z
)
]
×
(
β
τ
x
t
-
z
)
k
h
(
β
τ
x
t
-
z
)
=
0
.
The first equation of (A.54) is
(A.55)
∑
t
=
1
n
[
y
t
-
σ
^
(
β
τ
x
;
β
)
-
σ
^
′
(
β
τ
x
;
β
)
(
β
τ
x
t
-
β
τ
x
)
]
22
×
k
h
(
β
τ
x
t
-
β
τ
x
)
=
0
.
Taking derivatives with respect to
β
(
1
)
on both sides, direct observations lead to
(A.56)
∂
σ
^
(
β
τ
x
;
β
)
∂
β
(
1
)
=
[
A
(
β
τ
x
)
]
-
1
[
B
1
(
β
τ
x
)
+
B
2
(
β
τ
x
)
+
B
3
(
β
τ
x
)
]
,
where
(A.57)
A
(
β
τ
x
)
=
1
n
∑
t
=
1
n
k
h
(
β
τ
x
t
-
β
τ
x
)
,
B
1
(
β
τ
x
)
=
-
1
n
∑
t
=
1
n
k
h
(
β
τ
x
t
-
β
τ
x
)
J
τ
(
x
t
-
x
)
σ
^
′
(
β
τ
x
;
β
)
,
B
2
(
β
τ
x
)
=
-
1
n
∑
t
=
1
n
k
h
(
β
τ
x
t
-
β
τ
x
)
×
(
β
τ
x
t
-
β
τ
x
)
∂
σ
^
′
(
β
τ
x
;
β
)
∂
β
(
1
)
,
B
3
(
β
τ
x
)
=
1
n
h
∑
t
=
1
n
k
h
′
(
β
τ
x
t
-
β
τ
x
)
J
τ
(
x
t
-
x
)
222222
×
[
y
t
-
σ
^
(
β
τ
x
;
β
)
-
σ
^
′
(
β
τ
x
;
β
)
(
β
τ
x
t
-
β
τ
x
)
]
,
and
k
h
′
(
·
)
=
(
1
/
h
)
k
h
′
(
·
/
h
)
. Then, we have
(A.58)
∂
σ
^
(
β
τ
x
;
β
)
∂
β
(
1
)
=
[
A
(
β
τ
x
)
]
-
1
B
1
(
β
τ
x
)
+
[
A
(
β
τ
x
)
]
-
1
B
2
(
β
τ
x
)
+
[
A
(
β
τ
x
)
]
-
1
B
3
(
β
τ
x
)
.
The proof of Lemma A.8 is summarized in the following three steps.
Step 1. It is the analysis of term
[
A
(
β
τ
x
)
]
-
1
B
1
(
β
τ
x
)
.
First we analyze
[
A
(
β
τ
x
)
]
-
1
B
1
(
β
τ
x
)
, which can be decomposed as follows:
(A.59)
[
A
(
β
τ
x
)
]
-
1
B
1
(
β
τ
x
)
=
σ
^
′
(
β
τ
x
;
β
)
J
τ
[
x
-
g
(
β
τ
x
)
]
-
σ
^
′
(
β
τ
x
;
β
)
J
τ
·
{
[
∑
t
=
1
n
k
h
(
β
τ
x
t
-
β
τ
x
)
]
-
1
×
∑
t
=
1
n
k
h
(
β
τ
x
t
-
β
τ
x
)
x
t
-
g
(
β
τ
x
)
[
∑
t
=
1
n
k
h
(
β
τ
x
t
-
β
τ
x
)
]
-
1
}
.
Note that
∑
t
=
1
n
k
h
(
β
τ
x
t
-
β
τ
x
)
]
-
1
∑
t
=
1
n
k
h
(
β
τ
x
t
-
β
τ
x
)
x
t
is the Nadaraya-Watson estimate for
g
(
β
τ
x
)
, so it can be shown that
(A.60)
sup
θ
∈
Θ
|
[
∑
t
=
1
n
k
h
(
β
τ
x
t
-
β
τ
x
)
]
-
1
×
∑
t
=
1
n
k
h
(
β
τ
x
t
-
β
τ
x
)
x
t
-
g
(
β
τ
x
)
[
∑
t
=
1
n
k
h
(
β
τ
x
t
-
β
τ
x
)
]
-
1
|
=
O
p
{
h
+
(
log
n
n
h
)
1
/
2
}
.
Combining (A.59) and (A.60) with Lemma A.7, we have
(A.61)
sup
θ
∈
Θ
|
[
A
(
β
τ
x
)
]
-
1
B
1
(
β
τ
x
)
-
σ
′
(
β
τ
x
;
β
)
J
τ
[
x
-
g
(
β
τ
x
)
]
|
=
O
p
{
h
+
(
log
n
n
h
3
)
1
/
2
}
.
Step 2. The analysis of term
[
A
(
β
τ
x
)
]
-
1
B
2
(
β
τ
x
)
.
By condition (A1) and Lemma A.1, it can be shown that
(A.62)
sup
θ
∈
Θ
|
[
A
(
β
τ
x
)
]
-
1
-
f
-
1
(
β
τ
x
)
|
=
O
p
{
h
+
(
log
n
n
h
)
1
/
2
}
.
According to the symmetric assumption for the kernel function, we have
(A.63)
sup
θ
∈
Θ
|
B
2
(
β
τ
x
)
|
≤
h
sup
θ
∈
Θ
|
∂
σ
^
′
(
β
τ
x
;
β
)
∂
β
(
1
)
|
[
sup
θ
∈
Θ
|
S
n
,
1
(
β
τ
x
;
β
)
-
μ
1
f
(
β
τ
x
)
|
]
=
O
p
{
h
2
+
(
h
log
n
n
)
1
/
2
}
.
It is not difficult to get that
(A.64)
sup
θ
∈
Θ
|
[
A
(
β
τ
x
)
]
-
1
B
2
(
β
τ
x
)
|
≤
[
sup
θ
∈
Θ
|
[
A
(
β
τ
x
)
]
-
1
-
f
-
1
(
β
τ
x
)
|
+
f
-
1
(
β
τ
x
)
sup
θ
∈
Θ
|
[
A
(
β
τ
x
)
]
-
1
-
f
-
1
(
β
τ
x
)
|
]
[
sup
θ
∈
Θ
|
B
2
(
β
τ
x
)
|
]
,
which implies that
(A.65)
sup
θ
∈
Θ
|
[
A
(
β
τ
x
)
]
-
1
B
2
(
β
τ
x
)
|
=
O
p
{
h
2
+
(
h
log
n
n
)
1
/
2
}
.
Step 3. It is the analysis of term
[
A
(
β
τ
x
)
]
-
1
B
3
(
β
τ
x
)
.
We will deal with
B
3
(
β
τ
x
)
, which can be rewritten as follows:
(A.66)
B
3
(
β
τ
x
)
=
∑
i
=
1
4
I
i
(
β
τ
x
)
,
where
(A.67)
I
1
(
β
τ
x
)
=
1
n
h
J
τ
∑
t
=
1
n
k
h
′
(
β
τ
x
t
-
β
τ
x
)
(
x
t
-
x
)
22222222
×
[
y
t
-
σ
(
β
τ
x
t
;
β
)
]
,
I
2
(
β
τ
x
)
=
1
n
h
J
τ
∑
t
=
1
n
k
h
′
(
β
τ
x
t
-
β
τ
x
)
(
x
t
-
x
)
22222222
×
(
β
τ
x
t
-
β
τ
x
)
2
[
1
2
σ
′′
(
β
τ
x
)
+
o
(
1
)
]
,
I
3
(
β
τ
x
)
=
1
n
h
J
τ
∑
t
=
1
n
k
h
′
(
β
τ
x
t
-
β
τ
x
)
(
x
t
-
x
)
×
[
σ
(
β
τ
x
;
β
)
-
σ
^
(
β
τ
x
;
β
)
]
,
I
4
(
β
τ
x
)
=
1
n
h
J
τ
∑
t
=
1
n
k
h
′
(
β
τ
x
t
-
β
τ
x
)
(
x
t
-
x
)
22222222
×
(
β
τ
x
t
-
β
τ
x
)
22222222
×
[
σ
′
(
β
τ
x
;
β
)
-
σ
^
′
(
β
τ
x
;
β
)
]
.
In the following part, we will prove that every term
I
i
(
β
τ
x
)
is of order
o
p
(
1
)
uniformly.
By the law of large numbers, we have
(A.68)
1
n
∑
t
=
1
n
[
y
t
-
σ
(
β
τ
x
t
;
β
)
]
⟶
p
0
.
By condition (A3), we can get that
(A.69)
|
y
t
-
σ
(
β
1
τ
x
t
;
β
1
)
-
[
y
t
-
σ
(
β
2
τ
x
t
;
β
2
)
]
|
=
|
σ
(
β
2
τ
x
t
;
β
2
)
-
σ
(
β
1
τ
x
t
;
β
1
)
|
≤
M
∥
x
t
∥
∥
β
1
-
β
2
∥
.
According to Lemma 1 and Theorem 1 of Andrews [11], we can show that
(A.70)
sup
θ
∈
Θ
|
1
n
∑
t
=
1
n
[
y
t
-
σ
(
β
τ
x
t
;
β
)
]
|
=
o
p
(
1
)
.
Denote
(A.71)
F
n
l
=
1
n
h
∥
J
∥
∑
t
=
1
n
sup
θ
∈
Θ
|
k
h
′
(
β
τ
x
t
-
β
τ
x
)
|
∥
x
t
-
x
∥
l
+
1
,
2222222222222222222222222
l
=
0,1
,
2
,
…
.
Furthermore, we can get that
(A.72)
E
(
F
n
l
)
=
1
h
∥
J
∥
∫
sup
θ
∈
Θ
|
k
h
′
(
β
τ
x
t
-
β
τ
x
)
|
2222222222
×
∥
x
t
-
x
h
∥
l
+
1
h
l
+
1
p
(
x
t
)
d
x
t
=
h
l
∥
J
∥
p
(
x
)
∫
sup
θ
∈
Θ
|
k
h
′
(
β
τ
v
)
|
∥
v
∥
l
+
1
d
v
+
h
l
+
1
∥
J
∥
∫
sup
θ
∈
Θ
|
k
h
′
(
β
τ
v
)
|
∥
v
∥
l
+
1
v
p
′
(
t
v
h
+
x
)
d
v
=
O
(
h
l
)
,
which implies
F
n
l
=
O
p
(
h
l
)
.
For
I
1
(
β
τ
x
)
, the Abel inequality implies
(A.73)
sup
θ
∈
Θ
|
I
1
(
β
τ
x
)
|
≤
M
h
sup
t
|
J
τ
k
h
′
(
β
τ
x
t
-
β
τ
x
)
(
x
t
-
x
)
|
×
sup
θ
∈
Θ
|
1
n
∑
t
=
1
n
[
y
t
-
σ
(
β
τ
x
t
;
β
)
]
|
=
o
p
(
1
)
.
For
I
2
(
β
τ
x
)
, we have
(A.74)
sup
θ
∈
Θ
|
I
2
(
β
τ
x
)
|
≤
F
n
2
[
1
2
sup
θ
∈
Θ
|
σ
′′
(
β
τ
x
;
β
)
|
+
o
(
1
)
]
=
O
p
(
h
2
)
.
For
I
3
(
β
τ
x
)
, we have
(A.75)
sup
θ
∈
Θ
|
I
3
(
β
τ
x
)
|
≤
F
n
0
(
sup
θ
∈
Θ
|
σ
(
β
τ
x
;
β
)
-
σ
^
(
β
τ
x
;
β
)
|
)
=
O
p
{
h
2
+
(
log
n
n
h
)
1
/
2
}
.
For
I
4
(
β
τ
x
)
, we have
(A.76)
sup
θ
∈
Θ
|
I
4
(
β
τ
x
)
|
≤
F
n
1
(
sup
θ
∈
Θ
|
σ
′
(
β
τ
x
;
β
)
-
σ
^
′
(
β
τ
x
;
β
)
|
)
=
O
p
(
h
)
O
p
{
h
+
(
log
n
n
h
3
)
1
/
2
}
=
O
p
{
h
2
+
(
log
n
n
h
)
1
/
2
}
.
Consequently, we have shown that
(A.77)
sup
θ
∈
Θ
|
B
3
(
β
τ
x
)
|
=
O
p
{
h
2
+
(
log
n
n
h
)
1
/
2
}
.
Together with (A.62), we can obtain that
(A.78)
sup
θ
∈
Θ
|
[
A
(
β
τ
x
)
]
-
1
B
3
(
β
τ
x
)
|
=
O
p
{
h
2
+
(
log
n
n
h
)
1
/
2
}
.
For (A.58), we can get that
(A.79)
sup
θ
∈
Θ
|
∂
σ
^
(
β
τ
x
;
β
)
∂
β
(
1
)
-
σ
′
(
β
τ
x
;
β
)
J
τ
[
x
-
g
(
β
τ
x
)
]
|
≤
sup
θ
∈
Θ
|
[
A
(
β
τ
x
)
]
-
1
B
1
(
β
τ
x
)
-
σ
′
(
β
τ
x
;
β
)
J
τ
[
x
-
g
(
β
τ
x
)
]
|
+
sup
θ
∈
Θ
|
[
A
(
β
τ
x
)
]
-
1
B
2
(
β
τ
x
)
|
+
sup
θ
∈
Θ
|
[
A
(
β
τ
x
)
]
-
1
B
3
(
β
τ
x
)
|
.
Combination of (A.61), (A.65), and (A.78) can prove the first result.
Lemma A.9.
Under the conditions of Theorem 3, it holds that
(A.80)
sup
θ
∈
Θ
|
∂
m
(
β
τ
x
;
θ
)
∂
θ
-
∂
m
^
(
β
τ
x
;
θ
)
∂
θ
-
H
(
β
τ
x
;
θ
)
|
=
O
p
{
h
2
+
(
log
n
n
h
)
1
/
2
}
,
where
(A.81)
H
(
β
τ
x
;
θ
)
=
(
O
1
×
(
p
+
1
)
,
m
′
(
β
τ
x
;
θ
)
g
τ
(
β
τ
x
)
J
)
τ
.
Proof.
Let
σ
0
(
z
;
β
)
=
1
,
σ
^
0
(
z
;
β
)
=
1
. Note that
(A.82)
∂
m
(
z
;
θ
)
∂
θ
-
∂
m
^
(
z
;
θ
)
∂
θ
=
(
∂
m
(
z
;
θ
)
∂
ω
τ
-
∂
m
^
(
z
;
θ
)
∂
ω
τ
,
∂
m
(
z
;
θ
)
∂
β
(
1
)
τ
-
∂
m
^
(
z
;
θ
)
∂
β
(
1
)
τ
)
τ
,
where
(A.83)
∂
m
(
z
;
θ
)
∂
ω
i
-
∂
m
^
(
z
;
θ
)
∂
ω
i
=
σ
(
z
;
β
)
σ
i
(
z
;
β
)
×
{
1
[
∑
k
=
0
p
ω
k
σ
^
k
(
z
;
β
)
]
2
-
1
[
∑
k
=
0
p
ω
k
σ
k
(
z
;
β
)
]
2
}
+
{
1
[
∑
k
=
0
p
ω
k
σ
^
k
(
z
;
β
)
]
2
-
1
[
∑
k
=
0
p
ω
k
σ
k
(
z
;
β
)
]
2
}
×
{
σ
^
(
z
;
β
)
σ
^
i
(
z
;
β
)
-
σ
(
z
;
β
)
σ
i
(
z
;
β
)
}
+
1
[
∑
k
=
0
p
ω
k
σ
k
(
z
;
β
)
]
2
×
{
σ
^
(
z
;
β
)
σ
^
i
(
z
;
β
)
-
σ
(
z
;
β
)
σ
i
(
z
;
β
)
}
,
∂
m
(
z
;
θ
)
∂
β
(
1
)
-
∂
m
^
(
z
;
θ
)
∂
β
(
1
)
=
{
∂
σ
(
z
;
β
)
∂
β
(
1
)
1
∑
k
=
0
p
ω
k
σ
k
(
z
;
β
)
-
∂
σ
^
(
z
;
β
)
∂
β
(
1
)
1
∑
k
=
0
p
ω
k
σ
^
k
(
z
;
β
)
}
+
{
σ
^
(
z
;
β
)
[
∑
k
=
0
p
ω
k
(
∂
σ
^
k
(
z
;
β
)
/
∂
β
(
1
)
)
]
[
∑
k
=
0
p
ω
k
σ
^
k
(
z
;
β
)
]
2
-
σ
(
z
;
β
)
[
∑
k
=
0
p
ω
k
(
∂
σ
k
(
z
;
β
)
/
∂
β
(
1
)
)
]
[
∑
k
=
0
p
ω
k
σ
k
(
z
;
β
)
]
2
}
.
Combining Theorem 3 and Lemma A.8, we can obtain that
(A.84)
sup
θ
∈
Θ
|
∂
m
(
β
τ
x
;
θ
)
∂
ω
-
∂
m
^
(
β
τ
x
;
θ
)
∂
ω
|
=
O
p
{
h
2
+
(
log
n
n
h
)
1
/
2
}
,
sup
θ
∈
Θ
|
∂
m
(
β
τ
x
;
θ
)
∂
β
(
1
)
-
∂
m
^
(
β
τ
x
;
θ
)
∂
β
(
1
)
-
m
′
(
β
τ
x
;
θ
)
J
τ
g
(
β
τ
x
)
|
=
O
p
{
h
2
+
(
log
n
n
h
)
1
/
2
}
.
Consequently, the desired result directly follows from (A.84).
Lemma A.10.
Under the conditions of Theorem 3, if
n
h
2
→
∞
and
n
h
4
→
0
, it holds that
(A.85)
n
∂
L
^
n
(
θ
)
∂
θ
|
θ
=
θ
0
=
1
n
∑
t
=
1
n
λ
t
+
o
p
(
1
)
,
where
(A.86)
λ
t
=
-
2
π
(
x
t
)
[
∂
ε
t
∂
θ
+
H
(
β
0
τ
x
t
;
θ
0
)
h
t
]
×
f
-
1
(
β
0
τ
x
t
)
[
1
+
o
p
(
1
)
]
∑
k
=
0
p
ω
k
0
σ
k
(
β
0
τ
x
t
;
β
0
)
1
n
∑
i
=
1
n
K
h
(
β
0
τ
x
i
-
β
0
τ
x
t
)
ξ
i
+
2
π
(
x
t
)
[
∂
ε
t
∂
θ
+
H
(
β
0
τ
x
t
;
θ
0
)
h
t
]
×
f
-
1
(
β
0
τ
x
t
)
[
1
+
o
p
(
1
)
]
[
∑
k
=
0
p
ω
k
0
σ
k
(
β
0
τ
x
t
;
β
0
)
]
2
σ
(
β
0
τ
x
t
;
β
0
)
×
1
n
∑
k
=
0
p
ω
k
0
∑
i
=
1
n
K
h
(
β
0
τ
x
i
-
β
0
τ
x
t
)
ξ
i
,
k
+
π
(
x
t
)
{
(
1
-
ε
t
2
h
t
)
2
ε
t
[
1
h
t
∂
ε
t
∂
θ
+
H
(
β
0
τ
x
t
;
θ
0
)
]
+
1
h
t
∂
h
t
∂
θ
(
1
-
ε
t
2
h
t
)
}
.
Proof.
Note that
(A.87)
∂
L
^
n
(
θ
)
∂
θ
=
1
n
∑
t
=
1
n
{
1
h
t
(
θ
)
∂
h
t
(
θ
)
∂
θ
[
1
-
ε
^
t
2
(
θ
)
h
t
(
θ
)
]
+
2
ε
^
t
(
θ
)
h
t
(
θ
)
∂
ε
^
t
(
θ
)
∂
θ
[
1
-
ε
^
t
2
(
θ
)
h
t
(
θ
)
]
}
π
(
x
t
)
,
where
(A.88)
ε
^
t
(
θ
)
=
ε
t
(
θ
)
+
h
t
(
θ
)
[
m
(
β
τ
x
t
;
θ
)
-
m
^
(
β
τ
x
t
;
θ
)
]
,
∂
ε
^
t
(
θ
)
∂
θ
=
∂
ε
t
(
θ
)
∂
θ
+
h
t
(
θ
)
[
∂
m
(
β
τ
x
t
;
θ
)
∂
θ
-
∂
m
^
(
β
τ
x
t
;
θ
)
∂
θ
]
+
∂
h
t
(
θ
)
∂
θ
[
m
(
β
τ
x
t
;
θ
)
-
m
^
(
β
τ
x
t
;
θ
)
]
.
It can be easily got that
(A.89)
∂
L
^
n
(
θ
)
∂
θ
-
∂
L
n
(
θ
)
∂
θ
=
1
n
∑
t
=
1
n
π
(
x
t
)
{
2
h
t
(
θ
)
[
ε
^
t
(
θ
)
∂
ε
^
t
(
θ
)
∂
θ
-
ε
t
(
θ
)
∂
ε
t
(
θ
)
∂
θ
]
+
1
h
t
2
(
θ
)
∂
h
t
(
θ
)
∂
θ
[
ε
t
2
(
θ
)
-
ε
^
t
2
(
θ
)
]
}
=
2
n
∑
t
=
1
n
π
(
x
t
)
{
1
h
t
(
θ
)
ε
^
t
(
θ
)
[
∂
ε
^
t
(
θ
)
∂
θ
-
∂
ε
t
(
θ
)
∂
θ
]
-
1
h
t
(
θ
)
∂
ε
t
(
θ
)
∂
θ
[
ε
^
t
(
θ
)
-
ε
t
(
θ
)
]
[
∂
ε
^
t
(
θ
)
∂
θ
-
∂
ε
t
(
θ
)
∂
θ
]
}
+
1
n
∑
t
=
1
n
π
(
x
t
)
1
h
t
2
(
θ
)
∂
h
t
(
θ
)
∂
θ
×
(
h
t
2
(
θ
)
[
m
^
(
β
τ
x
t
;
θ
)
-
m
(
β
τ
x
t
;
θ
)
]
2
2
ε
t
(
θ
)
h
t
(
θ
)
[
m
^
(
β
τ
x
t
;
θ
)
-
m
(
β
τ
x
t
;
θ
)
]
-
h
t
2
(
θ
)
[
m
^
(
β
τ
x
t
;
θ
)
-
m
(
β
τ
x
t
;
θ
)
]
2
)
=
1
n
∑
t
=
1
n
π
(
x
t
)
{
∂
h
t
(
θ
)
∂
θ
[
m
(
β
τ
x
t
;
θ
)
-
m
^
(
β
τ
x
t
;
θ
)
]
2
+
2
∂
ε
t
(
θ
)
∂
θ
[
m
(
β
τ
x
t
;
θ
)
-
m
^
(
β
τ
x
t
;
θ
)
]
}
+
2
n
∑
t
=
1
n
π
(
x
t
)
×
[
ε
t
(
θ
)
+
h
t
(
θ
)
(
m
(
β
τ
x
t
;
θ
)
-
m
^
(
β
τ
x
t
;
θ
)
)
]
×
[
∂
m
(
β
τ
x
t
;
θ
)
∂
θ
-
∂
m
^
(
β
τ
x
t
;
θ
)
∂
θ
]
=
2
n
∑
t
=
1
n
π
(
x
t
)
{
[
m
(
β
τ
x
t
;
θ
)
-
m
^
(
β
τ
x
t
;
θ
)
]
H
(
β
τ
x
t
;
θ
)
ε
t
(
θ
)
+
[
∂
ε
t
(
θ
)
∂
θ
+
H
(
β
τ
x
t
;
θ
)
h
t
(
θ
)
]
×
[
m
(
β
τ
x
t
;
θ
)
-
m
^
(
β
τ
x
t
;
θ
)
]
}
+
1
n
∑
t
=
1
n
π
(
x
t
)
∂
h
t
(
θ
)
∂
θ
[
m
(
β
τ
x
t
;
θ
)
-
m
^
(
β
τ
x
t
;
θ
)
]
2
+
2
n
∑
t
=
1
n
π
(
x
t
)
ε
t
(
θ
)
[
∂
m
(
β
τ
x
t
;
θ
)
∂
θ
-
∂
m
^
(
β
τ
x
t
;
θ
)
∂
θ
-
H
(
β
τ
x
t
;
θ
)
]
+
2
n
∑
t
=
1
n
π
(
x
t
)
h
t
(
θ
)
[
m
(
β
τ
x
t
;
θ
)
-
m
^
(
β
τ
x
t
;
θ
)
]
×
[
∂
m
(
β
τ
x
t
;
θ
)
∂
θ
-
∂
m
^
(
β
τ
x
t
;
θ
)
∂
θ
-
H
(
β
τ
x
t
;
θ
)
]
.
Invoking Theorem 4, Lemma A.9, and the law of large numbers for
(
1
/
n
)
∑
t
=
1
n
π
(
x
t
)
h
t
(
θ
)
, together with the facts that
n
-
1
/
2
∑
t
=
1
n
π
(
x
t
)
ε
t
=
O
p
(
1
)
and
∥
(
1
/
h
t
(
θ
)
)
(
∂
h
t
(
θ
)
/
∂
θ
τ
)
∥
<
M
, we can prove that the latter three terms of the right-hand side of the last equation are of order
o
p
(
n
1
/
2
)
if
n
h
2
→
∞
and
n
h
8
→
0
hold.
Thus, we can obtain that
(A.90)
n
∂
L
^
n
(
θ
)
∂
θ
|
θ
=
θ
0
=
2
n
∑
t
=
1
n
π
(
x
t
)
[
∂
ε
t
∂
θ
+
H
(
β
0
τ
x
t
;
θ
0
)
h
t
]
×
[
m
(
β
0
τ
x
t
;
θ
0
)
-
m
^
(
β
0
τ
x
t
;
θ
0
)
]
+
1
n
∑
t
=
1
n
π
(
x
t
)
{
(
1
-
ε
t
2
h
t
)
2
ε
t
[
1
h
t
∂
ε
t
∂
θ
+
H
(
β
0
τ
x
t
;
θ
0
)
]
+
1
h
t
∂
h
t
∂
θ
(
1
-
ε
t
2
h
t
)
}
+
o
p
(
1
)
.
To analyze
∂
L
^
n
(
θ
)
/
∂
θ
, we will show the asymptotic representation for
m
(
z
;
θ
)
-
m
^
(
z
;
θ
)
. By the fact that
(
1
/
n
)
∑
t
=
1
n
d
t
(
z
;
β
)
(
β
τ
x
t
-
z
)
=
0
, applying the similar argument used in Lemma A.7, we can obtain that
(A.91)
σ
^
(
z
;
β
)
-
σ
(
z
;
β
)
=
f
-
1
(
z
)
1
n
∑
i
=
1
n
K
h
(
β
τ
x
i
-
z
)
ξ
i
(
β
)
[
1
+
o
p
(
1
)
]
+
O
p
(
h
2
)
,
σ
^
k
(
z
;
β
)
-
σ
k
(
z
;
β
)
=
f
-
1
(
z
)
1
n
∑
i
=
1
n
K
h
(
β
τ
x
i
-
z
)
ξ
i
,
k
(
β
)
[
1
+
o
p
(
1
)
]
+
O
p
(
h
2
)
,
where
ξ
i
(
β
)
=
y
i
-
σ
(
β
τ
x
i
;
β
)
,
ξ
i
,
k
(
β
)
=
y
i
-
k
2
-
σ
k
(
β
τ
x
i
;
β
)
,
ξ
i
=
ξ
i
(
β
0
)
and
ξ
i
,
k
=
ξ
i
,
k
(
β
0
)
.
Recalling the definition of (6) and (7), we have
(A.92)
m
^
(
z
;
θ
)
-
m
(
z
;
θ
)
=
σ
^
(
z
;
β
)
-
σ
(
z
;
β
)
∑
k
=
0
p
ω
k
σ
k
(
z
;
β
)
-
σ
(
z
;
β
)
∑
k
=
0
p
ω
k
[
σ
^
k
(
z
;
β
)
-
σ
k
(
z
;
β
)
]
[
∑
k
=
0
p
ω
k
σ
k
(
z
;
β
)
]
2
+
o
p
(
n
-
1
/
2
)
=
f
-
1
(
z
)
[
1
+
o
p
(
1
)
]
∑
k
=
0
p
ω
k
σ
k
(
z
;
β
)
1
n
∑
i
=
1
n
K
h
(
β
τ
x
i
-
z
)
ξ
i
(
β
)
-
f
-
1
(
z
)
[
1
+
o
p
(
1
)
]
[
∑
k
=
0
p
ω
k
σ
k
(
z
;
β
)
]
2
×
σ
(
z
;
β
)
n
∑
k
=
0
p
ω
k
∑
i
=
1
n
K
h
(
β
τ
x
i
-
z
)
ξ
i
,
k
(
β
)
+
O
p
(
h
2
)
+
o
p
(
n
-
1
/
2
)
.
Finally, we have
(A.93)
n
∂
L
^
n
(
θ
)
∂
θ
|
θ
=
θ
0
=
-
2
n
∑
t
=
1
n
π
(
x
t
)
[
∂
ε
t
∂
θ
+
H
(
β
0
τ
x
t
;
θ
0
)
h
t
]
×
f
-
1
(
β
0
τ
x
t
)
[
1
+
o
p
(
1
)
]
∑
k
=
0
p
ω
k
0
σ
k
(
β
0
τ
x
t
;
β
0
)
×
1
n
∑
i
=
1
n
K
h
(
β
0
τ
x
i
-
β
0
τ
x
t
)
ξ
i
+
2
n
∑
t
=
1
n
π
(
x
t
)
[
∂
ε
t
∂
θ
+
H
(
β
0
τ
x
t
;
θ
0
)
h
t
]
×
f
-
1
(
β
0
τ
x
t
)
[
1
+
o
p
(
1
)
]
[
∑
k
=
0
p
ω
k
0
σ
k
(
β
0
τ
x
t
;
β
0
)
]
2
σ
(
β
0
τ
x
t
;
β
0
)
×
1
n
∑
k
=
0
p
ω
k
0
∑
i
=
1
n
K
h
(
β
0
τ
x
i
-
β
0
τ
x
t
)
ξ
i
,
k
+
2
n
∑
t
=
1
n
π
(
x
t
)
[
∂
ε
t
∂
θ
+
H
(
β
0
τ
x
t
;
θ
0
)
h
t
]
×
[
O
p
(
h
2
)
+
o
p
(
n
-
1
/
2
)
]
+
1
n
∑
t
=
1
n
π
(
x
t
)
{
(
1
-
ε
t
2
h
t
)
2
ε
t
[
1
h
t
∂
ε
t
∂
θ
+
H
(
β
0
τ
x
t
;
θ
0
)
]
+
1
h
t
∂
h
t
∂
θ
(
1
-
ε
t
2
h
t
)
}
+
o
p
(
1
)
=
1
n
∑
t
=
1
n
λ
t
+
o
p
(
1
)
,
where
λ
t
is given in (A.86). The third term above is of order
o
p
(
1
)
by applying the law of large numbers to
(
1
/
n
)
∑
t
=
1
n
π
(
x
t
)
[
∂
ε
t
/
∂
θ
+
H
(
β
0
τ
x
t
;
θ
0
)
h
t
]
and
n
h
4
→
0
.
Lemma A.11.
Under the conditions of Theorem 3, it holds that
(A.94)
-
∂
2
L
^
n
(
θ
)
∂
θ
∂
θ
τ
|
θ
=
θ
~
=
Σ
I
+
o
p
(
1
)
,
where
Σ
I
is given in (A.101).
Proof.
Note that
(A.95)
∂
2
L
^
n
(
θ
)
∂
θ
∂
θ
τ
=
1
n
∑
t
=
1
n
π
(
x
t
)
×
{
[
2
ε
^
t
2
(
θ
)
h
t
(
θ
)
-
1
]
1
h
t
2
(
θ
)
∂
h
t
(
θ
)
∂
θ
∂
h
t
(
θ
)
∂
θ
τ
-
2
ε
^
t
(
θ
)
h
t
2
(
θ
)
∂
h
t
(
θ
)
∂
θ
∂
ε
^
t
(
θ
)
∂
θ
τ
+
2
h
t
(
θ
)
∂
ε
^
t
(
θ
)
∂
θ
∂
ε
^
t
(
θ
)
∂
θ
τ
+
2
ε
^
t
(
θ
)
h
t
(
θ
)
∂
2
ε
^
t
(
θ
)
∂
θ
∂
θ
τ
-
2
ε
^
t
(
θ
)
h
t
2
(
θ
)
∂
ε
^
t
(
θ
)
∂
θ
∂
h
t
(
θ
)
∂
θ
τ
[
2
ε
^
t
2
(
θ
)
h
t
(
θ
)
-
1
]
}
=
1
n
∑
t
=
1
n
π
(
x
t
)
×
{
[
2
ε
t
2
(
θ
)
h
t
(
θ
)
-
1
]
1
h
t
2
(
θ
)
∂
h
t
(
θ
)
∂
θ
∂
h
t
(
θ
)
∂
θ
τ
-
2
ε
t
(
θ
)
h
t
2
(
θ
)
∂
h
t
(
θ
)
∂
θ
×
[
∂
ε
t
(
θ
)
∂
θ
τ
+
h
t
(
θ
)
H
τ
(
β
τ
x
t
;
θ
)
]
+
2
h
t
(
θ
)
[
∂
ε
t
(
θ
)
∂
θ
+
h
t
(
θ
)
H
(
β
τ
x
t
;
θ
)
]
×
[
∂
ε
t
(
θ
)
∂
θ
τ
+
h
t
(
θ
)
H
τ
(
β
τ
x
t
;
θ
)
]
+
2
ε
t
(
θ
)
h
t
(
θ
)
[
∂
2
ε
t
(
θ
)
∂
θ
∂
θ
τ
+
h
t
(
θ
)
(
∂
2
m
(
β
τ
x
t
;
θ
)
∂
θ
∂
θ
τ
-
∂
2
m
^
(
β
τ
x
t
;
θ
)
∂
θ
∂
θ
τ
)
]
-
2
ε
t
(
θ
)
h
t
2
(
θ
)
×
[
∂
ε
t
(
θ
)
∂
θ
+
h
t
(
θ
)
H
(
β
τ
x
t
;
θ
)
]
∂
h
t
(
θ
)
∂
θ
τ
[
2
ε
t
2
(
θ
)
h
t
(
θ
)
-
1
]
}
+
o
p
(
1
)
=
1
n
∑
t
=
1
n
D
t
(
θ
)
+
o
p
(
1
)
,
where
(A.96)
D
t
(
θ
)
=
1
n
∑
t
=
1
n
π
(
x
t
)
×
{
[
2
ε
t
2
(
θ
)
h
t
(
θ
)
-
1
]
1
h
t
2
(
θ
)
∂
h
t
(
θ
)
∂
θ
∂
h
t
(
θ
)
∂
θ
τ
+
2
h
t
(
θ
)
[
∂
ε
t
(
θ
)
∂
θ
+
h
t
(
θ
)
H
(
β
τ
x
t
;
θ
)
]
×
[
∂
ε
t
(
θ
)
∂
θ
τ
+
h
t
(
θ
)
H
τ
(
β
τ
x
t
;
θ
)
]
[
2
ε
t
2
(
θ
)
h
t
(
θ
)
-
1
]
}
.
Thus, we have
(A.97)
∂
2
L
^
n
(
θ
)
∂
θ
∂
θ
τ
|
θ
=
θ
~
=
1
n
∑
t
=
1
n
D
t
(
θ
~
)
+
o
p
(
1
)
.
Based on the fact that
θ
~
converges in probability to
θ
0
, we can obtain that
(A.98)
1
n
∑
t
=
1
n
D
t
(
θ
~
)
=
1
n
∑
t
=
1
n
D
t
(
θ
0
)
+
o
p
(
1
)
.
Then, by the law of large numbers, we have
(A.99)
1
n
∑
t
=
1
n
D
t
(
θ
0
)
=
E
[
D
t
(
θ
0
)
]
+
o
p
(
1
)
.
Therefore, we get
(A.100)
-
∂
2
L
^
n
(
θ
)
∂
θ
∂
θ
τ
|
θ
=
θ
~
=
Σ
I
+
o
p
(
1
)
,
where
(A.101)
Σ
I
=
-
E
{
π
(
x
t
)
(
1
h
t
2
∂
h
t
∂
θ
∂
h
t
∂
θ
τ
+
2
h
t
[
∂
ε
t
∂
θ
+
h
t
H
(
β
0
τ
x
t
;
θ
0
)
]
×
[
∂
ε
t
∂
θ
τ
+
h
t
H
τ
(
β
0
τ
x
t
;
θ
0
)
]
1
h
t
2
∂
h
t
∂
θ
∂
h
t
∂
θ
τ
)
}
.
Proof of Theorem 6.
We have shown that
(A.102)
n
(
θ
^
n
-
θ
0
)
=
{
-
∂
2
L
^
n
(
θ
)
∂
θ
∂
θ
τ
|
θ
=
θ
~
}
-
1
{
n
∂
L
^
n
(
θ
)
∂
θ
|
θ
=
θ
0
}
.
According to Lemmas A.10 and A.11, we can obtain the following asymptotic expansion:
(A.103)
n
(
θ
^
n
-
θ
0
)
=
Σ
I
-
1
(
1
n
∑
t
=
1
n
λ
t
)
+
o
p
(
1
)
.
To analyse the asymptotic normality of
n
(
θ
^
n
-
θ
0
)
, we only need to show the asymptotic normality of
(
1
/
n
)
∑
t
=
1
n
λ
t
.
Recalling the definition of
ξ
i
and
ξ
i
,
k
, hence we have
E
(
ξ
i
)
=
0
and
E
(
ξ
i
,
k
)
=
0
, together with the assumptions of
{
e
t
}
, which imply
E
(
λ
t
)
=
0
.
We have to verify that for some
r
>
2
,
E
|
λ
t
|
r
<
∞
, which can be obtained with condition (A.20). Then, by the central limit theorem for strongly mixing sequences (see, e.g., Bosq [13], Theorem 1.7), we can show that
(A.104)
1
n
∑
t
=
1
n
λ
t
⟶
d
N
(
0
,
Σ
)
,
where
(A.105)
Σ
=
∑
t
=
1
n
Cov
(
λ
1
,
λ
t
)
.
Finally, by the Slutsky theorem, we have
n
(
θ
^
n
-
θ
0
)
→
d
N
(
0
,
Σ
I
-
1
Σ
Σ
I
-
1
)
.