VQ 10 x log10(2.13)
5.2 Context of This Chapter
esti-5.2. CONTEXT OF THIS CHAPTER 83
Degradation by Compression
Degradation by Transmission Error
—>Network --> —
VideoStreaming OriginalEncoderServer
Video
End-User Terminal
Quality Estimation
- -r
\ Degraded
Video
Estimated Quality
Figure 5.2: Framework of the typical NR model.
mation. In other words, if a method based on the NR model can estimate MSE accurately, it can be considered that this method has high enough performance for the NR model. Therefore, in this chapter, accuracy of
MSE estimation is discussed.
In the case that a transmission error occurs, some regions in video frames are degraded since the bitstream cannot be decoded correctly. When im-pairment regions have been detected, most of commercially available video
decoders apply an error-concealment process to these regions. Generally, the pixels at a corresponding location in the previous frame are replaced
in order to conceal video-quality degradation. Video frames to which the error concealment has been applied are output to a display device. As a result, video-quality degradation would be recovered to some degree, and in order to achieve accurate video-quality evaluation, the effectiveness of the error concealment would have to be taken into consideration.
In the proposed method, impairment regions in video frames due to transmission errors are first detected. Then, effectiveness of the error
con-cealment applied to the impairment regions is evaluated and calculated the number of macroblocks in which the error concealment is not effective.
Two kinds of evaluation methods for the error-concealment effectiveness are introduced. One is based on a motion level of the degraded video. The other is based on luminance discontinuity at boundaries of the impairment regions. The detailed explanation will be described in the following section.
Original Video
Degradation by Compression
Video Encoder
Degradation by Transmission Error
FR Quality Estimation
-
Streaming Server
Network Degraded
Video
The Proposed Method Presents This Function
NR Quality Estimation
Overall Quality Estimation
_÷Estimated Quality
Figure 5.3: Framework of the NR model in the proposed method.
5.3. PROPOSED METHOD 85
5.3
Proposed Method
5.3.1 Detecting Impairment Dependency
Macroblocks by Analyzing Coding
First of all, the proposed method detects impairment macroblocks which cannot correctly be decoded. The impairment macroblocks are detected by syntax-error existence in the bitstream coded by the video-coding stan-dards. When a syntax error is detected in a macroblock, successive mac-roblocks from this macroblock to the macroblock just before the next slice or the next picture are considered as the impairment macroblocks. In
addition, even when there are no syntax errors (i.e., no transmission er-rors), macroblocks which need the information of the detected impairment
macroblocks in the decoding process are also considered as the impair-ment macroblocks. In order to detect them, a coding-dependency map describing macroblock positions in which decoding errors have occurred is utilized. This map can be used to calculate the accurate number of impairment macroblocks. In Fig.5.4, for example, the upper frame would represent an I-picture and its Region A would represent an area that could
not be decoded correctly because of a transmission error (i.e., it is a map of impairment macroblocks). The lower frame in Fig.5.4 would represent
the consecutively decoded P-picture without transmission errors. Here, even though there have been no transmission errors, macroblocks in the lower frame which refer to Region A in inter-frame prediction would not
be correctly decoded (i.e., video quality would be degraded in these mac-roblocks despite no transmission errors). All other macmac-roblocks would be
correctly decoded. In this manner, the impairment macroblocks are able to be accurately detected.
5.3.2 Evaluation of Error-Concealment Effectiveness Using
Mo-tion Information
After detecting impairment macroblocks, the error concealment is exe-cuted by replacing the impairment regions with the pixels at an appropriate
I-Picture
with transmission error
P-Picture
without transmission error
Figure 5.4: An example of impairment macroblocks caused by transmission errors.
region having the macroblock size in a previous frame. Let (Ecx, EC) be
a vector from the impairment macroblock to a macroblock-size area fetched from the previous frame for the replacement by error-concealment process.
This vector is generally estimated from the neighboring macroblocks in spatial positions by some statistical observations. When simply repeating the pixels at the same spatial positions in the previous frame is adopted as
the error-concealment method, the vector (Ecx, EC) becomes zero.
Pre-cise implementation of error-concealment approaches are not specified in
video-coding standards. The vector (EC,, ECU) is also not specified in the
video-coding standards. Video decoders themselves calculate this vector and use it for the error concealment. Since the proposed method is imple-mented in video decoders, it assumes that the proposed method can access
5.3. PROPOSED METHOD 87
this vector. The error concealment works effectively, when the vector of
(ECX, ECy) has a correlation with motion which the macroblock originally
had.
The proposed method utilizes the following approach as a quality-degradation
criterion. When the vector (ECX, ECy) is similar to a motion vector in a
correctly decoded latest macroblock in the same spatial position, then it is considered that the error concealment works well. The similarity measure is written in
MV— EC ,1 MVy — ECy < Th,,,(5.1)
where (MV,, MVy) is the motion vector in a correctly decoded latest
mac-roblock in the same spatial position and Th7-fly is the threshold for the similarity. For a macroblock including transmission errors, the vector
(MV,, MVy) cannot be obtained. In this case, the vector (MV,, MVy)
is obtained from a macroblock in the same spatial position in the correctly decoded latest frame in the past. When the vector for the error conceal-ment becomes zero, the following equation is applied:
MVy < Thmv.
(5.2)
The threshold-parameter decision is discussed in the next section.
When successive transmission errors (burst errors) are generated in a
long period of time, successive multiple frames are not decoded correctly
and appropriate (MV,, MVy) cannot be obtained. In this case, however,
it must be considered that the transmission error is fatal and the video quality is evaluated as the worst. This chapter does not consider the burst transmission errors which result in fatal degradation, but takes into account of the situation of randomly generated transmission errors. For the burst errors, the number of successive frames which are not decoded correctly should be adopted as a parameter for detection of the fatal degradation.
If a macroblock in the same spatial position in the correctly decoded
latest frame has been encoded with intra-mode encoding, Eq.(5.1) and (5.2) cannot be used since a motion vector (M17,, MVy) will not be available.
In this case, however, it can also be assumed that intra-mode encoding without inter-frame prediction was adopted when there was either too much motion to use inter-frame prediction or there would be too little inter-frame
correlation. Therefore, it is also assumed that error concealment which employs pixels in the previous frame has been ineffective. As a result, any
impairment macroblock for which Eq. (5.1) or (5.2) is not satisfied or for
which a motion vector at the same position in the previous frame is not available is considered to be an error-concealment-ineffective macroblock.
5.3.3 Evaluation of Error-Concealment Effectiveness Using
Lu-minance Discontinuity at Impairment-Macroblock
Bound-aries
For another evaluation of error-concealment effectiveness, luminance discontinuity is calculated as an average of the absolute difference of the luminance values at the boundary between an impairment macroblock and its neighboring macroblocks. When, as is illustrated in Fig.5.5, Pk repre-sents a luminance value in the impairment macroblock along the boundary,
k represents one in the neighboring macroblock, and the number of pixels along the boundary is K, luminance discontinuity D is expressed as:
1K-1
D = — E Pk Qk
K k=0
(5.3)
The value of D is calculated at edges of each impairment macroblocks when neighboring macroblocks are correctly decoded, or when any neighboring macroblock is also an impairment macroblock and the error-concealed data for these adjacent impairment macroblocks are not fetched from a contin-uous region in the previous frame. Therefore, the value of K changes according to the decoding situation of the surrounding macroblocks. For example, when all four surrounding macroblocks are correctly decoded, the
value of K becomes 60 (16 x 2+14 x 2), and when only one edge of the
im-pairment macroblock is the candidate for the discontinuity calculation, the value of K becomes 16.
If luminance discontinuity D is smaller than a predetermined thresh-old Thi, in an impairment macroblock, error concealment is considered to be effective. A comparison between the luminance discontinuity and
5.3. PROPOSED METHOD 89
Figure 5.5:
Impairment Macroblock
Neighboring Macroblock
Pixels along a boundary of an impairment macroblock.
the threshold is applied to every impairment macroblock.
parameter decision is also discussed in the next section.
The
threshold-5.3.4 MSE Estimation by the Number of
Ineffective Macroblocks
Two kinds of evaluation methods for the error-concealment effective-ness, described in Section 5.3.2 and 5.3.3, are applied to the impairment macroblocks. If one of the evaluations judges as ineffective, the pro-posed method concludes that the error concealment to the impairment macroblock is not effective. Then, the number of the error-concealment-ineffective macroblocks are calculated.
It is assumed that there is a correlation between ]VISE and the size of
regions in which error concealment is estimated to have been ineffective.
Therefore, the proposed method estimates MSE on the basis of the number of the error-concealment-ineffective macroblocks. MSE values with respect to decoded frames having no transmission error versus degraded frames are considered to be estimated. An MSE value is calculated as the following equation:
M S E
1 N-1 H-1 vv_i
--- E E E (Pw,hn13w,h,n)2(5.4)
147)(1/xNn=o h=0 w=0
where W is frame width, H is frame height, N is the number of frames, Pw ,h,n is a luminance value of the decoded frames having no transmission
errors, and PL h,n is a luminance value of the degraded frames. In the
performance evaluation in the next section, it is shown that MSE calculated
by Eq. (5.4) and the number of error-concealment-ineffective macroblocks
have high correlation.