SeamlessSwitching ofScalable VideoBitstreams forEfficientStreamingSeamlessSwitching ofScalable VideoBitstreams forEfficientStreaming
Xiaoyan Sun, Feng Wu,Shipeng Li, Wen, Gao,and Ya-Qin ZhangXiaoyan Sun, Feng Wu,Shipeng Li, Wen, Gao,and Ya-Qin Zhang
OutlineOutline
•Introduction
•Seamless Switching Scheme AmongScalable Bitstreams
•Switching Down Between Two Bitstreams
•Switching Up Between Two Bitstreams
•Experimental Results
•Discussions
IntroductionIntroduction
•Why are scalable bitstreams needed?Switching between non-scalablebitstreams
–Drifting error
–Key frames
–Large storage requirements
•Multiple non-scalable bitstreams
•Key frames
Non-scalable bitstream1
Non-scalable bitstream2
drifting error
Non-scalable bitstream1
Non-scalable bitstream2
key frame
IntroductionIntroduction
•SP-frames scheme (proposed in H.264)
–Extra bitstreams are used.
–The size of S12 is similar to that of I-frame.
–When switching down, S12 may deterioratethe networks.
P
S2
P
P
S1
P
S12
bitstream1
bitstream2
SP frame
extra bitstream
IntroductionIntroduction
•Why are multiple scalable bitstreamsneeded?Single scalable bitstream
–FGS
–Low coding efficiency in FGS bitstreams.
•Motion prediction is based on the lowestquality base layer.
–Using multiple scalable bitstreams,coding efficiency is higher.
Base layer
Enhancement layer 1
Enhancement layer 2
Enhancement layer 3
…
Base layer
Enhancement layer 1
Enhancement layer 2
Enhancement layer 3
…
Frame n
Frame n+1
Seamless Switching SchemeAmong Scalable BitstreamsSeamless Switching SchemeAmong Scalable Bitstreams
•Seamless switching (Definition)
–The quality in each scalable bitstream issmooth.
–The switching among scalablebitstreams is drifting-free.
–Immediately switching from the currentscalable bitstream to one operated atlower rates without any delay.
Seamless Switching SchemeAmong Scalable BitstreamsSeamless Switching SchemeAmong Scalable Bitstreams
Scalable
Bitstream 1
Scalable
Bitstream 2
Scalable
Bitstream 1
Scalable
Bitstream 3
Bandwidth
Time
Scalable
Bitstream 1
Scalable
Bitstream 2
Scalable
Bitstream 3
Seamless Switching SchemeAmong Scalable BitstreamsSeamless Switching SchemeAmong Scalable Bitstreams
switching
SF switching frame
(switching-up point)
High-bit-rate scalable bitstream
Low-bit-rate scalable bitstream
Switching Down BetweenTwo BitstreamsSwitching Down BetweenTwo Bitstreams
•MVs are estimated in SB-H and areapplied to both SB-H and SB-L.
•The quantization information of SB-Lis coded in SB-H bitstream.
–Overhead bits: 3*MB_number+5
•Instead of original frames, base layerframes in SB-H are used to encodebase layer frames in SB-L.
First QP
Range of difference
☆Switching down at everywhere and with no overhead
Switching Down BetweenTwo BitstreamsSwitching Down BetweenTwo Bitstreams
•SF Decoder
VLD
Q-1
IDCT
MC
MC
DCT
QL-1
QL
IDCT
ref SB-L
refBase
SB-H
Base layer
SB-L
Base layer
Video
MVs
Switching Down BetweenTwo BitstreamsSwitching Down BetweenTwo Bitstreams
•Quality loss of SB-L base layer
–The reconstructed frames from SB-Hbase layer instead of the original videoare used as the input of the SB-L baselayer.
–The same set of MVs for SB-H baselayer is used to encode the SB-L baselayer.
Switching Up Between TwoBitstreamsSwitching Up Between TwoBitstreams
n-1
n
n+1
Switching
point
SB-L
Base layer
SB-H
Base layer
+
=
Extra bitstream
☆Considerable amount of
overhead bits
Switching Up Between TwoBitstreamsSwitching Up Between TwoBitstreams
•SF Encoder
IDCT
Qs-1
DCT
Qs
BP
VLC
DCT
Qs
SB-H
Base layer
SB-L
Base layer
Extra bitstream
SF frame
n-1
n+1
n
☆n’ is used instead of n
to avoid drifting error
=
S
B
-
L
B
a
s
e
+
E
x
t
r
a
b
i
t
s
t
r
e
a
m
n’
n
Switching Up Between TwoBitstreamsSwitching Up Between TwoBitstreams
•SF Decoder
VLD
Q-1
IDCT
MC
refBase
SB-H
Base layer
SB-L
Base layer
Video
MVs
DCT
Qs-1
Qs
IDCT
BP
VLD
Extra
bitstream
Switching Up Between TwoBitstreamsSwitching Up Between TwoBitstreams
•The SF frame is simpler than the SPframe.
•The SF frame gets better qualitythan the SP frame.
Experimental ResultsExperimental Results
•SF scheme is applied on MBPFGS
–MBPFGS is modified from PFGS
•Frame base -> Macroblock base
•
–
–10fps
•An I-frame is inserted every 1’s in a non-scalable bitstream
•An SF frame is inserted every 1’s in a SFbitstream
32kbps
64kbps
80kbps
112kbps
MBFGS-H bitstream
MBFGS-L bitstream
Experimental ResultsExperimental Results
•Comparisons of extra bitstream
–
– Average PSNR loss caused by SFframes < 0.1 dB
Sequence
SF (Qs=3)
Lossless
News
32959.11
147878.1
Foreman
34918.33
173569.3
Coastguard
39824.78
158157.7
(Bits)
Experimental ResultsExperimental Results
•Rate-distortion (static)
–Non-scalable bitstream (JVT H.26L codec)
•Two bitstream with different quality
•Switching at I-frame
–FGS and MPFGS
•Only one bitstream
Non-scalable
FGS
MPFGS
SF
Inaccurate MVsand input video
Experimental ResultsExperimental Results
•PSNR comparisons (dynamic channel)
–Foreman
–Coastguard
72 kbps
152 kbps
72 kbps
152 kbps
DiscussionsDiscussions
•The SF scheme can be extended to the case withmultiple scalable bitstreams.
–(N-1) extra bitstreams are required.
•N*(N-1) extra bitstreams are required in SP frames.
–High complexity of the decoder in decoding the high-bit-rate scalable bitstream.
•Three times of the MPEG-4 decoder.
•Complexity scalability.
–Large size of the overhead bits inside the high bit-ratebitstream.
•The extra bitstream for switching up is large.
–if the bitstreams switch up and down once for everysecond, the extra bitstream will cost 30-40 kbps.
–An intelligent server should be able to control switchingtimes.