WO2017080603A1

WO2017080603A1 - Frame alignment technique for live stream television

Info

Publication number: WO2017080603A1
Application number: PCT/EP2015/076443
Authority: WO
Inventors: Giuseppe CRISCI
Original assignee: Huawei Technologies Co., Ltd.
Priority date: 2015-11-12
Filing date: 2015-11-12
Publication date: 2017-05-18
Also published as: CN108353073A

Abstract

The invention relates to a method for frame synchronization in a user's equipment in a real time media streaming system. The method comprises the following steps: sending a channel change request to a retransmission server, receiving a channel change response from the retransmission server in response to the channel change request wherein the channel change response comprises a frame offset information that indicates a frame offset between a frame of a reference media stream and a frame of a corresponding media stream received from the retransmission server to the user's equipment, calculating a frame rate acceleration duration based on the frame offset information and accelerating a frame rate of the corresponding media stream for the calculated frame rate acceleration duration to synchronize the frame of the media stream provided from the retransmission server to the frame of the reference media stream.

Description

FRAME ALIGNMENT TECHNIQUE FOR LIVE STREAM TELEVISION

TECHNICAL FIELD The present invention is in the field of media live stream techniques and particular internet protocol, IP, real time media live stream techniques. The present invention especially relates to a method for frames synchronization in a user' s equipment in such a real time media streaming system, a method for frame synchronization in a retransmission server in such a real time media streaming system, a method for frame synchronization in such a real time media streaming system, a user's equipment in such a real time media streaming system, a retransmission server in such a real time media streaming system, a real time media streaming system and a computer program product for implementing a method for frame synchronization when carried out on a computing device.

In particular, the present invention suggests providing a frame synchronization method by aligning group of picture, GOP, frames.

BACKGROUND

Media refers to content that uses a combination of different content forms. This includes media that use rudimentary computer displays such as text-only but - in case of multimedia - also includes a combination of text, audio, picture, animation, video, or interactive content forms. Multimedia can be recorded and played, displayed, dynamic, interacted with or accessed by information content processing devices, such as computerized and electronic devices, and can be part of a live performance.

A media streaming system is a system, which receives media data delivered by a provider and presents them to a user's equipment of an end-user. Thus, a user's equipment can start to present media data before the entire file has been transmitted.

Today, a media can be streamed either live or on demand. Live streams are generally provided by a means called "true streaming". True streaming sends the information straight to the user's equipment without saving the content. On-demand streaming is provided by a means called progressive streaming. Progressive streaming saves the media content at the user' equipment and then it is played from that location. On- demand streams are often saved at the user' s equipment and/or servers for an extended amount of time, so the end-user basically chooses the time to play the media and the specific point of interest - so he could fast-forward or rewind to a desired point of interest anytime. In contrast, the live streams are only available at one time only, e.g., during the football game.

Thus, during live media streaming the content is delivered live - so to say in real-time - over an appropriate physical channel, e.g. over the internet. Live streaming uses encoder-technology to digitize media content for delivery over the physical channel in real-time. An audio content itself might be compressed using audio codec such as MPEG-1 or MPEG-2 audio layer III, MP3 or Vorbis or advanced audio coding, AAC. A video content itself might be compressed using a video codec such as MPEG-4 Part 10 advanced video coding, MPEG-4 AVC, also known as H.264, or high efficiency video coding HEVC, also known as MPEG-H part 2 (H.265), or VP8.

The encoded audio and video streams are than assembled in a container bit stream such as MP4 or flash video, FLV or WebM or advanced streaming format, ASF, or Internet Streaming Media Alliance, ISMA.

The bit stream is delivered from a media content provider to a user' s equipment using a transport protocol, such as Microsoft media server, MMS or real-time transport protocol, RTP. The user's equipment may interact with the streaming content provider using a control protocol, such as MMS or real-time- streaming protocol, RTSP.

The RTSP is a network control protocol designed for use in entertainment and communications systems to control streaming media servers. The protocol is used for establishing and controlling media sessions between end points. The transmission of streaming data itself is not a task of RTSP.

Nowadays, RTSP servers use the RTP in conjunction with a real-time control protocol, RTCP for media stream delivery. The RTP is a network protocol for delivering audio and video over IP networks. RTP is used extensively in communication and entertainment systems that involve streaming media, such as telephony, video teleconference applications, television services and web-based push-to-talk features.

RTP is used in conjunction with the RTCP. While RTP carries the media streams (e.g., audio and video), RTCP is used to monitor transmission statistics and quality of service, QoS, and aids synchronization of multiple streams. It is known from paper "Extensions to RTCP for Rapid Synchronization" from P. Yang, published on March 8, 2009, that RTP provides video delivery services with a real-time characteristics, such as video broadcasts in which receivers can frequently switch among different multicast media streams. In order to achieve a rapid synchronization and reduce the synchronization delay between these multicast streams, a retransmission server is implemented in between the media content provider and the user' s equipment. The retransmission server is configured to send a unicast stream that corresponds to the multicast media stream from the media content provider to the requesting user's equipments that temporally replaces the multicast session. The compression of the media content leads to a provision of media in a frame structure. In video coding, a group of pictures, GOP, structure, specifies the order in which intra- and inter-frames are arranged. The GOP is a group of successive pictures within a coded video stream. Each coded video stream consists of successive GOPs. From the pictures contained in it, the visible frames are generated by the user's equipment.

In order to correctly decode the media content, the very first frame that the user's equipment must obtain from the retransmission server is a so called I-frame. I-frame is the least compressible frame but does not require other video frames to decode. This means that the retransmission server according to the RTCP needs to start a unicast stream towards the user's equipment with an I-frame for the ongoing GOP frame, broadcasted at the time, the user joins the new channel. In order to keep channel change time as short as possible, the retransmission server will not wait for the next I- frame to come; it will always forward the last I-frame from its cache memory. During live media streaming it is currently impossible to align and/or adjust user's equipments to play the same frames at the same time. Depending on the moment, a specific user joins a channel that provides the live media content, a user- specific delay within the GOP is experienced based on the need that always an I-frame is delivered first. This causes random delays among different users which can be as long as a full GOP.

A GOP might have time durations of several seconds. Thus, in worst case scenarios, two users that receive exactly the same media live stream can actually experience a live stream delay up to several seconds between each other. The typical consequence of this user- specific delay is that during a sports event, such as a football live television stream, some end-users will hear the neighbors already cheering for a goal, but the goal has not yet happened on their screen. Another problem caused by the known GOP delivery is that the user's equipments with their user- specific consistent delay need to buffer the incoming packets corresponding to their user- specific consistent delay for as long as the live stream channel is watched, thus impacting the effective buffer capability of the user's equipment.

SUMMARY

In view of the above-mentioned problems and disadvantages, the present invention aims to improve the state of the art, particularly the frame- synchronization mechanism described above.

The present invention has the objective to align media streams for end-users in media- streaming system and to effectively reduce the buffer requirements in the user's equipment. The present invention should be easily implemented into existing live stream media systems. The present invention also intends to overcome all the above- mentioned disadvantages. The object of the present invention is achieved by the solution provided in the enclosed independent claims. Advantages implementation forms of the present invention are further defined in the depended claims. In particular, the present invention leverages a frame synchronization mechanism for a live-TV media stream.

A first aspect of the invention provides a method for frames synchronization in a user' s equipment in a real time media streaming system. The method comprises the following steps: Sending a channel change request to a retransmission server; receiving a channel change response from the transmission server in response to the channel change request, wherein the channel change response comprises a frame offset information that indicates a frame offset between a frame of a reference media stream and a frame of a corresponding media stream received by the user equipment from the retransmission server, calculating a frame rate acceleration duration based on the frame offset information and accelerating a frame rate of the corresponding media stream for the calculated frame rate acceleration duration to synchronize the frame of the corresponding media stream provided from the retransmission server to the frame of the reference media stream.

Alternatively, the reference media stream is accelerated for the acceleration duration instead of the corresponding media stream. In this case, the acceleration duration is stored in the user' s equipment and applied to the reference media stream instead of the corresponding media stream during the channel change procedure in the retransmission server.

A retransmission server is an entity between the user's equipment and the media content provider which performs rapid channel change request, preferably due to the RTCP. According to the RTCP, the channel change request is preferably a rapid synchronization request, RSR, and the channel change response is preferably a rapid synchronization indication, RSI. The retransmission server provides a corresponding unicast multimedia stream to the user's equipment to fasten the channel change process.

A user's equipment is preferably a set- top box, STB. The STB is an information appliance device that generally contains a live stream media input and displays output to a television set and an external source of signal, turning the source signal into content in a form that can then be displayed on the television screen or other display device on the end-user's side. According to the provided live-stream media content delivery via IP, the STB is a small computing device, also called IPTV-receiver, providing two-way communications on an IP network and decoding the video streaming media. STB for IP-related media streaming has a built-in home network interface that can be Ethernet, Wireless Network according to IEEE 802.11 g, n, ac or one of the existing wire home networking technologies such as HomePNA or the ITU- T G.hn standard, which provides a way to create a high-speed - up to lGbit/s - local area network, LAN, using existing home wiring. This type of live streaming service is preferably distinct from classical internet television and involves third-party media content provision over the Internet that is not controlled by the local system operator.

A reference media stream is preferably an IP multicast media stream. IP multicast is a technique for one-to-many communication over an IP infrastructure in a network. The user's equipments send channel change requests, for example in the case of Internet television when the user changes from one TV channel to another. Multicast uses network infrastructure efficiently by requiring a multimedia source to send a data packet only once, even if it needs to be delivered to a larger number of receivers via the user datagram protocol, UDP.

The reference media stream is preferably provided by a media content provider. A media content provider is a media streaming server that delivers the streaming media data over IP that supports RTP, RTSP and RTCP. It refers either to a dedicated computer appliance or to specialized application software.

According to the invention, the user sends a channel change request and obtains frame- offset information that basically represents a frame offset between a reference media stream, provided from the media content server, and a media stream provided by the retransmission server upon joining the retransmission server. This frame-offset information is delivered to the user's equipment within a channel change response. The user's equipment is configured to read the offset frame information and is able to calculate a frame rate acceleration duration based on the frame offset information. Based on the frame-offset, the user's equipment is configured to accelerate a frame rate of the media stream for a specific duration in order to align the reference media stream and the media stream, so that the frames of both media streams are aligned.

After the frame rate acceleration duration all users that watch this media stream of the media content provider, e.g. a live football game over IPTV, obtain the GOP-frame of the media stream at the same time. This avoids the above-described consistent user- specific delay when joining the channel. Thus, in case neighbored users watch the same IPTV live football game, a time-aligned media stream is provided and both neighbors see a goal at the same time.

Additionally, after the frame rate acceleration duration, the user's equipment does now not buffer the GOP-frames to compensate the user- specific time delay, since the time- delay does not occur after the acceleration duration and thus, the user's equipment does not permanently need to buffer the stream which relaxes the buffer of the user' s equipment.

In a first implementation form of the method according to the first aspect, the frame of the reference media stream is a frame of a GOP sequence of the reference media stream, preferably an I- frame and the frame of the corresponding media- stream from the retransmission server is a corresponding frame of a corresponding GOP sequence of the media-stream, preferably an I-frame. Alternatively, the frame of the GOP sequence of the reference media stream and its corresponding frame of a corresponding GOP sequence of the media- stream may be any frame which allows the decoder to reconstruct the picture without additional information, or any frame which contains the full picture information.

The compressed video stream comprises a plurality of GOP-sequences according to the H.264 or H.265 standardization. Thus, each GOP-sequence comprises frames of different frame-types. One of these frame-types is a so called I-frame. I-frame is an intra-coded picture which means that the picture is coded independently of all other pictures in the GOP. Each GOP begins in decoding order with an I-frame. I-frames contain the full picture-information and do not require any additional information to reconstruct the picture. Typically, encoders use GOP structures that cause each I-frame to be a clean random access point, such that any errors within the GOP structure are corrected by the next I-frame.

Generally, the more I-frames the video stream has, the more editable it is. However, having more I-frames substantially increases bit rate needed to code the video. Thus, to obtain a compression character in the video stream, not all frames in the GOP- sequence are I-frames. There are also P-frames that are predictive coded pictures which means that they contain motion-compensated difference information relative to previously decoded pictures, e.g. an I-frame. There are also B-frames that are bipredictive coded pictures which means that they contain motion-compensated difference information relative to previously decoded pictures, e.g. an I-frame or a P- frame.

In a second implementation form of the method according to the first aspect as such or according to the first implementation form of the first aspect, the frame rate acceleration is limited to a predefined percentage, preferably less than 10 percent, more preferably between 7 percent to 9 percent of the frame rate of the reference media stream from the media contents server. Accelerating the media stream leads to playing at higher frame rates. This acceleration can hardly be noticed by the end-users if the predetermined threshold of less than 10 percent is respected. Such a limitation of the frame rate acceleration is suggested in order to avoid affecting the user' s quality of experience, QoE, in that the user is unable to follow the video stream. However, the threshold value is preferably detected during a test phase and can be applied accordingly. Thus, the media stream is accelerated according to the frame rate acceleration duration calculation.

In a third implementation form of the method according to the first aspect as such or according to any previous implementation form of the first aspect, the method further comprises determining, by the user's equipment, the frame rate of the corresponding media stream received from the retransmission server, wherein the determined frame rate is equal to the frame rate of the reference media stream. Since the frame rate of the media stream from the retransmission server is known to the user' s equipment, an alignment of the I-frame from the reference media stream to the I-frame of the corresponding media stream by acceleration of the frame rate of the corresponding media stream from the retransmission server is achieved in an easy manner.

According to a second aspect of the invention, a method for frame synchronization in a retransmission server in a real time media streaming system is provided. The method comprises the following steps: receiving a channel change request from a user's equipment; determining a frame offset between a frame of a reference media stream and a frame of a corresponding media stream to be provided to the user' s equipment; generating a frame offset information indicating the frame offset, and sending a channel change response to the user's equipment in response to the received channel change request, wherein the channel change response comprises the offset information.

In a first implementation form of the method according to the second aspect, the frame of the reference media stream is a frame of a GOP sequence of the reference media stream, preferably an I-frame and the frame of the corresponding media- stream from the retransmission server is a corresponding frame of a corresponding GOP sequence of the corresponding media-stream, preferably an I-frame. Alternatively, the frame of the GOP sequence of the reference media stream and its corresponding frame of a corresponding GOP sequence of the media- stream may be any frame which allows the decoder to reconstruct the picture without additional information, or any frame which contains the full picture information.

In a second implementation form of the method according to the second aspect as such or according to the first implementation form of the second aspect, the frame rate acceleration is limited to a predefined percentage, preferably less than 10 percent, more preferably between 7 percent to 9 percent of the frame rate of the reference media stream from the media contents server.

In a third implementation form of the method according to the second aspect as such or according to any previous implementation form of the second aspect, the method further comprises determining, by the retransmission server, the frame rate of the reference media stream received from the content media server and provision of this determined frame rate to the user' s equipment. The method of the second aspect or its implementation forms achieves all advantages described above for the method of the first aspect or its implementation forms.

According to a third aspect of the invention, a method for frame synchronization in a real time media streaming system is provided. The method comprises the following steps: sending a channel change request from a user's equipment to a retransmission server; determining a frame offset between a frame of a reference media stream and a frame of a corresponding media stream to be provided to the user's equipment; generating a frame offset information indicating the frame offset; receiving a channel change response from the retransmission server in response to the channel change request, wherein the channel change response comprises a frame offset information that indicates a frame offset between a frame of a reference media stream and a frame of a corresponding media stream received by the user equipment from the retransmission server; calculating a frame rate acceleration duration based on the frame offset information; and accelerating a frame rate of the corresponding media stream for the calculated frame rate acceleration duration to synchronize the frame of the media stream provided from the retransmission server to the frame of the reference media stream.

In a first implementation form of the method according to the third aspect, the frame of the reference media stream is a frame of a GOP sequence of the reference media stream, preferably an I- frame and the frame of the corresponding media- stream from the retransmission server is a corresponding frame of a corresponding GOP sequence of the media-stream, preferably an I-frame. Alternatively, the frame of the GOP sequence of the reference media stream and its corresponding frame of a corresponding GOP sequence of the media- stream may be any frame which allows the decoder to reconstruct the picture without additional information, or any frame which contains the full picture information. In a second implementation form of the method according to the third aspect as such or according to the first implementation form of the third aspect, the frame rate acceleration is limited to a predefined percentage, preferably less than 10 percent, more preferably between 7 percent to 9 percent of the frame rate of the media stream from the media contents server.

In a third implementation form of the method according to the third aspect as such or according to any previous implementation form of the third aspect, the method further comprises determining, by the user's equipment, the frame rate of the media stream received from the retransmission server, wherein the determined frame rate is equal to the frame rate of the reference media stream.

In a fourth implementation form of the method according to the third aspect as such or according to any previous implementation form of the third aspect, the method further comprises determining, by the retransmission server, the frame rate of the reference media stream received from the content media server and provision of this determined frame rate to the user' s equipment.

The method of the third aspect or its implementation forms achieves all advantages described above for the method of the first aspect or its implementation forms.

According to a fourth aspect of the invention, a user' s equipment in a real time media stream system is provided. The user's equipment comprises: a sending means configured to send a channel change request to a retransmission server; a receiving means configured to receive a channel change response from the retransmission server in response to the channel change request, wherein the channel change response comprises a frame offset information that indicates a frame offset between a frame of a reference media stream and a frame of a corresponding media stream provided from the retransmission server; and a processing means configured to calculate a frame rate acceleration duration based on the frame offset information and further configured to accelerate a frame rate of the corresponding media stream for the calculated frame rate acceleration duration to synchronize the frame of the media stream provided from the retransmission server to the frame of the reference media stream. Alternatively, the reference media stream is accelerated for the acceleration duration instead of the corresponding media stream. In this case, the acceleration duration is stored in the user' s equipment and applied to the reference media stream instead of the corresponding media stream during the channel change procedure in the retransmission server.

In a first implementation form of the user' s equipment according to the fourth aspect, the processing means is further configured to determine the frame rate of the media stream provided from the retransmission server which is equal to the frame rate of the reference media stream.

In a second implementation form of the user's equipment according to the fourth aspect, the processing means is further configured to limit the frame rate acceleration to a predefined percentage, preferably less than 10 percent, more preferably between 7 percent and 9 percent, of the frame rate of the reference media stream from the media content server.

In a third implementation form of the of the user's equipment according to the fourth aspect as such or according to any previous implementation form of the fourth aspect, the processing means of user's equipment is further configured to determine the frame rate of the media stream received from the retransmission server, wherein the determined frame rate is being equal to the frame rate of the reference media stream.

The user equipment of the fourth aspect or its implementation forms achieves all advantages described above for the method of the first aspect or its implementation forms.

According to a fifth aspect of the invention, a retransmission server in a real time media streaming system is provided. The retransmission server comprises: a receiving means configured to receive a channel change request from a user's equipment; a processing means configured to determine a frame offset between a frame of a reference media stream and a frame of a corresponding media stream provided to the user's equipment and further configured to generate a frame offset information indicating the frame offset; and a sending means configured to send a channel change response to the user's equipment in response to the channel change request, wherein the channel change response comprises the offset information.

In a first implementation form of the retransmission server according to the fifth aspect, the processing means is configured to insert the frame offset information as a one octet information in the channel change response.

According to the RTCP, the channel change request is preferably a rapid synchronization request, RSR, and the channel change response is preferably a rapid synchronization indication, RSI. The RSI is preferably structured according to paper "Extensions to RTCP for Rapid Synchronization" from P. Yang, published on March 8, 2009. The RSI preferably comprises a reserved-field of the size of one octet. This reserved- field is preferably used to contain the frame-offset information to the user's equipment. Thus, the RTCP does not need to be adapted; the existing structures can be used without further message-types or fields to provide the frame-offset information.

In a second implementation form of the retransmission server according to the fifth aspect, the processing means is further configured to determine the frame rate of the referenced media stream provided from the content media server and to provide it to the user' s equipment.

The retransmission server of the fifth aspect or its implementation forms achieves all advantages described above for the method of the first aspect or its implementation forms. According to a sixth aspect of the invention, a real time media stream system for frame alignment is provided. The system comprises: a user's equipment according to the fourth aspect of the invention as such or according to any of the implementation form of the fourth aspect; a retransmission server according to the fifth aspect of the invention as such or according to any of the implementation forms of the fifth aspect; and a media content server configured to provide a media stream. The user's equipment is configured to send a channel change request to the retransmission server, the retransmission server is configured to send a channel change response based on the channel change request to the user' s equipment, wherein the channel change response comprises a frame offset information that indicates a frame offset between a frame of the reference media stream from the media content server and a frame of a corresponding media stream provided from the retransmission server to the user's equipment, the user's equipment is configured to calculate a frame rate acceleration duration based on the offset information, and the user's equipment is further configured to accelerate a frame rate of the media stream for the calculated frame rate acceleration duration to synchronize the frame of the media stream provided from the retransmission server to the frame of the reference media stream from the media content server. Alternatively, the reference media stream is accelerated for the acceleration duration instead of the corresponding media stream. In this case, the acceleration duration is stored in the user' s equipment and applied to the reference media stream instead of the corresponding media stream during the channel change procedure in the retransmission server.

In a first implementation form of the system according to the sixth aspect, the media streaming system is a RTCP, based media streaming system.

In a second implementation form of the system according to the sixth aspect as such or according to the first implementation form of the system according to the sixth aspect, the media content server is configured to provide a multicast reference media stream; and the retransmission server is configured to provide a unicast corresponding media stream. In a third implementation form of the system according to the sixth aspect as such or according to any of the previous implementation forms of the system according to the sixth aspect, the processing means is configured to insert the frame offset information as a one octet information in the channel change response. In a fourth implementation form of the system according to the sixth aspect as such or according to any of the previous implementation forms of the system according to the sixth aspect, the processing means of the retransmission server is further configured to determine the frame rate of the referenced media stream provided from the content media server and to provide it to the user' s equipment.

In a fifth implementation form of the system according to the sixth aspect as such or according to any of the previous implementation forms of the system according to the sixth aspect, the processing means of the user's equipment is further configured to determine the frame rate of the media stream provided from the retransmission server which is equal to the frame rate of the reference media stream.

In a sixth implementation form of the system according to the sixth aspect as such or according to any of the previous implementation forms of the system according to the sixth aspect, the frame of the reference media stream is a frame of a GOP sequence of the reference media stream, preferably an I-frame and the frame of the corresponding media- stream from the retransmission server is a corresponding frame of a corresponding GOP sequence of the media-stream, preferably an I-frame. Alternatively, the frame of the GOP sequence of the reference media stream and its corresponding frame of a corresponding GOP sequence of the media- stream may be any frame which allows the decoder to reconstruct the picture without additional information, or any frame which contains the full picture information.

In a seventh implementation form of the system according to the sixth aspect as such or according to any of the previous implementation forms of the system according to the sixth aspect, the frame rate acceleration is limited to a predefined percentage, preferably less than 10 percent, more preferably between 7 percent to 9 percent of the frame rate of the reference media stream from the media contents server. The system of the sixth aspect or its implementation forms achieves all advantages described above for the method of the first aspect or its implementation forms.

A seventh aspect of the present invention provides a computer program product for implementing, when carried out on a computing device, a method for frame synchronization according to the first aspect and any of its implementation forms or the second aspect and any of its implementation forms or the third aspect and any of its implementation forms. By implementing the method via the computer program product, all its advantages can be achieved.

It has to be noted that all devices, elements, units and means described in the present application could be implemented in the software or hardware elements or any kind of combination thereof. All steps which are performed by the various entities described in the present application as well as the functionalities described to be performed by the various entities are intended to mean that the respective entity is adapted to or configured to perform the respective steps and functionalities. Even if, in the following description of specific embodiments, a specific functionality or step to be full formed by eternal entities is not reflected in the description of a specific detailed element of that entity which performs that specific step or functionality, it should be clear for a skilled person that these methods and functionalities can be implemented in respective software or hardware elements, or any kind of combination thereof. BRIEF DESCRIPTION OF THE DRAWINGS

The above described aspects and implementation forms of the present invention will be explained in the following description of specific embodiments in relation to the enclosed drawings, in which:

Fig. 1 shows a flow chart of a method according to a first embodiment of the present invention.

Fig. 2 shows a flow chart of a method according to a second embodiment of the present invention.

Fig. 3 shows a flow chart of a method according to a third embodiment of the present invention. Fig. 4 shows a user's equipment according to an embodiment of the present invention.

Fig. 5 shows a retransmission server according to an embodiment of the present invention.

Fig. 6 shows a system according to an embodiment of the present invention.

Fig. 7 shows a flow diagram of a method according to a fourth specific embodiment of the present invention.

Fig. 8 shows a frame alignment of a system according to specific embodiment of the present invention.

DETAILED DESCRIPTION OF THE EMBODIMENTS

The embodiments according to Fig. 1 to Fig. 6 describe basic embodiments of methods and entities of a frame alignment in a real time media system. These basic concepts are explained in greater details with respect to the embodiments in Fig. 7 and Fig. 8. Thus, the explanations of Fig. 7 and Fig. 8 are also valid for the basic concepts according to Fig. 1 to Fig. 6.

Fig. 1 shows a flow chart of a method according to a first embodiment of the invention. The method 100 is used for frame synchronization in a user's equipment 400 in a real time media streaming system 600. The method 100 according to Fig. 1 comprises the step of sending 101 a channel change request to a retransmission server 500. The method 100 further comprises the step of receiving 102 a channel change response from the retransmission server 500 in response to the channel change request, wherein the channel change response comprises a frame offset information that indicates a frame offset between a frame of a reference media stream and a frame of a corresponding media stream received from the retransmission server 500 to the user's equipment 400. The method 100 further comprises the step of calculating 103 a frame rate acceleration duration based on the frame offset information. The method 100 further comprises the step of accelerating 104 a frame rate of the corresponding media stream for the calculated frame rate acceleration duration to synchronize the frame of the corresponding media stream provided from the retransmission server 500 to the frame of the reference media stream.

Fig. 2 shows a flow chart of a method 200 according to a second embodiment of the invention. The method 200 is used for frame synchronization in a retransmission server 500 in a real time media streaming system 600. The method 200 according to Fig. 2 comprises the step of receiving 201 a channel change request from a user's equipment 400. The method 200 further comprises the step of determining 202 a frame offset between a frame of a reference media stream and a frame of a corresponding media stream to be provided to the user's equipment 400. The method 200 further comprises the step of generating 203 a frame offset information indicating the frame offset. The method 200 further comprises the step of sending 204 a channel change response to the user's equipment 400 in response to the received channel change request, wherein the channel change response comprises the frame offset information.

Fig. 3 shows a flow chart of a method 300 according to a third embodiment of the invention. The method 300 is used for frame synchronization in a real time media streaming system 600. The method 300 according to Fig. 3 comprises the step of sending 301 a channel change request from a user's equipment 100 to a retransmission server 500. The method 300 further comprises the step of determining 302 a frame offset between a frame of a reference media stream and a frame of a corresponding media stream to be provided to the user's equipment 400. The method 300 further comprises the step of generating 303 a frame offset information indicating the frame offset. The method 300 further comprises the step of receiving 304 a channel change response from the retransmission server 500 in response to the channel change request, wherein the channel change response comprises a frame offset information that indicates the frame offset between a frame of a reference media stream and a frame of a corresponding media stream received from the retransmission server 500 to the user's equipment 400. The method 300 further comprises the step of calculating 305 frame rate acceleration duration based on the frame offset information. The method 300 further comprises the step of accelerating 306 a frame rate of the corresponding media stream for the calculated frame rate acceleration duration to synchronize the frame of the corresponding media stream provided from the retransmission server 500 to the frame of the reference media stream. Fig. 4 shows a user's equipment 400 in a real time media streaming system 600 according to an embodiment of the present invention. The user's equipment 400 comprises a sending means 401 configured to send a channel change request to a retransmission server 500; a receiving means 402 configured to receive a channel change response from the retransmission server 500 in response to the channel change request, wherein the channel change response comprises a frame offset information that indicates a frame offset between a frame of a reference media stream and a frame of a corresponding media stream provided from the retransmission server 500; and a processing means 403 configured to calculate a frame rate acceleration duration based on the frame offset information and further configured to accelerate a frame rate of the corresponding media stream for the calculated frame rate acceleration duration to synchronize the frame of the corresponding media stream provided from the retransmission server 500 to the frame of the reference media stream. Fig. 5 shows a retransmission server 500 in a real time media streaming system 600 according to an embodiment of the present invention. The retransmission server 500 comprises a receiving means 501 configured to receive a channel change request from a user's equipment 400; a processing means 503 configured to determine a frame offset between a frame of a reference media stream and a frame of a corresponding media stream provided to the user's equipment 400 and further configured to generate a frame offset information indicating the frame offset; and a sending means 502 configured to send a channel change response to the user's equipment 400 in response to the channel change request, wherein the channel change response comprises the frame offset information.

Fig. 6 shows a real time media streaming system 600 for frame alignment according to a fourth embodiment of the present invention. The system 600 comprises a user's equipment 400 according to Fig. 4. The system 600 further comprises a retransmission server 500 according to Fig. 5. The system 600 comprises a media content server 601 configured to provide a reference media stream 106.

Fig. 7 shows a system 600 of a fourth specific embodiment. As explained above, the statements provided hereinafter are applicable to the basic concepts as defined in Fig. 1 to Fig. 6. The system 600 comprises a user's equipment 400 according to Fig. 4. The user's equipment 400 may be, for instance, a STB, especially an IPTV-receiver. The STB is a small computer providing two-way communications to an IP network, preferably to the retransmission server 500, and is used for decoding the video streaming media provided from the retransmission server 500. IP set-top boxes have a built-in home network interface that can be Ethernet, Wireless LAN according to IEEE 802.11 g, n, ac, or one of the existing wire home networking technologies. The STB 400 is an information appliance device that generally contains a TV-tuner input and displays output to a television set and an external source of signal, turning the source signal into content in a form that can then be displayed on the television screen or other display device.

The system 600 further comprises a retransmission server 500 according to Fig. 5. The retransmission server 500 is preferably a fast channel change server, FCC, 500. With the advent of IPTV, user experience with channel surfing has taken a step back with channel change potentially measured in up to 24 seconds. A FCC 500 brings the zap time back to less than a second. This enables IPTV operators to offer premium high definition, HD, quality IPTV services with the highest user experience. The QoE highly increases when using an FCC 500. The FCC 500 functionality is implemented at the network edge which dramatically offloads a unicast corresponding media stream 107 from the FCC 500 and avoids costly and time-consuming network upgrades. This approach further improves the QoE by reducing latency and data congestion. The FCC 500 provides live UDP retransmission, which eliminates packet loss. According to Fig. 6, the FCC 500 is arranged in between the user's equipment 400 and the content provider 601.

The system 600 comprises a media content server 601 configured to provide a reference media stream 106. A media content provider is a media streaming server that delivers a multicast reference media stream 106 over IP that supports RTP, RTSP and RTCP. It refers either to a dedicated computer appliance or to specialized application software.

The system 600 supports the RTP and RTCP standard. Its basic functionality and packet structure is defined in request for comments, RFC, No. 3550. RTCP provides out-of-band statistics and control information for an RTP session. It partners with RTP in the delivery and packaging of multimedia data, but does not transport any media data itself. The primary function of RTCP is to provide feedback on the quality of service, QoS, in media distribution by periodically sending statistics information to participants in a streaming multimedia session.

According to Fig. 7, the media content provider 601 provides multiple multicast reference media streams 107 to the FCC 500. When the user performs an IPTV channel change, the STB 400 sends the unicast channel change request RSR as based on the RTCP to the FCC server 500 according to method step 301. The FCC 500 determines a frame offset 705 according to step 302 between the reference media stream 602 provided from the content server 601 and the moment when the user joins the new channel. The FCC 500 then generates a frame offset information according to step 303. Preferably, the frame offset information is the number of frames between an I- frame of the reference media stream 107 and the corresponding I- frame of the corresponding media stream 106 that represents the moment when the user joins the new channel. The frame offset information is preferably one octet information and can thus be contained in the channel change response RSI, preferably in the reserved-field octet of the RSI by the FCC 500.

Subsequently, the FCC 500 replies to the RSR with the RSI to the user's equipment 400 that includes the frame offset information as generated in step 303. The FCC 500 also unicasts the corresponding multimedia stream 106 of the new channel starting with the corresponding I- frame to the STB 400 in step 603.

Upon reception of the frame offset information in the RSI, the STB 400 will calculate a frame rate acceleration duration in step 305. The frame rate acceleration duration is basically a speedup time that is needed to reach the frame alignment between the I- frame of the reference media stream 106 and the corresponding I-frame in the corresponding media stream 107. The STB 400 is able to read the frame offset in the RSI message. Upon reception of the frame offset information and after looking at the frame rate of the content, the STB 400 will calculate for how many seconds it has to play at higher frame rate in order to align with the reference media stream 106. It then plays the received unicast media stream (step 603) with an accelerating frame rate according to step 306.

The unicast corresponding media stream 106 provided from the FCC 500 is provided at a faster speed than the normal multicast reference media stream 107 frame rate from the content provider 601 to keep pace with the subsequent multicast stream 107. In case the FCC 500 decides that the unicast corresponding media stream 107 is keeping pace with the multicast stream 107 in the STB 400 it notifies the STB 400 to join the multicast group of the new channel according to an SCN message in step 604. The STB 400 then notifies the content provider 601 according to step 605 with a multicast Internet Group Management Protocol, IGMP, join-message. Meanwhile, the FCC 500 slows down the sending speed of the unicast corresponding media stream 106 according to step 606. When the STB 400 receives the multicast reference media stream 107 from the content provider 601 it sends an SN- value (Sequence Number value) of the first multicast packet to the FCC 500 according to step 607. The SN-value is an identification number for identifying the data packet, which is being transmitted. The FCC 500 stops providing the unicast stream in step 608.

In an alternative embodiment, the acceleration of the frame rate is applied to the reference media stream 107 instead of the corresponding media stream 106. In this case, the STB 400 stores the calculated frame rate acceleration duration and applies the acceleration at the time, when the content provider 601 provides the reference media stream 107 to the STB 400.

After acceleration of the frame rate, the I-frames of the corresponding media stream 106 is aligned to the I-frame of the reference media stream 107. Thus, the media content provider 601 offers a media stream that is time-aligned to all users connected to that IPTV channel. Additionally, a buffering of the reference media stream 106 in order to time-align the media stream 106 according to a user-specific delay can be avoided and thus the buffer capabilities are relaxed in the STB 400. In a preferred embodiment, the STB 400 provides acknowledgement information to the FCC 500. For improvement reasons the FCC 500 might determine the frame- alignment and send another frame-offset information in case the frame- alignment is out of a predetermined range, e.g. caused due to latencies in the media streaming system 600. In Fig. 8, a frame synchronization scheme is shown for three different STB 400, 404 and 405. All STB 400, 404, 405 obtain the multicast reference media stream 106 from the content provider 601 after the fast channel change is applied using the FCC 500. Alternatively, the STB 400, 404, 405 obtain the unicast corresponding media stream 107 from the FCC 500.

The STB 400, 404 and 405 obtain the real-time live IPTV corresponding media stream 106. The corresponding stream 106 is divided into different GOPs, wherein the first GOP 701 is indicated by a first I-frame 703 and a second GOP 702 is indicated by another I-frame 703. Each GOP 701, 702 contains 18 frames, comprising the I-frame 703 at the beginning of each GOP 701, 702. The remaining 17 frames in each GOP 701, 702 are P- frames. Clearly, this is an example and the GOPs may also include in alternative or in addition frames which are different than I or P frames.

The STB 400 joins the IPTV channel when the GOP 701 sixth frame is about to be transmitted in the real time stream 107. As the sixth frame is not an I-frame, the FCC 500 sends a unicast stream 106 to the STB 400 starting from the last available I-frame 703. This would cause the constant offset time of five frames between the STB 400 and the reference media stream 106 form the content provider 601. A second STB 404 joins the channel when the GOP 701 fourteenth frame is about to be transmitted, therefore an offset 705 of thirteen frames exists between the second user's equipment 404 and the reference media stream 602. Finally, a third STB 405 joins the channel when the second GOP 702 ninth frame is about to be transmitted, therefore an frame offset 705 of eight frames is generated between this third user 405 and the reference media stream 601. Thus, STB 400 and STB 404 would have a relative time delay between each other of eight frames. Using a frame rate of 50 frames per second, the time-delay between the STB 400 and the STB 404 would be 8:50 = 0,16 seconds. More generally, the time- delay will be given by the ratio between the delay in terms of the number of frames and the frame-rate of the reference stream 107. Since GOP-times can be several seconds, this user-specific time is to be avoided using the inventive frame alignment.

To avoid this frame offset the user's equipments 400, 404 and 405 obtain a frame offset information to calculate a frame rate acceleration duration to reduce the frame offset 705 and to align the I-frame 703 to the I-frame of the reference media stream 107. The frame rate value is provided by the FCC 500 to each STB 400.

The limitation of the acceleration frame rate may be chosen to be around 8 percent of the frame rate of the reference media stream 107 or the corresponding media stream 106. Once the desired acceleration rate has been set, it is possible to calculate the number of additional frames per second and the time required by the user equipment to catch up the delay by accelerating the corresponding media stream 106. The required time is given by the ratio between the delay (in number of frames) and the calculated additional frames per second.

As an example, the compensation of a frame offset of ten frames at a frame rate of 50 frames per second should be applied. A limited acceleration rate of eight percent should be used to have an expectable QoE value. Thus, the STB 400 accelerates the frame rate from 50 frames per second to 54 frames per second for frame acceleration duration of 2.5 seconds to achieve the GOP frame alignment. Thus, the frame offset is compensated. The STB 400 buffer utilization is optimized as the buffer does not have to constantly store up to three seconds for as long as the user keeps on watching this new IPTV channel. The STB 400 is capable to play at higher frame rates compared to the standard frame rates used in the reference media stream 106.

The present invention has been described in conjunction with various embodiments as examples as well as implementations. However, other variations can be understood and effected by those persons skilled in the art and practicing the claimed invention, from the studies of the drawings, this disclosure and the independent claims. In the claims as well as in the description the word "comprising" does not exclude other elements or steps and the indefinite article "a" or "an" does not exclude a plurality. A single element or other unit may fulfill the functions of several entities or items recited in the claims. The mere fact that certain measures are recited in the mutual different dependent claims does not indicate that a combination of these measures cannot be used in an advantageous implementation.

Claims

What we claim is: 1. A method (100) for frame synchronization in a user's equipment (400) in a real time media streaming system (600), the method (100) comprises the following steps: sending (101) a channel change request to a retransmission server (500);

receiving (102) a channel change response from the retransmission server (500) in response to the channel change request, wherein the channel change response comprises a frame offset information that indicates a frame offset (705) between a frame of a reference media stream (106) and a frame of a corresponding media stream (107) received by the user equipment (400) from the retransmission server (500);

calculating (103) a frame rate acceleration duration based on the frame offset information; and

accelerating (104) a frame rate of the corresponding media stream (107) for the calculated frame rate acceleration duration to synchronize the frame of the corresponding media stream (107) provided from the retransmission server (500) to the frame of the reference media stream (106).

2. The method (100) according to claim 1, wherein

the frame of the reference media stream (106) is a frame of a group of picture, GOP, sequence of the reference media stream (106), preferably an I-frame; and

the frame of the corresponding media stream (107) from the retransmission server (500) is a corresponding frame of a corresponding GOP sequence of the media stream (107), preferably an I-frame.

3. The method (100) according to one of the claims 1 or 2, wherein

the frame rate acceleration is limited to a predefined percentage, preferably less than 10 percent, more preferably between 7 percent to 9 percent, of the frame rate of the reference media stream (106).

4. The method (100) according to one of the preceding claims, further comprising: determining (108), by the user's equipment (400), the frame rate of the corresponding media stream (107) received from the retransmission server (500), wherein the determined frame rate is equal to the frame rate of the reference media stream (106).

5. A method (200) for frame synchronization in a retransmission server (500) in a real time media streaming system (600), the method comprises the following steps: receiving (201) a channel change request from a user's equipment (400);

determining (202) a frame offset (705) between a frame of a reference media stream (106) and a frame of a corresponding media stream (107) to be provided to the user's equipment (400);

generating (203) a frame offset information indicating the frame offset (705); and

sending (204) a channel change response to the user's equipment (400) in response to the received channel change request, wherein the channel change response comprises the frame offset information.

6. A method (300) for frame synchronization in a real time media streaming system (600), the method comprises the following steps:

sending (301) a channel change request from a user's equipment (100) to a retransmission server (500);

determining (302) a frame offset (705) between a frame of a reference media stream (106) and a frame of a corresponding media stream (107) to be provided to the user's equipment (400);

generating (303) a frame offset information indicating the frame offset (705); receiving (304) a channel change response from the retransmission server (500) in response to the channel change request, wherein the channel change response comprises a frame offset information that indicates the frame offset (705) between a frame of a reference media stream (106) and a frame of a corresponding media stream (107) received by the user's equipment (400) from the retransmission server (500); calculating (305) a frame rate acceleration duration based on the frame offset information; and

accelerating (306) a frame rate of the corresponding media stream (107) for the calculated frame rate acceleration duration to synchronize the frame of the corresponding media stream (107) provided from the retransmission server (500) to the frame of the reference media stream (106).

7. A user's equipment (400) in a real time media streaming system (600), the user's equipment (400) comprising:

a sending means (401) configured to send a channel change request to a retransmission server (500);

a receiving means (402) configured to receive a channel change response from the retransmission server (500) in response to the channel change request, wherein the channel change response comprises a frame offset information that indicates a frame offset (705) between a frame of a reference media stream (106) and a frame of a corresponding media stream (107) provided from the retransmission server (500); and a processing means (403) configured to calculate a frame rate acceleration duration based on the frame offset information and further configured to accelerate a frame rate of the corresponding media stream (106) for the calculated frame rate acceleration duration to synchronize the frame of the corresponding media stream (106) provided from the retransmission server (500) to the frame of the reference media stream (107).

8. The user's equipment (400) according to claim 7, wherein

the processing means (403) is further configured to determine the frame rate of the corresponding media stream (107) provided from the retransmission server (500) which is equal to the frame rate of the reference media stream (106).

9. The user's equipment (400) according to one of the claims 7 or 8, wherein

the processing means (403) is further configured to limit the frame rate acceleration to a predefined percentage, preferably less than 10 percent, more preferably between 7 percent to 9 percent, of the frame rate of the reference media stream (106).

10. A retransmission server (500) in a real time media streaming system (600), the retransmission server (500) comprising:

a receiving means (501) configured to receive a channel change request from a user's equipment (400); a processing means (503) configured to determine a frame offset (705) between a frame of a reference media stream (106) and a frame of a corresponding media stream (107) provided to the user's equipment (400) and further configured to generate a frame offset information indicating the frame offset (705); and

a sending means (502) configured to send a channel change response to the user' s equipment (400) in response to the channel change request, wherein the channel change response comprises the frame offset information.

11. The retransmission server (500) according to claim 10, wherein

the processing means (503) is configured to insert the frame offset information as a one octet information in the channel change response.

12. A real time media streaming system (600) for frame alignment, the system (600) comprises:

a user's equipment (400) according to one of the claims 7 to 9;

a retransmission server (500) according to one of the claims 10 to 11; and a media content server (601) configured to provide a reference media stream

(106).

13. The system (600) according to claim 12, wherein

the system (600) is a real time control protocol, RTCP, based media streaming system (600).

14. The system (600) according to one of the claims 12 or 13, wherein

the media content server (601) is configured to provide a reference multicast media stream (106); and

the retransmission server (500) is configured to provide a corresponding unicast media stream (107).

15. Computer program product for implementing, when carried out on a computing device, a method for frame synchronization according to one of the claims 1 to 6.