CN100539710C - Method based on the converting plane video into stereoscopic video of optical flow field - Google Patents

Method based on the converting plane video into stereoscopic video of optical flow field Download PDF

Info

Publication number
CN100539710C
CN100539710C CNB2007101176542A CN200710117654A CN100539710C CN 100539710 C CN100539710 C CN 100539710C CN B2007101176542 A CNB2007101176542 A CN B2007101176542A CN 200710117654 A CN200710117654 A CN 200710117654A CN 100539710 C CN100539710 C CN 100539710C
Authority
CN
China
Prior art keywords
flow field
optical flow
frame
threshold
video
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Fee Related
Application number
CNB2007101176542A
Other languages
Chinese (zh)
Other versions
CN101087437A (en
Inventor
戴琼海
李涛
刘烨斌
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangdong Shengyang Information Technology Industry Co., Ltd.
Original Assignee
Tsinghua University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tsinghua University filed Critical Tsinghua University
Priority to CNB2007101176542A priority Critical patent/CN100539710C/en
Publication of CN101087437A publication Critical patent/CN101087437A/en
Application granted granted Critical
Publication of CN100539710C publication Critical patent/CN100539710C/en
Expired - Fee Related legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Abstract

The present invention relates to method, belong to method based on the converting plane video into stereoscopic video of optical flow field based on the converting plane video into stereoscopic video of optical flow field; This method comprises: the optical flow field that calculates given each frame of planar video sequence: be proportional to the absolute value of optical flow field vector according to parallax value, by the parallax value that is not obtained corresponding point in the optical flow field of present frame by the optical flow field vector of filtering; Obtain new view with present frame, present frame and described disparity map are synthetic, as the left and right view of present frame; With the synthetic stereo video frame that obtains of the left and right view of each frame; All stereo video frame are formed stereoscopic video sequence.The present invention in the method complexity and realize having obtained between the effect good compromise, advantage be implement very simple.

Description

Method based on the converting plane video into stereoscopic video of optical flow field
Technical field
The invention belongs to technical field of computer multimedia, particularly a kind of technology that the common plane video is transferred to three-dimensional video-frequency.
Technical background
Planar video transfers the three-dimensional video-frequency technology to more and more to be paid attention to, but because it realizes bigger, the also comparatively imperfect so far solution of difficulty, is a problem that still is in the exploratory stage.
Seen the people of 3D film all can stay deep impression to its sense of reality, the sort of sense assignee on the spot in person is difficult to forget, and now, three-dimensional video-frequency is just more and more promoted in worldwide and in a plurality of industries and is favored, such as, the large-scale shopping place and the public place of entertainment of some key cities of China have begun to arrange three-dimensional player, advertisement or other promotional videos made in advance with these three-dimensional player plays, give very strong visual impact, can attract very much spectators' eyeball.
Three-dimensional video-frequency must comprise two video sequences, is respectively left view sequence and right view sequence, and two sequences are seen with right eye for respectively spectators' left eye and seen.The people watches why the world has third dimension, be to have a little different and have a parallax (disparity) because left eye and right eye are seen the visual angle in the world, and each frame of stereopsis two sequence correspondences frequently also has parallax, and this also is to make spectators watch three-dimensional video-frequency can produce relief reason as if on the spot in person.
Above-mentioned parallax (disparity) is meant the horizontal displacement corresponding to two picture elements of same world point of left view and right view, and the parallax value of all pixels is formed disparity map (disparity map).The theorem of computer vision field is pointed out, the degree of depth (depth) of the parallax of certain point and its pairing world point is inversely proportional to, and that is to say, and is more little from the parallax value of watching point a little far away more, the parallax of infinite point is 0, and the depth value of being had a few has been formed depth map (depth map).If given left view (right view) and disparity map just can obtain right view (left view).
Optical flow method is the important method of present movement image analysis, and it is meant pattern movement velocity in the time varying image.The variation of image has been expressed in light stream, because it has comprised the information of target travel, but therefore the observed person is used for determining the motion conditions of target.By the definition of the light stream optical flow field of can extending out, optical flow field is meant a kind of two dimension (2D) instantaneous velocity field that all picture elements constitute in the image.Optical flow field is by the calculating to adjacent two frames (M frame and M+1 frame), obtains the motion conditions from the M frame to the M+1 frame of the every bit of M frame, it is emphasized that the motion here refers to the motion in the image, is meant the offset of same object point at consecutive frame.When calculating optical flow field, require the difference of adjacent frame can not be too big, otherwise can't calculate.About the calculating of optical flow field, very ripe and simple algorithm has been arranged at present.
Present existing converting plane video into stereoscopic video method is a kind of method that is called frame time-delay (Frame Delay), and its step is as follows:
A) obtain the sports ground (motion field) of each frame;
B) analyze the image motion situation of this frame by the sports ground of each frame;
C) motion conditions according to this frame directly finds a frame and this frame to form left and right sides view in former sequence;
D) obtain the left and right sides view of all frames, thereby realize converting plane video into stereoscopic video.
There are following two problems in above-mentioned existing method:
(1) problem of vertical parallax: such as the upwards emission of a rocket, no matter how to choose with existing method, also can only choose two figure that vertical parallax is arranged, promptly have only the skew on the vertical direction and do not have skew on the horizontal direction, two figure like this show at three-dimensional display, not only do not have any third dimension, also can allow the people dizzy and tired.
(2) the do more physical exercises situation of object: if the A object is to left movement in the video, the B object moves right, and no matter how chooses with existing method, and the parallax that also can only be one of them object is right and parallax another object is wrong.
Summary of the invention
The objective of the invention is for overcoming the existing methods weak point, a kind of method of the converting plane video into stereoscopic video based on optical flow field is provided, this method is simple, and effect is better, obtains good compromise between method complexity and effect.
The method based on the converting plane video into stereoscopic video of optical flow field that the present invention proposes is characterized in that, may further comprise the steps:
1) optical flow field of given each frame of planar video sequence of calculating:
(11) to the calculating of the first frame optical flow field: first frame and second frame calculate the optical flow field vector of each picture element of the optical flow field of first frame; Utilize the improper optical flow field vector that surpasses first threshold in this optical flow field of first threshold filtering;
(12) to the calculating of second frame to the optical flow field vector of frame second from the bottom: a back frame of present frame and present frame calculates the optical flow field vector of all picture elements of optical flow field of present frame; Obtain the ratio that all optical flow field vectors of present frame surpass the optical flow field vector of this threshold value according to first threshold, again this ratio and predefined second threshold value are compared, to judge whether present frame is boundary frame, if not, then utilize the improper optical flow field vector that surpasses first threshold in this optical flow field of first threshold filtering; If then abandon described optical flow field, and recomputate optical flow field, and utilize the improper optical flow field vector that surpasses first threshold in this optical flow field of first threshold filtering by present frame and former frame;
(13) to the calculating of last frame optical flow field: last frame and former frame calculate optical flow field, and utilize the optical flow field vector that surpasses first threshold in this optical flow field of first threshold filtering;
2) be proportional to the absolute value of optical flow field vector according to parallax value, by the parallax value that is not obtained corresponding point in the optical flow field of present frame by the optical flow field vector of filtering; Utilized interpolation to obtain by the parallax value of the pairing point of optical flow field vector of filtering in this present frame, the parallax value that present frame is had a few is formed complete disparity map;
3) obtain new view with present frame, present frame and described disparity map are synthetic, as the left and right view of present frame;
4) with the synthetic stereo video frame that obtains of the left and right view of each frame;
5) all stereo video frame are formed stereoscopic video sequence.
Characteristics effect of the present invention:
Method of the present invention is the disparity map that obtains each frame by the optical flow field of former each frame of sequence (optical flow), can obtain new right view sequence by each frame and the disparity map thereof of former sequence.This method has solved the problem that prior art exists, and simple, obtains good compromise between method complexity and effect.
Though this method can not perfectly be applicable to various situations, can obtain average effect preferably.
Embodiment
The method based on the converting plane video into stereoscopic video of optical flow field that the present invention proposes is described in detail as follows in conjunction with the embodiments:
The inventive method may further comprise the steps:
1) optical flow field of given each frame of planar video sequence of calculating specifically comprises:
11) to the calculating of the first frame optical flow field: first frame and second frame calculate the optical flow field vector of each picture element of the optical flow field of first frame
Figure C200710117654D0005102916QIETU
(vector that optical flow field is every is represented the moving displacement vector of corresponding picture element from first frame to next frame, and its absolute value is unit with the number of pixels, and the concrete computational methods of optical flow field are very ripe, are not described in detail in this);
Utilize first threshold T 1The optical flow field vector that surpasses first threshold in this optical flow field of filtering, the error that several problems cause below avoiding may occurring in the calculating of optical flow field:
The problem of mistake in computation: the optical flow field vector value of certain point should be 0 or near 0 originally, but because mistake in computation may draw a paranormal very big value, the optical flow field vector value of these points is improper optical flow field vector.
Occlusion issue a: point that in the M frame, exists, because the variation at visual angle may disappear in the M+1 frame, such point can't calculate light stream, being reflected on the result is exactly that the value calculated is very big, and the optical flow field vector value of such point is improper optical flow field vector.
First threshold T 1Be used to remove the improper optical flow field vector that above-mentioned situation occurs, concrete value is: determine (needing the abundant different video of test) by abundant test in advance, by the size of continuous adjustment T1, choosing it has the value of filter effect preferably, and the present invention obtains T by experiment 1Be 80-120.Embodiment can choose T 1Be 100, promptly absolute value can be by filtering above the optical flow field vector of 100 length in pixels.
(12) to the calculating of second frame to the optical flow field vector of frame second from the bottom: a back frame of present frame and present frame calculates the optical flow field vector of all picture elements of optical flow field of present frame; According to first threshold T 1Obtain the ratio that all optical flow field vectors of present frame surpass the optical flow field vector of this threshold value, again with this ratio and the predefined second threshold value T 2Whether compare, be boundary frame to judge present frame, if not, then utilize the improper optical flow field vector that surpasses first threshold in this optical flow field of first threshold filtering; If then abandon described optical flow field, and recomputate optical flow field, and utilize the improper optical flow field vector that surpasses first threshold in this optical flow field of first threshold filtering by present frame and former frame;
Above-mentioned boundary frame is meant the frame of the position that is in scene sudden change.Because can take the photograph camera lens is transferred to another scene suddenly by a scene situation, to suppose whole video is divided into some fragments, each fragment is corresponding to a scene.(be made as the A section) in same video segment, the variation between the different frame is less, is reflected in optical flow field, and the optical flow field vector value that is exactly the great majority point is also little; And the frame between fragment and the fragment, such as the last frame M of Segment A and the first frame M+1 of next fragment (being made as the B section), between difference should be very big, be reflected in the optical flow field, the vector value that is exactly almost every bit is all very big.Therefore for two adjacent frame M and M+1, also can judge whether to be in the boundary of A section and B section by optical flow field; The present invention judges that the method for boundary frame is as follows:
The number that adds up to capable number and row of the pixel of frame M multiplies each other, that is: Width * Hight, and the two-dimensional array Array that then sets up a Width * Hight preserves the optical flow field vector absolute value of each pixel of frame M, adds up among the Array absolute value then greater than T 1Optical flow field vector number Number, obtain the ratio P that this number accounts for the total flow field vector number of frame again: P = Number Width × Height , If P 〉=T 2(T 2Be given proportion threshold value, i.e. second threshold value), think that then M is the last frame of certain video clips, promptly be boundary frame.If M is a boundary frame, then the optical flow field that calculates of itself and M+1 frame is abandoned, recomputate optical flow field with M frame and M-1 frame, do the motion conditions from M to the M-1 frame (opposite) that is of the optical flow field reflection that obtains although it is so, only pay close attention to the motion size but the present invention does not pay close attention to the direction of motion with the real motion direction; The optical flow field that obtains has like this solved the problem of boundary frame calculating optical flow field.
The above-mentioned second threshold value T 2Also be to determine (needing the abundant different video of test) by abundant test in advance, constantly adjust T 2Size judge in the hope of boundary frame being made accurately, the threshold value of choosing is reached does not judge the result who does not also fail to judge by accident.In the present invention by test T 2Can get 0.4-0.6, embodiment desirable 0.5.
2) be proportional to the absolute value of optical flow field vector according to parallax value, by the parallax value that is not obtained corresponding point in the optical flow field of present frame by the optical flow field vector of filtering; Utilized linear interpolation to obtain by the parallax value of the pairing point of optical flow field vector of filtering in this present frame, the parallax value that present frame is had a few is formed complete disparity map;
The absolute value that parallax value of the present invention is proportional to the optical flow field vector (promptly disparity ∝ | | V → OF | | ) obtain according to following analysis:
Motion in the frame of video is that the relative motion by video camera and real scene causes, can be divided into following four kinds of situations:
First kind of situation is: video camera is static, does not have moving object in the scene, and the optical flow field of frame is zero vector entirely.The value that makes disparity map have a few is 0, and promptly left view and right view are just the same.Satisfy disparity ∝ | | V → OF | | ;
Second kind of situation is: video camera is static, and moving object is arranged in the scene, and the image section of the motion in the optical flow field of frame has non-zero vector.
For second kind of situation, Yun Dong object often is in from the closer prospect of video camera in general, that is to say that depth value is smaller, and the speed of often in image, moving big more (no matter what direction of motion) from the near more object of video camera, therefore can be similar to and think that depth value is inversely proportional to the absolute value of light stream vector, promptly depth ∝ 1 | | V → OF | | , Be inversely proportional to depth value by parallax value again, just obtain disparity ∝ | | V → OF | | ;
The third is a camera motion, does not have moving object in the scene, and in this case, the optical flow field of frame major part is non-zero vector.
For this situation, the video camera of motion is taken static scene fully, will strictness satisfy relational expression: depth ∝ 1 | | V → OF | | (this is from intuitively being readily appreciated that, when people's eyes were looked about the environment of on every side static, the translational speed of near more place in people's the visual field was fast more) is inversely proportional to depth value by parallax value again, just obtains disparity ∝ | | V → OF | |
The 4th kind is camera motion, and moving object is arranged in the scene.
This is the most complicated a kind of situation, and camera motion not only, the object of scene the inside also have motion, in this case for static background parts disparity ∝ | | V → OF | | Hypothesis still set up (with the third situation 3), only the prospect for motion partly is false.
In sum, the present invention proposes disparity ∝ | | V → OF | | In very large range be reasonably, can realize effect preferably.
According to the above-mentioned criterion of being calculated disparity map by optical flow field: the present invention exists disparity ∝ | | V → OF | | Concrete enforcement on, determine that the rational proportion coefficient is as follows: disparity map is a gray-scale map, the scope that every bit can assignment is the maximum parallax of 0 to 255,255 expression, 0 expression does not have parallax.If setting the maximum disparity value is D Max(concrete numerical value can be set according to the stereoeffect of wanting, i.e. D MaxThe big more stereoeffect that obtains at last is just obvious more, but too big words will allow the people come over dizzy, and the present invention can get 200-230 by test, for example gets 220), and the maximum value of the vector of the optical flow field of certain frame is
Figure C200710117654D00076
Then proportionality coefficient just is k = D max | | V → OF | | max , Even the maximum of the vector of optical flow field corresponds to the maximum disparity value of wanting.
Therefore, according to above-mentioned proportionality coefficient, do not obtained the parallax value of corresponding point in the optical flow field of present frame by the optical flow field vector of filtering.
Utilized interpolation to obtain by the parallax value of the pairing point of optical flow field vector of filtering in this present frame, concrete interpolation method is: the parallax value of the parallax value Linear Estimation A of the point by the known parallax value adjacent with an A; All try to achieve as the parallax value of 8 points around the A among the embodiment, ask this parallax value of 8 arithmetic average to obtain the parallax value of A so.
The parallax value that present frame is had a few is formed complete disparity map.
3) with the synthetic new view that obtains of present frame, present frame and described disparity map, as the left and right view of present frame; The value of disparity map every bit is represented the horizontal displacement of this picture element between the view of the left and right sides, disparity map is added on the present frame, in other words each picture element of present frame is done corresponding displacement, just can obtain new view (during concrete operations, for example can be the frame of former sequence as left view, point on the left view is done corresponding displacement left according to disparity map and is just obtained right view, just newly schemes, and left and right sides view has so just been arranged).
4) with the synthetic stereo video frame that obtains of the left and right view of each frame, specific practice is to make that the value of pixel of the odd column of stereo video frame is the value of pixel of the odd column of left view correspondence, and the value of the pixel of the even column of stereo video frame is the value of pixel of the even column of right view correspondence.
All repeat above-mentioned institute in steps for each frame of sequence, until last frame.
5) all stereo video frame are formed stereoscopic video sequence.
Above-mentioned steps 3), 4), 5) concrete grammar all can adopt prior art to realize.

Claims (4)

1, a kind of method of the converting plane video into stereoscopic video based on optical flow field is characterized in that, may further comprise the steps:
1) optical flow field of given each frame of planar video sequence of calculating:
(11) to the calculating of the first frame optical flow field: first frame and second frame calculate the optical flow field vector of each picture element of the optical flow field of first frame; Utilize the improper optical flow field vector that surpasses first threshold in this optical flow field of first threshold filtering;
(12) to the calculating of second frame to the optical flow field vector of frame second from the bottom: a back frame of present frame and present frame calculates the optical flow field vector of all picture elements of optical flow field of present frame; Obtain the ratio that all optical flow field vectors of present frame surpass the optical flow field vector of this threshold value according to first threshold, again this ratio and predefined second threshold value are compared, to judge whether present frame is boundary frame, if not, then utilize the improper optical flow field vector that surpasses first threshold in this optical flow field of first threshold filtering; If then abandon described optical flow field, and recomputate optical flow field, and utilize the improper optical flow field vector that surpasses first threshold in this optical flow field of first threshold filtering by present frame and former frame;
(13) to the calculating of last frame optical flow field: last frame and former frame calculate optical flow field, and utilize the optical flow field vector that surpasses first threshold in this optical flow field of first threshold filtering;
2) be proportional to the absolute value of optical flow field vector according to parallax value, by the parallax value that is not obtained corresponding point in the optical flow field of present frame by the optical flow field vector of filtering; Utilized linear interpolation to obtain by the parallax value of the pairing point of optical flow field vector of filtering in this present frame, the parallax value that present frame is had a few is formed complete disparity map;
3) with the left view of present frame as present frame, the synthetic right view of new view that obtain of present frame and described disparity map as present frame;
4) with the synthetic stereo video frame that obtains of the left and right view of each frame;
5) all stereo video frame are formed stereoscopic video sequence.
2, the method for claim 1 is characterized in that, the span of described first threshold Tl is: 80-120.
3, the method for claim 1 is characterized in that the span of the described second threshold value T2 is: 0.4-0.6.
4, the method for claim 1 is characterized in that the proportionality coefficient that described parallax value is proportional to the optical flow field vector is: k = D max | | V → OF | | max , Wherein, D MaxBe the maximum disparity value,
Figure C200710117654C00022
Be the maximum value of the vector of optical flow field, span is: 200-230.
CNB2007101176542A 2007-06-21 2007-06-21 Method based on the converting plane video into stereoscopic video of optical flow field Expired - Fee Related CN100539710C (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CNB2007101176542A CN100539710C (en) 2007-06-21 2007-06-21 Method based on the converting plane video into stereoscopic video of optical flow field

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CNB2007101176542A CN100539710C (en) 2007-06-21 2007-06-21 Method based on the converting plane video into stereoscopic video of optical flow field

Publications (2)

Publication Number Publication Date
CN101087437A CN101087437A (en) 2007-12-12
CN100539710C true CN100539710C (en) 2009-09-09

Family

ID=38938139

Family Applications (1)

Application Number Title Priority Date Filing Date
CNB2007101176542A Expired - Fee Related CN100539710C (en) 2007-06-21 2007-06-21 Method based on the converting plane video into stereoscopic video of optical flow field

Country Status (1)

Country Link
CN (1) CN100539710C (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101917636A (en) * 2010-04-13 2010-12-15 上海易维视科技有限公司 Method and system for converting two-dimensional video of complex scene into three-dimensional video

Families Citing this family (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101271578B (en) * 2008-04-10 2010-06-02 清华大学 Depth sequence generation method of technology for converting plane video into stereo video
US9659382B2 (en) 2008-05-28 2017-05-23 Thomson Licensing System and method for depth extraction of images with forward and backward depth prediction
JP5305094B2 (en) * 2009-03-19 2013-10-02 ソニー株式会社 Video signal processing apparatus, stereoscopic video display apparatus, stereoscopic video transmission display system, and video signal processing method
CN101699512B (en) * 2009-10-30 2011-09-21 无锡景象数字技术有限公司 Depth generating method based on background difference sectional drawing and sparse optical flow method
CN102036090B (en) * 2010-12-16 2012-05-02 清华大学 Television signal conversion device for digital television terminal
CN102685517B (en) * 2011-03-09 2015-01-28 联想(北京)有限公司 Device, method and system for converting 2D image to 3D image
US8203605B1 (en) 2011-05-11 2012-06-19 Google Inc. Point-of-view object selection
CN102368824B (en) * 2011-09-16 2013-11-20 清华大学 Video stereo vision conversion method
CN102547350B (en) * 2012-02-02 2014-04-16 北京大学 Method for synthesizing virtual viewpoints based on gradient optical flow algorithm and three-dimensional display device
CN109272479B (en) * 2017-07-12 2021-11-02 天津大学 Stereo image quality evaluation method based on stereo perception characteristic
CN112132915B (en) * 2020-08-10 2022-04-26 浙江大学 Diversified dynamic time-delay video generation method based on generation countermeasure mechanism

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101917636A (en) * 2010-04-13 2010-12-15 上海易维视科技有限公司 Method and system for converting two-dimensional video of complex scene into three-dimensional video

Also Published As

Publication number Publication date
CN101087437A (en) 2007-12-12

Similar Documents

Publication Publication Date Title
CN100539710C (en) Method based on the converting plane video into stereoscopic video of optical flow field
US10715782B2 (en) 3D system including a marker mode
US9215452B2 (en) Stereoscopic video display apparatus and stereoscopic video display method
CN102170577B (en) Method and system for processing video images
US20140111627A1 (en) Multi-viewpoint image generation device and multi-viewpoint image generation method
CN101257641A (en) Method for converting plane video into stereoscopic video based on human-machine interaction
CN106504190A (en) A kind of three-dimensional video-frequency generation method based on 3D convolutional neural networks
EP4115606A1 (en) Systems and methods for end to end scene reconstruction from multiview images
AU2018249563B2 (en) System, method and software for producing virtual three dimensional images that appear to project forward of or above an electronic display
CN102547350B (en) Method for synthesizing virtual viewpoints based on gradient optical flow algorithm and three-dimensional display device
CN101840574B (en) Depth estimation method based on edge pixel features
US20100302234A1 (en) Method of establishing dof data of 3d image and system thereof
CN102098528A (en) Method and device for converting planar image into stereoscopic image
CN103428515A (en) Video format determination device, video format determination method, and video display device
CN102710949B (en) Visual sensation-based stereo video coding method
US10122987B2 (en) 3D system including additional 2D to 3D conversion
CN106231287A (en) A kind of bore hole 3D rendering method for designing strengthening Consumer's Experience
CN103414907B (en) Method capable of improving 3D displayer crosstalk
Jin et al. Validation of a new full reference metric for quality assessment of mobile 3DTV content
CN103024419A (en) Video image processing method and system
CN104661014B (en) The gap filling method that space-time combines
CN109345444A (en) The super-resolution stereo-picture construction method of depth perception enhancing
CN102567992B (en) Image matching method of occluded area
US10277880B2 (en) 3D system including rendering with variable displacement
CN105578171A (en) Image signal processing method and device

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
C14 Grant of patent or utility model
GR01 Patent grant
ASS Succession or assignment of patent right

Owner name: GUANGDONG SHENGYANG INFORMATION TECHNOLOGY INDUSTR

Free format text: FORMER OWNER: TSINGHUA UNIVERSITY

Effective date: 20130305

C41 Transfer of patent application or patent right or utility model
COR Change of bibliographic data

Free format text: CORRECT: ADDRESS; FROM: 100084 HAIDIAN, BEIJING TO: 528300 FOSHAN, GUANGDONG PROVINCE

TR01 Transfer of patent right

Effective date of registration: 20130305

Address after: 528300, first floor, C building, Shunde Creative Industry Park, 41 Fengxiang Road, Daliang District, Foshan, Guangdong, Shunde

Patentee after: Guangdong Shengyang Information Technology Industrial Co., Ltd.

Address before: 100084 Beijing City, Haidian District Tsinghua Yuan

Patentee before: Tsinghua University

C56 Change in the name or address of the patentee
CP03 Change of name, title or address

Address after: 528300, building 107, building C, creative Pioneer Park, 41 Fengxiang Road, Shunde District, Foshan, Guangdong, Daliang

Patentee after: Guangdong Shengyang Information Technology Industry Co., Ltd.

Address before: 528300, first floor, C building, Shunde Creative Industry Park, 41 Fengxiang Road, Daliang District, Foshan, Guangdong, Shunde

Patentee before: Guangdong Shengyang Information Technology Industrial Co., Ltd.

CF01 Termination of patent right due to non-payment of annual fee

Granted publication date: 20090909

Termination date: 20170621

CF01 Termination of patent right due to non-payment of annual fee