Blanking inter-frame transitions of a 3D signal转让专利
申请号 : US13377132
文献号 : US08553072B2
文献日 : 2013-10-08
发明人 : Timothy A. Tabor
申请人 : Timothy A. Tabor
摘要 :
权利要求 :
I claim:
说明书 :
The present invention is a U.S. National Stage Application corresponding to PCT Patent Application No. PCT/US2011/025262, filed Feb. 17, 2009, which claims the benefit of priority to U.S. Provisional Application No. 61/416,708, filed Nov. 23, 2010, entitled “3D VIDEO CONVERTER,” the entire contents of the above identified patent applications are incorporated by reference herein.
1. The Field of the Invention
This invention relates to systems, methods, and computer program products related to presentation of three-dimensional video content.
2. Background and Relevant Art
Three-dimensional (3D) display technology involves presenting two-dimensional images in a manner that the images appear to the human brain to be three-dimensional. The process typically involves presenting “left” image data to the left eye, and “right” image data to the right eye. When received, the brain perceives this data as a 3D image. 3D display technology generally incorporates the use of a filtering device, such as glasses, which filters displayed image data to the correct eye. Filtering devices can be passive, meaning that image data is filtered passively (e.g., by color code or by polarization), or active, meaning that the image data is filtered actively (e.g., by shuttering).
Traditional display devices, such as computer monitors, television sets, and portable display devices, typically are either incapable of producing suitable image data for 3D viewing, or produce an inferior 3D viewing experience. For instance, viewing 3D content from traditional display devices, generally results in blurry images and/or images that have “ghosting” effects, both of which may cause headache, discomfort, and even nausea in the viewer. This is true even for display devices that incorporate more recent technologies, such as Liquid Crystal Display (LCD), Plasma, Light Emitting Diode (LED), Organic Light Emitting Diode (OLED), etc.
Recently, 3D display devices designed specifically for displaying 3D content have become increasingly popular. These 3D display devices are generally used in connection with active filtering devices (e.g., shuttering glasses) to produce 3D image quality not previously available from traditional display devices. These 3D display devices, however, are relatively expensive when compared to traditional display devices.
As a result, consumers who desire to view 3D content are faced with the purchase of expensive 3D display devices, even when they may already have traditional display devices available. Accordingly, there a number of considerations to be made regarding the display of 3D content.
Implementations of the present invention solve one or more problems in the art with systems, methods, and computer program products configured to enable users to view 3D content on a broad range of display devices. When viewing 3D content using one or more implementations of the present invention, the viewer can experience a level of quality that can match or even exceed the quality of specialized 3D display devices. Accordingly, implementations of the present invention can alleviate or eliminate the need to purchase a 3D-specific display device by allowing a viewer to view 3D content on traditional display devices.
For example, one implementation can include a method of blanking an inter-frame overlap of a three-dimensional video signal. The method can involve receiving a three-dimensional video signal comprising at least a first video frame and a second video frame. The method can also involve determining a frame overlap interval in which at least a portion of the first video frame and at least a portion of the second video frame are to be displayed concurrently at a display device during a transition between the first video frame and the second video frame. Additionally, the method can involve generating an inter-frame blanking signal based on the determined frame overlap interval. The inter-frame blanking signal can comprise one or more instructions for instructing a blanking device to blank a display of the three-dimensional video signal during the transition.
In addition, a method of generating an inter-frame blanking signal can involve generating a first blanking instruction that directs a blanking device to blank a first portion of a view of a display device during at least a portion of a display of a first video frame. The method can also involve identifying a frame overlap time period in which at least a portion of the first video frame and at least a portion of a second video frame are to be displayed concurrently at the display device. Additionally, the method can involve generating a second blanking instruction corresponding to the frame overlap time period that directs the blanking device to simultaneously blank the first portion and a second portion of the view of the display device during at least a portion of the frame overlap time period. Furthermore, the method can involve generating a third blanking instruction which directs the blanking device to blank the second portion of the view of the display device during at least a portion of a display of the second video frame.
This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter.
Additional features and advantages of exemplary implementations of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by the practice of such exemplary implementations. The features and advantages of such implementations may be realized and obtained by means of the instruments and combinations particularly pointed out in the appended claims. These and other features will become more fully apparent from the following description and appended claims, or may be learned by the practice of such exemplary implementations as set forth hereinafter.
In order to describe the manner in which the above-recited and other advantages and features of the invention can be obtained, a more particular description of the invention briefly described above will be rendered by reference to specific embodiments thereof which are illustrated in the appended drawings. It should be noted that the figures are not drawn to scale, and that elements of similar structure or function are generally represented by like reference numerals for illustrative purposes throughout the figures. Understanding that these drawings depict only typical embodiments of the invention and are not therefore to be considered to be limiting of its scope, the invention will be described and explained with additional specificity and detail through the use of the accompanying drawings in which:
Implementations of the present invention extend to systems, methods, and computer program products configured to enable users to view 3D content on a broad range of display devices. When viewing 3D content using one or more implementations of the present invention, the viewer can experience a level of quality that can match or even exceed the quality of specialized 3D display devices. Accordingly, implementations of the present invention can alleviate or eliminate the need to purchase a 3D-specific display device by allowing a viewer to view 3D content on traditional display devices.
Specialized 3D display devices attempt to provide an enhanced 3D viewing experience by modifying physical characteristics of the display device. For example, specialized 3D display device may include modification that increase frame rate and reduce the frame overlap interval of the display device. The frame rate refers to the number of unique video frames the display device can render in a given amount of time (e.g., one second). Frame overlap interval refers to the period of time that elapses when transitioning between two frames. During the frame overlap interval, the display device displays at least a portion of two or more video frames concurrently. Longer frame overlap intervals are perceptible to the human eye, and can lead to a degraded viewing experience. For example, longer frame overlap intervals can cause motion blurring or ghosting. These effects are a particular problem when viewing 3D video content.
One or more implementations of the present invention provide for the blanking of some or all of the frame overlap interval from the user's view. This involves processing a 3D video signal, determining frame overlap interval information, and generating an inter-frame blanking signal. The inter-frame blanking signal can instruct a blanking device (e.g., shuttering glasses) to synchronously blank portions of the user's view while the user is viewing displayed 3D content. Thus, one or more implementations allow for viewing of 3D content on a broad range of display devices, including devices that that may have lower frame rates and longer frame overlap intervals.
For example,
The shuttering device 116 can, in one or more implementations, comprise stereoscopic shuttering glasses that include a plurality of shuttering components 118, 120. The shuttering components 118, 120 can comprise one or more liquid crystal layers. In one or more implementations, the shuttering components correspond to lenses or portions of lenses on the shuttering glasses 116. The liquid crystal layers can have the property of becoming opaque (or substantially opaque) when voltage is applied (or, alternatively, when voltage is removed). Otherwise, the liquid crystal layers can have the property of being transparent (or substantially transparent) when voltage is removed (or, alternatively, when voltage is applied). As discussed more fully herein after, a blanking signal can control the shuttering glasses 116.
Similarly, in state 106, the display device 108 can display a second video frame 114. As illustrated, the second video frame 114 can correspond to 3D video content intended for view by the viewer's right eye. Thus, in this instance, the shuttering device 116 can use a shuttering component 118 to blank the portion of the viewer's view of the display device 108 corresponding to the viewer's left eye.
One will appreciate that states 102 and 106 are not limited to displaying left and right frames in the manner illustrated. For instance, in state 102, the display device 108 can display right eye content, and the shuttering device 116 can use the shuttering component 118 to blank the viewer's left eye. Furthermore, in state 106, the display device 108 can display left eye content, and the shuttering device 116 can use the shuttering component 120 blank the viewer's right eye.
State 104 illustrates a portion of a frame overlap interval in which the display device 108 is transitioning between displaying two video frames. For instance, the display device 108 might be transitioning from display of the video frame 110 to the video frame 114 or from the video frame 114 to the video frame 110. Thus, in one or more implementations, the display device 108 displays an inter-frame overlap 112, in which the display device 108 concurrently displays at least a portion of at least two different video frames (e.g., video frame 110 and video frame 114). In this instance, the shuttering device 116 can concurrently use both shuttering components 118 and 120 to blank portions of the viewer's view of the display device 108 corresponding to both eyes. By blanking both eyes, the shuttering device 116 can prevent the user from viewing all or part of the inter-frame overlap 112 during all or part of the frame overlap interval.
One will appreciate that the appropriate shuttering of a single eye, as in states 102 and 106, when combined with the synchronous display of appropriate 3D video content, can provide the illusion that two-dimensional images are actually three-dimensional. Furthermore, inter-frame blanking, or the synchronous shuttering of both eyes during frame overlap intervals, can enhance the clarity of the perceived 3D image by eliminating undesirable effects such as motion blurring and ghosting. Thus, the disclosed inter-frame blanking techniques can allow for viewing of 3D content on display devices that may have lower frame rates and/or longer frame overlap intervals.
Included in
One will appreciate that while the displayed video content 202 comprises a series of alternating left and right video frames (in any order), one or more implementations extend to any sequence of video frames. In one implementation, for example, the displayed video content 202 comprises differing sequences of left and right video frames (e.g., left, left, right, right). In another implementation, the displayed video content 202 comprises only video frames intended for viewing with both eyes. In yet another implementation, the displayed video content 202 comprises a combination of different video frame types. One combination, for instance, can include both video frames intended for viewing with both eyes, as well as video frames intended for viewing with a single eye.
In the illustrated embodiment, the inter-frame blanking signal 204 can include a blanking instruction 230 (blank right) which instructs the blanking device to blank the right eye view of the displayed video content 202. This instruction may correspond with the displayed left video frame 220. Additionally, the inter-frame blanking signal 204 can include a blanking instruction 224 (blank left) which instructs the blanking device to blank the right eye view of the displayed video content 202. This instruction may correspond with the displayed right video frame 224.
Furthermore, the inter-frame blanking signal 204 can include a blanking instruction 232 (blank both) which instructs the blanking device to blank both the right and left eye views of the displayed video content 202. This instruction may correspond with the displayed inter-frame overlap 222 during time interval 208 (a frame overlap interval). Of course, the inter-frame blanking signal 204 can also include other blanking instructions, such as blanking instructions 236 and 238 corresponding to time intervals 212 and 214. Also, as indicated herein above, blanking instructions 230-238 can blank all the time in their corresponding time intervals, a portion of the time in their corresponding time intervals, or more time than is in their corresponding time intervals.
One will appreciate in light of the disclosure herein that the inter-frame blanking signal 204 can include any appropriate sequence of blanking instructions that correspond to the displayed video content 202. For instance, if displayed video content 202 includes a different sequence of left and right video frames, the inter-frame blanking signal 204 can include an appropriate different sequence of blanking instructions. Furthermore, the inter-frame blanking signal 204 can depart from the illustrated implementations. For example, inter-frame blanking signal 204 can refrain from blanking during one or more time intervals corresponding to a transition. Furthermore, inter-frame blanking signal 204 can include any number of other blanking instructions, such as blanking instructions that do no blanking (e.g., when displaying a video frame intended for viewing with both eyes).
In at least one implementation, generation of the inter-frame blanking signal 204 occurs prior to the display of video content. In such an implementation, a processing device processes a 3D video signal to determine inter-frame overlap information. The processing device then generates the inter-frame blanking signal 204 based on the determined inter-frame overlap information. The processing device can then send the inter-frame blanking signal 204 to a blanking device. The blanking device can receive the inter-frame blanking signal 204 prior to, or concurrently with, the display of the displayed video content 202.
Illustratively, processing the 3D video signal can include processing the 3D video signal in view of physical characteristics or parameters of an intended display device. For instance, processing the 3D video signal can include determining a frame size of video frames of a particular 3D video signal (e.g., 720 interlaced or progressive vertical lines, 1080 interlaced or progressive vertical lines, etc.). Furthermore, processing the 3D video signal can include determining the frame rate and response time of the intended display device. In addition, processing the 3D video signal can include using this information to calculate the inter-frame overlap interval, and the timing of the displayed video content on the intended display device.
A display device may have a frame rate of 60 Hz, meaning that it displays sixty video frames per second. This display device may also have a frame overlap interval of 25 milliseconds (ms), meaning that inter-frame overlap occurs for 25 ms when transitioning between two video frames (e.g., left video frame 220 and right video frame 210). Further, the frame size may be 720p, meaning that each video frame includes 720 lines of vertical data. Of course, these parameters are merely illustrative and are not limiting.
Based on the foregoing information, the processing device can generate a proper sequence and timing of blanking instructions for the particular 3D video signal and the exemplary display device. These instructions can include a blanking instruction (e.g., blank left or blank right) for blanking the appropriate eye while the display device will display a left or right video frame individually. After the display device will have drawn the 720th line of the individual video frame, the instructions can also include a blanking instruction (e.g., blank both) for blanking both eyes during the transition to the next video frame. In this instance, the blanking instruction may instruct the blanking device to blank both eyes for a period of 25 ms. Then, the blanking instructions can instruct the blanking device to blank the other eye while the display device will display another individual video frame.
Turning now to
In at least one implementation, the video processing device 302 receives a 3D video signal from a media device. The media device can comprise any number of devices capable of transmitting a 3D video signal. For example,
In any event, the video processing device 302 can also generate an inter-frame blanking signal (e.g., inter-frame blanking signal 204). The video processing device 302 can send the 3D video signal, either in its original form or in a modified form, to the display device 306 for display to one or more users (e.g., as displayed video content 202). Prior to or concurrently with sending the 3D video signal, the video processing device 302 can also send the generated inter-frame blanking signal to the blanking device(s) 304.
The video processing device 302 can communicate with the display device 306 and the blanking device(s) 304 in any appropriate manner. For instance, an appropriate wired mechanism, such as High Definition Media Interface (HDMI), component, composite, coaxial, network, optical, and the like can couple the video processing device 302 and the display device together. Additionally, or alternatively, an appropriate wireless mechanism, such as BLUETOOTH, Wi-Fi, etc., can couple he video processing device 302 and the display device 306 together. Likewise, any appropriate wired or wireless mechanism (e.g., BLUETOOTH, infrared, etc.) can couple the video processing device 302 and the blanking device(s) 304 together.
The display device 306 can be one of a broad range of display devices, including display devices that have relatively low frame rates and/or relatively long frame overlap intervals. The display device 306 can also be a display device exhibiting relatively high frame rates and/or relatively short frame overlaps. Of course, the display device 306 can exhibit many combinations of frame rates and frame overlap intervals. The display device 306 can even be a specialized 3D display device.
The one or more blanking devices 304 can be any blanking device configured to interoperate with video processing device 302, and respond to instructions received via an inter-frame blanking signal. Blanking device(s) 304 can be a single device or a plurality of devices (e.g., when there is a plurality of users). In at least one implementation, the blanking device(s) 304 comprises shuttering glasses that include one or more shuttering components that selectively block a user's view of the display device 306. Generally, the blanking device(s) 304 is capable of selectively blanking a left eye view, a right eye view, and a view from both eyes. Of course, the blanking device(s) 304 can also refrain from blanking any part of the user's view.
The video processing device 302 can communicate the inter-frame blanking signal to the blanking device(s) 304 in any number of formats, including both analog and digital signals. For instance, when communicating using analog signals, blanking instructions can comprise various frequencies and/or amplitudes of light, electromagnetic waves, sound, etc. Alternatively, when communicating using digital signals, blanking instructions can comprise data packets. Of course, the video processing device 302 and the blanking device(s) 304 can communicate using any combination of analog and digital signals. In one or more implementations, the communication is uni-directional (i.e., from the video processing device 302 to the blanking device(s) 304), while in other embodiments the communication is bi-directional.
The video receiver component 402 can receive a 3D video signal from any appropriate source, such as a media device. The video transmitter component 404 can send the 3D video signal to the display device 306, either in its original form or in a modified form. One will appreciate that these components can be combined as a single component, or can even be eliminated altogether (e.g., when the video processing device 302 is integrated with the display device 306). The video receiver component 402 can receive a plurality of 3D video signal formats, and the video transmitter component 404 can transmit a plurality of 3D video signal formats, including a universal 3D video signal format.
The processing component 406 can process the received 3D video signal in any appropriate manner, regardless of the original received 3D video signal format. In particular, the processing component 406 can determine frame overlap interval information of the received 3D video signal and generate an inter-frame blanking signal for a particular display device. The processing component 406 can also convert or modify the received 3D video signal. In some circumstances, for example, the processing component 406 can convert the received 3D video signal into universal 3D video signal format. One will appreciate that processing component 406 can include any number constituent components.
In one or more implementations, the processing component 406 can convert the received 3D video signal into a universal 3D video signal format by decoding the received 3D video signal into two frame buffers. This can include decoding right eye image data into one frame buffer and decoding left eye image data into the other frame buffer. By decoding image data into two frame buffers, the image data is readily available for encoding into the universal 3D video signal format. One will appreciate that the 3D video signal can encode the image data using various techniques, and that decoding can include constructing full image data in from a plurality of video frames.
Inter-frame blanking signal transmitter 408 can transmit the generated inter-frame blanking signal to one or more blanking devices (e.g., blanking device(s) 304). As discussed herein above, the blanking signal transmitter 408 can transmit wirelessly (e.g., Bluetooth or infrared) or with a wired connection. Also discussed herein above, the blanking signal transmitter 408 can employ any number of protocols, analog or digital. In some circumstances, the blanking signal transmitter 408 is incorporated with the video processing device 302 or even with the display device 306, while in other instances the blanking signal transmitter 408 is a separate device (e.g., a separate USB device).
One will appreciate that the video processing device 302 can take any of a variety of forms. As indicated, the video processing device 302 can, in one or more implementations, be integrated with the display device 306. In other implementations, the video processing device 302 can be a set-top box or other customized computing system. In yet other implementations, the video processing device 302 can be a general-purpose computing system (e.g., a laptop computer, a desktop computer, a tablet computer, etc.), or a special purpose computing system (e.g., a gaming console, a set-top box, etc.) that has been adapted to implement one or more disclosed features.
Accordingly,
Additionally, implementations of the present invention can also be described in terms of flowcharts comprising one or more acts in a method for accomplishing a particular result. Along these lines,
For example,
In addition,
In addition to the foregoing,
For example, the video processing device 302 can generate an inter-frame blanking signal 204 that includes a blanking instruction 230 which instructs the blanking device(s) 304 to blank the user's right eye view during display a left video frame 220. Alternatively, the video processing device 302 can generate an inter-frame blanking signal 204 including other blanking instructions. For instance, the video processing device 302 can generate a blanking instruction which instructs the blanking device(s) 304 to blank the user's left eye view during display a right video frame. Furthermore, the video processing device 302 can generate can generate an inter-frame blanking signal 204 that also includes a blanking instruction that instructs the blanking device(s) 304 to refrain from blanking any eye view during display a video frame intended for view by both eyes.
In addition,
Furthermore,
Alternatively, the video processing device 302 can generate an inter-frame blanking signal 204 including other blanking instructions. For instance, the video processing device 302 can generate an inter-frame blanking signal 204 that instructs the blanking device(s) 304 to blank the user's right eye view during display a left video frame. Furthermore, inter-frame blanking signal 204 could include a blanking instruction which instructs the blanking device(s) 304 to refrain from blanking any eye view during display a video frame intended for view by both eyes.
Accordingly,
The implementations of the present invention can comprise a special purpose or general-purpose computing systems. Computing systems may, for example, be handheld devices, appliances, laptop computers, desktop computers, mainframes, distributed computing systems, or even devices that have not conventionally considered a computing system, such as DVD players, Blu-Ray Players, gaming systems, and video converters. In this description and in the claims, the term “computing system” is defined broadly as including any device or system (or combination thereof) that includes at least one physical and tangible processor, and a physical and tangible memory capable of having thereon computer-executable instructions that may be executed by the processor.
The memory may take any form and may depend on the nature and form of the computing system. A computing system may be distributed over a network environment and may include multiple constituent computing systems. In its most basic configuration, a computing system typically includes at least one processing unit and memory. The memory may be physical system memory, which may be volatile, non-volatile, or some combination of the two. The term “memory” may also be used herein to refer to non-volatile mass storage such as physical storage media. If the computing system is distributed, the processing, memory and/or storage capability may be distributed as well. As used herein, the term “module” or “component” can refer to software objects or routines that execute on the computing system. The different components, modules, engines, and services described herein may be implemented as objects or processes that execute on the computing system (e.g., as separate threads).
Implementations of the present invention may comprise or utilize a special purpose or general-purpose computer including computer hardware, such as, for example, one or more processors and system memory, as discussed in greater detail below. Embodiments within the scope of the present invention also include physical and other computer-readable media for carrying or storing computer-executable instructions and/or data structures. Such computer-readable media can be any available media that can be accessed by a general purpose or special purpose computer system. Computer-readable media that store computer-executable instructions are physical storage media. Computer-readable media that carry computer-executable instructions are transmission media. Thus, by way of example, and not limitation, embodiments of the invention can comprise at least two distinctly different kinds of computer-readable media: computer storage media and transmission media.
Computer storage media includes RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store desired program code means in the form of computer-executable instructions or data structures and which can be accessed by a general purpose or special purpose computer.
A “network” is defined as one or more data links that enable the transport of electronic data between computer systems and/or modules and/or other electronic devices. When information is transferred or provided over a network or another communications connection (either hardwired, wireless, or a combination of hardwired or wireless) to a computer, the computer properly views the connection as a transmission medium. Transmissions media can include a network and/or data links which can be used to carry or desired program code means in the form of computer-executable instructions or data structures and which can be accessed by a general purpose or special purpose computer. Combinations of the above should also be included within the scope of computer-readable media.
Further, upon reaching various computer system components, program code means in the form of computer-executable instructions or data structures can be transferred automatically from transmission media to computer storage media (or vice versa). For example, computer-executable instructions or data structures received over a network or data link can be buffered in RAM within a network interface module (e.g., a “NIC”), and then eventually transferred to computer system RAM and/or to less volatile computer storage media at a computer system. Thus, it should be understood that computer storage media can be included in computer system components that also (or even primarily) utilize transmission media.
Computer-executable instructions comprise, for example, instructions and data which, when executed at a processor, cause a general purpose computer, special purpose computer, or special purpose processing device to perform a certain function or group of functions. The computer executable instructions may be, for example, binaries, intermediate format instructions such as assembly language, or even source code. Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the described features or acts described above. Rather, the described features and acts are disclosed as example forms of implementing the claims.
Those skilled in the art will appreciate that the invention may be practiced in network computing environments with many types of computer system configurations, including, personal computers, desktop computers, laptop computers, message processors, hand-held devices, multi-processor systems, microprocessor-based or programmable consumer electronics, network PCs, minicomputers, mainframe computers, mobile telephones, PDAs, pagers, routers, switches, and the like. The invention may also be practiced in distributed system environments where local and remote computer systems, which are linked (either by hardwired data links, wireless data links, or by a combination of hardwired and wireless data links) through a network, both perform tasks. In a distributed system environment, program modules may be located in both local and remote memory storage devices.
The present invention may be embodied in other specific forms without departing from its spirit or essential characteristics. The described embodiments are to be considered in all respects only as illustrative and not restrictive. The scope of the invention is, therefore, indicated by the appended claims rather than by the foregoing description. All changes which come within the meaning and range of equivalency of the claims are to be embraced within their scope.