WO2007057053A1 - Conditional updating of image data in a memory buffer - Google Patents
Conditional updating of image data in a memory buffer Download PDFInfo
- Publication number
- WO2007057053A1 WO2007057053A1 PCT/EP2005/056116 EP2005056116W WO2007057053A1 WO 2007057053 A1 WO2007057053 A1 WO 2007057053A1 EP 2005056116 W EP2005056116 W EP 2005056116W WO 2007057053 A1 WO2007057053 A1 WO 2007057053A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- data
- image data
- pixels
- memory buffer
- data values
- Prior art date
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
- G06F3/1454—Digital output to display device ; Cooperation and interconnection of the display device with other functional units involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
- G09G2310/00—Command of the display device
- G09G2310/04—Partial updating of the display screen
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
- G09G2340/00—Aspects of display data processing
- G09G2340/02—Handling of images in compressed format, e.g. JPEG, MPEG
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
- G09G5/00—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators
- G09G5/36—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators characterised by the display of a graphic pattern, e.g. using an all-points-addressable [APA] memory
- G09G5/39—Control of the bit-mapped memory
- G09G5/393—Arrangements for updating the contents of the bit-mapped memory
Definitions
- the present invention relates to buffering image data.
- Remote visualization means that an image to be displayed locally is copied or completely redirected to a remote data processing device, which is different from the local data processing device.
- Remote visualization allows, among others, remote system maintenance, e.g., by a hardware or software vendor, testing or performing helpdesk functions for end users, etc.
- Visualization may be realized by providing a graphics device, which transforms a digital representation of an image provided by, e.g., a central processing unit (CPU) of a local data processing device into signals which lead a monitor to display that image.
- the signals may be analog, e.g., in the case of CRT monitors, or digital as often in the case of LCD-displays.
- a redirection of graphic information may be performed by adding a separate capture device to the system that captures the image contents and transfers it to a predetermined remote system.
- the graphics device and the capture device may communicate with each other, e.g., via an internal bus of the respective local processing device.
- the graphics device uses a memory unit. This memory unit contains the data of the digital representation of the image to be displayed on a local monitor. In order to make these data remotely available, the data are first captured by a processing unit provided with the capture device, typically a card which is input as an additional component into a slot of the local computer.
- the image data are then stored into a memory buffer, which is also provided with the capture device.
- This memory buffer relates to a dedicated capture memory, which is - among others - needed to take into account the reduced bandwidth of a LAN connection.
- the memory buffer thus receives a copy of an image and provides an opportunity to perform compression on the image data.
- the image data in other words: the digital representation of an image, may be captured in different modes: one active mode is to address the graphics device and then simply copy the digital data from the video memory via a bus (e.g., PCI) into the memory buffer of the capture device. Another passive mode is to monitor the information when being transferred from the video memory of the graphics device to the local graphics display (local monitor). In case of analog VGA-signals, an additional A/D converter is then required to yield a transformation into a digital representation of the image. Digital DVI-signals, e.g., may be recorded directly.
- a bus e.g., PCI
- Another passive mode is to monitor the information when being transferred from the video memory of the graphics device to the local graphics display (local monitor).
- an additional A/D converter is then required to yield a transformation into a digital representation of the image.
- Digital DVI-signals e.g., may be recorded directly.
- Compression may be performed by reducing the information available per pixel of an image (data values associated with a pixel), e.g., the bit length or number of colours. It is also possible to apply reduction schemes such as those known from the JPEG or MPEG-formats, wherein data values of adjacent pixels are related to each other.
- a method of performing visualization of image data on a remote processing device includes steps of capturing data of a next image and providing the next image data to a processing unit.
- the capturing of next image data implies that previous image data are already present, e.g., in the capture device.
- the processing unit may thereby be located on the capture device, which is dedicated to perform the remote transfer.
- the capture device which performs the method steps, houses a video processing unit (VPU) as well as a central processing unit (CPU). Both may be part of a single device, i.e., a remote transfer controller.
- VPU video processing unit
- CPU central processing unit
- the processing unit being provided with captured next image data represents the CPU, while the next image data are captured by the VPU.
- an additional A/D converter may be placed upstream of the VPU.
- the invention is not limited to the specific implementation of different or unique processing units and associated memories on a dedicated capture device.
- Image data as referred to herein are digital representations of images, and comprise an arrangement of pixels, which are each associated with a data value, which represents, e.g., a colour.
- a pixel is in this respect a predefined coordinate position within an array of data, and further has a predefined length of bits. The data values are taken from a combination of the specific binary values currently associated with these bits.
- the processing unit selects a first subset of pixels of the next image data.
- the subset may according to an embodiment represent an arrangement of pixels organized in rows and columns.
- First data values, which are associated with these pixels of the subset, are then determined. This determination may comprise the step of providing the bit values, which define a respective data value of a pixel, into a dedicated position within a register of the processing unit.
- second data values are read out from a memory buffer and into the processing unit, e.g., into a similarly dedicated further position in said register.
- the second data values are associated with pixels of data of a previous image, wherein said pixels of the previous image data correspond to said pixels of the new image data.
- corresponding means, that the pixel location of a respective first data value within the next image data is the same as that of a second data value within the previous image data.
- the memory buffer which keeps the previous image data, is according to one embodiment a buffer located on the capture device. In an alternate embodiment, it relates to the graphics, or video memory located on the graphics device.
- the invention is not limited to the specific arrangement of this buffer with respect to the processing unit. However, a direct association of the memory buffer with the processing unit, i.e., a short traveling distance with respect to signal is preferred as the loading from and the writing to this buffer turns out to represent a bottleneck for the bandwidth of data transfer.
- a data value of the first image data is compared with a respective one of the second image data for each of the pixels in the subset. This means that a pixel- wise check is performed, whether the data values have changed or not. For example, both data values may be selected from the respective register positions and directly compared.
- the result of this comparison step performed for each of the pixels is stored by setting a flag bit.
- a flag bit there is one separate flag bit associated with each subset. For example, a logical "1" (flag bit set) indicates that a change has occurred, while an initial value of "0" (flag bit not set) indicates no change has occurred, yet.
- the parallel provision of first data values and reading of second data values is terminated.
- the incoming first data values are directly written into the memory buffer to overwrite the respective data values of the first image data, therein.
- at least the currently compared pixel data values which have lead to the detection of a data value change, are written to the memory buffer for updating the previous image. Pixel by pixel, and subset by subset, the "previous" image data are thus altered into the next image data.
- the direct write mode of first data values of the next image data into the memory buffer means that the reading step of second data values into the processing unit is effectively stopped. This stop of data transfer from the memory buffer to the processing unit is limited to the presently selected subset of pixels.
- each different subset of pixels, which is selected from the next image data is associated with its own flag bit.
- the steps relating to one subset might have already lead to the event of a direct write of first data values
- the corresponding steps relating to another subset may include a full read out of second values accompanied with a comparison with first data values for all pixels in the respective subset.
- the termination of the read step may considerably reduce the time needed to perform the memory update using the next image data.
- each subset which may also referred to as a block of pixels within image data
- the flag bits may be stored in a change matrix, which indicates the occurrence of changes in the respective blocks of image data with respect to a previous image.
- This change matrix is used to control the transfer to the remote processing device. Only those blocks represented by an associated flag bit, which is set (thus attaining for example a logical "1"), are transferred according to this embodiment.
- the current settings in the change matrix which may be stored in a buffer or register associated with the processing unit, correspond to the actually updated image in the memory buffer.
- time is reduced by only reading out (second) data values from the memory buffer until a change is detected, generating an event and stopping further read out of data values in response to this event. It is found that once compression is performed by means of a change matrix, which reduces traffic via LAN for example, the data transfer from the memory buffer to the processing unit for the purpose of comparison with new image data becomes a dominant contributor to the bandwidth.
- the bandwidth is defined here as data throughput per time unit.
- the effect of the invention thus is that data values are either read out from the memory buffer into the processing unit, or are written from the memory unit into the memory buffer. Consequently, only one transfer bandwidth of images (and not two) between the memory buffer and the processing unit is needed - with the exception of those one or few data values of a subset, which have lead to the detection of an change event, which were read out, compared and written.
- embodiments of the invention can be partly or entirely embodied or supported by one or more suitable software programs, which can be stored on or otherwise provided by any kind of data carrier, and which might be executed in or by any suitable data processing unit.
- Software programs or routines can be preferably applied with respect to operating the kernel functions of the processing unit on a capture device (of a remote management unit, RMU), e.g., a remote controller. The same is valid if the remote controller is located within a graphics device having direct access to the video memory.
- FIGs.1 and 2 show block diagrams of a remote system connected to a local system for the purpose of remote image display according to prior art
- FIG. 3 illustrate the flow of data through a capture device according to prior art
- FIG. 4 illustrate a flow of data through capture device including a CPU, a VPU, a memory buffer and a remote interface according to a first embodiment of the invention
- FIG. 4B illustrate a flow of data through capture device including a CPU, a VPU, a memory buffer, a remote interface, a compression device and a decompression device according to a further embodiment of the invention.
- Fig. 5 shows a schematic representation of an image divided into subsets, an associated change matrix and a counter table for flags within the change matrix.
- FIG. 1 shows an embodiment of a remote processing device (computer) 10 connected to a local processing device (computer, or server) for the purpose of remote visualization of an image, the arrangement being known in the art.
- the remote processing device 10 is connected with a remote display 12.
- a CPU 20 of the local processing device (right hand side of the sinuous line) is connected to an internal bus 25 such as a PCI-bus or PCI-Express bus.
- a graphics device 52 is also connected to the internal bus.
- the graphics device comprises a video memory 62 for storing a digital representation of the image data to be displayed on a local monitor 22.
- the display of an image is managed by means of a display controller 37 arranged on the graphics device 52. Further components 21a, 21 b of the local system may also be connected to the internal bus 25.
- a connection of the local processing device with the remote processing device 10 is established by the capture device 30.
- the capture device 30 is also connected to the internal bus 25 and comprises a capture memory 32 and a remote interface 33. Capturing of image data is accomplished by addressing - and accessing - the video memory 62 via the internal bus 25 with help of a capture device controller 35 (remote transfer controller, not shown in Fig. 1 ) and retrieving full image data.
- the full image data are buffered in the capture memory 32, compressed and forwarded to the remote data processing unit via remote interface 33.
- the flow of redirected image data is indicated by the dashed line in the figures.
- An example flow of image data generated in the CPU 20 intended for local display is indicated by the dot-dashed line
- Fig. 2 shows an alternate embodiment known in the art.
- the capture device is arranged to retrieve image data from a connection between the display controller 37 and the local monitor 22.
- the data transferred herein are less redirected than snarfed, i.e., duplicated for local and remote display.
- the format of transferred data considerably differs between the embodiments of Figs.1 and 2.
- the embodiment shown in Fig. 2 may involve the snarfing of analog VGA signals, which are to be converted into digital data representing the image by an A/D converter 29, first.
- Fig. 3 shows a capture device 30, which may be embodied as shown in Fig. 2, in more detail.
- the remote transfer controller 35 in this embodiment comprises a video processing unit 40 (VPU) and a central processing unit 41 (CPU).
- the VPU 40 is designed to capture image data and to process the incoming data. In particular it performs bit- or pixel-wise comparison of image data in order to detect changes. It further arranges the reading from and writing to the capture memory 32.
- the CPU 41 of the capture device 30 controls the flow of image data, for example it initiates the capture of image data and organizes the compression of data to be transferred (numeral 70 in Fig. 3) via remote interface 33.
- the capture memory 32 which may be, e.g., one DRAM memory chip, comprises three or more memory buffers 45, 46, 47.
- a conventionally known flow of image data is described as follows: as indicated by the arrows in Fig. 3, data of a first captured image are processed by VPU 40, and stored as "previous" image data in a first memory buffer 45. Then, a next image is captured, processed and stored as next image data in the second memory buffer 46.
- Both image data - the previous and the next - are then retrieved from the respective memory buffers 45, 46 by CPU 41.
- Data values corresponding to pixels from both images, which are associated with each other, are compared to detect a change.
- a difference image is thus formed, which is stored in the third buffer 47.
- the capture and storage of one image takes 10 - 20 ms of time.
- the calculation of the difference image lasts for 50 - 200 ms depending on the image size.
- the transferal of the compressed image via remote interface 33 (e.g., LAN) takes another 20 ms.
- One image update at the remote location thus amounts to 80 - 240 ms according to this illustrative embodiment.
- Roughly 5 - 10 frames per second may thus be realized at the remote location, which may be sufficient to indicate the addition of text on the remote monitor 12.
- changes such as smooth mouse movements developing to abrupt jumps thereby are displeasingly followed by an observer according to this two-pass-dataflow.
- FIG.4A An embodiment of the invention is shown in Fig.4A.
- Next image data are converted into digital data by A/D converter 29, and in case of a first image are directly written into memory buffer 49 by means of VPU 40.
- These initially stored image data will then be referred to as "previous" image data, since a further image is captured by VPU 40.
- Being digitized, the next image data are transferred successively to the VPU 40 as a contiguous stream of captured digital data 82.
- VPU 40 The read out of data from the memory buffer is coordinated with the incoming data from the A/D converter 29.
- digital data of the next image form bytes or words, which contain data values with respect to pixels of the next image.
- What is read out from the memory buffer are digital data corresponding to respective pixel positions of the previous image.
- the data values formed from the digital data are then compared in VPU 40 in order to detect a change or not.
- the images (previous and next) are subdivided into the same subsets of pixels.
- an image being formed from 1024 (rows) * 768 (columns) pixels may contain subsets, or blocks, formed from 16 * 16 pixels. Accordingly, one such image has 64 * 48 blocks. The event of detecting a change is handled with respect to each subset, or block, separately.
- a change may be signaled when the data values of corresponding pixels in the previous and next image are not the same, i.e., they differ.
- a relaxed condition may be set up, stating that the difference between these data values must exceed a predetermined threshold level in order to represent a change.
- This latter condition represents an advantageous aspect, as it enables further compression of data to be transmitted to the remote location. Small fluctuations of, e.g., colours from image to image will herein be disregarded from an assessment as a change.
- a change is tracked for each subset separately. Until a change is detected with regard to one subset, digital data 82 are captured and digital data 83 are read into the VPU 40. Once a change is detected, the data transfer mode is also changed. For this specific subset, the read out of digital data 84 from the memory buffer 49 is stopped along with the comparison step. Rather, the captured digital data 82, or data values corresponding to respective pixels of the subset, are directly written as digital data 83 into the memory buffer 49. More precisely, buffer locations in memory buffer 49, which keep digital data related to associated pixels of the previous image, are overwritten with the digital data 83 of the next image, regardless of whether they represent a further change or not.
- the communication between the memory buffer 49 and the VPU merely comprises either reading digital data 84 or writing digital data 83. Only, those specific data values, which correspond to the event of a detected change, are read in to VPU 40 as "aged" digital data 84 as well as (re-)written as updated digital data 83 into the memory buffer 83. If only one data value at a time is read in and compared in the VPU 40, there is consequently one pixel out of 256 (16 times 16 in a block according to this embodiment), which necessitates twice a bandwidth.
- the present embodiment essentially refers to a one-pass procedure of updating image data.
- a further advantage of this embodiment is that memory is saved as the memory buffer is simply updated by the VPU 40. As a result, less bandwidth and slower as well as cheaper memory chips are necessary.
- the embodiment detailed above may further be refined by means of a change matrix 75.
- the detection of change may be realized by setting a flag bit, e.g., from logical "0" to "1" or vice versa.
- the resulting 64 * 48 flag bits may be arranged in a corresponding change matrix. This change matrix indicates those subsets of an image in which changes have taken place.
- the change matrix may be stored in memory buffer 49, in a separate storage area, or in registers of the VPU 40 or CPU 41.
- the latter CPU 41 also controls compression as explained above.
- the change matrix 75 which is each time updated according to the present method, may therefore easily be exploited for the purpose of compression, e.g., transferring only digital data from changed blocks to a remote location (along with the information, which block the digital data actually refer to).
- Particulars of a change matrix for the purpose of data compression are disclosed in the Patent EP 1168154 B1 to Agilent Technologies, Inc., the specific features of which may advantageously be combined with the invention and are thus incorporated herein by reference.
- the task of remote data transfer is facilitated by CPU 41 via communication lines 72 to the VPU 40 and 71 to the remote interface 33, e.g., a LAN-, or WAN- interface, wherein CPU 41 initiates interface 33 to retrieve data from memory buffer 49 in dependence of entries (flag bits) within change matrix.
- the remote interface 33 e.g., a LAN-, or WAN- interface
- digital data of an image 88 are captured, or read in by the VPU 40 successively pixel by pixel 90 and row by row 92.
- flag bits of different subsets 94 may be maintained alternately until a final row of respective subsets is reached. Maintenance of multiple flag bits within a change matrix 75 becomes particularly advantageous herein, when data from one subset are not contiguously arranged in the incoming data stream from A/D converter 29, or a DVI interface, etc.
- Each flag bit 96 in the change matrix 75 is associated with a counter 98 in the counter table 77.
- the counter 98 helps to impede aging of subsets 94, which are repeatedly not updated according to the present method.
- a threshold value of 25, as an arbitrary example value may be predefined, and with each run through an image capture and update, the counter of a flag bit 96, or subset 94, is increased by 1 , as an example. If the number of 25 is reached for a subset, that has no changed pixel value, the flag bit is set and the update is performed for this subset 94 irrespective of whether a change presently occurs or not.
- This approach with counters may be advantageous as in the case, that similar but no identical data values between adjacent images are admitted, a slight drift in data values may occur leading to an inconsistency in image data.
- admittance no setting of a flag bit
- An advantageous embodiment refers to an application, wherein additional screens are slowly faded in to or out of the image, respectively.
- a further refinement may be made by compressing the digital data 83 already when being written from the VPU 40 to the memory buffer 49, and decompressing the digital data 84, when being read out from the memory buffer 49 into the VPU 40.
- a so-called run length encoding RLE
- data values of consecutive pixels having the same level may be stored as this level added by the count of consecutive pixels, for which this condition holds.
- a data value range (or similar colour range) is provided, which extends the run length as explained above to those pixels, which merely have similar - but not necessarily the same - data values, or colours respectively.
- An approximate data value is chosen instead and attributed to each of the pixels involved, along with the pixel count, which reduces the necessary memory space to store the image information.
- the approximate data value may be a mean value of the data values involved, a median value or the most frequent one of the data values, etc.
- the value range may further be adapted (e.g., shifted) to include subsequently read in data values, until one subsequent data value inevitably falls out of this range as the difference to a smallest or largest one becomes too large.
- a counter is reset, e.g. to zero, the previous count and the approximate data value are stored, and a new value range is selected for further subsequently read in data values.
- This method of compression of data in-between one image may be realized by an appropriate software implemented with regard to the VPU 40, CPU 41 or another controller 35, which could replace both processing units 40, 41 shown in this embodiment.
- Reference numerals 85a, 85b indicate those compression units or decompression units, respectively, arranged in hardware or software.
- a decision unit and a value range adjusting unit such as that shown in Fig. 1 of the EP application (reference numerals 14, 18 therein) may be realized as ASIC's (application specific integrated circuits) between the processing units 40, or 41 , and memory buffer 49 according to the present embodiment.
- a still further embodiment relates to the memory buffer 49.
- the buffer 49 into the graphics memory of a graphics device.
- the VPU may "capture" (select) data from a first frame buffer in the memory and compare these data with those of a second frame buffer, which represents the buffer 49 shown in Fig. 4. Again, time for capture and memory space is saved to considerable amounts.
- Examples of advantageous buffer management configurations are provided in the European patent application EP05108275.8 of Agilent Technologies, Inc, the specific features of which, particularly shown in Figs. 3, 4, 5, and 7 and corresponding description therein may be combined with the present invention and are thus incorporated herein by reference.
Abstract
The invention refers to updating memory buffer with image data to be provided to a visualization device (10), comprising capturing first data of a next image and providing said data to a processing unit (35, 40, 41); reading second data values from a memory buffer (49) into the processing unit (35, 40, 41), the second data values being associated with pixels of data of a previous image, comparing pixel by pixel (90) a respective first data value associated with a pixel from the next image data with a respective second data value associated with a pixel from the previous image data, setting a flag bit (96) in response to said comparison, if for one pair of corresponding pixels (90) according to both images the data values are not the same, or a difference between the data values exceeds a pre-defined threshold, continuing with comparing a next pair of pixels (90) as long as the flag bit (96) is not set or all pixels are compared, and stop continuing with comparing a next pair of pixels (90) and at least partly updating the memory buffer (49) with data based on the first data values, if the flag bit (96) is set.
Description
DESCRIPTION
CONDITIONAL UPDATING OF IMAGE DATA IN A MEMORY BUFFER
TECHNICAL FIELD
[0001] The present invention relates to buffering image data.
BACKGROUND ART
[0002] In the past two decades, the growth in the sector of personal computers (PC's) has been accompanied by an increase in networking capabilities. Multiple computers including specific devices such as printers or other peripherals are grouped into networks such as local area networks (LAN) or wide area networks (WAN). Those networks are optionally controlled by dedicated server computers (i.e., one or more servers). As the networks become more and more locally distributed, the need to perform control of individual components in a remote fashion increases in importance.
[0003] This becomes particularly valid in the case of system maintenance, e.g., with respect to servers. Efforts and costs may be reduced when the graphic output, or visual response to an action, is redirected to a remote computer for display thereon. The person performing system maintenance may then carry out the necessary tasks, or may obtain a system diagnosis, etc., from his remote location without the need to move to, e.g., the server room and log on to the server computer of interest. Servers, computers and other logic devices in a network are referred to herein as processing devices.
[0004] Remote visualization means that an image to be displayed locally is copied or completely redirected to a remote data processing device, which is different from the local data processing device. Remote visualization allows, among others, remote system maintenance, e.g., by a hardware or software vendor, testing or performing helpdesk functions for end users, etc.
[0005] Visualization may be realized by providing a graphics device, which transforms a digital representation of an image provided by, e.g., a central processing unit (CPU) of a local data processing device into signals which lead a monitor to
display that image. The signals may be analog, e.g., in the case of CRT monitors, or digital as often in the case of LCD-displays. A redirection of graphic information may be performed by adding a separate capture device to the system that captures the image contents and transfers it to a predetermined remote system.
[0006] The graphics device and the capture device may communicate with each other, e.g., via an internal bus of the respective local processing device. The graphics device uses a memory unit. This memory unit contains the data of the digital representation of the image to be displayed on a local monitor. In order to make these data remotely available, the data are first captured by a processing unit provided with the capture device, typically a card which is input as an additional component into a slot of the local computer.
[0007] The image data are then stored into a memory buffer, which is also provided with the capture device. This memory buffer relates to a dedicated capture memory, which is - among others - needed to take into account the reduced bandwidth of a LAN connection. The memory buffer thus receives a copy of an image and provides an opportunity to perform compression on the image data.
[0008] The image data, in other words: the digital representation of an image, may be captured in different modes: one active mode is to address the graphics device and then simply copy the digital data from the video memory via a bus (e.g., PCI) into the memory buffer of the capture device. Another passive mode is to monitor the information when being transferred from the video memory of the graphics device to the local graphics display (local monitor). In case of analog VGA-signals, an additional A/D converter is then required to yield a transformation into a digital representation of the image. Digital DVI-signals, e.g., may be recorded directly.
[0009] Compression may be performed by reducing the information available per pixel of an image (data values associated with a pixel), e.g., the bit length or number of colours. It is also possible to apply reduction schemes such as those known from the JPEG or MPEG-formats, wherein data values of adjacent pixels are related to each other.
[0010] Another approach is to transfer only data values of those pixels towards a
remote location which have changed in respective next image data with regard to previous image data. U.S. Patent Application No. 6,701 ,380 B2, to Schneider, W.J. et al., discloses a method of performing visualization of image data on a remote processing device. Therein, image data are divided into blocks of pixels, and the change of data values associated with pixels inside one block between two subsequent images is indicated by setting a flag bit in a register. The flag bit is associated with this specific block and is later used to select this block upon transferral of data values associated with the pixels of this block towards the remote location.
SUMMARY OF THE INVENTION
[0011] It is an object of the invention to provide an improved buffering of image data.
[0012] The object is solved by the independent claims. Further embodiments are shown by the dependent claims.
[0013] According to embodiments of the present invention a method of performing visualization of image data on a remote processing device includes steps of capturing data of a next image and providing the next image data to a processing unit. The capturing of next image data implies that previous image data are already present, e.g., in the capture device.
[0014] The processing unit may thereby be located on the capture device, which is dedicated to perform the remote transfer. According to one embodiment, the capture device, which performs the method steps, houses a video processing unit (VPU) as well as a central processing unit (CPU). Both may be part of a single device, i.e., a remote transfer controller. In this context, the processing unit being provided with captured next image data represents the CPU, while the next image data are captured by the VPU. In the case of analog video signals (e.g., VGA signals), an additional A/D converter may be placed upstream of the VPU. However, the invention is not limited to the specific implementation of different or unique processing units and associated memories on a dedicated capture device. One embodiment relates to implementing the components, which perform or enable the method steps as described herein, directly on the graphics device of a local computer.
[0015] Image data as referred to herein are digital representations of images, and comprise an arrangement of pixels, which are each associated with a data value, which represents, e.g., a colour. A pixel is in this respect a predefined coordinate position within an array of data, and further has a predefined length of bits. The data values are taken from a combination of the specific binary values currently associated with these bits.
[0016] The processing unit, for example, selects a first subset of pixels of the next image data. The subset may according to an embodiment represent an arrangement of pixels organized in rows and columns. First data values, which are associated with these pixels of the subset, are then determined. This determination may comprise the step of providing the bit values, which define a respective data value of a pixel, into a dedicated position within a register of the processing unit.
[0017] In parallel, but not necessarily simultaneously, second data values are read out from a memory buffer and into the processing unit, e.g., into a similarly dedicated further position in said register. The second data values are associated with pixels of data of a previous image, wherein said pixels of the previous image data correspond to said pixels of the new image data. The term "corresponding" means, that the pixel location of a respective first data value within the next image data is the same as that of a second data value within the previous image data.
[0018] The memory buffer, which keeps the previous image data, is according to one embodiment a buffer located on the capture device. In an alternate embodiment, it relates to the graphics, or video memory located on the graphics device. The invention is not limited to the specific arrangement of this buffer with respect to the processing unit. However, a direct association of the memory buffer with the processing unit, i.e., a short traveling distance with respect to signal is preferred as the loading from and the writing to this buffer turns out to represent a bottleneck for the bandwidth of data transfer.
[0019] Next, a data value of the first image data is compared with a respective one of the second image data for each of the pixels in the subset. This means that a pixel- wise check is performed, whether the data values have changed or not. For example, both data values may be selected from the respective register positions and directly
compared.
[0020] The result of this comparison step performed for each of the pixels is stored by setting a flag bit. According to a preferred embodiment, there is one separate flag bit associated with each subset. For example, a logical "1" (flag bit set) indicates that a change has occurred, while an initial value of "0" (flag bit not set) indicates no change has occurred, yet.
[0021] According to this embodiment, there is one flag bit for the current subset of pixels. Once this flag bit is set, the parallel provision of first data values and reading of second data values is terminated. Starting from this event, the incoming first data values are directly written into the memory buffer to overwrite the respective data values of the first image data, therein. Further, at least the currently compared pixel data values, which have lead to the detection of a data value change, are written to the memory buffer for updating the previous image. Pixel by pixel, and subset by subset, the "previous" image data are thus altered into the next image data.
[0022] The direct write mode of first data values of the next image data into the memory buffer means that the reading step of second data values into the processing unit is effectively stopped. This stop of data transfer from the memory buffer to the processing unit is limited to the presently selected subset of pixels. According to one embodiment, each different subset of pixels, which is selected from the next image data, is associated with its own flag bit. Thus, while the steps relating to one subset might have already lead to the event of a direct write of first data values, the corresponding steps relating to another subset may include a full read out of second values accompanied with a comparison with first data values for all pixels in the respective subset.
[0023] The termination of the read step may considerably reduce the time needed to perform the memory update using the next image data. In case each subset, which may also referred to as a block of pixels within image data, is associated with a flag bit, the flag bits may be stored in a change matrix, which indicates the occurrence of changes in the respective blocks of image data with respect to a previous image. This change matrix is used to control the transfer to the remote processing device. Only those blocks represented by an associated flag bit, which is set (thus attaining for
example a logical "1"), are transferred according to this embodiment. The current settings in the change matrix, which may be stored in a buffer or register associated with the processing unit, correspond to the actually updated image in the memory buffer.
[0024] Therefore, time is reduced by only reading out (second) data values from the memory buffer until a change is detected, generating an event and stopping further read out of data values in response to this event. It is found that once compression is performed by means of a change matrix, which reduces traffic via LAN for example, the data transfer from the memory buffer to the processing unit for the purpose of comparison with new image data becomes a dominant contributor to the bandwidth. The bandwidth is defined here as data throughput per time unit.
[0025] The effect of the invention thus is that data values are either read out from the memory buffer into the processing unit, or are written from the memory unit into the memory buffer. Consequently, only one transfer bandwidth of images (and not two) between the memory buffer and the processing unit is needed - with the exception of those one or few data values of a subset, which have lead to the detection of an change event, which were read out, compared and written.
[0026] Only those data values are read and written, which lead to the detection of a change. In case that more data values have to be read into the processing unit and compared therein simultaneously, for example all data values of pixels organized in a row (i.e., a subgroup) of a block or subset, more than just one data value is read and rewritten into the memory buffer according to this embodiment. In this context, the phrase "read and rewrite" necessarily includes an update of data values of previous image data in the memory buffer.
[0027] As will be described in the following, previously known approaches needed twice the transfer bandwidth, plus an additional write step the into the memory buffer with regard to the next image, which is thereby intermediately stored.
[0028] It is noted that embodiments of the invention can be partly or entirely embodied or supported by one or more suitable software programs, which can be stored on or otherwise provided by any kind of data carrier, and which might be
executed in or by any suitable data processing unit. Software programs or routines can be preferably applied with respect to operating the kernel functions of the processing unit on a capture device (of a remote management unit, RMU), e.g., a remote controller. The same is valid if the remote controller is located within a graphics device having direct access to the video memory.
BRIEF DESCRIPTION OF DRAWINGS
[0029] Other objects and many of the attendant advantages of embodiments of the present invention will be readily appreciated and become better understood by reference to the following more detailed description of embodiments in connection with the accompanied drawings. Features that are substantially or functionally equal or similar will be referred to by the same reference signs.
[0030] Figs.1 and 2 show block diagrams of a remote system connected to a local system for the purpose of remote image display according to prior art;
[0031] Fig. 3 illustrate the flow of data through a capture device according to prior art;
[0032] Fig. 4 illustrate a flow of data through capture device including a CPU, a VPU, a memory buffer and a remote interface according to a first embodiment of the invention;
[0033] Fig. 4B illustrate a flow of data through capture device including a CPU, a VPU, a memory buffer, a remote interface, a compression device and a decompression device according to a further embodiment of the invention; and
[0034] Fig. 5 shows a schematic representation of an image divided into subsets, an associated change matrix and a counter table for flags within the change matrix.
DETAILED DESCRIPTION OF THE INVENTION
[0035] Fig. 1 shows an embodiment of a remote processing device (computer) 10 connected to a local processing device (computer, or server) for the purpose of remote visualization of an image, the arrangement being known in the art. The remote processing device 10 is connected with a remote display 12. A CPU 20 of the local
processing device (right hand side of the sinuous line) is connected to an internal bus 25 such as a PCI-bus or PCI-Express bus. A graphics device 52 is also connected to the internal bus. The graphics device comprises a video memory 62 for storing a digital representation of the image data to be displayed on a local monitor 22. The display of an image is managed by means of a display controller 37 arranged on the graphics device 52. Further components 21a, 21 b of the local system may also be connected to the internal bus 25.
[0036] A connection of the local processing device with the remote processing device 10 is established by the capture device 30. The capture device 30 is also connected to the internal bus 25 and comprises a capture memory 32 and a remote interface 33. Capturing of image data is accomplished by addressing - and accessing - the video memory 62 via the internal bus 25 with help of a capture device controller 35 (remote transfer controller, not shown in Fig. 1 ) and retrieving full image data. The full image data are buffered in the capture memory 32, compressed and forwarded to the remote data processing unit via remote interface 33. The flow of redirected image data is indicated by the dashed line in the figures. An example flow of image data generated in the CPU 20 intended for local display is indicated by the dot-dashed line
[0037] Fig. 2 shows an alternate embodiment known in the art. Herein, the capture device is arranged to retrieve image data from a connection between the display controller 37 and the local monitor 22. The data transferred herein are less redirected than snarfed, i.e., duplicated for local and remote display. As the data have already been processed by the display controller, the format of transferred data considerably differs between the embodiments of Figs.1 and 2. In particular, the embodiment shown in Fig. 2 may involve the snarfing of analog VGA signals, which are to be converted into digital data representing the image by an A/D converter 29, first.
[0038] Fig. 3 shows a capture device 30, which may be embodied as shown in Fig. 2, in more detail. The remote transfer controller 35 in this embodiment comprises a video processing unit 40 (VPU) and a central processing unit 41 (CPU). The VPU 40 is designed to capture image data and to process the incoming data. In particular it performs bit- or pixel-wise comparison of image data in order to detect changes. It further arranges the reading from and writing to the capture memory 32. The CPU 41
of the capture device 30 controls the flow of image data, for example it initiates the capture of image data and organizes the compression of data to be transferred (numeral 70 in Fig. 3) via remote interface 33.
[0039] The capture memory 32, which may be, e.g., one DRAM memory chip, comprises three or more memory buffers 45, 46, 47. A conventionally known flow of image data is described as follows: as indicated by the arrows in Fig. 3, data of a first captured image are processed by VPU 40, and stored as "previous" image data in a first memory buffer 45. Then, a next image is captured, processed and stored as next image data in the second memory buffer 46.
[0040] Both image data - the previous and the next - are then retrieved from the respective memory buffers 45, 46 by CPU 41. Data values corresponding to pixels from both images, which are associated with each other, are compared to detect a change. A difference image is thus formed, which is stored in the third buffer 47. The capture and storage of one image takes 10 - 20 ms of time. The calculation of the difference image lasts for 50 - 200 ms depending on the image size. Having applied conventional compression techniques, the transferal of the compressed image via remote interface 33 (e.g., LAN) takes another 20 ms. One image update at the remote location thus amounts to 80 - 240 ms according to this illustrative embodiment. Roughly 5 - 10 frames per second may thus be realized at the remote location, which may be sufficient to indicate the addition of text on the remote monitor 12. However, changes such as smooth mouse movements developing to abrupt jumps thereby are displeasingly followed by an observer according to this two-pass-dataflow.
[0041] An embodiment of the invention is shown in Fig.4A. Next image data are converted into digital data by A/D converter 29, and in case of a first image are directly written into memory buffer 49 by means of VPU 40. These initially stored image data will then be referred to as "previous" image data, since a further image is captured by VPU 40. Being digitized, the next image data are transferred successively to the VPU 40 as a contiguous stream of captured digital data 82.
[0042] Simultaneously, interchangeably, or adjacently to the capture, digital data 84 are read out with respect to the previous image from the memory buffer 49 and into the
VPU 40. The read out of data from the memory buffer is coordinated with the incoming
data from the A/D converter 29. In particular, digital data of the next image form bytes or words, which contain data values with respect to pixels of the next image. What is read out from the memory buffer are digital data corresponding to respective pixel positions of the previous image. The data values formed from the digital data are then compared in VPU 40 in order to detect a change or not.
[0043] According to this embodiment, the images (previous and next) are subdivided into the same subsets of pixels. For example, an image being formed from 1024 (rows) * 768 (columns) pixels may contain subsets, or blocks, formed from 16 * 16 pixels. Accordingly, one such image has 64 * 48 blocks. The event of detecting a change is handled with respect to each subset, or block, separately.
[0044] A change may be signaled when the data values of corresponding pixels in the previous and next image are not the same, i.e., they differ. Alternatively, a relaxed condition may be set up, stating that the difference between these data values must exceed a predetermined threshold level in order to represent a change. This latter condition represents an advantageous aspect, as it enables further compression of data to be transmitted to the remote location. Small fluctuations of, e.g., colours from image to image will herein be disregarded from an assessment as a change.
[0045] As mentioned above, a change is tracked for each subset separately. Until a change is detected with regard to one subset, digital data 82 are captured and digital data 83 are read into the VPU 40. Once a change is detected, the data transfer mode is also changed. For this specific subset, the read out of digital data 84 from the memory buffer 49 is stopped along with the comparison step. Rather, the captured digital data 82, or data values corresponding to respective pixels of the subset, are directly written as digital data 83 into the memory buffer 49. More precisely, buffer locations in memory buffer 49, which keep digital data related to associated pixels of the previous image, are overwritten with the digital data 83 of the next image, regardless of whether they represent a further change or not.
[0046] Accordingly, the communication between the memory buffer 49 and the VPU merely comprises either reading digital data 84 or writing digital data 83. Only, those specific data values, which correspond to the event of a detected change, are read in to VPU 40 as "aged" digital data 84 as well as (re-)written as updated digital data 83
into the memory buffer 83. If only one data value at a time is read in and compared in the VPU 40, there is consequently one pixel out of 256 (16 times 16 in a block according to this embodiment), which necessitates twice a bandwidth. The present embodiment essentially refers to a one-pass procedure of updating image data.
[0047] A further advantage of this embodiment, visible from Fig.4, is that memory is saved as the memory buffer is simply updated by the VPU 40. As a result, less bandwidth and slower as well as cheaper memory chips are necessary.
[0048] The embodiment detailed above may further be refined by means of a change matrix 75. The detection of change may be realized by setting a flag bit, e.g., from logical "0" to "1" or vice versa. As each subset or block has its own flag bit, the resulting 64 * 48 flag bits may be arranged in a corresponding change matrix. This change matrix indicates those subsets of an image in which changes have taken place.
[0049] The change matrix may be stored in memory buffer 49, in a separate storage area, or in registers of the VPU 40 or CPU 41. The latter CPU 41 also controls compression as explained above. The change matrix 75, which is each time updated according to the present method, may therefore easily be exploited for the purpose of compression, e.g., transferring only digital data from changed blocks to a remote location (along with the information, which block the digital data actually refer to). Particulars of a change matrix for the purpose of data compression are disclosed in the Patent EP 1168154 B1 to Agilent Technologies, Inc., the specific features of which may advantageously be combined with the invention and are thus incorporated herein by reference.
[0050] The task of remote data transfer is facilitated by CPU 41 via communication lines 72 to the VPU 40 and 71 to the remote interface 33, e.g., a LAN-, or WAN- interface, wherein CPU 41 initiates interface 33 to retrieve data from memory buffer 49 in dependence of entries (flag bits) within change matrix.
[0051] As shown in Fig. 5, digital data of an image 88 are captured, or read in by the VPU 40 successively pixel by pixel 90 and row by row 92. As a row 92 extends in an image beyond one subset 94, flag bits of different subsets 94 may be maintained alternately until a final row of respective subsets is reached. Maintenance of multiple
flag bits within a change matrix 75 becomes particularly advantageous herein, when data from one subset are not contiguously arranged in the incoming data stream from A/D converter 29, or a DVI interface, etc.
[0052] Associated with the change matrix is further a counter table 77. Each flag bit 96 in the change matrix 75 is associated with a counter 98 in the counter table 77. The counter 98 helps to impede aging of subsets 94, which are repeatedly not updated according to the present method. A threshold value of 25, as an arbitrary example value may be predefined, and with each run through an image capture and update, the counter of a flag bit 96, or subset 94, is increased by 1 , as an example. If the number of 25 is reached for a subset, that has no changed pixel value, the flag bit is set and the update is performed for this subset 94 irrespective of whether a change presently occurs or not.
[0053] This approach with counters may be advantageous as in the case, that similar but no identical data values between adjacent images are admitted, a slight drift in data values may occur leading to an inconsistency in image data. Therein, admittance (no setting of a flag bit) of slight differences, when comparing first with second data values saves transmittance bandwidth, and may thus simultaneously be advantageous. An advantageous embodiment refers to an application, wherein additional screens are slowly faded in to or out of the image, respectively.
[0054] It is clear that after a update the flag bit has to be reset, e.g., to a logical "0" in order to enable the next run through the method steps without a change being indicated.
[0055] As is visible from Fig.4B, a further refinement may be made by compressing the digital data 83 already when being written from the VPU 40 to the memory buffer 49, and decompressing the digital data 84, when being read out from the memory buffer 49 into the VPU 40. In an advantageous embodiment, a so-called run length encoding (RLE) may be performed. Therein, data values of consecutive pixels having the same level may be stored as this level added by the count of consecutive pixels, for which this condition holds. An improved method thereof is disclosed in the European patent application EP05102300.0 of Agilent Technologies, Inc., the specific features of which may be advantageously be combined with the invention and are thus
incorporated herein by reference.
[0056] According to the compression method described therein, a data value range (or similar colour range) is provided, which extends the run length as explained above to those pixels, which merely have similar - but not necessarily the same - data values, or colours respectively. An approximate data value is chosen instead and attributed to each of the pixels involved, along with the pixel count, which reduces the necessary memory space to store the image information. The approximate data value may be a mean value of the data values involved, a median value or the most frequent one of the data values, etc.
[0057] Upon execution of the method, the value range may further be adapted (e.g., shifted) to include subsequently read in data values, until one subsequent data value inevitably falls out of this range as the difference to a smallest or largest one becomes too large. A counter is reset, e.g. to zero, the previous count and the approximate data value are stored, and a new value range is selected for further subsequently read in data values.
[0058] This method of compression of data in-between one image may be realized by an appropriate software implemented with regard to the VPU 40, CPU 41 or another controller 35, which could replace both processing units 40, 41 shown in this embodiment. Reference numerals 85a, 85b indicate those compression units or decompression units, respectively, arranged in hardware or software. For example, a decision unit and a value range adjusting unit such as that shown in Fig. 1 of the EP application (reference numerals 14, 18 therein) may be realized as ASIC's (application specific integrated circuits) between the processing units 40, or 41 , and memory buffer 49 according to the present embodiment.
[0059] A still further embodiment relates to the memory buffer 49. Thereby, it is possible to integrate the buffer 49 into the graphics memory of a graphics device. This means that the VPU may "capture" (select) data from a first frame buffer in the memory and compare these data with those of a second frame buffer, which represents the buffer 49 shown in Fig. 4. Again, time for capture and memory space is saved to considerable amounts. Examples of advantageous buffer management configurations are provided in the European patent application EP05108275.8 of Agilent
Technologies, Inc, the specific features of which, particularly shown in Figs. 3, 4, 5, and 7 and corresponding description therein may be combined with the present invention and are thus incorporated herein by reference.
Claims
1. A method of updating a memory buffer (49) with image data for transmission to a visualization device (10), comprising:
(a) capturing first data of a next image and providing said data to a processing unit (35, 40, 41 );
(b) reading second data values from a memory buffer (49) into the processing unit (35, 40, 41 ), the second data values being associated with pixels of data of a previous image;
(c) comparing pixel by pixel (90) a respective first data value associated with a pixel from the next image data with a respective second data value associated with a pixel from the previous image data;
(d) setting a flag bit (96) in response to said comparison, if for one pair of corresponding pixels (90) according to both images the data values are not the same, or a difference between the data values exceeds a pre-defined threshold;
(e) continuing with comparing a next pair of pixels (90), as long as the flag bit (96) is not set or all pixels are compared; and
(f) stop continuing with comparing a next pair of pixels (90) and at least partly updating the memory buffer (49) with data based on the first data values, if the flag bit (96) is set.
2. Method according to claim 1 , comprising:
dividing the next and previous image data into a plurality of image subsets (94),
selecting a first subset (94) of pixels of the next image data and determining first data values associated with each pixel of the subset;
repeating steps (b) - (f) for each of said subsets of the next image data to update the previous image data with next image data in the memory buffer (49); and transferring the updated image data (88), or a portion thereof, from the memory buffer (49) to the visualization device (10).
3. Method according to claim 1 or any one of the above claims, wherein:
the first data values of pixels (90) of the next image are provided to the processing unit (35, 40, 41 ) simultaneously with reading in the second data values of pixels (90) of the previous image data into the processing unit (35, 40, 41 ).
4. Method according to claim 1 or any one of the above claims, further comprising associating each one separate flag bit (96) with each one separate subset (94) of the previous image data stored and then updated in the memory buffer (49).
5. Method according to claim 4 or any one of the above claims, further comprising storing said separate flag bits (96) in a matrix (75), which indicates changes of the previous image data.
6. Method according to claim 1 or any one of the above claims, wherein transferring the updated image data (88) from the memory buffer (49) to the remote processing device (10) includes compressing the image data.
7. Method according to claim 4 or any one of the above claims, wherein transferring the updated image data (88) from the memory buffer (49) to the remote processing device (10) includes compressing the image data (88) by selecting data values from pixels (90) of subsets (94) in dependence of a setting of a respective flag bit (96) associated with the subset (94).
8. Method according to claim 4 or any one of the above claims, further comprising:
repeating the capturing of further images, updating respective previous images and transferring the updated images (88) to the remote processing device (10);
associating each of said separate flag bits (96) with a separate counter (98),
adding a fixed value to each of said counters (98) each time the sequence of method steps is repeated for a respective next image; comparing each of said the counters (98) with a predefined threshold value, which represents a maximum age of a non-updated subset of image data (88); and
setting a flag bit (96) associated with a subset (94) in response to said comparison of the counter (98), for initiating an update of the previous image data (88) in the memory buffer (49) with the data values associated with pixels (90) of a subset (94) which has exceeded said maximum age.
9. Method according to claim 1 or any one of the above claims, wherein reading second data values from a memory buffer (49) into the processing unit (35, 40, 41 ) includes extracting and decompressing said data.
10. Method according to claim 9 or any one of the above claims, wherein the decompression is performed by a run length decoding (RLE), a first data value being stored in the memory buffer (49) by means of a fixed data value and a count of consecutive pixels, which are all associated with this data value.
11. Method according to claim 1 or any one of the above claims, wherein writing the third data values into the memory buffer (49) includes compressing the image data.
12. Method according to claim 11 or any one of the above claims, wherein the compression is performed by run length encoding (RLE), a second data value being written into the memory buffer (49) in step (f) by setting a fixed data value and a count of consecutive pixels, which are all associated with this fixed data value.
13. Method according to claim 12 or any one of the above claims, wherein the fixed data value is set by determining an approximate data value, said approximate data value being determined from multiple data values of consecutive pixels (90), which reside within a value range defined by a lower and an upper threshold.
14. A software program or product, preferably stored on a data carrier, for controlling or executing the method of claim 1 or any one of the above claims, when run on a data processing system such as a computer.
15. An interface device for receiving incoming image data representing a sequence of images, comprising:
a capturing device (29) adapted for capturing first data of a next image,
a memory buffer (49) adapted for storing second data to be provided to an image device (10), and
a processing unit (35, 40, 41 ) adapted for:
reading the first data of the capturing device (29) and the second data values from the memory buffer (49), comparing pixel by pixel (90) a respective first data value associated with a pixel from the next image data with a respective second data value associated with a pixel from the previous image data;
setting a flag bit (96) in response to said comparison, if for one pair of corresponding pixels (90) according to both images the data values are not the same, or a difference between the data values exceeds a pre-defined threshold;
continuing with comparing a next pair of pixels (90) of the subset (94), as long as the flag bit (96) is not set or all pixels are compared; and
stop continuing with comparing a next pair of pixels (90) and at least partly updating the memory buffer (49) with data based on the first data values, if the flag bit (96) is set.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/EP2005/056116 WO2007057053A1 (en) | 2005-11-21 | 2005-11-21 | Conditional updating of image data in a memory buffer |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/EP2005/056116 WO2007057053A1 (en) | 2005-11-21 | 2005-11-21 | Conditional updating of image data in a memory buffer |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2007057053A1 true WO2007057053A1 (en) | 2007-05-24 |
Family
ID=36589091
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/EP2005/056116 WO2007057053A1 (en) | 2005-11-21 | 2005-11-21 | Conditional updating of image data in a memory buffer |
Country Status (1)
Country | Link |
---|---|
WO (1) | WO2007057053A1 (en) |
Cited By (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN101872293A (en) * | 2009-04-23 | 2010-10-27 | 威睿公司 | Method and system for copying a framebuffer for transmission to a remote display |
CN101873308A (en) * | 2009-04-23 | 2010-10-27 | 威睿公司 | Method and system for identifying drawing primitives for selective transmission to a remote display |
ES2474266A1 (en) * | 2013-07-25 | 2014-07-08 | Mar�a Cristina CARMONA DUARTE | Procedure to compare and quantify the similarity between the image drawn on a touch screen with digital images (Machine-translation by Google Translate, not legally binding) |
WO2016016607A1 (en) * | 2014-07-31 | 2016-02-04 | Displaylink (Uk) Limited | Managing display data for display |
CN107656108A (en) * | 2017-11-09 | 2018-02-02 | 光科技股份有限公司 | Suitable for the fast appearing methods of RCM and device of electric-power metering |
US10365881B2 (en) * | 2016-04-25 | 2019-07-30 | Seiko Epson Corporation | Image supplying apparatus, method for controlling image supplying apparatus, and program |
CN113228596A (en) * | 2018-12-04 | 2021-08-06 | 区块链联合香港有限公司 | Method and device for transmitting list information |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4958378A (en) * | 1989-04-26 | 1990-09-18 | Sun Microsystems, Inc. | Method and apparatus for detecting changes in raster data |
US5168269A (en) * | 1990-11-08 | 1992-12-01 | Norton-Lambert Corp. | Mouse driven remote communication system |
US20030169264A1 (en) * | 2002-01-04 | 2003-09-11 | Emerson Theodore F. | Operating system independent method and apparatus for graphical remote access having improved latency |
US6701380B2 (en) * | 1997-08-22 | 2004-03-02 | Avocent Redmond Corp. | Method and system for intelligently controlling a remotely located computer |
-
2005
- 2005-11-21 WO PCT/EP2005/056116 patent/WO2007057053A1/en active Application Filing
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4958378A (en) * | 1989-04-26 | 1990-09-18 | Sun Microsystems, Inc. | Method and apparatus for detecting changes in raster data |
US5168269A (en) * | 1990-11-08 | 1992-12-01 | Norton-Lambert Corp. | Mouse driven remote communication system |
US6701380B2 (en) * | 1997-08-22 | 2004-03-02 | Avocent Redmond Corp. | Method and system for intelligently controlling a remotely located computer |
US20030169264A1 (en) * | 2002-01-04 | 2003-09-11 | Emerson Theodore F. | Operating system independent method and apparatus for graphical remote access having improved latency |
Cited By (21)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10572214B2 (en) | 2009-04-23 | 2020-02-25 | Vmware, Inc. | Method and system for identifying drawing primitives for selective transmission to a remote display |
EP2244182A3 (en) * | 2009-04-23 | 2011-06-08 | VMWare, Inc. | Method and system for identifying drawing primitives for selective transmission to a remote display |
US11397553B2 (en) | 2009-04-23 | 2022-07-26 | Vmware, Inc. | Method and system for identifying drawing primitives for selective transmission to a remote display |
US11003412B2 (en) | 2009-04-23 | 2021-05-11 | Vmware, Inc. | Method and system for identifying drawing primitives for selective transmission to a remote display |
US9448760B2 (en) | 2009-04-23 | 2016-09-20 | Vmware, Inc. | Method and system for identifying drawing primitives for selective transmission to a remote display |
EP2244183A3 (en) * | 2009-04-23 | 2011-06-08 | VMWare, Inc. | Method and system for copying a framebuffer for transmission to a remote display |
AU2010201050B2 (en) * | 2009-04-23 | 2012-03-29 | VMware LLC | Method and system for copying a framebuffer for transmission to a remote display |
US8441494B2 (en) | 2009-04-23 | 2013-05-14 | Vmware, Inc. | Method and system for copying a framebuffer for transmission to a remote display |
CN101872293A (en) * | 2009-04-23 | 2010-10-27 | 威睿公司 | Method and system for copying a framebuffer for transmission to a remote display |
US8902242B2 (en) | 2009-04-23 | 2014-12-02 | Vmware, Inc. | Method and system for identifying drawing primitives for selective transmission to a remote display |
JP2010257454A (en) * | 2009-04-23 | 2010-11-11 | Vmware Inc | Method and system for copying framebuffer for transmission to remote display |
CN101873308A (en) * | 2009-04-23 | 2010-10-27 | 威睿公司 | Method and system for identifying drawing primitives for selective transmission to a remote display |
JP2010282609A (en) * | 2009-04-23 | 2010-12-16 | Vmware Inc | Method and system for identifying drawing primitive for selective transmission to remote display |
US10067732B2 (en) | 2009-04-23 | 2018-09-04 | Vmware, Inc. | Method and system for identifying drawing primitives for selective transmission to a remote display |
ES2474266A1 (en) * | 2013-07-25 | 2014-07-08 | Mar�a Cristina CARMONA DUARTE | Procedure to compare and quantify the similarity between the image drawn on a touch screen with digital images (Machine-translation by Google Translate, not legally binding) |
WO2016016607A1 (en) * | 2014-07-31 | 2016-02-04 | Displaylink (Uk) Limited | Managing display data for display |
US10365881B2 (en) * | 2016-04-25 | 2019-07-30 | Seiko Epson Corporation | Image supplying apparatus, method for controlling image supplying apparatus, and program |
CN107656108A (en) * | 2017-11-09 | 2018-02-02 | 光科技股份有限公司 | Suitable for the fast appearing methods of RCM and device of electric-power metering |
CN113228596A (en) * | 2018-12-04 | 2021-08-06 | 区块链联合香港有限公司 | Method and device for transmitting list information |
US11838349B2 (en) | 2018-12-04 | 2023-12-05 | Hong Kong Sunstar Technology Co., Limited | Method and device for transmitting list information |
CN113228596B (en) * | 2018-12-04 | 2024-04-02 | 香港星辰科技有限公司 | Method and device for transmitting list information |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US8255900B2 (en) | Virtual computer server apparatus, and update image detection method | |
US5943064A (en) | Apparatus for processing multiple types of graphics data for display | |
US7944451B2 (en) | Providing pixels from an update buffer | |
CN107493448B (en) | Image processing system, image display method and display device | |
US7423642B2 (en) | Efficient video frame capturing | |
WO2007057053A1 (en) | Conditional updating of image data in a memory buffer | |
CN106030652B (en) | Method, system and composite display controller for providing output surface and computer medium | |
CN114051145B (en) | Video compression processing method, device and medium | |
CN115297094B (en) | Video transmission control method, device, equipment and computer readable storage medium | |
JP2004280125A (en) | Video/graphic memory system | |
US11616895B2 (en) | Method and apparatus for converting image data, and storage medium | |
CN115209145A (en) | Video compression method, system, device and readable storage medium | |
CN113205779A (en) | Multi-screen synchronous control method, device and system suitable for electronic ink screen | |
CN107027069B (en) | Processing method, device and system, storage medium and the processor of image data | |
KR102619668B1 (en) | Apparatus and method of using a slice update map | |
US6313839B1 (en) | Method and apparatus for performing Z buffer depth comparison operations | |
JP2952780B2 (en) | Computer output system | |
US6717989B1 (en) | Video decoding apparatus and method for a shared display memory system | |
CN108024116B (en) | Data caching method and device | |
US9471956B2 (en) | Graphic remoting system with masked DMA and graphic processing method | |
US20080055286A1 (en) | Method And Apparatus For Displaying Bitmap Images | |
CN115174915A (en) | Image processing system and method for reducing network bandwidth occupation | |
EP1762928A1 (en) | Graphics device comprising remote transfer controller for remote visualization | |
CN114090168A (en) | Self-adaptive adjusting method for image output window of QEMU (QEMU virtual machine) | |
US5543822A (en) | Method for increasing the video throughput in computer systems |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application | ||
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 05813668 Country of ref document: EP Kind code of ref document: A1 |