WO2007057053A1 - Conditional updating of image data in a memory buffer - Google Patents

Conditional updating of image data in a memory buffer Download PDF

Info

Publication number
WO2007057053A1
WO2007057053A1 PCT/EP2005/056116 EP2005056116W WO2007057053A1 WO 2007057053 A1 WO2007057053 A1 WO 2007057053A1 EP 2005056116 W EP2005056116 W EP 2005056116W WO 2007057053 A1 WO2007057053 A1 WO 2007057053A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
image data
pixels
memory buffer
data values
Prior art date
Application number
PCT/EP2005/056116
Other languages
French (fr)
Inventor
Frank Klein
Original Assignee
Agilent Technologies, Inc.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Agilent Technologies, Inc. filed Critical Agilent Technologies, Inc.
Priority to PCT/EP2005/056116 priority Critical patent/WO2007057053A1/en
Publication of WO2007057053A1 publication Critical patent/WO2007057053A1/en

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/14Digital output to display device ; Cooperation and interconnection of the display device with other functional units
    • G06F3/1454Digital output to display device ; Cooperation and interconnection of the display device with other functional units involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay
    • GPHYSICS
    • G09EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09GARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2310/00Command of the display device
    • G09G2310/04Partial updating of the display screen
    • GPHYSICS
    • G09EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09GARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2340/00Aspects of display data processing
    • G09G2340/02Handling of images in compressed format, e.g. JPEG, MPEG
    • GPHYSICS
    • G09EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09GARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G5/00Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators
    • G09G5/36Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators characterised by the display of a graphic pattern, e.g. using an all-points-addressable [APA] memory
    • G09G5/39Control of the bit-mapped memory
    • G09G5/393Arrangements for updating the contents of the bit-mapped memory

Definitions

  • the present invention relates to buffering image data.
  • Remote visualization means that an image to be displayed locally is copied or completely redirected to a remote data processing device, which is different from the local data processing device.
  • Remote visualization allows, among others, remote system maintenance, e.g., by a hardware or software vendor, testing or performing helpdesk functions for end users, etc.
  • Visualization may be realized by providing a graphics device, which transforms a digital representation of an image provided by, e.g., a central processing unit (CPU) of a local data processing device into signals which lead a monitor to display that image.
  • the signals may be analog, e.g., in the case of CRT monitors, or digital as often in the case of LCD-displays.
  • a redirection of graphic information may be performed by adding a separate capture device to the system that captures the image contents and transfers it to a predetermined remote system.
  • the graphics device and the capture device may communicate with each other, e.g., via an internal bus of the respective local processing device.
  • the graphics device uses a memory unit. This memory unit contains the data of the digital representation of the image to be displayed on a local monitor. In order to make these data remotely available, the data are first captured by a processing unit provided with the capture device, typically a card which is input as an additional component into a slot of the local computer.
  • the image data are then stored into a memory buffer, which is also provided with the capture device.
  • This memory buffer relates to a dedicated capture memory, which is - among others - needed to take into account the reduced bandwidth of a LAN connection.
  • the memory buffer thus receives a copy of an image and provides an opportunity to perform compression on the image data.
  • the image data in other words: the digital representation of an image, may be captured in different modes: one active mode is to address the graphics device and then simply copy the digital data from the video memory via a bus (e.g., PCI) into the memory buffer of the capture device. Another passive mode is to monitor the information when being transferred from the video memory of the graphics device to the local graphics display (local monitor). In case of analog VGA-signals, an additional A/D converter is then required to yield a transformation into a digital representation of the image. Digital DVI-signals, e.g., may be recorded directly.
  • a bus e.g., PCI
  • Another passive mode is to monitor the information when being transferred from the video memory of the graphics device to the local graphics display (local monitor).
  • an additional A/D converter is then required to yield a transformation into a digital representation of the image.
  • Digital DVI-signals e.g., may be recorded directly.
  • Compression may be performed by reducing the information available per pixel of an image (data values associated with a pixel), e.g., the bit length or number of colours. It is also possible to apply reduction schemes such as those known from the JPEG or MPEG-formats, wherein data values of adjacent pixels are related to each other.
  • a method of performing visualization of image data on a remote processing device includes steps of capturing data of a next image and providing the next image data to a processing unit.
  • the capturing of next image data implies that previous image data are already present, e.g., in the capture device.
  • the processing unit may thereby be located on the capture device, which is dedicated to perform the remote transfer.
  • the capture device which performs the method steps, houses a video processing unit (VPU) as well as a central processing unit (CPU). Both may be part of a single device, i.e., a remote transfer controller.
  • VPU video processing unit
  • CPU central processing unit
  • the processing unit being provided with captured next image data represents the CPU, while the next image data are captured by the VPU.
  • an additional A/D converter may be placed upstream of the VPU.
  • the invention is not limited to the specific implementation of different or unique processing units and associated memories on a dedicated capture device.
  • Image data as referred to herein are digital representations of images, and comprise an arrangement of pixels, which are each associated with a data value, which represents, e.g., a colour.
  • a pixel is in this respect a predefined coordinate position within an array of data, and further has a predefined length of bits. The data values are taken from a combination of the specific binary values currently associated with these bits.
  • the processing unit selects a first subset of pixels of the next image data.
  • the subset may according to an embodiment represent an arrangement of pixels organized in rows and columns.
  • First data values, which are associated with these pixels of the subset, are then determined. This determination may comprise the step of providing the bit values, which define a respective data value of a pixel, into a dedicated position within a register of the processing unit.
  • second data values are read out from a memory buffer and into the processing unit, e.g., into a similarly dedicated further position in said register.
  • the second data values are associated with pixels of data of a previous image, wherein said pixels of the previous image data correspond to said pixels of the new image data.
  • corresponding means, that the pixel location of a respective first data value within the next image data is the same as that of a second data value within the previous image data.
  • the memory buffer which keeps the previous image data, is according to one embodiment a buffer located on the capture device. In an alternate embodiment, it relates to the graphics, or video memory located on the graphics device.
  • the invention is not limited to the specific arrangement of this buffer with respect to the processing unit. However, a direct association of the memory buffer with the processing unit, i.e., a short traveling distance with respect to signal is preferred as the loading from and the writing to this buffer turns out to represent a bottleneck for the bandwidth of data transfer.
  • a data value of the first image data is compared with a respective one of the second image data for each of the pixels in the subset. This means that a pixel- wise check is performed, whether the data values have changed or not. For example, both data values may be selected from the respective register positions and directly compared.
  • the result of this comparison step performed for each of the pixels is stored by setting a flag bit.
  • a flag bit there is one separate flag bit associated with each subset. For example, a logical "1" (flag bit set) indicates that a change has occurred, while an initial value of "0" (flag bit not set) indicates no change has occurred, yet.
  • the parallel provision of first data values and reading of second data values is terminated.
  • the incoming first data values are directly written into the memory buffer to overwrite the respective data values of the first image data, therein.
  • at least the currently compared pixel data values which have lead to the detection of a data value change, are written to the memory buffer for updating the previous image. Pixel by pixel, and subset by subset, the "previous" image data are thus altered into the next image data.
  • the direct write mode of first data values of the next image data into the memory buffer means that the reading step of second data values into the processing unit is effectively stopped. This stop of data transfer from the memory buffer to the processing unit is limited to the presently selected subset of pixels.
  • each different subset of pixels, which is selected from the next image data is associated with its own flag bit.
  • the steps relating to one subset might have already lead to the event of a direct write of first data values
  • the corresponding steps relating to another subset may include a full read out of second values accompanied with a comparison with first data values for all pixels in the respective subset.
  • the termination of the read step may considerably reduce the time needed to perform the memory update using the next image data.
  • each subset which may also referred to as a block of pixels within image data
  • the flag bits may be stored in a change matrix, which indicates the occurrence of changes in the respective blocks of image data with respect to a previous image.
  • This change matrix is used to control the transfer to the remote processing device. Only those blocks represented by an associated flag bit, which is set (thus attaining for example a logical "1"), are transferred according to this embodiment.
  • the current settings in the change matrix which may be stored in a buffer or register associated with the processing unit, correspond to the actually updated image in the memory buffer.
  • time is reduced by only reading out (second) data values from the memory buffer until a change is detected, generating an event and stopping further read out of data values in response to this event. It is found that once compression is performed by means of a change matrix, which reduces traffic via LAN for example, the data transfer from the memory buffer to the processing unit for the purpose of comparison with new image data becomes a dominant contributor to the bandwidth.
  • the bandwidth is defined here as data throughput per time unit.
  • the effect of the invention thus is that data values are either read out from the memory buffer into the processing unit, or are written from the memory unit into the memory buffer. Consequently, only one transfer bandwidth of images (and not two) between the memory buffer and the processing unit is needed - with the exception of those one or few data values of a subset, which have lead to the detection of an change event, which were read out, compared and written.
  • embodiments of the invention can be partly or entirely embodied or supported by one or more suitable software programs, which can be stored on or otherwise provided by any kind of data carrier, and which might be executed in or by any suitable data processing unit.
  • Software programs or routines can be preferably applied with respect to operating the kernel functions of the processing unit on a capture device (of a remote management unit, RMU), e.g., a remote controller. The same is valid if the remote controller is located within a graphics device having direct access to the video memory.
  • FIGs.1 and 2 show block diagrams of a remote system connected to a local system for the purpose of remote image display according to prior art
  • FIG. 3 illustrate the flow of data through a capture device according to prior art
  • FIG. 4 illustrate a flow of data through capture device including a CPU, a VPU, a memory buffer and a remote interface according to a first embodiment of the invention
  • FIG. 4B illustrate a flow of data through capture device including a CPU, a VPU, a memory buffer, a remote interface, a compression device and a decompression device according to a further embodiment of the invention.
  • Fig. 5 shows a schematic representation of an image divided into subsets, an associated change matrix and a counter table for flags within the change matrix.
  • FIG. 1 shows an embodiment of a remote processing device (computer) 10 connected to a local processing device (computer, or server) for the purpose of remote visualization of an image, the arrangement being known in the art.
  • the remote processing device 10 is connected with a remote display 12.
  • a CPU 20 of the local processing device (right hand side of the sinuous line) is connected to an internal bus 25 such as a PCI-bus or PCI-Express bus.
  • a graphics device 52 is also connected to the internal bus.
  • the graphics device comprises a video memory 62 for storing a digital representation of the image data to be displayed on a local monitor 22.
  • the display of an image is managed by means of a display controller 37 arranged on the graphics device 52. Further components 21a, 21 b of the local system may also be connected to the internal bus 25.
  • a connection of the local processing device with the remote processing device 10 is established by the capture device 30.
  • the capture device 30 is also connected to the internal bus 25 and comprises a capture memory 32 and a remote interface 33. Capturing of image data is accomplished by addressing - and accessing - the video memory 62 via the internal bus 25 with help of a capture device controller 35 (remote transfer controller, not shown in Fig. 1 ) and retrieving full image data.
  • the full image data are buffered in the capture memory 32, compressed and forwarded to the remote data processing unit via remote interface 33.
  • the flow of redirected image data is indicated by the dashed line in the figures.
  • An example flow of image data generated in the CPU 20 intended for local display is indicated by the dot-dashed line
  • Fig. 2 shows an alternate embodiment known in the art.
  • the capture device is arranged to retrieve image data from a connection between the display controller 37 and the local monitor 22.
  • the data transferred herein are less redirected than snarfed, i.e., duplicated for local and remote display.
  • the format of transferred data considerably differs between the embodiments of Figs.1 and 2.
  • the embodiment shown in Fig. 2 may involve the snarfing of analog VGA signals, which are to be converted into digital data representing the image by an A/D converter 29, first.
  • Fig. 3 shows a capture device 30, which may be embodied as shown in Fig. 2, in more detail.
  • the remote transfer controller 35 in this embodiment comprises a video processing unit 40 (VPU) and a central processing unit 41 (CPU).
  • the VPU 40 is designed to capture image data and to process the incoming data. In particular it performs bit- or pixel-wise comparison of image data in order to detect changes. It further arranges the reading from and writing to the capture memory 32.
  • the CPU 41 of the capture device 30 controls the flow of image data, for example it initiates the capture of image data and organizes the compression of data to be transferred (numeral 70 in Fig. 3) via remote interface 33.
  • the capture memory 32 which may be, e.g., one DRAM memory chip, comprises three or more memory buffers 45, 46, 47.
  • a conventionally known flow of image data is described as follows: as indicated by the arrows in Fig. 3, data of a first captured image are processed by VPU 40, and stored as "previous" image data in a first memory buffer 45. Then, a next image is captured, processed and stored as next image data in the second memory buffer 46.
  • Both image data - the previous and the next - are then retrieved from the respective memory buffers 45, 46 by CPU 41.
  • Data values corresponding to pixels from both images, which are associated with each other, are compared to detect a change.
  • a difference image is thus formed, which is stored in the third buffer 47.
  • the capture and storage of one image takes 10 - 20 ms of time.
  • the calculation of the difference image lasts for 50 - 200 ms depending on the image size.
  • the transferal of the compressed image via remote interface 33 (e.g., LAN) takes another 20 ms.
  • One image update at the remote location thus amounts to 80 - 240 ms according to this illustrative embodiment.
  • Roughly 5 - 10 frames per second may thus be realized at the remote location, which may be sufficient to indicate the addition of text on the remote monitor 12.
  • changes such as smooth mouse movements developing to abrupt jumps thereby are displeasingly followed by an observer according to this two-pass-dataflow.
  • FIG.4A An embodiment of the invention is shown in Fig.4A.
  • Next image data are converted into digital data by A/D converter 29, and in case of a first image are directly written into memory buffer 49 by means of VPU 40.
  • These initially stored image data will then be referred to as "previous" image data, since a further image is captured by VPU 40.
  • Being digitized, the next image data are transferred successively to the VPU 40 as a contiguous stream of captured digital data 82.
  • VPU 40 The read out of data from the memory buffer is coordinated with the incoming data from the A/D converter 29.
  • digital data of the next image form bytes or words, which contain data values with respect to pixels of the next image.
  • What is read out from the memory buffer are digital data corresponding to respective pixel positions of the previous image.
  • the data values formed from the digital data are then compared in VPU 40 in order to detect a change or not.
  • the images (previous and next) are subdivided into the same subsets of pixels.
  • an image being formed from 1024 (rows) * 768 (columns) pixels may contain subsets, or blocks, formed from 16 * 16 pixels. Accordingly, one such image has 64 * 48 blocks. The event of detecting a change is handled with respect to each subset, or block, separately.
  • a change may be signaled when the data values of corresponding pixels in the previous and next image are not the same, i.e., they differ.
  • a relaxed condition may be set up, stating that the difference between these data values must exceed a predetermined threshold level in order to represent a change.
  • This latter condition represents an advantageous aspect, as it enables further compression of data to be transmitted to the remote location. Small fluctuations of, e.g., colours from image to image will herein be disregarded from an assessment as a change.
  • a change is tracked for each subset separately. Until a change is detected with regard to one subset, digital data 82 are captured and digital data 83 are read into the VPU 40. Once a change is detected, the data transfer mode is also changed. For this specific subset, the read out of digital data 84 from the memory buffer 49 is stopped along with the comparison step. Rather, the captured digital data 82, or data values corresponding to respective pixels of the subset, are directly written as digital data 83 into the memory buffer 49. More precisely, buffer locations in memory buffer 49, which keep digital data related to associated pixels of the previous image, are overwritten with the digital data 83 of the next image, regardless of whether they represent a further change or not.
  • the communication between the memory buffer 49 and the VPU merely comprises either reading digital data 84 or writing digital data 83. Only, those specific data values, which correspond to the event of a detected change, are read in to VPU 40 as "aged" digital data 84 as well as (re-)written as updated digital data 83 into the memory buffer 83. If only one data value at a time is read in and compared in the VPU 40, there is consequently one pixel out of 256 (16 times 16 in a block according to this embodiment), which necessitates twice a bandwidth.
  • the present embodiment essentially refers to a one-pass procedure of updating image data.
  • a further advantage of this embodiment is that memory is saved as the memory buffer is simply updated by the VPU 40. As a result, less bandwidth and slower as well as cheaper memory chips are necessary.
  • the embodiment detailed above may further be refined by means of a change matrix 75.
  • the detection of change may be realized by setting a flag bit, e.g., from logical "0" to "1" or vice versa.
  • the resulting 64 * 48 flag bits may be arranged in a corresponding change matrix. This change matrix indicates those subsets of an image in which changes have taken place.
  • the change matrix may be stored in memory buffer 49, in a separate storage area, or in registers of the VPU 40 or CPU 41.
  • the latter CPU 41 also controls compression as explained above.
  • the change matrix 75 which is each time updated according to the present method, may therefore easily be exploited for the purpose of compression, e.g., transferring only digital data from changed blocks to a remote location (along with the information, which block the digital data actually refer to).
  • Particulars of a change matrix for the purpose of data compression are disclosed in the Patent EP 1168154 B1 to Agilent Technologies, Inc., the specific features of which may advantageously be combined with the invention and are thus incorporated herein by reference.
  • the task of remote data transfer is facilitated by CPU 41 via communication lines 72 to the VPU 40 and 71 to the remote interface 33, e.g., a LAN-, or WAN- interface, wherein CPU 41 initiates interface 33 to retrieve data from memory buffer 49 in dependence of entries (flag bits) within change matrix.
  • the remote interface 33 e.g., a LAN-, or WAN- interface
  • digital data of an image 88 are captured, or read in by the VPU 40 successively pixel by pixel 90 and row by row 92.
  • flag bits of different subsets 94 may be maintained alternately until a final row of respective subsets is reached. Maintenance of multiple flag bits within a change matrix 75 becomes particularly advantageous herein, when data from one subset are not contiguously arranged in the incoming data stream from A/D converter 29, or a DVI interface, etc.
  • Each flag bit 96 in the change matrix 75 is associated with a counter 98 in the counter table 77.
  • the counter 98 helps to impede aging of subsets 94, which are repeatedly not updated according to the present method.
  • a threshold value of 25, as an arbitrary example value may be predefined, and with each run through an image capture and update, the counter of a flag bit 96, or subset 94, is increased by 1 , as an example. If the number of 25 is reached for a subset, that has no changed pixel value, the flag bit is set and the update is performed for this subset 94 irrespective of whether a change presently occurs or not.
  • This approach with counters may be advantageous as in the case, that similar but no identical data values between adjacent images are admitted, a slight drift in data values may occur leading to an inconsistency in image data.
  • admittance no setting of a flag bit
  • An advantageous embodiment refers to an application, wherein additional screens are slowly faded in to or out of the image, respectively.
  • a further refinement may be made by compressing the digital data 83 already when being written from the VPU 40 to the memory buffer 49, and decompressing the digital data 84, when being read out from the memory buffer 49 into the VPU 40.
  • a so-called run length encoding RLE
  • data values of consecutive pixels having the same level may be stored as this level added by the count of consecutive pixels, for which this condition holds.
  • a data value range (or similar colour range) is provided, which extends the run length as explained above to those pixels, which merely have similar - but not necessarily the same - data values, or colours respectively.
  • An approximate data value is chosen instead and attributed to each of the pixels involved, along with the pixel count, which reduces the necessary memory space to store the image information.
  • the approximate data value may be a mean value of the data values involved, a median value or the most frequent one of the data values, etc.
  • the value range may further be adapted (e.g., shifted) to include subsequently read in data values, until one subsequent data value inevitably falls out of this range as the difference to a smallest or largest one becomes too large.
  • a counter is reset, e.g. to zero, the previous count and the approximate data value are stored, and a new value range is selected for further subsequently read in data values.
  • This method of compression of data in-between one image may be realized by an appropriate software implemented with regard to the VPU 40, CPU 41 or another controller 35, which could replace both processing units 40, 41 shown in this embodiment.
  • Reference numerals 85a, 85b indicate those compression units or decompression units, respectively, arranged in hardware or software.
  • a decision unit and a value range adjusting unit such as that shown in Fig. 1 of the EP application (reference numerals 14, 18 therein) may be realized as ASIC's (application specific integrated circuits) between the processing units 40, or 41 , and memory buffer 49 according to the present embodiment.
  • a still further embodiment relates to the memory buffer 49.
  • the buffer 49 into the graphics memory of a graphics device.
  • the VPU may "capture" (select) data from a first frame buffer in the memory and compare these data with those of a second frame buffer, which represents the buffer 49 shown in Fig. 4. Again, time for capture and memory space is saved to considerable amounts.
  • Examples of advantageous buffer management configurations are provided in the European patent application EP05108275.8 of Agilent Technologies, Inc, the specific features of which, particularly shown in Figs. 3, 4, 5, and 7 and corresponding description therein may be combined with the present invention and are thus incorporated herein by reference.

Abstract

The invention refers to updating memory buffer with image data to be provided to a visualization device (10), comprising capturing first data of a next image and providing said data to a processing unit (35, 40, 41); reading second data values from a memory buffer (49) into the processing unit (35, 40, 41), the second data values being associated with pixels of data of a previous image, comparing pixel by pixel (90) a respective first data value associated with a pixel from the next image data with a respective second data value associated with a pixel from the previous image data, setting a flag bit (96) in response to said comparison, if for one pair of corresponding pixels (90) according to both images the data values are not the same, or a difference between the data values exceeds a pre-defined threshold, continuing with comparing a next pair of pixels (90) as long as the flag bit (96) is not set or all pixels are compared, and stop continuing with comparing a next pair of pixels (90) and at least partly updating the memory buffer (49) with data based on the first data values, if the flag bit (96) is set.

Description

DESCRIPTION
CONDITIONAL UPDATING OF IMAGE DATA IN A MEMORY BUFFER
TECHNICAL FIELD
[0001] The present invention relates to buffering image data.
BACKGROUND ART
[0002] In the past two decades, the growth in the sector of personal computers (PC's) has been accompanied by an increase in networking capabilities. Multiple computers including specific devices such as printers or other peripherals are grouped into networks such as local area networks (LAN) or wide area networks (WAN). Those networks are optionally controlled by dedicated server computers (i.e., one or more servers). As the networks become more and more locally distributed, the need to perform control of individual components in a remote fashion increases in importance.
[0003] This becomes particularly valid in the case of system maintenance, e.g., with respect to servers. Efforts and costs may be reduced when the graphic output, or visual response to an action, is redirected to a remote computer for display thereon. The person performing system maintenance may then carry out the necessary tasks, or may obtain a system diagnosis, etc., from his remote location without the need to move to, e.g., the server room and log on to the server computer of interest. Servers, computers and other logic devices in a network are referred to herein as processing devices.
[0004] Remote visualization means that an image to be displayed locally is copied or completely redirected to a remote data processing device, which is different from the local data processing device. Remote visualization allows, among others, remote system maintenance, e.g., by a hardware or software vendor, testing or performing helpdesk functions for end users, etc.
[0005] Visualization may be realized by providing a graphics device, which transforms a digital representation of an image provided by, e.g., a central processing unit (CPU) of a local data processing device into signals which lead a monitor to display that image. The signals may be analog, e.g., in the case of CRT monitors, or digital as often in the case of LCD-displays. A redirection of graphic information may be performed by adding a separate capture device to the system that captures the image contents and transfers it to a predetermined remote system.
[0006] The graphics device and the capture device may communicate with each other, e.g., via an internal bus of the respective local processing device. The graphics device uses a memory unit. This memory unit contains the data of the digital representation of the image to be displayed on a local monitor. In order to make these data remotely available, the data are first captured by a processing unit provided with the capture device, typically a card which is input as an additional component into a slot of the local computer.
[0007] The image data are then stored into a memory buffer, which is also provided with the capture device. This memory buffer relates to a dedicated capture memory, which is - among others - needed to take into account the reduced bandwidth of a LAN connection. The memory buffer thus receives a copy of an image and provides an opportunity to perform compression on the image data.
[0008] The image data, in other words: the digital representation of an image, may be captured in different modes: one active mode is to address the graphics device and then simply copy the digital data from the video memory via a bus (e.g., PCI) into the memory buffer of the capture device. Another passive mode is to monitor the information when being transferred from the video memory of the graphics device to the local graphics display (local monitor). In case of analog VGA-signals, an additional A/D converter is then required to yield a transformation into a digital representation of the image. Digital DVI-signals, e.g., may be recorded directly.
[0009] Compression may be performed by reducing the information available per pixel of an image (data values associated with a pixel), e.g., the bit length or number of colours. It is also possible to apply reduction schemes such as those known from the JPEG or MPEG-formats, wherein data values of adjacent pixels are related to each other.
[0010] Another approach is to transfer only data values of those pixels towards a remote location which have changed in respective next image data with regard to previous image data. U.S. Patent Application No. 6,701 ,380 B2, to Schneider, W.J. et al., discloses a method of performing visualization of image data on a remote processing device. Therein, image data are divided into blocks of pixels, and the change of data values associated with pixels inside one block between two subsequent images is indicated by setting a flag bit in a register. The flag bit is associated with this specific block and is later used to select this block upon transferral of data values associated with the pixels of this block towards the remote location.
SUMMARY OF THE INVENTION
[0011] It is an object of the invention to provide an improved buffering of image data.
[0012] The object is solved by the independent claims. Further embodiments are shown by the dependent claims.
[0013] According to embodiments of the present invention a method of performing visualization of image data on a remote processing device includes steps of capturing data of a next image and providing the next image data to a processing unit. The capturing of next image data implies that previous image data are already present, e.g., in the capture device.
[0014] The processing unit may thereby be located on the capture device, which is dedicated to perform the remote transfer. According to one embodiment, the capture device, which performs the method steps, houses a video processing unit (VPU) as well as a central processing unit (CPU). Both may be part of a single device, i.e., a remote transfer controller. In this context, the processing unit being provided with captured next image data represents the CPU, while the next image data are captured by the VPU. In the case of analog video signals (e.g., VGA signals), an additional A/D converter may be placed upstream of the VPU. However, the invention is not limited to the specific implementation of different or unique processing units and associated memories on a dedicated capture device. One embodiment relates to implementing the components, which perform or enable the method steps as described herein, directly on the graphics device of a local computer. [0015] Image data as referred to herein are digital representations of images, and comprise an arrangement of pixels, which are each associated with a data value, which represents, e.g., a colour. A pixel is in this respect a predefined coordinate position within an array of data, and further has a predefined length of bits. The data values are taken from a combination of the specific binary values currently associated with these bits.
[0016] The processing unit, for example, selects a first subset of pixels of the next image data. The subset may according to an embodiment represent an arrangement of pixels organized in rows and columns. First data values, which are associated with these pixels of the subset, are then determined. This determination may comprise the step of providing the bit values, which define a respective data value of a pixel, into a dedicated position within a register of the processing unit.
[0017] In parallel, but not necessarily simultaneously, second data values are read out from a memory buffer and into the processing unit, e.g., into a similarly dedicated further position in said register. The second data values are associated with pixels of data of a previous image, wherein said pixels of the previous image data correspond to said pixels of the new image data. The term "corresponding" means, that the pixel location of a respective first data value within the next image data is the same as that of a second data value within the previous image data.
[0018] The memory buffer, which keeps the previous image data, is according to one embodiment a buffer located on the capture device. In an alternate embodiment, it relates to the graphics, or video memory located on the graphics device. The invention is not limited to the specific arrangement of this buffer with respect to the processing unit. However, a direct association of the memory buffer with the processing unit, i.e., a short traveling distance with respect to signal is preferred as the loading from and the writing to this buffer turns out to represent a bottleneck for the bandwidth of data transfer.
[0019] Next, a data value of the first image data is compared with a respective one of the second image data for each of the pixels in the subset. This means that a pixel- wise check is performed, whether the data values have changed or not. For example, both data values may be selected from the respective register positions and directly compared.
[0020] The result of this comparison step performed for each of the pixels is stored by setting a flag bit. According to a preferred embodiment, there is one separate flag bit associated with each subset. For example, a logical "1" (flag bit set) indicates that a change has occurred, while an initial value of "0" (flag bit not set) indicates no change has occurred, yet.
[0021] According to this embodiment, there is one flag bit for the current subset of pixels. Once this flag bit is set, the parallel provision of first data values and reading of second data values is terminated. Starting from this event, the incoming first data values are directly written into the memory buffer to overwrite the respective data values of the first image data, therein. Further, at least the currently compared pixel data values, which have lead to the detection of a data value change, are written to the memory buffer for updating the previous image. Pixel by pixel, and subset by subset, the "previous" image data are thus altered into the next image data.
[0022] The direct write mode of first data values of the next image data into the memory buffer means that the reading step of second data values into the processing unit is effectively stopped. This stop of data transfer from the memory buffer to the processing unit is limited to the presently selected subset of pixels. According to one embodiment, each different subset of pixels, which is selected from the next image data, is associated with its own flag bit. Thus, while the steps relating to one subset might have already lead to the event of a direct write of first data values, the corresponding steps relating to another subset may include a full read out of second values accompanied with a comparison with first data values for all pixels in the respective subset.
[0023] The termination of the read step may considerably reduce the time needed to perform the memory update using the next image data. In case each subset, which may also referred to as a block of pixels within image data, is associated with a flag bit, the flag bits may be stored in a change matrix, which indicates the occurrence of changes in the respective blocks of image data with respect to a previous image. This change matrix is used to control the transfer to the remote processing device. Only those blocks represented by an associated flag bit, which is set (thus attaining for example a logical "1"), are transferred according to this embodiment. The current settings in the change matrix, which may be stored in a buffer or register associated with the processing unit, correspond to the actually updated image in the memory buffer.
[0024] Therefore, time is reduced by only reading out (second) data values from the memory buffer until a change is detected, generating an event and stopping further read out of data values in response to this event. It is found that once compression is performed by means of a change matrix, which reduces traffic via LAN for example, the data transfer from the memory buffer to the processing unit for the purpose of comparison with new image data becomes a dominant contributor to the bandwidth. The bandwidth is defined here as data throughput per time unit.
[0025] The effect of the invention thus is that data values are either read out from the memory buffer into the processing unit, or are written from the memory unit into the memory buffer. Consequently, only one transfer bandwidth of images (and not two) between the memory buffer and the processing unit is needed - with the exception of those one or few data values of a subset, which have lead to the detection of an change event, which were read out, compared and written.
[0026] Only those data values are read and written, which lead to the detection of a change. In case that more data values have to be read into the processing unit and compared therein simultaneously, for example all data values of pixels organized in a row (i.e., a subgroup) of a block or subset, more than just one data value is read and rewritten into the memory buffer according to this embodiment. In this context, the phrase "read and rewrite" necessarily includes an update of data values of previous image data in the memory buffer.
[0027] As will be described in the following, previously known approaches needed twice the transfer bandwidth, plus an additional write step the into the memory buffer with regard to the next image, which is thereby intermediately stored.
[0028] It is noted that embodiments of the invention can be partly or entirely embodied or supported by one or more suitable software programs, which can be stored on or otherwise provided by any kind of data carrier, and which might be executed in or by any suitable data processing unit. Software programs or routines can be preferably applied with respect to operating the kernel functions of the processing unit on a capture device (of a remote management unit, RMU), e.g., a remote controller. The same is valid if the remote controller is located within a graphics device having direct access to the video memory.
BRIEF DESCRIPTION OF DRAWINGS
[0029] Other objects and many of the attendant advantages of embodiments of the present invention will be readily appreciated and become better understood by reference to the following more detailed description of embodiments in connection with the accompanied drawings. Features that are substantially or functionally equal or similar will be referred to by the same reference signs.
[0030] Figs.1 and 2 show block diagrams of a remote system connected to a local system for the purpose of remote image display according to prior art;
[0031] Fig. 3 illustrate the flow of data through a capture device according to prior art;
[0032] Fig. 4 illustrate a flow of data through capture device including a CPU, a VPU, a memory buffer and a remote interface according to a first embodiment of the invention;
[0033] Fig. 4B illustrate a flow of data through capture device including a CPU, a VPU, a memory buffer, a remote interface, a compression device and a decompression device according to a further embodiment of the invention; and
[0034] Fig. 5 shows a schematic representation of an image divided into subsets, an associated change matrix and a counter table for flags within the change matrix.
DETAILED DESCRIPTION OF THE INVENTION
[0035] Fig. 1 shows an embodiment of a remote processing device (computer) 10 connected to a local processing device (computer, or server) for the purpose of remote visualization of an image, the arrangement being known in the art. The remote processing device 10 is connected with a remote display 12. A CPU 20 of the local processing device (right hand side of the sinuous line) is connected to an internal bus 25 such as a PCI-bus or PCI-Express bus. A graphics device 52 is also connected to the internal bus. The graphics device comprises a video memory 62 for storing a digital representation of the image data to be displayed on a local monitor 22. The display of an image is managed by means of a display controller 37 arranged on the graphics device 52. Further components 21a, 21 b of the local system may also be connected to the internal bus 25.
[0036] A connection of the local processing device with the remote processing device 10 is established by the capture device 30. The capture device 30 is also connected to the internal bus 25 and comprises a capture memory 32 and a remote interface 33. Capturing of image data is accomplished by addressing - and accessing - the video memory 62 via the internal bus 25 with help of a capture device controller 35 (remote transfer controller, not shown in Fig. 1 ) and retrieving full image data. The full image data are buffered in the capture memory 32, compressed and forwarded to the remote data processing unit via remote interface 33. The flow of redirected image data is indicated by the dashed line in the figures. An example flow of image data generated in the CPU 20 intended for local display is indicated by the dot-dashed line
[0037] Fig. 2 shows an alternate embodiment known in the art. Herein, the capture device is arranged to retrieve image data from a connection between the display controller 37 and the local monitor 22. The data transferred herein are less redirected than snarfed, i.e., duplicated for local and remote display. As the data have already been processed by the display controller, the format of transferred data considerably differs between the embodiments of Figs.1 and 2. In particular, the embodiment shown in Fig. 2 may involve the snarfing of analog VGA signals, which are to be converted into digital data representing the image by an A/D converter 29, first.
[0038] Fig. 3 shows a capture device 30, which may be embodied as shown in Fig. 2, in more detail. The remote transfer controller 35 in this embodiment comprises a video processing unit 40 (VPU) and a central processing unit 41 (CPU). The VPU 40 is designed to capture image data and to process the incoming data. In particular it performs bit- or pixel-wise comparison of image data in order to detect changes. It further arranges the reading from and writing to the capture memory 32. The CPU 41 of the capture device 30 controls the flow of image data, for example it initiates the capture of image data and organizes the compression of data to be transferred (numeral 70 in Fig. 3) via remote interface 33.
[0039] The capture memory 32, which may be, e.g., one DRAM memory chip, comprises three or more memory buffers 45, 46, 47. A conventionally known flow of image data is described as follows: as indicated by the arrows in Fig. 3, data of a first captured image are processed by VPU 40, and stored as "previous" image data in a first memory buffer 45. Then, a next image is captured, processed and stored as next image data in the second memory buffer 46.
[0040] Both image data - the previous and the next - are then retrieved from the respective memory buffers 45, 46 by CPU 41. Data values corresponding to pixels from both images, which are associated with each other, are compared to detect a change. A difference image is thus formed, which is stored in the third buffer 47. The capture and storage of one image takes 10 - 20 ms of time. The calculation of the difference image lasts for 50 - 200 ms depending on the image size. Having applied conventional compression techniques, the transferal of the compressed image via remote interface 33 (e.g., LAN) takes another 20 ms. One image update at the remote location thus amounts to 80 - 240 ms according to this illustrative embodiment. Roughly 5 - 10 frames per second may thus be realized at the remote location, which may be sufficient to indicate the addition of text on the remote monitor 12. However, changes such as smooth mouse movements developing to abrupt jumps thereby are displeasingly followed by an observer according to this two-pass-dataflow.
[0041] An embodiment of the invention is shown in Fig.4A. Next image data are converted into digital data by A/D converter 29, and in case of a first image are directly written into memory buffer 49 by means of VPU 40. These initially stored image data will then be referred to as "previous" image data, since a further image is captured by VPU 40. Being digitized, the next image data are transferred successively to the VPU 40 as a contiguous stream of captured digital data 82.
[0042] Simultaneously, interchangeably, or adjacently to the capture, digital data 84 are read out with respect to the previous image from the memory buffer 49 and into the
VPU 40. The read out of data from the memory buffer is coordinated with the incoming data from the A/D converter 29. In particular, digital data of the next image form bytes or words, which contain data values with respect to pixels of the next image. What is read out from the memory buffer are digital data corresponding to respective pixel positions of the previous image. The data values formed from the digital data are then compared in VPU 40 in order to detect a change or not.
[0043] According to this embodiment, the images (previous and next) are subdivided into the same subsets of pixels. For example, an image being formed from 1024 (rows) * 768 (columns) pixels may contain subsets, or blocks, formed from 16 * 16 pixels. Accordingly, one such image has 64 * 48 blocks. The event of detecting a change is handled with respect to each subset, or block, separately.
[0044] A change may be signaled when the data values of corresponding pixels in the previous and next image are not the same, i.e., they differ. Alternatively, a relaxed condition may be set up, stating that the difference between these data values must exceed a predetermined threshold level in order to represent a change. This latter condition represents an advantageous aspect, as it enables further compression of data to be transmitted to the remote location. Small fluctuations of, e.g., colours from image to image will herein be disregarded from an assessment as a change.
[0045] As mentioned above, a change is tracked for each subset separately. Until a change is detected with regard to one subset, digital data 82 are captured and digital data 83 are read into the VPU 40. Once a change is detected, the data transfer mode is also changed. For this specific subset, the read out of digital data 84 from the memory buffer 49 is stopped along with the comparison step. Rather, the captured digital data 82, or data values corresponding to respective pixels of the subset, are directly written as digital data 83 into the memory buffer 49. More precisely, buffer locations in memory buffer 49, which keep digital data related to associated pixels of the previous image, are overwritten with the digital data 83 of the next image, regardless of whether they represent a further change or not.
[0046] Accordingly, the communication between the memory buffer 49 and the VPU merely comprises either reading digital data 84 or writing digital data 83. Only, those specific data values, which correspond to the event of a detected change, are read in to VPU 40 as "aged" digital data 84 as well as (re-)written as updated digital data 83 into the memory buffer 83. If only one data value at a time is read in and compared in the VPU 40, there is consequently one pixel out of 256 (16 times 16 in a block according to this embodiment), which necessitates twice a bandwidth. The present embodiment essentially refers to a one-pass procedure of updating image data.
[0047] A further advantage of this embodiment, visible from Fig.4, is that memory is saved as the memory buffer is simply updated by the VPU 40. As a result, less bandwidth and slower as well as cheaper memory chips are necessary.
[0048] The embodiment detailed above may further be refined by means of a change matrix 75. The detection of change may be realized by setting a flag bit, e.g., from logical "0" to "1" or vice versa. As each subset or block has its own flag bit, the resulting 64 * 48 flag bits may be arranged in a corresponding change matrix. This change matrix indicates those subsets of an image in which changes have taken place.
[0049] The change matrix may be stored in memory buffer 49, in a separate storage area, or in registers of the VPU 40 or CPU 41. The latter CPU 41 also controls compression as explained above. The change matrix 75, which is each time updated according to the present method, may therefore easily be exploited for the purpose of compression, e.g., transferring only digital data from changed blocks to a remote location (along with the information, which block the digital data actually refer to). Particulars of a change matrix for the purpose of data compression are disclosed in the Patent EP 1168154 B1 to Agilent Technologies, Inc., the specific features of which may advantageously be combined with the invention and are thus incorporated herein by reference.
[0050] The task of remote data transfer is facilitated by CPU 41 via communication lines 72 to the VPU 40 and 71 to the remote interface 33, e.g., a LAN-, or WAN- interface, wherein CPU 41 initiates interface 33 to retrieve data from memory buffer 49 in dependence of entries (flag bits) within change matrix.
[0051] As shown in Fig. 5, digital data of an image 88 are captured, or read in by the VPU 40 successively pixel by pixel 90 and row by row 92. As a row 92 extends in an image beyond one subset 94, flag bits of different subsets 94 may be maintained alternately until a final row of respective subsets is reached. Maintenance of multiple flag bits within a change matrix 75 becomes particularly advantageous herein, when data from one subset are not contiguously arranged in the incoming data stream from A/D converter 29, or a DVI interface, etc.
[0052] Associated with the change matrix is further a counter table 77. Each flag bit 96 in the change matrix 75 is associated with a counter 98 in the counter table 77. The counter 98 helps to impede aging of subsets 94, which are repeatedly not updated according to the present method. A threshold value of 25, as an arbitrary example value may be predefined, and with each run through an image capture and update, the counter of a flag bit 96, or subset 94, is increased by 1 , as an example. If the number of 25 is reached for a subset, that has no changed pixel value, the flag bit is set and the update is performed for this subset 94 irrespective of whether a change presently occurs or not.
[0053] This approach with counters may be advantageous as in the case, that similar but no identical data values between adjacent images are admitted, a slight drift in data values may occur leading to an inconsistency in image data. Therein, admittance (no setting of a flag bit) of slight differences, when comparing first with second data values saves transmittance bandwidth, and may thus simultaneously be advantageous. An advantageous embodiment refers to an application, wherein additional screens are slowly faded in to or out of the image, respectively.
[0054] It is clear that after a update the flag bit has to be reset, e.g., to a logical "0" in order to enable the next run through the method steps without a change being indicated.
[0055] As is visible from Fig.4B, a further refinement may be made by compressing the digital data 83 already when being written from the VPU 40 to the memory buffer 49, and decompressing the digital data 84, when being read out from the memory buffer 49 into the VPU 40. In an advantageous embodiment, a so-called run length encoding (RLE) may be performed. Therein, data values of consecutive pixels having the same level may be stored as this level added by the count of consecutive pixels, for which this condition holds. An improved method thereof is disclosed in the European patent application EP05102300.0 of Agilent Technologies, Inc., the specific features of which may be advantageously be combined with the invention and are thus incorporated herein by reference.
[0056] According to the compression method described therein, a data value range (or similar colour range) is provided, which extends the run length as explained above to those pixels, which merely have similar - but not necessarily the same - data values, or colours respectively. An approximate data value is chosen instead and attributed to each of the pixels involved, along with the pixel count, which reduces the necessary memory space to store the image information. The approximate data value may be a mean value of the data values involved, a median value or the most frequent one of the data values, etc.
[0057] Upon execution of the method, the value range may further be adapted (e.g., shifted) to include subsequently read in data values, until one subsequent data value inevitably falls out of this range as the difference to a smallest or largest one becomes too large. A counter is reset, e.g. to zero, the previous count and the approximate data value are stored, and a new value range is selected for further subsequently read in data values.
[0058] This method of compression of data in-between one image may be realized by an appropriate software implemented with regard to the VPU 40, CPU 41 or another controller 35, which could replace both processing units 40, 41 shown in this embodiment. Reference numerals 85a, 85b indicate those compression units or decompression units, respectively, arranged in hardware or software. For example, a decision unit and a value range adjusting unit such as that shown in Fig. 1 of the EP application (reference numerals 14, 18 therein) may be realized as ASIC's (application specific integrated circuits) between the processing units 40, or 41 , and memory buffer 49 according to the present embodiment.
[0059] A still further embodiment relates to the memory buffer 49. Thereby, it is possible to integrate the buffer 49 into the graphics memory of a graphics device. This means that the VPU may "capture" (select) data from a first frame buffer in the memory and compare these data with those of a second frame buffer, which represents the buffer 49 shown in Fig. 4. Again, time for capture and memory space is saved to considerable amounts. Examples of advantageous buffer management configurations are provided in the European patent application EP05108275.8 of Agilent Technologies, Inc, the specific features of which, particularly shown in Figs. 3, 4, 5, and 7 and corresponding description therein may be combined with the present invention and are thus incorporated herein by reference.

Claims

1. A method of updating a memory buffer (49) with image data for transmission to a visualization device (10), comprising:
(a) capturing first data of a next image and providing said data to a processing unit (35, 40, 41 );
(b) reading second data values from a memory buffer (49) into the processing unit (35, 40, 41 ), the second data values being associated with pixels of data of a previous image;
(c) comparing pixel by pixel (90) a respective first data value associated with a pixel from the next image data with a respective second data value associated with a pixel from the previous image data;
(d) setting a flag bit (96) in response to said comparison, if for one pair of corresponding pixels (90) according to both images the data values are not the same, or a difference between the data values exceeds a pre-defined threshold;
(e) continuing with comparing a next pair of pixels (90), as long as the flag bit (96) is not set or all pixels are compared; and
(f) stop continuing with comparing a next pair of pixels (90) and at least partly updating the memory buffer (49) with data based on the first data values, if the flag bit (96) is set.
2. Method according to claim 1 , comprising:
dividing the next and previous image data into a plurality of image subsets (94),
selecting a first subset (94) of pixels of the next image data and determining first data values associated with each pixel of the subset;
repeating steps (b) - (f) for each of said subsets of the next image data to update the previous image data with next image data in the memory buffer (49); and transferring the updated image data (88), or a portion thereof, from the memory buffer (49) to the visualization device (10).
3. Method according to claim 1 or any one of the above claims, wherein:
the first data values of pixels (90) of the next image are provided to the processing unit (35, 40, 41 ) simultaneously with reading in the second data values of pixels (90) of the previous image data into the processing unit (35, 40, 41 ).
4. Method according to claim 1 or any one of the above claims, further comprising associating each one separate flag bit (96) with each one separate subset (94) of the previous image data stored and then updated in the memory buffer (49).
5. Method according to claim 4 or any one of the above claims, further comprising storing said separate flag bits (96) in a matrix (75), which indicates changes of the previous image data.
6. Method according to claim 1 or any one of the above claims, wherein transferring the updated image data (88) from the memory buffer (49) to the remote processing device (10) includes compressing the image data.
7. Method according to claim 4 or any one of the above claims, wherein transferring the updated image data (88) from the memory buffer (49) to the remote processing device (10) includes compressing the image data (88) by selecting data values from pixels (90) of subsets (94) in dependence of a setting of a respective flag bit (96) associated with the subset (94).
8. Method according to claim 4 or any one of the above claims, further comprising:
repeating the capturing of further images, updating respective previous images and transferring the updated images (88) to the remote processing device (10);
associating each of said separate flag bits (96) with a separate counter (98),
adding a fixed value to each of said counters (98) each time the sequence of method steps is repeated for a respective next image; comparing each of said the counters (98) with a predefined threshold value, which represents a maximum age of a non-updated subset of image data (88); and
setting a flag bit (96) associated with a subset (94) in response to said comparison of the counter (98), for initiating an update of the previous image data (88) in the memory buffer (49) with the data values associated with pixels (90) of a subset (94) which has exceeded said maximum age.
9. Method according to claim 1 or any one of the above claims, wherein reading second data values from a memory buffer (49) into the processing unit (35, 40, 41 ) includes extracting and decompressing said data.
10. Method according to claim 9 or any one of the above claims, wherein the decompression is performed by a run length decoding (RLE), a first data value being stored in the memory buffer (49) by means of a fixed data value and a count of consecutive pixels, which are all associated with this data value.
11. Method according to claim 1 or any one of the above claims, wherein writing the third data values into the memory buffer (49) includes compressing the image data.
12. Method according to claim 11 or any one of the above claims, wherein the compression is performed by run length encoding (RLE), a second data value being written into the memory buffer (49) in step (f) by setting a fixed data value and a count of consecutive pixels, which are all associated with this fixed data value.
13. Method according to claim 12 or any one of the above claims, wherein the fixed data value is set by determining an approximate data value, said approximate data value being determined from multiple data values of consecutive pixels (90), which reside within a value range defined by a lower and an upper threshold.
14. A software program or product, preferably stored on a data carrier, for controlling or executing the method of claim 1 or any one of the above claims, when run on a data processing system such as a computer.
15. An interface device for receiving incoming image data representing a sequence of images, comprising:
a capturing device (29) adapted for capturing first data of a next image,
a memory buffer (49) adapted for storing second data to be provided to an image device (10), and
a processing unit (35, 40, 41 ) adapted for:
reading the first data of the capturing device (29) and the second data values from the memory buffer (49), comparing pixel by pixel (90) a respective first data value associated with a pixel from the next image data with a respective second data value associated with a pixel from the previous image data;
setting a flag bit (96) in response to said comparison, if for one pair of corresponding pixels (90) according to both images the data values are not the same, or a difference between the data values exceeds a pre-defined threshold;
continuing with comparing a next pair of pixels (90) of the subset (94), as long as the flag bit (96) is not set or all pixels are compared; and
stop continuing with comparing a next pair of pixels (90) and at least partly updating the memory buffer (49) with data based on the first data values, if the flag bit (96) is set.
PCT/EP2005/056116 2005-11-21 2005-11-21 Conditional updating of image data in a memory buffer WO2007057053A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/EP2005/056116 WO2007057053A1 (en) 2005-11-21 2005-11-21 Conditional updating of image data in a memory buffer

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/EP2005/056116 WO2007057053A1 (en) 2005-11-21 2005-11-21 Conditional updating of image data in a memory buffer

Publications (1)

Publication Number Publication Date
WO2007057053A1 true WO2007057053A1 (en) 2007-05-24

Family

ID=36589091

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/EP2005/056116 WO2007057053A1 (en) 2005-11-21 2005-11-21 Conditional updating of image data in a memory buffer

Country Status (1)

Country Link
WO (1) WO2007057053A1 (en)

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101872293A (en) * 2009-04-23 2010-10-27 威睿公司 Method and system for copying a framebuffer for transmission to a remote display
CN101873308A (en) * 2009-04-23 2010-10-27 威睿公司 Method and system for identifying drawing primitives for selective transmission to a remote display
ES2474266A1 (en) * 2013-07-25 2014-07-08 Mar�a Cristina CARMONA DUARTE Procedure to compare and quantify the similarity between the image drawn on a touch screen with digital images (Machine-translation by Google Translate, not legally binding)
WO2016016607A1 (en) * 2014-07-31 2016-02-04 Displaylink (Uk) Limited Managing display data for display
CN107656108A (en) * 2017-11-09 2018-02-02 光科技股份有限公司 Suitable for the fast appearing methods of RCM and device of electric-power metering
US10365881B2 (en) * 2016-04-25 2019-07-30 Seiko Epson Corporation Image supplying apparatus, method for controlling image supplying apparatus, and program
CN113228596A (en) * 2018-12-04 2021-08-06 区块链联合香港有限公司 Method and device for transmitting list information

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4958378A (en) * 1989-04-26 1990-09-18 Sun Microsystems, Inc. Method and apparatus for detecting changes in raster data
US5168269A (en) * 1990-11-08 1992-12-01 Norton-Lambert Corp. Mouse driven remote communication system
US20030169264A1 (en) * 2002-01-04 2003-09-11 Emerson Theodore F. Operating system independent method and apparatus for graphical remote access having improved latency
US6701380B2 (en) * 1997-08-22 2004-03-02 Avocent Redmond Corp. Method and system for intelligently controlling a remotely located computer

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4958378A (en) * 1989-04-26 1990-09-18 Sun Microsystems, Inc. Method and apparatus for detecting changes in raster data
US5168269A (en) * 1990-11-08 1992-12-01 Norton-Lambert Corp. Mouse driven remote communication system
US6701380B2 (en) * 1997-08-22 2004-03-02 Avocent Redmond Corp. Method and system for intelligently controlling a remotely located computer
US20030169264A1 (en) * 2002-01-04 2003-09-11 Emerson Theodore F. Operating system independent method and apparatus for graphical remote access having improved latency

Cited By (21)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10572214B2 (en) 2009-04-23 2020-02-25 Vmware, Inc. Method and system for identifying drawing primitives for selective transmission to a remote display
EP2244182A3 (en) * 2009-04-23 2011-06-08 VMWare, Inc. Method and system for identifying drawing primitives for selective transmission to a remote display
US11397553B2 (en) 2009-04-23 2022-07-26 Vmware, Inc. Method and system for identifying drawing primitives for selective transmission to a remote display
US11003412B2 (en) 2009-04-23 2021-05-11 Vmware, Inc. Method and system for identifying drawing primitives for selective transmission to a remote display
US9448760B2 (en) 2009-04-23 2016-09-20 Vmware, Inc. Method and system for identifying drawing primitives for selective transmission to a remote display
EP2244183A3 (en) * 2009-04-23 2011-06-08 VMWare, Inc. Method and system for copying a framebuffer for transmission to a remote display
AU2010201050B2 (en) * 2009-04-23 2012-03-29 VMware LLC Method and system for copying a framebuffer for transmission to a remote display
US8441494B2 (en) 2009-04-23 2013-05-14 Vmware, Inc. Method and system for copying a framebuffer for transmission to a remote display
CN101872293A (en) * 2009-04-23 2010-10-27 威睿公司 Method and system for copying a framebuffer for transmission to a remote display
US8902242B2 (en) 2009-04-23 2014-12-02 Vmware, Inc. Method and system for identifying drawing primitives for selective transmission to a remote display
JP2010257454A (en) * 2009-04-23 2010-11-11 Vmware Inc Method and system for copying framebuffer for transmission to remote display
CN101873308A (en) * 2009-04-23 2010-10-27 威睿公司 Method and system for identifying drawing primitives for selective transmission to a remote display
JP2010282609A (en) * 2009-04-23 2010-12-16 Vmware Inc Method and system for identifying drawing primitive for selective transmission to remote display
US10067732B2 (en) 2009-04-23 2018-09-04 Vmware, Inc. Method and system for identifying drawing primitives for selective transmission to a remote display
ES2474266A1 (en) * 2013-07-25 2014-07-08 Mar�a Cristina CARMONA DUARTE Procedure to compare and quantify the similarity between the image drawn on a touch screen with digital images (Machine-translation by Google Translate, not legally binding)
WO2016016607A1 (en) * 2014-07-31 2016-02-04 Displaylink (Uk) Limited Managing display data for display
US10365881B2 (en) * 2016-04-25 2019-07-30 Seiko Epson Corporation Image supplying apparatus, method for controlling image supplying apparatus, and program
CN107656108A (en) * 2017-11-09 2018-02-02 光科技股份有限公司 Suitable for the fast appearing methods of RCM and device of electric-power metering
CN113228596A (en) * 2018-12-04 2021-08-06 区块链联合香港有限公司 Method and device for transmitting list information
US11838349B2 (en) 2018-12-04 2023-12-05 Hong Kong Sunstar Technology Co., Limited Method and device for transmitting list information
CN113228596B (en) * 2018-12-04 2024-04-02 香港星辰科技有限公司 Method and device for transmitting list information

Similar Documents

Publication Publication Date Title
US8255900B2 (en) Virtual computer server apparatus, and update image detection method
US5943064A (en) Apparatus for processing multiple types of graphics data for display
US7944451B2 (en) Providing pixels from an update buffer
CN107493448B (en) Image processing system, image display method and display device
US7423642B2 (en) Efficient video frame capturing
WO2007057053A1 (en) Conditional updating of image data in a memory buffer
CN106030652B (en) Method, system and composite display controller for providing output surface and computer medium
CN114051145B (en) Video compression processing method, device and medium
CN115297094B (en) Video transmission control method, device, equipment and computer readable storage medium
JP2004280125A (en) Video/graphic memory system
US11616895B2 (en) Method and apparatus for converting image data, and storage medium
CN115209145A (en) Video compression method, system, device and readable storage medium
CN113205779A (en) Multi-screen synchronous control method, device and system suitable for electronic ink screen
CN107027069B (en) Processing method, device and system, storage medium and the processor of image data
KR102619668B1 (en) Apparatus and method of using a slice update map
US6313839B1 (en) Method and apparatus for performing Z buffer depth comparison operations
JP2952780B2 (en) Computer output system
US6717989B1 (en) Video decoding apparatus and method for a shared display memory system
CN108024116B (en) Data caching method and device
US9471956B2 (en) Graphic remoting system with masked DMA and graphic processing method
US20080055286A1 (en) Method And Apparatus For Displaying Bitmap Images
CN115174915A (en) Image processing system and method for reducing network bandwidth occupation
EP1762928A1 (en) Graphics device comprising remote transfer controller for remote visualization
CN114090168A (en) Self-adaptive adjusting method for image output window of QEMU (QEMU virtual machine)
US5543822A (en) Method for increasing the video throughput in computer systems

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application
NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 05813668

Country of ref document: EP

Kind code of ref document: A1