WO2008099040A2

WO2008099040A2 - Method for analysing, viewing and processing digital biomedical images

Info

Publication number: WO2008099040A2
Application number: PCT/ES2008/000079
Authority: WO
Inventors: Maria Gloria Bueno Garcia; Roberto Gonzalez Morales; Manuel Torres Prado
Original assignee: Universidad De Castilla-La Mancha
Priority date: 2007-02-14
Filing date: 2008-02-14
Publication date: 2008-08-21
Also published as: ES2303790B1; ES2303790A1; WO2008099040A3

Abstract

The invention relates to a method for analysing, viewing and processing digital biomedical images, in which: at least one digital biomedical image in a first size Igi is stored in a remote server (20, 21); the biomedical image is rescaled to a second size image Im (1); an initial value is established in order to associate the second size image Im (1) with a third size image Iv (4) for the initial execution of the method; a fragment (2) to be viewed is selected from the second size image Im or the third size image Iv, said fragment being determined by initial (xi, yi) and final (xf, yf) coordinates; a magnification (3) is selected for said fragment; a request containing at least the initial and final coordinates and the magnification for the second size image Im (1) or the third size image Iv (4) is sent to the remote server; a fragment of the first size image Ig, corresponding to said initial and final coordinates and magnification, is recovered by means of rescaling and the fragment is viewed on the third size image Iv.

Description

ANALYSIS PROCEDURE. VISUALIZATION AND PROCESSING OF BIOMEDICAL DIGITAL IMAGES.

OBJECT OF THE INVENTION The present invention relates to a method for the remote analysis of biomedical digital images for viewing, processing and diagnosis. The method of the invention performs the analysis, visualization and processing of biomedical images of different formats, sizes and for different pathologies.

BACKGROUND OF THE INVENTION

At present there is no tool or procedure that analyzes, visualizes, manages and processes automatically and also remotely and independently of the digitalization format, biomedical digital images, with dimensions of several Gbytes, as certainly are the images digitized in Biomedicine and particular in the specialty of Pathological Anatomy.

Currently, the only thing that exists is a remote viewing methodology, to visualize images up to the order of 2 Gbyte, with fragment division protocols. The methodology is known by Zoomifyer. Zoomifyer was developed in the USA. and it allows to visualize images of up to 1, 5 Gbytes of formats BMPs, TIFs, JPGs, and GIFs, in web environment. The method is to copy

The image many times at different levels of resolution, dividing it in turn into hundreds of JPG files; This is what is known as pyramidal strip structure. This technology has a number of drawbacks:

The division of the image into thousands of pieces deteriorates the image (loss and corruption of files).

The division of the image means that when viewed at different magnifications it is pixelated into pieces. There is a limitation up to the order of 2 Gbytes. There is limitation of the type of format. - The original image Ia transforms JPGs into an image, which can lead to a loss due to compression, which makes it difficult to process the image with exact results. The analysis of the image is limited only to the visualization. Zoomifyer makes use of US5968120, ('Method and system for providing on-line interactivity over a server-client network ') for the remote client-server transmission prior to the fragmentation of images by means of the pyramidal methodology, previously creating different files of the same image region at different resolutions to be able to visualize it. This pyramidal methodology is described in US Patent US-A-5760783, ('Method and system for providing texture using a selected portion of a texture map'). The problems and limitations of this methodology are those listed previously.

There are other patented methodologies related to procedures for visualizing images but all with limitations to achieve the necessary objectives for analysis (both visualization and processing) and management of Biomedical images. The following patent documents are mentioned by way of example: Korean patent application KR-A-20040104070, ('Interactive Multimedia Image Viewer Using Flash / Server Side Technology and Datábase'), allows to analyze images only for viewing but in images of video that carries a limited capacity of 500Mb. European patent application EP-1646201 -A1 ('Client dependent image processing for browser- based image document viewer for handheld client devices') allows you to view static images, stored on a disk drive, without analysis and processing features and with the same limitation of handling images greater than 500Mb.

Applied to the medical field there is, for example, the Japanese patent application JP-A-2005312683 ('Medical Image Distribution System and Its Viewer Terminal Equipmenf), which describes a storage and visualization system to perform only preprocessed and contrast of the image, to enhance or improve its resolution facilitating its visualization. In addition to being limited by the processing, it is limited in the type of image, associated with the radiological image and therefore its dimensions and its flexibility from the point of view of remote analysis. Also, US Patent Application US-A1 -2002/0133373, ('Integration of radiology information into an application sen / ice provider DICOM image archive and / or web based viewer'), describes a system that manages image information by integrating Ia DICOM image along with the radiological information. Limited only to the radiological image, not biomedical, whose standard to DICOM is not defined; Furthermore, the radiological image does not have the problems of dimensions of the biomedical image or the problems of color management. So This tool itself has unilateral capabilities, that is, it does not allow sharing between several users (medical specialists).

DESCRIPTION OF THE INVENTION The invention relates to a method of analysis, visualization and processing of biomedical digital images according to claim 1. Preferred embodiments of the procedure are defined in the dependent claims.

The present invention provides a method of analysis, visualization and processing that solves the problem of automatically analyzing (managing), visualizing and processing biomedical digital images, without limit of dimensions (Gbytes), of various formats and it is also possible to do it remotely, by means of the inventive matter included in the attached independent claims. The invention relates to a method of analysis, visualization and processing of biomedical digital images that as a methodology has its own architecture that allows the decompression of any image in any format - jpeg, jpeg2000, tiff, bmp, ppm, pgm, gif, svs - and dimension to a single decompressed file free of header and compression information that keeps the 'raw' information obtained by digitizing the image, for later viewing and processing.

According to a first aspect of the invention, this refers to a method of analysis, visualization and processing of biomedical digital images, comprising: - storing at least one biomedical digital image in a first size / _g or image on a remote server original; re-scaling said original image to an image in a second size I _n ;;, selecting in said image in the second size / _m a fragment to be displayed - preferably it will be enlarged - whose fragment is determined by initial coordinates and coordinated finals; select an increase for said fragment to be displayed, the increase being defined by a user; send to said remote server a request containing at least said initial coordinates and said final coordinates and said increase in the image in the second size / _m ; recover by re-scaling a fragment of the biomedical digital image in the first corresponding size I ₉ said initial coordinates and said final coordinates and said increase and display it in an image to a third size l _v .

Preferably said second size of the image is smaller than the first size of the original digital biomedical image.

The selection of the fragment to be displayed can also be made in said image in the third size / „or image to be displayed. In this case, an initial value can be established to relate said image in the second size l _m with said image in the third size l _v , for the first execution of the procedure. In this case, the sending to said remote server of a request containing at least said initial coordinates and said final coordinates and said increase will be referred to on the image in the third size l _v . As just described, this procedure of analysis, visualization and processing of biomedical digital images allows dealing with huge images (biomedical digital image at the first size I ₉ ) without dimension limit and without loss or deterioration, avoiding data inconsistency and duplication of data. information because it maintains a single image and manages its access for viewing and processing through its relationship with another thumbnail image (biomedical digital image at the second size / _m ).

Thus, in the process of the invention, a user - through, for example, the corresponding user interface - selects the fragment or the image region that is intended to be displayed, and with what level of magnification or zoom that fragment is to be displayed. This is done on the biomedical digital image at a second size l _m resulting from rescaling the original biomedical digital image that is at a first size I ₉ . The request that includes the magnification or zoom applied and the initial and final coordinates of said fragment within the image in the second size / _m , is sent to a remote server that returns the image fragment to be displayed from the original image to the First size I ₉ , this original biomedical digital image is not fragmented or divided, but is a single file without size limitation, stored on a server. The size of the file is not what should really be displayed; this allows to manage images of values of dimensions of the Gigabyte. Through the procedure outlined only access that file, the region from which the fragment or region to be managed or visualized (small fragment) must be extracted, in order to show it to the user. In this way the memory to be used and the fragment to be transmitted is of small dimensions (eg, approximately 800x800). When treating the original biomedical digital image as a single file, limitations of size and duplication of information are avoided, which makes the procedure of the invention capable of solving the problem of the management and analysis (visualization and processing) of images of larger enormous dimensions 2 gigabytes In this way, there is also no need to maintain a database that contains all the information of correspondence of tiles that requires, for example, Zoomifyer for the visualization of the image.

The method of the invention can also comprise the detection in said fragment - fraction of I ₉ or globality of I ₉ in case of 1 ^* increase - of the biomedical digital image to the first size I ₉ of one or more regions of interest by:

- statistical analysis of said fragment, determining at least four centroids or values with greater frequency in color coordinates;

- Clustering by fuzzy logic by grouping pixels in different classes, one class for each centroid, minimizing the color distance of the value of each pixel to the centroid of the class;

- determination of membership values of each pixel in a class, detecting the one or more regions of interest.

Said one or more regions of interest can also be analyzed using architectural and morphological criteria, producing a set of specific descriptors for each type of biomedical digital image, so that they provide relevant information to the user (specialist) and thus facilitate the final diagnosis of the sample. (represented by the biomedical digital image).

A series of morphometric and / or densitometric descriptors can be applied to said one or more regions of interest detected. The process of analysis and processing of the invention also allows inserting comments, as well as their subsequent management (modification of comments, acceptance, inclusion of new comments) and recovery thereof with their corresponding re-positioning, to the increase in which it was made , as an integral part of the images. Preferably the process of analysis and processing of the invention includes management and processing tools that are associated with said biomedical image.

The increase for the fragment to be displayed can be 2 *, 10 ^* , 20 ^* , 40 ^* or 80 ^* ; It can be selected by the user and / or come selected by default.

In case the increase is greater than 20 *, said fragment is preferably divided into six or more classes. In this case, another clustering by diffuse logic is preferably done iteratively in the image of two or more regions of interest of the same hue to detect other regions of interest. The design of the preferred architecture for the procedure is through a Web service, which also allows incorporating such functionalities added to the visualization and processing (remote and parallel), such as: insertion of comments in a collaborative environment and image processing; collaborative environment in which different users can share comments and opinions about certain regions of the image in real time.

These functionalities are preferably executed remotely; Any Biomedical digital image can be analyzed including Biomedical images that are sent remotely.

Preferably, the relevant diagnostic information is included in medical forms, also complementing the management of information databases of clinical-medical interest.

In addition, preferably all communication of files and sending information, both image and management, is done through secure communication protocols (ssh). The progressive use of this secure transmission protocol allows the transmission of new cases (of any size) remotely without posing problems.

There is currently no tool that provides collaborative tools for the insertion and management of diagnostic comments in biomedical images. Being possible to access and manage these comments simultaneously and in real time through the web by several user specialists, in addition to sharing opinions in real time before or during the diagnostic notes.

According to the characteristics that have been defined in the foregoing for the process of the present invention, this allows: - The visualization of images with different dimensions reaching the order of GBytes.

- The integration of several different formats: Jpeg2000, .jpeg, .tif., .Bmp, .ppm, .raw, .pgn, .gif, svs etc. Although the use of image formats such as .bmp, .jpeg, .ppm and .gif is extended, the Jpeg2000 format is a new feature of this procedure, as well as the integration of all of them for the management and processing of biomedical images.

In addition, according to the preferred implementation of the invention, the procedure includes: - Collaborative tools for the insertion and management of comments in biomedical images: it is possible to access and manage these comments simultaneously and in real time through the web by several user specialists, in addition to sharing opinions in real time before or during diagnostic annotations. - Multiple processing of different biomedical diagnostic images: the procedure integrates a set of methods for the processing of diagnostic images, adapting each processing to the analysis needs of said images. It allows to analyze histopathology, oncological radiotherapy, neurology and breast pathology images, through the processing of TMA (Tissue MicroArray), H&E (Hematoxylin and Eosin), Cytologies, CT (Axial Tomography

Computerized), MRI (Magnetic Resonance), Angiographies and Mammograms.

- Visualization to different magnification or zoom for images of different sizes adapting to the specific zooms used, particularly, in the pathology specialty to favor the analysis of the images. The response time of this zoom is immediate for any type of images whatever their dimensions.

BRIEF DESCRIPTION OF THE DRAWINGS

Next, a series of drawings that help to better understand the invention and that expressly relate to an embodiment of said invention that is presented as a non-limiting example thereof is described very briefly.

Figure 1 shows a general scheme of an application in which the method of the invention is implemented. Figure 2 shows how biomedical images can be viewed at different zooms. And the possibility of measuring areas of interest.

Figure 3 shows a screen of the application corresponding to a Preprocessing of the image. Figures 4, 5 and 6 show a screen corresponding to the Insertion,

Management and Recovery of comments, respectively.

Figures 7, 8A-8B and 9A-9B, show examples of processing biomedical images, H&E for cell detection, DO7 images and TMA images, respectively. Figure 10 shows examples of the integration of the diagnostic information into other information relevant to the diagnosis and prognosis.

Figure 11 shows the physical elements that make up a possible embodiment for the system that implements the method of the invention.

DESCRIPTION OF A PREFERRED EMBODIMENT OF THE INVENTION

Figure 1 shows a general scheme of an application in which the method of the invention is implemented. In this figure 1 indicative elements of the description of the invention are shown, such as:

- Image re-scaled to the second size I _n , (1) of the original biomedical image I ₉ .

- Selection in said image in the second size l _m or in the third size / „of a fragment (2) to be displayed, fragment is determined by initial coordinates xi, yi and final coordinates xf, yf.

- The panel for the selection of an increase (3) between 2 ^* , 10 ^* , 20 ^* , 40 ^* or 80 ^* . - Zone of visualization of the image in the third size l _v (4), image recovered from the original image I ₉ by rescaling from the coordinates (xi, yi) e (xf, yf) and the magnification (3 ) selected:

The processing and measurement panels (5) where the method of the invention includes tools for processing the biomedical image. The procedure thus allows the user to select and perform the processing of the image or fragment displayed at a third size l _v .

-The comment panels (6) where the procedure of the invention includes collaborative tools for the insertion and management of diagnostic comments in the biomedical images. Inside the comment panel (6) a field (6a) appears with the coordinates of the position of the comment in the image.

Figure 2 shows how biomedical images can be viewed at different zoom or magnification (3). And the possibility of measuring areas of interest. In the illustrative example, it is shown how the fragment (2) selected in

The image at the second size I _n , (1) for a magnification (3) selected of 40 *, said fragment of the biomedical digital image in the first size I ₉ corresponding to said initial coordinates and said coordinates has been recovered final and said increase and has been visualized to a third size l _v (4). In addition, Figure 2 illustrates a work session to make measurements of regions of interest, selecting the points (A, B) of interest to be measured on the image at the third size l _v (4). The measurements are applied when using panel analysis options (5) and are considered as processing and values of diagnostic interest. Figure 3 shows a screen of the application corresponding to an image processing consisting of a variation of the contrast on the image to the third size l _v (4), obtained by using panel analysis options (5).

Figures 4, 5 and 6 illustrate collaborative tools of the procedure, through comments associated with the biomedical images. These are tools used simultaneously and in real time via the web by several user specialists.

The application has the possibility of inserting comments (see figure 4), selecting the area of interest where you want to make a comment with the mouse.

The tool also allows the integral management of these comments, and thus it is possible to delete, modify or include new comments (see figure 5). And also recover a comment with its re-positioning to the increase in which it was made (see figure 6).

The insertion of comments is carried out by means of a mouse click in the area of interest where the fragment of I ₉ has been visualized to a third size l _v (4). As shown in Figure 6, the management of comments is carried out by selecting with a mouse click on the areas or regions of interest, after which a form appears to insert a comment. This can be modified, deleted and / or add additional comments. It is also possible to recover the area to which the comment refers in the original format in which it was inserted, is say, to the corresponding image magnification.

The recovery of comments by locating them in the inserted area and at the specified zoom is carried out by selecting the coordinates of the comment to be recovered by clicking on the panel (6). Figures 7, 8A-8B and 9A-9B show examples of processing of biomedical digital images, of different types and formats. That is, it can be detected in l _g or in the fragment displayed in l _v one or more regions of interest.

Figure 7 illustrates the detection of regions of interest in images stained with Hematoxylin-Eosin (H&E) by clustering by fuzzy logic to four centroids statistically determined in I ₉ or fragment visualized in l _v in HSI color model and jointly visualizing two of said centroids corresponding to nuclei and nucleoli.

Figure 8A illustrates the analysis of DO7 images for processing at 40 ^* (Figure 8B) detecting the regions of interest according to their staining levels. Five centroids have been identified by clustering by minimizing the color distance of the value of each pixel to the centroid of the class.

Figure 9A illustrates the analysis of microarray images (TMA) for processing at 10 ^* (Figure 9B) detecting the regions of interest by clustering by fuzzy logic to four statistically determined centroids in I ₉ or fragment visualized in l _v in model of CIELab color.

Figure 10 shows examples of the integration of the diagnostic information into other information relevant to the diagnosis and prognosis. It is shown through two illustrations:

Figure 10B shows the analysis of said one or more regions of interest by means of architectural, morphometric and / or densitometric criteria, producing a set of specific descriptors, for each type of biomedical digital image, to help diagnose said biomedical digital image. .

As an illustrative example, Figure 10B shows the number of regions of interest detected in the analyzed area, the dimension of said regions, the region of interest that has acquired staining of the applied reagent, the color coordinates of said region of interest.

Figure 10A shows the analysis performed on the descriptors, which can be integrated into a relevant clinical history and information corresponding to a patient to whom said biomedical digital image belongs. History The diagnosis produced by the image processing is integrated into a database implemented and integrated into the procedure, which contains other information relevant to the joint diagnosis, from patient characteristics to characteristics of the sample to be analyzed from which the Ia has been acquired. biomedical image I ₉ .

The application is designed following a model of three independent layers, but interconnected:

- Domain Layer: represents the logical content of the application, being able to say that it is the computational part of it. - Presentation Layer: it is oriented to present the information to the user.

- Persistence Layer: stores the context on the operation of the viewer on a server.

Based on this three-layer model, the design of a database and the classes necessary to implement the domain layer are elaborated in parallel.

The viewer of the application is implemented as a Web Service to accommodate, remotely, users using this server as "engine".

It also includes options for image processing and operations that facilitate actions such as the change between the format of images or their storage.

The viewfinder utilities implemented in a preferred embodiment of the invention, illustrated in Figures 1 to 10, are:

- Image management.

- Image preprocessing: zoom, change brightness and contrast. - Edition: comment management, point of interest measurements.

Both the loading, as the display and preprocessing of an image is done quickly in a few seconds.

Figure 11 shows the physical elements that make up a possible embodiment for the system that implements the method of the invention, for use of the application remotely and parallel processing.

It has three essential components in its architecture:

Application Web Server 10. The main function of the server is to remotely manage the possible requests that users can simultaneously make (request to store new images in the cluster, visualization, analysis or processing of a certain image). In turn, it maintains a database management system 11 with the information corresponding to the images themselves and to the collaborative part of the application environment. - Cluster 20 with MPP (Massively Parallel Processors) architecture. The cluster enables remote execution of various parallel processing tasks (detection of areas of interest, descriptors, general algorithms). Communication between nodes has been designed using MPI (message passing interface). The cluster is also used as a mass storage device 21 for the images managed by the application, as well as for the results obtained after processing. The images can be stored directly in the cluster through external physical means (through a digitalization equipment, such as a digital microscope 30) or through the request of the clients 40. The connection between the cluster and the server It has been designed using a secure connection protocol for several purposes:

- Update in the server database the possible new cases that enter by direct connection to the cluster (scanning equipment). Request to execute processing tasks for a given image.

- Transmission of complete images.

- Display tasks.

Clients 40. Present the information to the user and enable the interaction of the same with the proposed application. It communicates with the web server through a secure remote communication protocol.

The invention has been described according to a preferred embodiment thereof, but for the person skilled in the art it will be evident that multiple variations can be introduced in said preferred embodiment without exceeding the object of the claimed invention.

Claims

1. A method of analysis, visualization and processing of biomedical digital images, characterized in that it comprises: - storing at least one biomedical digital image in a first size / _g on a remote server (20, 21); rescaling said biomedical image to an image in a second size I _n , (1); - establish an initial value to relate said image in the second size / _m (1) with an image in a third size l _v (4) for an initial execution of the procedure; select in said image in the second size I _n , (1) or in said image in the third size l _v (4) a fragment (2) to be displayed, whose fragment is determined by initial coordinates (xi, yi) and final coordinates (xf, yf); - select an increase (3) for said fragment to be displayed; send to said remote server a request containing at least said initial coordinates and said final coordinates and said increase in the image in the second size / _m (1) or in the image in the third size l _v (4); recover by re-scaling a fragment of the biomedical digital image in the first size I ₉ corresponding to said initial coordinates and said final coordinates and said increase and display it in said image to a third size l _v (4).

2. Method according to claim 1, characterized in that it further comprises detecting in said fragment of the biomedical digital image at the first size I ₉ one or more regions of interest by:

3. Method according to claim 2, characterized in that said one or more regions of interest are also analyzed by means of architectural, morphometric and / or densitometric criteria, producing a set of specific descriptors, for each type of biomedical digital image, to aid in a diagnosis of said biomedical digital image.

4. Method according to claim 3, characterized in that an analysis of the descriptors can also be integrated into a relevant clinical history and information corresponding to a patient to whom said biomedical digital image belongs.

5. Procedure according to any of the preceding claims, characterized in that it also includes inserting comments as well as their subsequent management -modification of comments, acceptance, inclusion of new comments-, and recovery thereof with their re-positioning to the increase in which He did, as an integral part of biomedical digital images.

6. Method of analysis and processing of biomedical digital images according to any of the preceding claims, characterized in that said increase is 2 ^* , 10 ^* , 20 ^* , 40 ^* or 80 ^* .

7. Method according to any of the preceding claims, characterized in that in case of an increase greater than 20 ^* said fragment is divided into six or more classes.

8. Method according to any of the preceding claims, characterized in that in case of an increase greater than 20 ^* another clustering is done iteratively by diffuse logic in the image of two or more regions of interest of the same hue to detect other regions of interest.

9. A program product comprising means of program instructions for carrying out the procedure defined in any of claims 1-8.

10. A product according to claim 9, stored in a program support medium.