WO1991015829A1

WO1991015829A1 - Method and apparatus for creating a single composite image of a plurality of separate radiographic images

Info

Publication number: WO1991015829A1
Application number: PCT/US1991/002405
Authority: WO
Inventors: John Wai-Chu Wong; Tin Chi Leung; Mary Vogelsang Graham
Original assignee: Washington University
Priority date: 1990-04-11
Filing date: 1991-04-10
Publication date: 1991-10-17
Also published as: AU7689891A

Abstract

A method and apparatus for generating a single, composite, anatomically aligned radiographic image from a prescription or reference image and a plurality of separate radiographic images includes an electronic display (30), a general purpose computer (20), a hard disk drive (22) for storing electronic data representative of the individual image and a mouse (28) to permit the on-screen designation of anatomical points for each image by an operator. The method includes the steps of designating a plurality of landmark points on each image, aligning the image to correlate to a reference image through alignment of the landmark points, scanning the image to detect the irradiated area, creating a summary image, and displaying the summary image over the reference image.

Description

METHOD AND APPARATUS FOR CREATING A SINGLE COMPOSITE IMAGE OF A PLURALITY OF SEPARATE RADIOGRAPHIC IMAGES

Background and Summary of the Invention

Historically, radiation treatments have been prescribed for, administered to, and verified for patients through the use of rad ographic film.

For a typical patient, a doctor will prepare a pres¬ cription for the patient*s treatment with the use of a simulation or SIM film. This reference image shows the appropriate anatomical view of the patient which the prescribing doctor will then mark up to provide guidance to the radiotherapy technologists for the placement of radiopague blocks to mask out all but that portion of the anatomy desired to be treated. This prescription or SIM film will then be used by the radiotherapy personnel to design one or more radiopague blocks for application to the patient dur¬ ing the treatment sessions. For the typical patient for which radio¬ therapy has been prescribed, a daily session extend¬ ing over several weeks can be expected. In order to monitor and verify the area of treatment and dosage received, typical protocol now prevalent in the medi¬ cal industry provides for the exposing of several radiographic films at the beginning of the treatment in order to verify the placement of the various radi¬ opague blocks which surround and define the area of a treatment. Then, another film would typically be taken during each treatment in order to verify that the blocks are being placed in the proper location and that the patient is receiving proper treatment over the prescribed area of the anatomy. If errors in placement of the blocks or in placement of the patient or in any other regard are found during the course of treatment, then the physician must pres¬ cribe corrective treatment in an attempt to apply correct dosages to the proper portions of the pa- tient's anatomy.

Because of the time delay reguired to take the film, process it, and present it to the attending physician, it is present practice for a physician to review the treatment or port film immediately before the first one or two sessions and then subseguently view a film taken to verify treatment only once a week. During these weekly reviews, the physician would typically display the multiple number of film images side by side on a light box to make visual comparisons between the treatment image and a veri¬ fication image in order to verify that the radiopague blocks were generally in the correct positions to mask that portion of the anatomy reguired to be masked and radiate that portion of the anatomy re- quired to be radiated. Although this was the best that could be done with the available technology for many years, this methodology had many drawbacks. The film is ex¬ posed typically for only a few seconds out of a treat- ent length of 30-40 seconds. Thus, there is no as¬ surance that the patient has not moved during the course of treatment and such movement would not be captured on film or discovered by the physician. Any error in treatment would be discovered after the fact as film would typically be reviewed by the attending physician after the treatment has been given. Prior to treatment when film for verification is exposed, the technicians might perhaps take extra special care to properly place the patient and blocks but the pa- tient or blocks might move after the film is exposed. Irradiating various portions of the body which are not susceptible to immobilization necessarily creates errors in treatment which are undetected. All of these inaccuracies, and others, render the use of film for verifying treatment as an undesirable alter¬ native but one which has been previously the only methodology available for that purpose.

With the advent of technology, x-ray treat¬ ment machines are being developed and have become available which are more sophisticated in applying the field to the patient. In other words, newer machines are capable of conformal shaping of the field to suit the desired anatomical area of treat¬ ment as opposed to utilitizing additional external blocks to modify or block those portions of the beam which would otherwise reach the patient. Also, the number of fields used to treat a particular portion of a patient is being expanded in order to achieve greater selectivity of the tissue desired to be irradiated. To achieve this, newer machines have computer-controlled multiple collimating or blocking leaves to shape the beams and are also capable of rapidly reorienting these beams during treatment rather than using the typical co-planar orientation used with older machines. With these newer method¬ ologies, the use of film becomes problematic in that many films will have to be used, within an interval of seconds, for verification of each beam.

As a substitute for film, one of the in- ventors herein is the co-inventor of the invention disclosed in the parent application hereto which per¬ mits the generation of an on-line electronic image fully representative of each treatment session and which records in real time the relative position of the patient's anatomy with that of the blocks and the beam as it is used to irradiate the patient. As a result, much more accurate treatment verification can take place for each treatment session through the ac¬ quisition of images at any time during the treatment. Thus, for the first time, the physician is presented with a wealth of information in multiple images re¬ cording the actual treatment at any point in time that the beam irradiates a patient. However, it is unreasonable to expect that physicians or technicians will be able to take full advantage of this wealth of information merely due to the time required to wade through it all and analyze it for purposes of deter¬ mining the treatment applied to a particular patient over the course of a 4-6 week series of treatments. In the prior art, some attention has been paid to aligning several frames of film in an attempt to determine how a patient's treatment has progressed. Fiducial points which are external to the patient and which appear in the radiographic images have been used to locate the patient in the field for treatment and others have used these for lining up a series of images. Also, the blocks themselves have been used in an attempt to line up multiple frames. Still others have chosen one or two landmark points on a prescription or simulation film to compare with a port film recording treatment to determine whether the actual treatment received in any one given session was equivalent to that which was prescribed. However, for the reasons expressed above, the patient's posi- tion in the field can vary such that lining up more than two or three frames of film results in an in¬ comprehensible composite image. As discussed above, in the prior art, a series of film images are typical¬ ly viewed serially and separately with no real serious attempt at creating a composite image.

In order to solve these and other problems in the prior art, and for the first time to enable a physician to take full advantage of the multitude of images generated by the later technology as evidenced by the invention disclosed in the parent application, the inventors herein have succeeded in developing a method and apparatus for lining up a multiple number of radiographic images to create a composite image which is significantly important and helpful to a physician in not only determining the treatment the patient has already received but to also design sub¬ sequent treatment in order to correct for any misap¬ plications of dosage. Essentially, the invention comprises the use of a general purpose computer, dis- play, and storage medium such as a disk drive which permits the individual images to be sequentially dis¬ played for viewing by a technician or the like. The technician may then use a mouse or other pointer to locate three or more anatomical landmark positions on the image, typically points on the patient's skeletal formation, which correspond to those same points se¬ lected by the prescribing physician. These landmarks are thus determined and the operator repeats this designation of landmark points through the entire series of images. Once these landmarks have been designated, each treatment image is compared with the prescription image and the computer automatically rotates, magnifies, or translates each treatment image to bring the images into correlation. Image magnification can be fixed or pre-determined if the position of the imaging device with respect to the patient is known. At this point, the computer pro¬ gram scans the treatment image and detects the area of treatment as opposed to the area masked from treatment, and stores that information in a summary file. The next treatment image is then processed in the same manner such that the areas of treatment for each treatment session are accumulated in the summary file. Thus, the summary file describes how the block positions have cumulated throughout the treatment. At any point in the treatment, a technician or phy¬ sician may call up the reference image with the sum¬ mary file superimposed thereon to thereby present a composite image representative of the accumulated treatment history for that particular portion of the patient's anatomy.

For the first time, a physician can now focus his attention on the variation in positioning between the blocks and the surrounding anatomy that has been identified in the treatment prescription, a variation which is critical in determining the dosage of treatment applied to the patient. Initial studies have demonstrated that there is a range of applica¬ tion of treatment defining a gradient from fully treated areas to fully blocked areas of the anatomy. Furthermore, the images may be processed to designate portions of tissue receiving, for example, 0%, 20%, 40%, 60%, 80%, and 100% of treatment areas. The data is susceptible to other statistical analysis, as de- sired. Thus, for the first time a physician is pre¬ sented with data corresponding to the actual treat¬ ment which the patient has received, clearly identi¬ fying those portions of tissue which have mistakenly received irradiation in an amount greater or less than desired. In some radiotherapy treatment ses¬ sions, this can be critical. For example, in radia¬ tion therapy protocols using adjoining fields, fields which overlap produce a spike of dosage while fields which are not "abutted" leave a gap in treatment. While various protocols exist in the prior art to smooth out this treatment, the method and apparatus of the present invention accurately verifies the ex¬ act dosages received by adjoining fields which will lead to the development of new protocols and will also dramatically increase the accuracy at which radiotherapy may be applied.

While the principal advantages and features of this invention have been described above, a fuller understanding of the invention may be obtained by re- ferring to the drawings and detailed description of the preferred embodiment which follow. Brief Description of the Drawings

Figure 1 depicts the electronic display system of the present invention; Figure 2 is the flow chart for the program to create the composite summary image;

Figure 3 is the flow chart for the program which reviews and edits the treatment summary image;

Figure 4 is the flow chart for the program which adds a new portal image (treatment image) to the summary image; and Figure 5 is the flow chart for the program which deletes a portal image (treatment image) from the summary image.

Detailed Description of the Preferred Embodiment As shown in Figure 1, a general purpose computer 20 has a hard disk drive 22 or other kind of floppy disk drive or the like which receives image data through a disk 24 which is inserted therein, or on-line from the imaging device disclosed in the parent application. A keyboard 26 and mouse 28 pro¬ vide operator control through computer 20 of the dis¬ play 30.

As shown in Figure 1, a patient's torso 32 is partially blocked by two blocks 34, 36 and the composite image depicted therein also shows a par¬ tially shaded area 38, 40 which illustrates the tran¬ sition between tissue totally blocked from irradia¬ tion by blocks 34, 36 and tissue which receives full dosage irradiation through the series of treatments. Thus, the composite or summary treatment image has been anatomically aligned with the prescription or simulation image to visually display to a physician those portions of the patient's anatomy which have received no treatment, ..full treatment, and some por- tion in between. For our purposes of illustrating the capabilities of the present invention, only one shaded portion 38, 40 has been shown. However, it is understood that a summary image of 10 port or treat¬ ment images may show as many as ten different shades each of which is illustrative of a different level of treatment. For example, one area of tissue may have been blocked from irradiation all ten times, while another area of tissue may never have been blocked, these two areas defining the outer limits of possible exposure. In between these two outer limits are found tissue areas having been exposed for some por¬ tion or percentage of the total treatment time.

A flow chart for the application program which creates the composite image is shown in Figure 2. As shown in Figure 2, the program first goes through a routine which initiates a new file, or loads a pre-existing summary file and then permits an oper¬ ator to designate three anatomical points in each image. With the teachings of this invention, the operator designates those three prominent points on the patient's skeletal anatomy which have been se¬ lected by the prescribing physician as being readily ascertainable from image-to-image. After the three points for a treatment image have been designated, a transformation matrix is calculated which electroni¬ cally "shifts" the image and the values of the pixels for that image into as close a one-to-one correspon¬ dence with a reference image as can be obtained. After being shifted, the treatment image is scanned to determine that portion of the anatomy which has been treated and, for each pixel determined to be in the treated area, an arbitrary value of 1 is added to the corresponding pixel in the summary image. Thus, a summary image file is created which accumulates pixel intensities with an arbitrary value of 1 for each treatment image. Thus, for ten treatment images, the summary image would have some pixels with an in¬ tensity of ten for those areas irradiated all ten times, down to a summary pixel value of zero for all areas which are not irradiated for all ten treatments. The rest of the program then smooths the data, deter¬ mines specific statistical boundaries (as desired) and makes the data available for display as well as being stored. For a complete display, the program combines the reference image which is the prescrip- tion or simulation image with the summary or treat¬ ment image and overlays them. This is necessary as the simulation image is the only image with anatomi¬ cal information while the treatment summary image is merely a composite of the relative intensity to be added to various pixels in the simulation image in order to display a gradient of treatment from least irradiated to most irradiated.

The program which is depicted by the flow chart in Figure 3 permits an operator to review and edit the treatment summary image. The first or mid¬ dle branch of the flow chart loads the proper treat¬ ment summary image into the machine and permits an operator to select either the left or the right branch for editing the summary. The left branch permits an operator to reselect three anatomical reference points for the reference image and then reselect correspond¬ ing new anatomical reference points in each of the portal or treatment images. This left branch also utilizes the program of flow chart shown in Figure 4 for reprocessing each of the images previously stored in memory. This branch of the review process permits an operator to select three different anatomical ref¬ erence points for the reference image in order to correct for previous error, or in order to make a second try at selecting anatomical points which are either more prominent or are more reliably present in the same physical orientation throughout the treat¬ ment. The right branch of the review and edit treatment summary flow chart shown in Figure 3 per¬ mits an operator to correct any one or more of the portal or treatment images for which the anatomical reference points have been inaccurately selected by the operator who input the data into the reference image. This right branch also incorporates the pro¬ gram of the flow chart of Figure 5 which deletes the particular treatment image corresponding to the old data with the incorrectly selected anatomical refer- ence points. Thus, if an operator suspects or notices that any one of the portal or treatment images has been loaded into the machine by an operator who made a mistake or inaccurately selected the anatomical reference points, he can correct the reference image by editing out the old treatment image for that par¬ ticular treatment and re-entering the same treatment image with reselected anatomical reference points.

There are various changes and modifications which may be made to the invention as would be ap- parent to those skilled in the art. However, these changes or modifications are included in the teaching of the disclosure, and it is intended that the in¬ vention be limited only by the scope of the claims appended hereto.

Claims

What Is Claimed Is:

1. A device for correlating a plurality of radiographic images with a prescription image and creating a single composite image therefrom, each of said radiographic images representing a separate radiation treatment session for substantially the same portion of a patient's anatomy wherein one or more radiopaque blocks are used and are depicted in said separate images for shielding surrounding por¬ tions of said patient's anatomy, said prescription image representing the same portion of the patient's anatomy wherein the position of said one or more radiopaque blocks is determined by a prescribing phy¬ sician, said device comprising means for electroni¬ cally storing said prescription image and said plur- ality of radiographic images, means for sequentially displaying each of said prescription and said radio¬ graphic images, means for permitting an operator to designate a plurality of anatomical landmarks on each of said images, means for automatically aligning said radiographic images with said prescription image by aligning said anatomical landmarks, and means for displaying said aligned images in a single composite image to thereby display the variation in the pa¬ tient's anatomy irradiated through the course of said treatment sessions.

2. The device of Claim 1 further comprising means for an operator to re-designate said anatomical landmarks for any one or more of said radiographic images.

3. The device of Claim 1 further comprising means for an operator to re-designate said anatomical landmarks on the prescription image.

4. The device of Claim 1 wherein said plural¬ ity of anatomical landmarks comprises at least three points.

5. The device of Claim 4 wherein said at least three points are on the patient's skeleton.

6. The device of Claim 1 wherein said auto¬ matic aligning means includes a least squares fit¬ ting of the anatomical points in the radiographic image to the anatomical points in the prescription image.

7. The device of Claim 1 further comprising means for creating a summary image from said plural¬ ity of radiographic images, said summary image being representative of those portions of the anatomy ir- radiated for each of said treatment sessions.

8. The device of Claim 7 wherein said summary image creating means comprises means for scanning each of said aligned radiographic images, means for detecting the irradiated portion of the patient's anatomy in each of said images, means for storing a pre-determined value for each of said irradiated por¬ tions, and means for creating a summary image from said stored pre-determined values; said displaying means including means for combining said summary image with said prescription image for display.

9. The device of Claim 8 wherein each of said radiographic images is comprised of a plurality of pixels, and said storing means comprises means for storing a pre-determined value for each pixel de- tected by the detecting means as being in the irradi¬ ated portion.

10. A device for correlating a plurality of radiographic images with a prescription image and creating a single composite image therefrom, each of said radiographic images representing a separate radi- ation treatment session for substantially the same portion of a patient's anatomy wherein one or more radiopaque blocks are used and are depicted in said separate images for shielding surrounding portions of said patient's anatomy, said prescription image rep- resenting the same portion of the patient's anatomy wherein the position of said one or more radiopaque blocks is determined by a prescribing physician, said device comprising means for sequentially inputting and displaying the prescription image and said plur- ality of radiographic images, means for permitting an operator to designate a plurality of anatomical land¬ marks on each of said images, means for automatically aligning each of said radiographic images with said prescription image by aligning said anatomical land- marks, means for detecting the irradiated portion of the patient's anatomy in each of said radiographic images, means for creating a summary image represen¬ tative of said detected irradiated anatomical por¬ tions, and means for displaying said prescription image overlaid with said summary image to thereby display the variation in the patient's anatomy ir¬ radiated through the course of said treatment ses¬ sions.

11. The device of Claim 10 wherein said summary image creating means comprises means for scanning each of said aligned radiographic images, said radiographic images being comprised of a plurality of pxxels, means for detecting those pixels of each radiographic image which represent irradiated portions of the patient's anatomy, means for storing a pre-determined fixed value corresponding to the pixel position for each of said irradiated portions, and means for summing said pre-determined fixed values for all of said radio¬ graphic images at each of said pixel locations, said summed pre-determined fixed values being representa¬ tive of the intensity for their associated pixel in the summary image; said displaying means including means for combining said summary image with said pre¬ scription image for display.

12. The device of Claim 11 further comprising means for determining boundaries of said summary image which comprise pre-determined percentages of said sums of said pre-determined fixed values.

13. A method for generating a single, composite, anatomically aligned radiographic image from a pre¬ scription image and a plurality of separate radio¬ graphic images comprising the steps of: sequentially displaying on an electronic display means each of said prescription image and said separate radiographic images, each of said radiographic images representing a separate radiation treatment session for substantially the same portion of a patient's anatomy wherein one or more radiopaque blocks are used and are depicted therein for shield¬ ing surrounding portions of said patient's anatomy; designating a plurality of anatomical landmarks on each of said prescription and radio- graphic images by an operator, said anatomical land- —lb-

marks being coincident for each of said images; aligning each of said radiographic images with said prescription image by aligning said operator designated anatomical landmarks; and displaying said aligned images in a single composite image to thereby display the varia¬ tion in the patient's anatomy irradiated through the course of said treatment sessions.

14. The method of Claim 13 further comprising the step of creating a summary image from said plur¬ ality of radiographic images, said summary image being representative of the irradiated portions of the patient's anatomy.

15. The method of Claim 14 wherein the step of creating the summary image comprises the steps of: scanning each of said aligned radio¬ graphic images, said radiographic images each being comprised of a plurality of pixels; detecting those pixels of each radio¬ graphic image which represent irradiated portions of the patient's anatomy; storing a pre-determined fixed value representative of the pixel position for each of said radiated portions; and summing said pre-determined fixed values at each of said pixel locations, said sum of said pre-determined fixed values being representative of the intensity for their associated pixel.