WO2014085918A1

WO2014085918A1 - System and method of viewing digital medical images

Info

Publication number: WO2014085918A1
Application number: PCT/CA2013/001049
Authority: WO
Inventors: Vernon COLACO; Vittorio Accomazzi
Original assignee: International Medical Solutions, Inc.
Priority date: 2012-12-03
Filing date: 2013-12-18
Publication date: 2014-06-12
Also published as: WO2014085918A4

Abstract

A system for viewing medical images may use a web browser to preload the images on a portable viewing device prior to a user requesting the images. This system may reduce the time that the user has to wait for viewing the images, and may allow viewing of the images on the portable device in an offline mode in which the device is able to visualize and manipulate the images while not connected to the Internet. Such an approach permits a zero foot print viewer to be utilized without the need to install additional software on the portable viewing device beyond a compatible web browser. Such an approach also permits the portable viewing device to display medical images with the web browser when there is no active digital network connection for the portable viewing device.

Description

SYSTEM AND METHOD OF VIEWING DIGITAL MEDICAL IMAGES

Background

Medical Images can be very large: a single X-ray image can be 10Mb or more, and typically physician and nurses wish to view several of them at the same time. Due to the size of each image and the number of images to viewed together, downloading the data into the web browser make take a significant amount of time. Furthermore the physician or nurses might need to review the images while they are in an area of the hospital without internet connection or with lower speed connections.

Currently there are several application which allow a viewer (such as a physician, imaging technician or nurse) to view medical images using a Web Browser without installing any applications or software on the viewer's device. This class of application is referred to "Zero Footprint."

The current solutions on the market typically involve a Web Server which may be interfaced to an archive which contains the images to be viewed (referred as a DICOM archive, since DICOM is the format in which the images are stored); the Web Server may transform the images from DICOM format to jpeg format, based on some user preferences (for instance, using pre-set brightness and contrast settings) and then sent the image to a Web Browser used by the viewer for displaying the images. This architecture is referred as "Server Side Rendering" because the generation or rendering of the image being displayed in the browser is done on the Web Server.

The introduction of a new web standard referred to as HTML5 provides capabilities that may be built into a web page that will permit the web page to generate the images within the browser, and therefore to implement "Client Side Rendering." US 13/541,719 protect some aspect of this client side rendering which allows the web browser to download an entire DICOM images (split in several components) and generate the images to display to the viewer within the browser. Benefits of this approach may be lower infrastructure cost and higher frame rate. However, this approach does not fully address the issues that exist with regard to the accessing and displaying of digital medical images. Improvements to the conventional approaches to accessing and viewing digital medical images are desirable.

Summary

The system and method of the present disclosure may facilitate the viewing of digital medical images by using the Web to preload the images on a viewing device prior to user request. This may reduce the time that the user has to wait for viewing the images, as well as allow the viewing device to be used in an "off line" mode in which the device is able to visualize and manipulate the images while not connected to the internet.

One aspect of this invention is to provide this "Offline mode" using only web technologies, and therefore technologies which do not require any installation on the device. This approach is similar to but distinct from the technology and approach disclosed in U.S. Patent Application Serial No. US 13/541,719.

The present application proposes a method to avoid having the users or viewers of the images having to wait for the images to download: the system and method of the present disclosure may allow for pre-fetching the data (DICOM Image information) in advance, prior to the users needing to view the images. The data may be stored in a browser cache or in the browser's Local Storage if the browser supports HTML5's Local Store APIs (for instance, the HTML5 FileSystem APIs). This process is referred to as "Pre-fetch" of the data. Conventional web viewing applications today are using Sever Side rendering, which may be expensive and not capable of providing fast feedback to the user as the user is viewing the images. Furthermore, these conventional applications cannot work offline. Pre-Fetch as implemented in the system of the present application also enables the "Offline" mode of the viewing device: once the digital image data is stored in the local storage of the device, an HTML page will be fully functional even without internet connection. As outlined in U.S. Patent Application Serial No. 13/541,719, using HTML5 enables a web browser to provide client side rendering for DICOM Images. The present application protects another aspect of the Html5 Zero foot print viewer, specifically the pre-fetch and offline mode. Brief Description of the Drawings

The accompanying drawing figures, which are incorporated in and constitute a part of the description, illustrate several aspects of the present disclosure and together with the description, serve to explain the principles of the present disclosure. A brief description of the figures is as follows:

FIG. 1 is a schematic view of a system according to the present disclosure shown in an internet-connected Pre-Fetch state.

FIG. 2 is a schematic view of a system according to the present disclosure shown in a non-internet connected Offline-Mode state.

FIG. 3 is a schematic view of a system according to the present disclosure illustrating a client side rendering state.

DETAILED DESCRIPTION OF THE INVENTION

Reference will now be made in detail to exemplary aspects of the present disclosure which are illustrated in the accompanying drawings. Wherever possible, the same reference numbers will be used throughout the drawings to refer to the same or like parts.

In the present application, the following reference numbers will be used to refer to the following elements in the drawings:

10 is an internet/web server running IMS software exemplification

12 is an image (data) exemplification

14 is a computer/end-server exemplification.

16 is an HTML5 internal file/folder exemplification.

18 is an end user exemplification.

20 is an image display exemplification.

22 is a JavaScript "window" exemplification.

24 is a webpage exemplification.

26 is a browser exemplification.

28 is a display device exemplification.

30 is an active digital network connection, such as but not limited to an Internet connection.

32 is an inactive/broken internet connection. 34 is a DICOM archive.

One purpose of the system and method of the present disclosure is to facilitate the review of digital medical images using a web browser. Medical images can be very large: a single X-ray image can be 10MB or more, and typically a physician, imaging technician, or nurses will want to view several images at the same time. Therefore downloading the data for these digital images into a web browser may take a significant amount of time. Furthermore the physician, imaging technician or nurses might need to review the images while they are in an area of a health care facility such as but not limited to a hospital where this is only limited or non-existent internet connectivity. One of the aspect of system and method of the present disclosure is to provide an "Offline mode" using only Web Technologies, and therefore technologies which do not require any installation on the device. The present disclosure is based on U.S. Patent Application Serial No. 13/541,719, the disclosure of which is incorporated herein by reference in its entirety, and protects a new and novel aspect of the data downloading, namely, use of pre-fetch and enablement of offline viewing capability.

Using rules based on the hospital workflow the system of the present disclosure may be able to identify which images have to be reviewed by each user, and then using local storage (such as but not limited to that capability provided by HTML5) pre download the images on the viewing device that each user may be accessing. HTML5 local storage allows the web page (and specifically the JavaScript code contained in the web page) to download the images and then access these images even when the device is not connected to the internet.

Furthermore, HTML5 can notify to the JavaScript when the device is connected or disconnected to the internet. It should be noted that the system and method of the present disclosure will require that any software be installed on the target device and is based only on HTML5 technologies.

The system and method of the present application differ from what currently exists. The system and method provide a mechanism in which digital medical images can be viewed quickly by the user and in which these images can be viewed without an internet connection at the time of viewing. Specifically the system and method of the present application allow the user to avoid waiting for the time to download the medical images to the viewing device. Conventional medical image web viewing applications do not use make use of the capabilities required to accomplish these sorts of improvements and typically these applications download the image to display on the browser, which is typically quite fast. There is no local storage of the image and therefore they don't need to pre-fetch the data to the browser or the local device. The pre-fetching of the current application is necessary when the actual DICOM image is desired to be downloaded to the viewing device, such as to permit offline viewing and manipulation of the image. These conventional web viewing applications are using sever side rendering where the image data is read and rendered remotely from the viewing device and only the rendered image data and not the raw data if the DICOM image is sent to the local browser or viewing application for display to a user. This approach can be expensive and doesn't provide fast feedback to the user. When a change to the image properties or display properties is requested, the manipulation of the data and rerendering of the image must be carried out at the server and then the new display data forwarded to the local viewing device.

Furthermore, these conventional applications cannot work offline. Because they rely on the server to manipulate and render the data of the DICOM file and only deliver a display image to the local viewing device, these conventional applications require a working Internet connection at all times during the viewing process. Pre-Fetch, as implemented in the system and method of the present application, may also enable an "Offline" mode of operation of a viewing device according the present disclosure: once the DICOM image data is stored in a local storage of the viewing device, the viewing device and html viewing page will be fully functional even without internet connection. As outlined in U.S. Patent Application Serial No. 13/541,719, using HTML5, it was possible to develop client side rendering for DICOM Images. What was not disclosed in this earlier application was the use of pre-fetch and offline to further exploit this client side rendering capability.

The elements of the system and method of the present disclosure may be as follows, although it is intended to limit the present application to just this exemplary set of elements of features:

1. DICOM Archive 34

2. Web Server 10

3. Viewing Device's Web Browser 26 It is anticipated that these elements may operate together in the following non- limiting exemplary fashion:

The web server 10 can communicate one or more images 12 which have to be viewed by the user using web server 10 and the communication may be carried in at least two different ways:

1. web browser 26 can periodically poll the Web Server requesting if new images have to be reviewed by the user currently using or associated with the device.

2. web browser 26 and web server 10 could implement the Web Sockets HTML5 specification which allows the server to push image files to a particular browser or viewing device, and notify the user of the viewing device that new images have arrived for the user to review

Once new images have to be reviewed by the user, web browser 26 may possibly display a notification to the user and/or start the pre-fetching of the data. This activity can be possibly implemented in a Web Worker (part of the HTML5 protocol) which may perform the activity in the background, leaving the user able to view a different set of images or perform different operation on the device while images are being retrieved from the DICOM archive.

The pre-fetching can be done in at least two ways :

1. Using the browser cache, as outlined in U.S. Patent Application Serial No. 13/541,719, the DICOM information can be broken down png and JSON files. These files can then be directly accessed from an HTML page. Therefore web browser 26 can just access these files and not display them. This will cause the web browser's cache to keep a temporary copy of them, so when the user selects the DICOM image, the png files necessary to display that image are immediately available on the local device. The time saved in this approach may be typically several seconds per image.

2. Using HTML5 local storage API, web browser 26 can download the png and JSON file and store the content using the browser's local Storage APIs. Particularly useful to this end may be the FileSystem APIs, which allows the local browser to store any type of data. This local storage may also be achieved through the use of HTML 5 local databases. The local storage APIs may provide greater flexibility in operation of the system according to the present disclosure but it is anticipated that other local storage approaches may be used within the scope of the present application.

It has to be noted that while both approaches to pre-fetching may function within the scope of the present disclosure, approach 2 may be the preferred approach for at least the following reasons:

A. Web browser 26 and more precisely the JavaScript application running on the Browser, doesn't have any control on the size and persistence of the browser's local cache. So even if the png file for an image is accessed and downloaded, approach 1 does not guarantee that the image file will be in the browser cache when needed.

B. Using the local storage API the JavaScript application running on web browser 26 does have direct control of the space allocated and its persistence.

Furthermore, as detailed in U.S. Patent Application Serial No. 13/541,719, once the png images are received by the browser, javascript code in the browser does have to perform a decoding step to retrieve the DICOM images' pixel value. Using the local storage API, the javascript code can store the actual pixel values rather than the png images, therefore performing more computation in the background and providing a faster feedback to the user when accessing the image.

C. Offline mode: local storage can be used to download all the studies that the physician or nurses will need to review and then have the system working offline. This is very useful in a hospital in which large areas might not be covered by WIFI or may not have Internet connections available for the viewing device. Using the data stored in the local cache on a viewing device, the system of the present disclosure is still able display the images and remain fully functional without an active Internet connection.

Finally, it should be noted that while Reason B requires web browser 26 to support the HTML5 local storage extension, Reason A doesn't require that this extension be supported by the web browser.

The system and method of the present disclosure can be used in conjunction with the medical image viewing system described in U.S. Patent Application Serial No. 13/541,719. A user, such as but not limited to a physician or nurses, will open a web browser for viewing one or a series of medical images. While the user is viewing some images, the browser in the background will download the next set of images to be viewed. Possibly the user can leave the browser open while performing other operation, for instance visiting with a patient to whom the prior image or set of images pertained, and the browser may be downloading the images for the next patient to be visited. Alternatively, the user may download all the images he/she needs to review onto a portable viewing device operating as part of the system of the present application and then the user may carry the portable viewing device into an area where Internet connectivity is not available. This is a user case for Offline mode.

A key aspect of the present disclosure is that it may allow a user, such as a doctor/surgeon/nurse, to view patient images which are pre-stored on a portable viewing device in an area not covered by Internet connection. This is viewing of images may be accomplished using a zero footprint viewer, meaning a viewer which runs inside a web browser. This is referred to herein as working in offline mode. A second relevant aspect of the present disclosure is that the data may be downloaded to the portable viewing device prior to the

doctor/surgeon/nurse using it. This means that the user does not have to wait for the

downloading time. This is referred to herein as pre-fetching. Pre-fetching may permit the entire DICOM image to be downloaded locally for display by the portable viewing device rather relying on a previously rendered jpeg approximation of the DICOM image.

Pre-fetching and offline mode are essentially different aspects of the same technical innovation: download the images that the doctor/surgeon/nurse will review ahead of time to a portable viewing device. The images may be stored on the local viewing device inside the HTML5 FileSystem which is a new extension of HTML and which allows a web page to request and manage an area of storage on the portable viewing device. The area of storage will be viewed only by the webpage. The present disclosure is related most preferably with a Zero Foot Print viewer application, meaning that a web page should preferably be run inside a web browser on the portable viewing device without installing any software components in the viewing device.

It should also be noted that use of pre-fetching as described herein may provide for better network utilization as many of the images or data needed related to the images may be downloaded in the background. This would allow the data transfer to proceed as bandwidth is available, since immediate download is not vital to the provision of medical services. Delivery of such image data on-demand may require that the image data download be given priority over other applications that may need to transfer data, resulting is slower response times for other applications and less efficient utilization of available bandwidth.

Due to state and federal regulations and historical confidentiality requirements, it is paramount for any application accessing medical records to keep the data relating to those records in a secure location. The data shall only be accessed to authorized user, and its access has to be controlled. The use of HTML5 or similar standards may assure that the storage allocated on the portable viewing device for a particular web site can be viewed only be accessed by that web site and therefore provides a control aspect with regard to the data. The security aspect can be enforced simply using https protocols and possible encryption of the data as it is stored on the portable viewing device. Access to the data could be then controlled with the user's password or similar standard methods.

As outlined in U.S. Patent Application Serial No. 13/541,719, it is not convenient for a web page (and more specifically the javascript code contained in a web page) to deal directly with a DICOM image. The parsing and decompression of the data associated with a DICOM image would be very difficult to implement and certainly would require tremendous system resources. It is therefore important to download the data converted into an "HTML friendly" format to the local storage of the portable viewing device, such as but not limited to JSON and png files as described in U.S. Patent Application Serial No. 13/541,719.

While the invention has been described with reference to preferred embodiments, it is to be understood that the invention is not intended to be limited to the specific embodiments set forth above. Thus, it is recognized that those skilled in the art will appreciate that certain substitutions, alterations, modifications, and omissions may be made without departing from the spirit or intent of the invention. Accordingly, the foregoing description is meant to be exemplary only, the invention is to be taken as including all reasonable equivalents to the subject matter of the invention, and should not limit the scope of the invention set forth in the following claims.

Claims

What is claimed is:

1. A system for viewing medical images, the system comprising:

a storage system containing a plurality of digital medical images; a web server connected to the storage system and permitting access to the plurality of digital medical images through a digital network;

a web browser installed on a viewing device and configured to access the plurality of digital medical images on the storage system through the web server and the digital network;

the web browser further configured to permit the viewing device to render the digital medical image and to permit a user to view the rendered digital medical images on the viewing device;

the web browser configured to pre-fetch at least one digital medical image and store the pre-fetched digital medical image in local storage on the viewing device prior to the user displaying the pre-fetched digital medical image on the viewing device;

the web browser further configured to permit the user to access and view the prefetched digital medical image without requiring additional data to be transferred to the viewing device via the digital network.

2. The system of claim 1, further comprising the viewing device is a portable viewing device.

3. The system of claim 1, further comprising the storage system is a DICOM storage system.

4. The system of claim 3, further comprising the web server provides the web browser with JSON and png files for the digital medical image being retrieved by the web browser.

5. The system of claim 1, further comprising the viewing device may be disconnected from the digital network after the digital medical image has been pre-fetched and the user is permitted to view the digital medical image while the viewing device is disconnected from the digital network.

6. The system of claim 1 , further comprising the web browser is configured to pre-fetch a plurality of digital medical images and the viewing device is configured to store a plurality of digital medical images for viewing by the user.

7. The system of claim 1, further comprising the viewing device is configured to store a digital medical image retrieved by the web browser without the need for additional software beyond the web browser to be installed on the viewing device.