DE102010042278A1 - Operation navigation system with structured light - Google Patents

Abstract

The invention discloses a surgical navigation system that uses spatially and temporally modulated light that is emitted onto an object (6, 8, 10, 12, O) in an operating area (1) in order to determine the type and / or position and / or the Determine the orientation of an object as a function of time.

Description

The invention relates to a surgical navigation system which detects the position and orientation of a medical instrument, a medical tool, an implant and / or a body part by means of structured light.

In the prior art, surgical navigation systems are known which assist the physician in an operation, for example when inserting an implant. Both optical and electromagnetic position sensing systems are used. The accuracy of the actual position determination is about 0.2 mm for the optical methods and about 1 mm for the electro-magnetic systems. If the accuracy of the entire application, i. H. including imaging, registration, object movement, etc., an accuracy of about 3 to 5 mm is achieved in both the optical and electromagnetic systems, with the optical systems achieving slightly better results.

In image-guided navigation, patient image data is compared with the current patient position via a so-called registration process. A distinction is made between intraoperative and preoperative imaging with the corresponding registration procedures. During preoperative registration, a three-dimensional data set about the volume of the surgical area, which was created, for example, by means of computed tomography, magnetic resonance tomography, PET, SPECT, etc., is generally loaded during navigation. Then the registration is done manually. During intraoperative procedures, the images are created in the operating room with tracking reference elements already arranged on the patient, whereby the registration can therefore take place automatically. For this purpose, in particular C-arms are used for the two-dimensional and three-dimensional imaging. It is also possible to use intraoperative CT systems or MR systems.

Optical navigation uses both passive and active tracking techniques. A visual contact between the instruments and the camera system is required, which makes practical handling difficult. By means of a stereo tracking camera both infrared light emitted and the reflections are received and assigned spatially by means of triangulation. In this passive method, reflective markers are applied to all navigational elements, the patient, and optionally to the intraoperative imaging system. The reflective marking elements are designed as disposable products and therefore generate high operating costs, which are about 50.00 Euro (as of 2010) per navigated operation. It is also possible to use active marking elements, for example infrared LEDs. These are problematic in terms of sterilization and the cables required for this are perceived by the surgical staff as disturbing. The use of batteries in turn leads to higher operating costs. Even with the electromagnetic navigation cables required for this purpose are considered by the surgical staff as disturbing.

Operational navigation systems are described in detail in Imaging Systems for Mecical Diagnostics, Arnulf Oppelt, Publicis Corporate Publishing.

The invention has as its object to overcome problems of the prior art and to provide an improved operation navigation system.

The object of the invention is achieved by using spatially and temporally modulated light emitted on one or more objects in an operating region to determine at least one of type, position and orientation of an object as a function of time.

A surgical navigation system according to the invention comprises a light emitting device which emits structured light onto an operating area with at least one object, wherein an illumination pattern of the structured light is spatially and temporally modulated. The operation navigation system includes an image pickup device that detects the light emitted from the object as a function of time. The term "emissive" may include reflecting and / or scattering light. The operation navigation system further comprises a position determination device which determines the type and / or the position and / or the orientation of the object from the light emitted by the object as a function of the illumination pattern in the coordinate system of the position determination device.

The temporally and locally modulated light can be used to continuously determine the type, the position and / or the orientation of an object in an operating area without necessarily having to attach special marking elements to the object. The structured light preferably has a wavelength which is invisible to humans, preferably a wavelength in the infrared range, for example between 780 nm and 3 μm. Since no marking elements are required in the surgical navigation system according to the invention, the operating costs can be reduced.

The operation navigation system comprises a database in which image information data, for example virtual three-dimensional model data, of the objects are stored. This ensures that the position-determining device recognizes the type and / or the position and / or the orientation of an object. Furthermore, geometrical data can be stored in the database with which, for example, the tip of an operating instrument or any other position that is medically relevant can be determined from the determined position and / or orientation of the object. This geometric data can be part of the virtual three-dimensional model. The database may have a geometric description of the object. Alternatively, data in the database may be determined based on a so-called teach-in method in which the object is captured by a suitable image capture device and the relevant data for the database is generated, for example, by feature extraction.

The position determination device can be designed to recognize at least one imaging system, a tool, an instrument, an implant, a body part, a reference object etc. as the aforementioned object. The database can be set up to define as object at least one imaging system, a tool, an instrument, an implant, a body part, a reference object, etc. The reference object can be arranged on a body part, any location in the operating area and / or in another location.

The position determination device can be designed to recognize any modality as an imaging system, and the database can be set up to define any modality as the imaging system. The modality may include, for example, an ultrasound system, an X-ray system, a C-arm X-ray system, an MRI system, a PET system, a SPECT system. This allows image-based navigation, where the position of objects in patient image data can be displayed.

The image recording device may comprise a stereoscopic camera. The two cameras of the stereoscopic camera can be offset from each other and / or inclined. By using the stereoscopic camera with at least two separate cameras, i. H. two separate combinations of lens and sensor, such as CCD chip, the position of an object in space can be determined by means of suitable mathematical methods, since the object is detected from two different positions or perspectives.

The light emitting device may comprise a projector having a rotating light mask which is illuminated, whereby the locally and temporally modulated light generated by the disc is delivered to the operating area. The rotating light mask can be illuminated with transmitted or incident light. When the rotating light mask is illuminated with transmitted light, the light mask has areas of higher transparency and areas of lower transparency. If the light mask is illuminated with incident light, the light mask has areas which are more reflective and less reflective, or more absorbent and less absorbent.

The light-emitting device can deliver a strip-shaped and / or a wave-shaped, for example sinusoidal, light intensity pattern to the surgical area. If a strip-shaped and / or substantially sinusoidal intensity pattern is emitted onto the surgical area, a distortion of the pattern results when the camera is observed by means of a camera from a direction different from the direction of projection. The substantially sinusoidal pattern is laterally displaced depending on the distance of the object in phase. From this phase shift, the distance and orientation to the camera can be calculated.

The image recording device may comprise a plurality of image recording systems, for example stereoscopic cameras, which capture the surgical area from different directions. Preferably, an image acquisition system is arranged in each case at a corner of the surgical area, so that a total of four image acquisition systems are used. This can reduce the problems of having a part of an object hidden in the line of sight to one of the cameras. With this arrangement, the type, position and / or orientation of each object in the operating area can be determined.

Furthermore, it is possible to use imaging systems, for example stereoscopic cameras, of different resolution in an operating area. The plurality of imaging systems may detect the surgical area at different resolutions. For example, a comparatively coarse resolution imaging system may be used to capture the entire surgical area. Furthermore, a relatively high resolution imaging system can be used to more accurately detect a particular area of the surgical area. The imaging systems can use structured light with different wavelengths so that the imaging systems are not mutually exclusive to disturb. Furthermore, the image acquisition systems may use structured light with a different temporal modulation, so that the image acquisition systems do not interfere with each other.

The operation navigation system may include a control device that is configured to control the light emitting device and the position determining device so that the position and orientation of the imaging system in the coordinate system of the surgical navigation system are determined. The imaging system may be, for example, an X-ray system with a C-arm. By means of the operation navigation system, the position of the detector and / or the emitter at one or a plurality of pivot position (s) can be determined.

The invention also relates to a method for determining the position of an object in an operating area. The object in the operating area is illuminated with structured light, wherein a pattern of illumination of the structured light is spatially and temporally modulated. The light emitted by the object is detected as a function of time. The position and / or the orientation of the object are determined from the light emitted by the object as a function of the illumination pattern. The method may be configured as previously explained with reference to the device.

The invention also relates to a computer program product that can be loaded into a memory of a computer having a processor or is loaded, the computer program product comprising means adapted to carry out the steps of the method described above.

The invention will be elucidated below with reference to the accompanying drawings which illustrate exemplary embodiments. Show it:

1 a schematic section through an operating area to which the present invention is applied;

2A a projector with a transmitted light illuminated light mask;

2 B a projector with a reflected-light light mask;

3A a spatially strip-shaped modulated light pattern;

3B a spatially sinusoidally modulated light pattern;

3C an object illuminated with a linear light pattern; and

4 a schematic plan view of a stereoscopic camera with a projector for structured light.

1 schematically shows a cross section of an exemplary operating environment 1 , The operating environment includes an operating table 2 on which a patient is 4 located. The operation becomes a first instrument 6 used. At the operating area is a first reference element 8th arranged. Further, on the body of the patient 4 a second instrument 10 positioned at which a second reference element 12 is attached. The operating area further comprises an X-ray system with a pivotable C-arm 14 on, on which an X-ray emitter 16 and an x-ray detector 18 are arranged. By means of the X-ray system with the C-arm 14 can take pictures of the interior of the patient's body 4 be generated before and / or during the operation.

The operation navigation system according to the invention has a position sensor 20 up, a light projector 22 and a camera 24 includes. The light projector gives a strip-shaped and / or wavy modulated light 26 on the surgery area 1 from. The structured light is temporally and locally modulated. The camera 24 captures that from the first instrument 6 , first reference element 8th , second instrument 10 and second reference element 12 emitted structured light as a function of time. The light projector 22 of the position sensor 22 can also detect the detector 18 X-ray system with the C-arm 14 illuminate. The camera 24 can that be from the detector 18 detect emitted structured light. It is understood that the light projector 22 also the emitter 16 of the C-arm, and the camera 24 that from the emitter 16 can detect emitted light. Consequently, the operation navigation system can also control the position of the emitter 16 and / or the detector 18 of the C-arm.

Once the structured light 26 impinging on an object, the shape of the structured light changes. This altered shape may be from the camera 24 be detected and by means of an evaluation, the position and the surface of an object can be determined.

The basics of determining the type, position and / or orientation of an object with the help of structured light are DE 10 2010 006 105.0 described, the contents of which are incorporated herein by reference.

2A shows the operation of a first embodiment of a light projector 22 , The light projector has a lighting unit 28 on, the light on a rotating light mask 30 emits. On the rotating light mask are linear areas 32 . 35 arranged. Alternatively or additionally, wavy areas can be formed on the light mask 33 . 34 be arranged. Preferably, the wave-shaped regions are sinusoidal. The linear areas 32 . 35 or the wave-shaped areas 33 . 34 are transparent, whereas the remaining areas of the rotating light mask 30 are not transparent. That of the lighting unit 28 emitted light passes through the line-shaped areas 32 . 35 or wavy areas 33 . 34 in the projection optics 25 and is used by this as a structured light 26 on a surface to be examined, for example, an operating area submitted.

A wavy or sinusoidal illumination pattern is laterally shifted in response to the distance of an object in phase. From this phase shift, the distance to the light projector 22 or to the camera 24 be calculated.

2 B shows a second embodiment of the light projector 22 , The light mask is as a rotating roller 31 formed with an incident illumination 29 is illuminated. The strip-shaped areas 32 . 35 and the wavy areas 33 . 34 are formed in this embodiment, reflective or light-emitting, while the remaining areas of the rotating roller are designed to absorb light. The light emitted by the line-shaped regions or wave-shaped regions is emitted by the projection optics 25 as structured light 26 delivered to a region to be examined, for example, the surgical area. All patterns of the plurality of patterns may be formed differently.

3A shows a line-shaped pattern 32 . 35 which is delivered to a flat object. 3B shows a wavy pattern, which is delivered to a flat object. 3C shows a line-shaped pattern, which is delivered to a round object O. It can be seen that the line-shaped pattern bulges in an area where it meets the round object. Due to the curvature of the linear pattern, the surgical navigation system according to the invention can measure at least one of the type of the object, the position of the object and the orientation of the object. Computer-aided methods for evaluating structured light are known to the person skilled in the art and consequently the evaluation of the structured light is not discussed further here.

4 shows a stereoscopic camera 36 with a light projector 22 , a first camera 38 and a second camera 40 , The first camera 38 has a first imager 42 and the second camera 40 has a second imager 44 on. The light projector 22 has a light emitting device 46 which generates temporally and locally modulated light. The temporally and locally modulated light is applied to an object O in the form of linear areas 32 . 35 issued. The linear areas 32 . 35 be through the first camera 38 and the second camera 40 taken and to a position determining device 48 to hand over. The position determination device 48 creates a pictorial representation of the operating area displayed on a display device 56 is pictured. The position determination device 48 is with a database 54 connected, which contains representations of the objects. By means of the representations of the objects, the position determining device 48 determine the type of the object. The position determination device 48 can based on the in the database 54 contained data determine whether the object is an instrument, a tool, an implant, an imaging system, etc. Furthermore, the position determination device can be made of those in the database 54 contained data, determine the position of the object. In particular, the position determination device 48 the position of the tip of a medical instrument from those in the database 54 determine stored data.

The operation navigation system further comprises a control device 50 that with the light emitting device 46 and with the position determining device 48 connected is. This allows the delivery of structured light through the light emitting device 46 as well as capturing the images through the first camera 38 and the second camera 40 synchronize. The control device 50 is also with a C-arm control device 52 connected, which is adapted to cause a pivoting of the C-arm, so that the position of the detector or the emitter by means of the light emitting device 46 , the first camera 38 and the second camera 40 can be determined.

The invention proposes to use structured illumination to directly determine the position and orientation of objects in space. Separate optically visible marking elements, LEDs or electromagnetic coils are not absolutely necessary with this method.

Three-dimensional models of all instruments, tools, implants, and imaging modalities used in an operation, such as a C-arm X-ray system, an ultrasound machine, and optional reference structures are stored in a database. The visible Three-dimensional surfaces of all objects that are in the operating area are determined by the system in real time and assigned in real time to the corresponding three-dimensional models. The algorithms used can also identify an object if only a part of the object is optically visible. Consequently, the current position and orientation of all visible objects can be determined in real time. For example, the tip of an instrument located in the body of the patient can be displayed if the associated handle of the instrument can be optically detected and rigidly connected to the tip. The position and orientation of these objects can be faded in and updated in real time after a registration procedure into the image data obtained, for example, with an imaging modality. If the position and orientation of implants and instruments are determined at the same time, they can be faded in simultaneously with the image data and determined in their relative relationship to one another.

In addition, reference structures may be used to allow the location of the patient to be determined. Since during surgery a surgeon holds in hand the object whose position is to be detected, for example an instrument, only part of the surface of the object is visible. This negatively affects the recognizability of the object. Optionally, additional three-dimensional structures are provided on the objects to ensure detectability, even when a surgeon holds the object in his hand.

If intra-operative imaging is additionally used, the position and orientation of the coordinate system of the imaging in the space in relation to the coordinate system of the surgical navigation system can also be determined by the surgical navigation system. Consequently, the images of the intra-operative imaging can be displayed in the coordinate system of the surgical navigation system. In an X-ray system with a C-arm, this means that the position and orientation of the X-ray detector and / or the X-ray tube are determined. In the case of an ultrasound system, only the position and orientation of the ultrasound applicator need to be determined. In these cases, the relative relationship between the imaging volume and the imaging device whose position is determined intra-operatively must first be determined and preferably also stored.

Several stereoscopic cameras can be used as positioning systems which emit temporally and spatially modulated light of different wavelengths and which know their mutual position or determine by measurement. The invention may also be combined with conventional electro-magnetic and / or optical navigation systems. For example, the position of a flexible instrument in the body may be determined by electromagnetic navigation, where the position of the field generator for determining the position of the flexible instrument is determined by the method described herein. Thus, the positions and orientations of instruments in the body in the coordinate system of the surgical navigation system according to the invention can be determined and assigned to the images generated with an imaging modality. To improve the recognition security, the objects to be recognized may be provided with an infrared-reflective layer. The invention is not limited to stereoscopic cameras. Spatially independent cameras can also be used if their positions are known to each other exactly, for example if they are located at the corners of an operating room, operating table, operating area, etc., at fixed and / or known positions.

The invention has the advantage that objects can be tracked in real time by means of three-dimensional techniques. At the usual movement speed of objects in an operating range, a sampling rate of 20 to 30 Hz is sufficient. Optionally, a sampling rate of 50 Hz can be provided.

Furthermore, a projection of only four phase positions (sine) as well as a projection of a subcode to eliminate possible ambiguities in objects that are very extensive in height can be provided. Furthermore, algorithms for compensating for the motion blur can be implemented in fast-moving objects. The projection of the structured light preferably takes place in the infrared range. Further, the system of the invention may be implemented so that it can be easily implemented in an existing environment.

The invention also has the advantage that by means of three-dimensional detection techniques navigation-relevant objects in the operating area without marking elements can be located exactly. The invention is based on the finding that clear advantages in terms of costs, handling and further development potentials are opened up by the use of active triangulation by means of structured illumination in the entire three-dimensional space in the operating area. The three-dimensional detection of surgical equipment, such as drills, chisels, probes, etc., allows additional, disabling the surgeon aids, such as marking elements or cables to dispense.

Less expensive instruments can be used, which are not burdened by the additional costs for the marking elements. In conventional navigation systems, the marking elements must be changed after each use, resulting in high costs. Furthermore, the costs for registration elements can be saved on the X-ray system or on the ultrasound system, since the inventive operation navigation system can perform the registration of the X-ray system.

Image acquisition systems are known which allow a resolution of 5000 points per line. Based on an operating range of 1 m × 1 m, this means a resolution of well below 1 mm.

LIST OF REFERENCE NUMBERS

1

operation area

2

operating table

4

patient

6

First instrument

8th

First reference element

10

Second instrument

12

Second reference element

14

C-arm

16

emitter

18

detector

20

position sensor

22

overhead projector

24

camera

25

projection optics

26

Structured light

28

lighting unit

29

Barlight

30

light mask

31

Rotating roller

32

Line-shaped area

34

Wavy area

36

Stereoscopic camera

38

First camera

40

Second camera

O

object

42

First imager

44

Second imager

46

A light emitting device

48

Location facility

50

control device

52

C-arm controller

54

Database

56

display unit

QUOTES INCLUDE IN THE DESCRIPTION

This list of the documents listed by the applicant has been generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Cited patent literature

• DE 102010006105 [0032]

Claims (14)

Operation navigation system, comprising - a light emitting device ( 22 . 46 ), the structured light on an operating area ( 1 ) with at least one object ( 6 . 8th . 10 . 12 , O), wherein at least one illumination pattern ( 26 . 32 . 33 . 34 . 35 ) of the structured light is spatially and temporally modulated; An image recording device ( 24 ; 38 . 40 ), that of the object ( 6 . 8th . 10 . 12 , O) light detected as a function of time; and - a position determination device ( 48 ), the type and / or the position and / or the orientation of the object ( 6 . 8th . 10 . 12 , O) determined from the light emitted by the object as a function of the illumination pattern ( 26 . 32 . 33 . 34 . 35 ) Operation navigation system according to claim 1, characterized by - a database ( 54 ), in which image information data of the objects are stored. Operation navigation system according to claim 1, characterized in that the position-determining device ( 48 ) is adapted to at least one imaging system ( 14 . 16 . 18 ), a tool ( 6 . 10 ), an instrument ( 6 . 10 ), an implant, a body part, a reference object ( 8th . 12 ) as an object, and the database ( 54 ) is set up as an object at least one imaging system ( 14 . 16 . 18 ), a tool ( 6 . 10 ), an instrument ( 6 . 10 ), an implant, a body part, a reference object ( 8th . 12 ), the reference object ( 8th . 12 ) can be arranged on a body part, any location in the operating area and / or on another object. Operation navigation system according to claim 3, characterized in that the position-determining device ( 48 ) is designed to recognize any modality as an imaging system, and the database is set up to define any modality as an imaging system, the modality comprising at least an ultrasound system, an X-ray system and a C-arm X-ray system ( 14 . 16 . 18 ). Operation navigation system according to one of claims 1 to 4, characterized in that the image recording device comprises a plurality of cameras which are in a fixed geometric relationship to each other and in particular a stereoscopic camera ( 36 ) having. Operation navigation system according to one of claims 1 to 5, characterized in that the light emitting device ( 22 ) a projector with a rotating light mask ( 30 . 31 ), which is illuminated, wherein the locally and temporally modulated light generated by the light mask on the operating area ( 1 ) is delivered. Operation navigation system according to one of claims 1 to 6, characterized in that the light emitting device ( 27 ) a strip-shaped ( 32 . 35 ) and / or wavy light intensity pattern ( 33 . 34 ) on the operating area. Operation navigation system according to claim 7, characterized in that the position-determining device ( 48 ) is designed, by means of evaluation of the basis of the strip-shaped ( 32 . 35 ) and / or wavelike light intensity pattern ( 33 . 34 ) from the object ( 6 . 8th . 10 . 12 , O) emitted light to determine the distance of the object. Operation navigation system according to one of claims 1 to 8, characterized in that the image recording device comprises a plurality of image recording systems ( 36 ), which covers the operating area ( 1 ) from different directions. Operation navigation system according to one of claims 1 to 9, characterized in that the image recording device comprises a plurality of image recording systems ( 36 ), which covers the operating area ( 1 ) with different resolutions. Operation navigation system according to one of claims 3 to 10, characterized by - a control device ( 50 ), which is designed to control the light-emitting device and the position-determining device so that the position and orientation of the imaging system in the coordinate system of the surgical navigation system is determined. Using spatially and temporally modulated light that is incident on an object ( 6 . 8th . 10 . 12 , O) in an operating area ( 1 ) is emitted in order to determine the type and / or the position and / or the orientation of an object ( 6 . 8th . 10 . 12 , O) as a function of time. Method for determining the position of an object ( 6 . 8th . 10 . 12 , O) in an operating area ( 1 ), comprising the following steps: - illuminating the object ( 6 . 8th . 10 . 12 , O) in the operating area with structured light, wherein a pattern of illumination of the structured light is spatially and temporally modulated; - Capture the object ( 6 . 8th . 10 . 12 , O) emitted light as a function of time; and determining the position and / or the orientation of the object ( 6 . 8th . 10 . 12 , O) from the object emitted light as a function of the illumination pattern ( 26 . 32 . 33 . 34 . 35 ). A computer program product loadable or loaded into a memory of a computer comprising means adapted to carry out the steps of the method of claim 13.

Images