"Inspection Management System and Graphical User Interface Therefor"
Field of the Invention
The present invention relates to an inspection management system and graphical user interface therefor. The invention is particularly useful in the administration of inspections of sub-sea offshore oil and gas facilities such as pipelines.
Throughout the specification, unless the context requires otherwise, the word "comprise" or variations such as "comprises" or "comprising", will be understood to imply the inclusion of a stated integer or group of integers but not the exclusion of any other integer or group of integers.
Background Art
The following discussion of the background to the invention is intended to facilitate an understanding of the present invention. However, it should be appreciated that the discussion is not an acknowledgement or admission that any of the material referred to was published, known or part of the common general knowledge in Australia as at the priority date of the application.
Inspections are performed to ensure the continued integrity of assets. The integrity of assets is essential to guarantee the continued safety of personnel and conservation of the surrounding environment, as well as, in some cases, ensuring that optimal production levels are maintained.
Original methods for conducting inspections were by hand. Data sheets where designed and created for each task on a structure. These were then filled out as each task was completed. For pipeline inspections, the inspector would note the location (KP, northings, eastings, and time) of each anomaly as well as describing it in as much detail as possible. The problem with this method, however, was that the flexibility of the process did little to assist in achieving consistency and conformity across multiple inspections. It also made reporting difficult and trend analysis impossible.
These original methods were subsequently replaced with software that mimicked the manual inspection process. Over time, these computerised inspection tools have become complicated software suites incorporating separate modules for planning inspections, conducting them, and reporting on them. However, in the main, present day computerised inspection tools, such as those mentioned, have been designed to accommodate the database back-end rather than the user interface. This generally introduces three problems:
1. A lack of flexibility in defining and recording pertinent information relating to an inspection task or event;
2. An inability for the inspector to intuitively obtain meaningful information regarding an event or task from the user interface; and
3. The need to enter extensive information into the computerised inspection tool's database may draw the inspector's attention away from the actual inspection for significant periods of time. This means that the inspection may not be as thorough as it could be or that particularly relevant information is missed.
It is an object of the present invention to alleviate at least some, if not all, of the above problems associated with database-centric inspection management tools.
Disclosure of the Invention
One aspect of the present invention relates to an inspection management system having a user interface whereby the user interface displays:
a visual representation of an entity to be inspected; and
at least one glyph, each glyph representative of a task or incident, at a location relative to the visual representation of the entity being inspected,
where the location of the glyph relative to the visual representation of the entity being inspected substantially corresponds to the physical location of the task or incident relative to the physical location of the entity being inspected.
Preferably, the size of the glyph displayed by the user interface relative to the size of the visual representation of the entity being inspected substantially corresponds to the physical size of the task or incident represented by the glyph relative to the physical size of the entity being inspected.
Preferably, when a glyph is activated, the user interface displays information regarding the task or incident the glyph represents.
More preferably, the information displayed can be modified.
More preferably, only one glyph can be activated at a time.
Preferably, the user interface updates the visual representation of the entity being inspected and the glyphs based on the recorded position of an inspecting entity.
More preferably, the user interface displays glyphs created while an inspection is being conducted at positions in front of or behind the inspecting entity.
More preferably, the inspecting entity is a remotely-operated vehicle.
More preferably, the inspecting entity is equipped with at least one of the following inspection tools: Corrosion Protection; Alternating Current Frequency Modulation; and Flooded Member Detection.
Preferably, the user interface distinguishes glyphs representing current event tasks or incidents from glyphs representing historical event tasks or incidents.
More preferably, the method of distinguishing glyphs representing current event tasks or incidents from glyphs representing historical event tasks or incidents is to render partially transparent glyphs representing historical event tasks or incidents.
Preferably, the user interface distinguishes glyphs representing anomalous event tasks or incidents from glyphs representing other event tasks or incidents.
Preferably, the user interface displays a list of incidents identified during an inspection.
Preferably, the inspection management system further comprises automatic information gathering means in data communication with an input device for obtaining information automatically about event tasks or incidents.
More preferably, the automatic information gathering means is also in control communication with the input device.
More preferably, the input device is at least one of the following: digital video disc recorder; video cassette recorder, survey computer.
Preferably, the inspection management system further comprises audio recording means for recording commentary dictated by an inspector in respect of the inspection.
More preferably, the audio recording means records commentary dictated by an inspector in respect of the inspection on a first audio track and the automatic information gathering means records commentary on the inspection on a second audio track.
Preferably, the visual representation of the entity being inspected is derived from live-video feed of the entity.
Preferably, the visual representation of the entity being inspected is displayed in stereo or three dimensional format.
Preferably, the inspection management system further comprises target definition means for defining inspection targets, each inspection target representative of an entity to be inspected.
Preferably, the inspection management system further comprises hierarchical means for defining a hierarchical relationship between inspection targets.
Preferably, the inspection management system further comprises task definition means for defining inspection tasks.
More preferably, the step of defining an inspection task includes defining data to be recorded in respect of the inspection task and a range of acceptable values for such data.
More preferably, the inspection management system further comprises task allocation means for assigning inspection tasks to inspection targets.
Preferably, the inspection management system further comprises span means for calculating the span of the entity being inspected.
Preferably, the inspection management system further comprises scour means for calculating the scour of the entity being inspected.
Preferably, the inspection management system further comprises time estimation means for estimating the time necessary to complete an inspection task.
Preferably, the inspection management system further comprises report generation means for generating a report on the inspection.
In accordance with a second aspect of the present invention there is a graphical user interface for an inspection management system, the interface displaying:
a visual representation of an entity; and
a glyph representative of a task or incident at a location relative to the visual representation of the entity being inspected;
wherein the location of the glyph relative to the visual representation of the entity being inspected substantially corresponds to the physical location of the task or incident relative to the physical location of the entity being inspected.
Preferably, the size of the glyph displayed by the user interface relative to the size of the visual representation of the entity being inspected substantially corresponds to the physical size of the task or incident represented by the glyph relative to the physical size of the entity being inspected.
Preferably, when a glyph is activated, the user interface displays information regarding the task or incident the glyph represents.
Preferably, the information displayed can be modified.
Preferably, only one glyph can be activated at a time.
Preferably, the user interface updates the visual representation of the entity being inspected and the glyphs based on the recorded position of an inspecting entity.
Preferably, the user interface displays glyphs created while an inspection is being conducted at positions in front of or behind the inspection entity.
Preferably, glyphs representing current event tasks or incidents are distinguished from glyphs representing historical event tasks or incidents.
More preferably, the method of distinguishing glyphs representing current event tasks or incidents from glyphs representing historical event tasks or incidents is to render partially transparent glyphs representing historical event tasks or incidents.
Preferably, glyphs representing anomalous event tasks or incidents are distinguished from glyphs representing other event tasks or incidents.
Preferably, the user interface further displays a list of incidents identified during an inspection.
Preferably, the visual representation of the entity inspected is derived from live- video feed of the entity.
Preferably, the visual representation of the entity being inspected is displayed in stereo or three dimensional format.
In accordance with a third aspect of the present invention there is a method of presenting information during an inspection comprising:
displaying a visual representation of an entity being inspected on a user interface;
displaying a glyph representative of a task or incident related to the entity being inspected on the user interface at a location relative to the visual representation of the entity being inspected such that the location of the glyph relative to the visual representation of the entity being inspected substantially corresponds to the physical location of the task or incident relative to the physical location of the entity being inspected.
Preferably, the step of displaying a glyph representative of a task or incident further comprises the step of adjusting the size of the glyph displayed by the user interface such that the size of the glyph relative to the size of the visual representation of the entity being inspected substantially corresponds to the physical size of the task or incident represented by the glyph relative to the physical size of the entity being inspected.
Preferably, the method of presenting information further comprises the step of displaying information on the user interface regarding a task or incident represented by a glyph upon activation of that glyph.
Preferably, the method of presenting information further comprises the step of updating the visual representation of the entity being inspected and the glyphs based on the recorded position of an inspecting entity.
More preferably, the method of presenting information further comprises the step of displaying on the user interface glyphs created while an inspection is being conducted at positions in front of or behind the inspecting entity.
Preferably, the method of presenting information further comprises the step of distinguishing glyphs representing current event tasks or incidents from glyphs representing historical tasks or incidents.
More preferably, the step of distinguishing glyphs further includes the step of rendering glyphs representing historical event tasks or incidents partially transparent.
Preferably, the method of presenting information further comprises the step of distinguishing glyphs representing anomalous event tasks or incidents from glyphs representing other event tasks or incidents.
Preferably, the method of presenting information further comprises the step of displaying a list of incidents identified during an inspection on the user interface.
Preferably, the method of presenting information further comprises the steps of:
obtaining live-video feed of the entity being inspected; and
deriving the visual representation of the entity being inspected from the live-video feed.
Preferably, the step of displaying a visual representation of the entity being inspected further comprises the step of rendering the visual representation of the entity into a stereo or three dimensional format.
Brief Description of the Drawings
The present invention will now be described, by way of example, with reference to the accompanying drawings, wherein:
Figure 1 is a schematic diagram of the inspection management system and its external interfaces.
Figure 2 is a first illustration of the user interface during an inspection displaying the structure being inspected.
Figure 3 is a second illustration of the user interface during an inspection displaying the tasks to be performed and the anomalies encountered during the inspection.
Figure 4 is a block diagram showing the general data structure.
Figures 5a - 5f are block diagram depictions of the automated inspection-video voice annotation function of the inspection management system.
Figure 6 is a block diagram depiction of the automated span, burial and scour profiling function of the inspection management system.
Figure 7 is a block diagram depiction of the structure navigation with recording interface function of the inspection management system.
Figure 8 is a block diagram depiction of the fully integrated reporting function of the inspection management system.
Figures 9a and 9b are block diagram depictions of the real-time visualisation and inspection interaction functions of the inspection management system.
Best Mode(s) for Carrying Out the Invention
The present invention relates to an inspection management system 10 having a graphical user interface 12. The inspection management system 10 described hereafter has been particularly adapted for use in the inspection of sub-sea offshore oil and gas facilities, such as pipelines.
The inspection management system 10 is in data communication with a survey computer 14 and an inspection tool computer 16. The inspection management system 10 is in data and control communication with an image recording device 18, in this case a video cassette recorder.
The inspection management system 10 also has a database 17. As shown in Figure 4, database 17 is in reciprocal communication with non-visual module 15. Non visual module 15 contains information relating to the various components and objects of the system. Non-visual module 15 is made up of data module 13 and I/O module 19. Data module 13 contains information relating to the various data structures used in the inspection management system 10 and their implementation. Data module 13 is made up of incident module 11 and generic module 25. Incident module 11 contains information relating to the customised data structures obtained for event tasks and incidents and is further made up of recorded data module 23 and data definitions module 21. Recorded data module 23 stores information entered in by a user during the course of an inspection. Data definitions module 21 stores the user-defined data structures relating to the various event tasks and incidents. Generic module 25 stores the data structures for the generic components used in the information management system 10 and other generic data structures. Non-visual module 15 is also in reciprocal communication with the user interface 12.
The inspection tool computer 16 is in data and control communication with inspection tools 20 installed on a remotely operated vehicle ("ROV") 22 through I/O module 19. The survey computer 14 is also in data communication with the inspection tools 20 installed on the ROV 22 through I/O module 19. The inspection tools 20 may include such components as Corrosion Protection, Alternating Current Frequency Modulation (crack detection and sizing) and Flooded Member Detection in addition to a video camera 24.
The video camera 24 provides raw video footage to monitors 26. The image recording device 18 are connected to monitors 26 in such a manner that the raw video footage provided to the monitors 26 can be recorded by the image recording device 18.
In use, the inspection management system 10 operates as follows. The following description should be read in conjunction with Figure 5 through Figure 9 which graphically illustrate some of the features described.
An inspector 28 defines inspection targets 30 in a workpack preparation stage. Usually, the inspection targets 30 are identified by diagrams produced by the owner of the facility being inspected. Details relating to each inspection target 30 are also defined. Typically, these details include:
• the type of the inspection target 30 (pipeline, wellhead, etc.);
• the dimensions of the inspection target 30;
• efficiency factors relating to the inspection target 30;
• references to blueprints of the inspection component 30;
• pictures of the inspection component; and
• special instructions to be carried out during inspection of the inspection target 30.
The inspector 28 then generates an inspection hierarchy by defining parent-child relationships between the inspection targets 30. The inspection hierarchy is stored in database 17.
Once the inspection hierarchy has been defined and stored in the database, the inspector 28 defines the inspection tasks 32. The process of defining the inspection tasks 32 includes specifying the data that is to be recorded against the inspection task 32 and the method by which that data is to be recorded. A range of acceptable values for each item of data must also be specified by the inspector 28, where appropriate.
At the time of defining an inspection task 32, the inspector 28 must choose a glyph 34 that will identify the inspection task 32 when displayed on the graphical user interface 12.
The inspector 28 must also specify the expected estimated time to complete each inspection task 32. The expected estimated time is compared with the efficiency factor that relates to that inspection target 30, if any, to produce a revised expected estimated time. The revised expected estimated time is obtained by multiplying the efficiency factor for the inspection target 30 by the expected estimated time. In this manner, the efficiency factor should take into consideration such issues as water current and accessibility. The total estimated time for an inspection can be obtained by summing the revised expected estimated times for each inspection target 30 in the inspection hierarchy.
Once all inspection tasks 32 have been defined, the inspection tasks 32 are grouped within data definitions module 21 into logical categories to provide a fast and intuitive way of finding them during an inspection. Thereafter, the inspector 28 traverses each inspection target 30 in the inspection hierarchy allocating inspection tasks 32 as required. The inspector 28 may then commence the inspection at any time.
In accordance with the present description of the inspection management system 10, an inspection is commenced by initiating a dive.
On initiation of a dive, the inspection management system 10 sets up the graphical user interface's 12 display and executes a sound file that announces the dive name and its status (ie. started).
The graphical user interface's 12 display contains a series of toolbars 34 and windows 36. In the centre of the graphical user interface 12 a visual representation 38 of the inspection target 30, defined as paramount in the inspection hierarchy, is displayed as determined by the inspection target's 30 type. The toolbars 34 and windows 36 are displayed close to the edges of the
graphical user interface 12 so as not to obscure the visual representation 38 of the inspection target 30.
The visual representation 38 of the inspection target 30 is displayed in three- dimensional format. The x-y plane 40 of the visual representation 38 is represented as a grid. The x-axis 42 represents the transverse distance to the inspection target 30. The y-axis 44 represents the length of or distance travelled along the inspection target 30.
Inspection tasks 32 allocated to the inspection target 30 are also displayed on the graphical user interface 12 by their glyphs 52. The position of the glyphs 52 are determined with reference to the actual or expected physical location recorded for the inspection task 32 as related to the x- and y-axes 42, 44. In this manner, each glyph 52 is shown on the graphical user interface 12 at a position relative to the visual representation 38 that substantially corresponds with the actual or expected physical location of the inspection task 32 relative to the physical location of the inspection target 30. Incidents are displayed in the same manner.
The inspector 28 also has the option of viewing historical inspection tasks 32 and incidents. When this option is chosen, glyphs 54 representing historical inspection tasks 32 and incidents relating to the inspection target 30 and stored in the database are displayed in the same manner as glyphs 52 representing current inspection tasks 32 and incidents. Glyphs 54 representing historical inspection tasks 32 and incidents are distinguished from glyphs 52 representing current inspection tasks 32 and incidents by their partial transparency. Glyphs 54 representing historical inspection tasks 32 and incidents can not be activated (discussed below).
Toolbar 34a is the incident toolbar. The glyphs displayed in the incident toolbar 34a vary according to the type of inspection target 30.
Toolbar 34b is the menu bar. From toolbar 34b the inspector 28 can take a frame or video capture, initiate an anomaly, and enter a text comment into the video log.
Toolbar 34c allows the inspector to control the status of the inspection. Toolbar 34c also allows the inspector 28 to specify the current video number and its length, as well as entering operational log details.
Window 36a shows a visual representation 56 of the distance between the inspection target 30, being a pipeline in this instance, and the seabed. If the inspection target 30 is buried in the seabed, a visual representation 56 of how deep the inspection target 30 is buried in comparison to the seabed is shown. The calculations that provide this visual representation 56 are discussed in more detail below.
Window 36b displays the current time counter of the image recording device 18.
Window 36c displays a list of the last user-definable incidents and tasks. The inspector 28 can select a task or incident from this list and the details relating thereto are displayed in window 36d.
Window 36d is joined to window 36c. Window 36c shows information recorded against an incident or task. This information can be modified.
Window 36e displays information regarding the inspection target 30 or special instructions relating to the inspection target 30, depending on what tab 46 has been pressed.
The position of an inspecting entity, in this case ROV 22, is displayed on the user interface 12 as line 50. When inspecting an inspection target 32, line 50 starts from the bottom of the user interface 12. As ROV 22 moves along inspection target 30, line 50 moves up the user interface 12 until it reaches a position representing a line of symmetry on the x-y plane 40 as displayed. When ROV 22 begins to reach the end of inspection target 32, line 50 moves downwards from this position until it again reaches the bottom of the user interface 12 (signalling that ROV 22 has reached the end of inspection target 30).
As ROV 22 travels along inspection target 30, once a second survey computer 14 feeds information relating to the ROV's 22 position to the inspection management system 10. This new positioning information is then used to update the visual representation 18 of the inspection target 32 and its associated glyphs 52, 54 in the following manner.
The ROV's 22 new position is compared to its old position for changes in distance travelled along the inspection target 32. These changes, if any, are then replicated to each glyph 52, 54 currently displayed, such that those glyphs 52, 54 below line 50 are moved even closer to the bottom of the user interface 12 and glyphs 52, 54 above the line 50 are moved even closer to line 50. If an event task 32 or incident, as a result of the change in position of the ROV 22, is no longer within the y-axis 44 range of the user interface 12, it's representative glyph 52, 54 is no longer displayed. Similarly, if, as a result of the change in position of the ROV 22, a new event task 32 or incident comes within the y-axis 44 range of the user interface 12, its representative glyph 52, 54 is displayed above line 50. This allows the user interface 12 to provide a real-time simulation of the inspection according to the position of the ROV 22.
Once the visual representation 18 and its associated glyphs 52, 54 have been updated, the inspection management system 10 generates a list of incidents. This list includes details of the incident type and position. A sub-set of this list is then displayed in window 36c. Also displayed in window 36c is a list of ongoing and planned incidents, including their type and their start time.
The inspector 28 also has the option of stopping an ongoing incident by double- clicking it or pressing the appropriate button on the ongoing incidents toolbar (not shown). Once the ongoing incident is stopped, a sound file is generated and executed announcing the incident type and the length of time that the incident has been ongoing.
The inspector 28 can activate any of the glyphs 52 currently displayed on the user interface 12, with the exception of historical glyphs 54. The method of activation can vary ranging from double or single clicking the glyph 52 with a mouse or
activating a hot-key associated with the glyph 52. When activated, details of the inspection task 32 or incident associated with the glyph 52 is displayed in window 36d. A sound file is also generated and executed that announces the chosen glyph's 52 full name. The inspector 28 may then enter in information relating to the inspection task 32 or incident based on the visual feed displayed on monitors 26. This information is stored in related data module 23.
If the inspector 28 enters information related to an inspection task 32 or incident that falls outside of the range of acceptable values defined for that field, the inspector 28 is informed of this fact. The inspector 28 is then asked to confirm that the value is correct and, if so, the inspection task 32 or incident is designated an anomaly and added to the anomaly collection. When next updated, the user interface 12 indicates that the glyph 52 representing the inspection task now represents an anomaly. Details of the anomaly, including its position, are then generated as a sound file and announced.
Only one incident or event task can be displayed in window 36d at any one time. This assists in providing an uncluttered user interface 12 to the inspector 28.
If the inspector 28 identifies a new incident that needs to be recorded, the inspector 28 chooses the appropriate incident glyph 52 from toolbar 34a. The inspector 28 can then place the incident glyph 52 at a position on the user interface 12 such that the position of the new incident glyph 52 relative to the visual representation 38 of the inspection target 30 substantially corresponds to the position of the incident relative to the inspection target 30. The method used to position and size the incident glyph 52 is the "click and drag" method commonly known to the person skilled in the art. The position of the new incident is calculated according to the position of its glyph 52 on the user interface 12 relative to the position of the ROV 22 and the visual representation 38 of the inspection target 30. The new incident is then added to the incident collection and a sound file generated and executed that announces the position and type of incident identified. Information displayed in window 36d is then replaced with information relating to the new incident (default values being displayed) as defined for that type in generic module 25.
The inspector 28 may also identify a need for additional inspection tasks 32. New inspection tasks 32 can be created regardless of the position of the ROV 22. Accordingly, inspection tasks 32 can be created either before or after the position of the ROV 22. New inspection tasks 32 are created and defined in the same manner as described above and stored in data definitions module 21. Once created, the user interface 12 is updated to include the new inspection tasks 32 and information pertaining to the new inspection task 32 is displayed in window 36d.
The system also generates a seabed profile that is displayed in window 36a. The seabed profile is generated by taking an image of the seabed profile from monitors 26 via image recording device 18 every predetermined number of seconds. The captured image is then processed to determine the depth of spanning or depth of burial of the pipeline. The degree of scour, if any, on each side of the pipeline can also be determined during the image processing procedure. The position of the seabed profile is obtained via survey computer 14.
If the seabed profile is a span, the inspection management system 10 checks the length of the span over successive captured images. If the length of the span, as determined by these successive captured images, is greater than a preset incident limit, the span is recorded as an anomaly and processed accordingly.
At any stage the inspector 28 may use the inspection management system 10 to record part of the live video feed sent to monitors 26 by issuing a record command to the image recording device 18. Typically this is done to allow video images to be incorporated in reports generated by the inspection management system 10. Recording stops when a stop command is issued to the image recording device 18. If desired, the inspector may view the recording or pause the recording by issuing a play or pause command, as appropriate, to the image recording device 18. In each case, as a command is issued to the image recording device 18 a sound file is created and executed announcing the command given. Images and video recorded by the image recording device 18 can be used in subsequent reports.
During an inspection of an inspection target 30, a glyph (not shown) representing an additional inspection target 30 may be displayed. The inspector 28 may then inspect this additional inspection target 30 by activating this glyph as described above. The user interface 12 is then updated to represent the new inspection target 30 and the event tasks 32 and incidents associated with the new inspection target 32.
If a new inspection target 30 is not selected during the inspection of an inspection target 30, upon completion of the inspection of the inspection target 30, the inspector 28 must indicate that inspection of the inspection target 30 has been completed and what inspection target 30 will be inspected next.
The inspector 28 may stop the dive at any time by activating the 'stop' button in window 34C. When a dive is stopped a sound file is generated and executed that announces the dive name and is status.
Once an inspection has been completed, the inspector 28 may generate a report on the inspection as a whole or on a particular inspection target 30. The report may also be limited according to particular inspection tasks 32 or incidents. The report can be generated in either Word or Excel format and video or still taken from video recorded by the image recording device 18 may be included therein.
In a first alternative arrangement of the invention as described, the audio produced by the announcement of the various sound files is also directed towards an audio channel separate from the audio channel the inspector 28 uses for his/her commentary. The audio directed towards this separate audio channel is recorded and can be used by the inspector 28 when reviewing the dive to verify aspects of the inspection that may not have been commented on as part of his /her audio commentary.
In a second alternative arrangement of the invention as described, the visual representation 38 of the inspection target 30 is derived from direct video feed of the inspection target 30 during the inspection.
In a third alternative arrangement of the invention as described, the visual representation 38 of the inspection target 30 is presented in stereo vision format.
In a fourth alternative arrangement, historical glyphs may be activated.
It should be appreciated by the person skilled in the art that the use of the invention is not limited to the inspection of subsea facilities. In particular, it has potential in the following fields:
• overhead helicopter inspection of powerlines;
• internal pipecrawler inspections of pipes;
• landlord inspections of rented homes;
• inspections of roads by the department of transport;
• dental and medical checkups;
• product conformance inspections (car, electronics, etc.);
• environmental inspections (rivers, lakes, etc.)