US20030200391A1

US20030200391A1 - Method for managing critial data in an N-way mirrored storage device

Info

Publication number: US20030200391A1
Application number: US10/440,833
Authority: US
Inventors: Stanley Krehbiel; William Delaney; Donald Humlicek; Gregory Yarnell; Joseph Moore
Original assignee: LSI Logic Corp
Current assignee: LSI Corp
Priority date: 2001-06-15
Filing date: 2003-05-19
Publication date: 2003-10-23
Also published as: US6671777B1

Abstract

Accordingly, the present invention is directed to a method of managing data in a storage system. A method of performing a write to data storage system, including a first storage device and a second storage device, may include writing a first set of header information to a first storage device and a second storage device. The first set of header information includes a first sequence number and a second sequence number, in which the first set of header information includes a first sequence number incremented to indicate a change from the second sequence number. Data is written to the first storage device and the second storage device. Then, a second set of header information is written to a first storage device and a second storage device. The second set of header information includes a first sequence number and a second sequence number, in which the second set of header information includes a second sequence number incremented to correspond to the first sequence number.

Description

FIELD OF THE INVENTION

The present invention generally relates to the field of data storage, and particularly to a method for managing critical data in an N-way mirrored storage device.

BACKGROUND OF THE INVENTION

Persistent, efficient and safe storage of electronic data is of the utmost importance. Every aspect of modem life, from electronic commerce, storage of business and personal information, and the like have as a necessary and integral component the use and storage of electronic data. One method utilized to ensure access to electronic data is the use of a storage system employing a plurality of separate electronic data storage devices. For example, a storage system may include a variety of disk drives arranged in a RAID format, with data mirrored across the plurality of disks. In this way, should one device become unavailable, data may be accessed from one of the other devices.

Users of an N-Way Mirrored storage system may write data using a variety of mechanisms. For example, a write may be performed using both non-critical and critical writes. Non-critical writes are typically write operations for which interruptions, or partially-completed operations, need not be detected and/or corrected. Critical writes typically include operations for which interruptions are detected and/or corrected. Correction may include, for instance, ensuring that copies of the data on a storage device are the same, even if the data represents only a partially completed critical write operation. However, during a critical write, an interruption may occur which may call into question the validity of the data. Such an interruption may defeat the very purpose of a data storage system, namely the establishment of a valid data set, especially in an environment in which devices may be added and/or removed, and in which recovery from interruption during an update of the storage devices may be achieved.

Therefore, it would be desirable to provide a method of managing data in a storage system.

SUMMARY OF THE INVENTION

Accordingly, the present invention is directed to a method of managing data in a storage system. In a first aspect of the present invention, a method of performing a write to data storage system including a first storage device and a second storage device includes writing a first set of header information to a first storage device and a second storage device. The first set of header information includes a first sequence number and a second sequence number, in which the first set of header information includes a first sequence number incremented to indicate a change from the second sequence number. Data is written to the first storage device and the second storage device. Then, a second set of header information is written to a first storage device and a second storage device. The second set of header information includes a first sequence number and a second sequence number, in which the second set of header information includes a second sequence number incremented to correspond to the first sequence number.

In a second aspect of the present invention, a data storage system includes a first data storage device and a second data storage device suitable for storing electronic data. A data storage controller suitable for performing a program of instructions is coupled to the first data storage device and the second data storage device. The program of instruction configures the data storage controller to write a first set of header information to the first storage device and the second storage device. The first set of header information includes a first sequence number incremented to indicate initiation of a write operation. Data is written to the first storage device and the second storage device. A second set of header information is written to the first storage device and the second storage device, wherein the second set of header information includes a second sequence number incremented to indicate completion of the data write.

In a third aspect of the present invention, a method, includes obtaining a database identifier from a data storage device included in a data storage system. The obtained database identifier is compared with a previously stored database identifier, wherein if the obtained data identifier does not correspond with a previously stored database identifier, the data storage device is not native to a data storage system It is to be understood that both the forgoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention as claimed. The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate an embodiment of the invention and together with the general description, serve to explain the principles of the invention.

BRIEF DESCRIPTION OF THE DRAWINGS

The numerous advantages of the present invention may be better understood by those skilled in the art by reference to the accompanying figures in which: [0008]
FIG. 1A is a block diagram illustrating an exemplary embodiment of the present invention wherein a storage system including up to N storage devices is communicatively coupled to a host system including an application suitable for utilizing the storage system; [0009]
FIG. 1B is a block diagram depicting an embodiment of the present invention is shown wherein three distinct pieces of information are provided to establish a valid data set and determine if recovery operations are needed; [0010]
FIG. 2 is a flow diagram of an exemplary method of the present invention wherein three atomic operations are performed to accomplish a critical write operation; [0011]
FIGS. 3A, 3B, [0012] 3C and 3D are illustrations depicting embodiments of the present invention wherein sequence numbers having values indicating a variety of conditions are shown;
FIGS. 4A and 4B are illustrations of embodiments of the present invention wherein invalid and/or stale data as indicated by sequence numbers of the present invention are shown; and [0013]
FIG. 5 is a flow diagram depicting an exemplary method of the present invention wherein a database ID is utilized to detect a non-native storage device. [0014]

DETAILED DESCRIPTION OF THE INVENTION

Reference will now be made in detail to the presently preferred embodiments of the invention, examples of which are illustrated in the accompanying drawings. [0015]
Referring generally now to FIGS. 1 through 5, exemplary embodiments of the present invention are shown. The present invention may be utilized in creating an N-Way-Mirrored storage device in which individual drives may be removed and inserted during normal operation, when the device is powered off, and the like. Additionally, the present invention allows for data from other devices of the same type to be imported into the storage device. [0016]
Referring now to FIG. 1A, an [0017] embodiment 100 of the present invention is shown wherein a data storage system operable to employ the present invention is described. A data storage system may include a storage device controller 104 and a number of storage devices, such as Drive One 106, Drive Two 108 and up to Drive N 110 as contemplated by a person of ordinary skill in the art. The storage device controller 104 receives data from a host system 112 and stores the data across to storage devices, such as the drives 106, 108 & 110, to ensure data integrity and access to the data should one of the storage devices become unavailable, such as through a drive malfunction, data corruption, and the like condition. However, when writing data to the drives, an interruption, error condition, newly accessible drive, and the like may be encountered, thereby calling into question the validity of the data. For example, by adding a new drive and/or encountering an interruption during a write to the storage devices, a storage device controller 104, host system 112, application 114, and the like, must determine which data set is valid.
The present invention may utilize three distinct pieces of information to establish a valid data set and determine if recovery operations are needed, as shown in the [0018] embodiment 150 depicted in FIG. 1B. The three pieces of information include a database ID 152, 154 & 156, and two sequence numbers, which for purposes of the present discussion are labeled “A” 158, 160 & 162 and “B” 164, 166 & 168. This data may be stored in a special location on each of the drives 170, 172 & 174 in the database 176. The sequence numbers are used to detect interruptions from a “critical write” operation. The information may also be utilized to detect if foreign data has been introduced and thus, whether a merge operation is required.
The database ID is a unique value that may be computed based on the storage device serial number and a time stamp when the database is created, and other like methods of creating a unique value as contemplated by a person of ordinary skill in the art without departing from the spirit and scope of the present invention. A database ID may be included to make it possible for an entire foreign database to be recognized and merged into the native system. For example, when one or more drives from a foreign system are inserted into a native system. [0019]
In general, users of an N-Way-Mirrored storage facility may write data using a variety of mechanisms. For example, it contemplated embodiments, a write may be performed using one of two mechanisms, namely non-critical and critical writes. Non-critical writes are typically write operations for which interruptions, or partially-completed operations, need not be detected/corrected. However, critical writes may include operations for which interruptions must be detected and/or corrected. Correction may include ensuring that all copies of the data on the N-Way-Mirrored storage devices are the same, even if the data represents only a partially completed critical write operation. [0020]
One application for which these two different types of writes may be employed is in a database system. Insertions of individual records, such as segments of data, into the database generally require that the data record is written, and then an index structure is modified to reference the data record. The write of the data record may usually be handled in a non-critical fashion, since an interruption of the operation will simply cause the system to restart before any index structure points to the new record (although critical data record writes are also contemplated by the present invention). In non-critical instances, it may not matter if an N-Way-Mirrored device contains mismatched copies of a data record. For example, since no index record points to the data record yet, the database will not try to access that data record. [0021]
However, in most instances, an index record update would generally be handled as a critical write. Interruption of the index record update may have to be detected at restart time to ensure that all copies of the data on the N-Way-Mirrored device are the same. Even if fully-mirrored matching data represents a partially-completed write, the database system may be prepared to recognize this condition and handle it accordingly, such as by utilizing a CRC computation/analysis (for detection) and rollback/rollforward (for data integrity). [0022]
Therefore, an important consideration in the storage of critical data is that, preferable, all N copies of the data are the same. Consequently, the return of a consistent result to a client is of the N-Way-Mirrored device may be ensured, regardless of which of the N copies is used to satisfy a read request. [0023]
Referring now to FIG. 2, an [0024] exemplary method 200 of the present invention is shown wherein three operations are performed to accomplish a critical write operation. To implement this model, a critical write is performed as three atomic operations, which allows detection of interruptions, identification of when the interruptions occurred and recovery from them.
In a normal operating mode, both sequence numbers on all drives will be the same. When a “critical write” operation is initiated [0025] 202, header information may be written to all drives 204 with the sequence number A incremented 206 by one. This is done as a single atomic operation before phase two is allowed to begin 208. Phase two is the data phase in which all data is written out to all drives 210. When the data is completely written to all drives, phase three is allowed to begin 212. In phase three, the header information is written again 214 and in this phase sequence number B is incremented 216 to once again match sequence number A. Only when all three phases have completed is the write operation allowed to complete 218 and return status to a client 220, such as an application 114 operating on a host system 112 as depicted in FIG. 1A.
Referring generally now to FIGS. 3A, 3B, [0026] 3C and 3D, exemplary embodiments 300, 330, 360 & 390 of the present invention are shown wherein an interrupt has occurred. An interruption may occur in any of the three phases described in FIG. 2. However, by utilizing the present invention, the occurrence of the interruption may be detected and recovered. Sequence number configurations corresponding to exemplary interruptions are shown in the corresponding figures.
For example, as shown in the [0027] embodiment 300 depicted in FIG. 3A, a sequence number configuration of an uninterrupted condition is shown. The interruptions may be detected when a device executes a start-of-day (i.e. restart) procedure by examining both sequence numbers “A” 302, 304, 306 & 308 and “B” 310, 312, 314 & 316 as stored on the drives. If the sequence numbers 302-316 on the drives 318, 320, 322 & 324 are the same, then an interruption has not occurred and the device may begin normal operation.
Referring now to FIG. 3B, an [0028] embodiment 330 of the present invention is shown wherein sequence numbers resulting from an interruption occurring in a first phase of a critical write are shown. If an interruption occurred during a first phase of a critical write, one or more of sequence number A's 332 & 334 of drives 340 & 342, which were updated but an interruption occurred before drives 344 & 346 could likewise be updated, will be greater than the other sequence number A's 336 & 338. In this instance, an update of headers, wherein sequence number A 332 & 334 of the drives 340 & 342 having the higher sequence number A value is rewritten to match previous values of sequence number 336 & 338 of the non-updated drives 344 & 346 is sufficient to correct database integrity. Further verification that the interruption occurred during the first phase may be found by examining the second set of sequence numbers, labeled sequence number B. For example, if sequence number B 348, 350, 352 & 354 of the drives matched the lower valued sequence number A 336 & 338.
Similarly, if an interruption occurred in phase three, all of the sequence number A's will match, but some of the sequence number B's will match the A's and some will not have been incremented. For example, as shown in FIG. 3C, an [0029] embodiment 360 of the present invention is shown wherein a first set of sequence numbers of a drive match each other and at least one sequence number of a second set of sequence numbers, with at least one additional sequence number of the second set of sequence number not matching another sequence number of the second set.
For instance, a set of four [0030] drives 362, 364, 366 & 368 may have a first set of sequence numbers 370, 372, 374 & 376 that match. A second sequence numbers, sequence number B's 378 & 380, of at least one drive, in this instance two drives 366 & 368, match the first sequence numbers, sequence number A 370, 372, 374 & 376.
However, other sequence numbers of the second sequence numbers were not incremented. In this instance, [0031] sequence numbers B 382 & 384 are one less than the other sequence numbers, such as sequence numbers A 370, 372, 374 & 376 as well as the other sequence number B 378 & 380 of the two incremented drives 366 & 368. As discussed in the previous embodiment 330 shown in FIG. 3B, an update of the headers, wherein sequence number A 370, 372, 374 & 376 and sequence number B 378 & 380 having the higher sequence number values, are rewritten to match previous values of sequence number B 382 & 384 to correct database integrity.
Referring now to FIG. 3D, an [0032] embodiment 390 of the present invention is shown wherein sequence numbers resulting from an interruption occurring in a second phase of a critical write are shown. An interruption in phase two is detected if all sequence number A's 392 a, 392 b, 392 c & 392 d match and all sequence number B's 394 a, 394 b, 394 c & 394 d match, but sequence number A's 392 a, 392 b, 392 c & 392 d and sequence number B's 394 a, 394 b, 394 c & 394 d do not match each other. In this case, a resynchronization of database may be performed. For example, resynchronizing may include performing full copies to all drives to ensure that they all have the same content.
Invalid data, such as stale data, may also be detected using the sequence numbers. For instance, as shown in the [0033] exemplary embodiment 400 illustrated in FIG. 4A, a plurality of drives including at least one drive having invalid data may be detected by utilizing the present invention. If a drive 402 contains a sequence number, such as sequence number A 404, that is more than one less than the largest corresponding sequence number, such as other sequence number A's 406, 408 & 410 contained in the storage device (the sequence number is more than one increment of the other sequence numbers), then the drive contains stale data. To correct this condition, the drive 402 and/or corresponding data segment to the sequence number 404 may be overwritten to match the valid drives. Although sequence number A is described, it should be apparent to a person of ordinary skill in the art that a similar analysis of sequence number B may be performed without departing from the spirit and scope of the present invention.
Likewise, the present invention may also be utilized to detect stale data as shown in the [0034] embodiment 450 depicted in FIG. 4B. If a drive 452 contains a sequence number A 454 that is one less than the largest sequence number A, such as other sequence number A's 456, 458 & 460 of at least one other drive 462, 464 & 466 of the system, and the system contains a drive (or drives) whose sequence number B 468, 470 & 472 matches the largest sequence number A 456, 458 & 460, then this drive 452 is also detected as containing stale data, and may be rewritten as described previously.
Referring now to FIG. 5, an [0035] exemplary method 500 of the present invention is shown wherein a database ID is utilized to detect a new drive. For example, a database ID may be used to determine if a drive has been imported from another database. If drives are detected from another (foreign) N-Way-Mirrored device, the drives may be grouped together and the imported data merged into the native system. When this operation is complete, the drives are then overwritten with the data from the native database and integrated into the N-Way-Mirrored device for all ensuing read/write operations.
For example, a storage system warm up [0036] 502 is initiated. Database Ids are obtained 504 from the corresponding drives, such as the drives and database Ids shown in FIG. 1B. The database ID from each drive is compared with stored ID values 506. The database ID is a unique value that may be computed based on the storage device serial number and a time stamp when the database is created, and other like methods of creating a unique value as contemplated by a person of ordinary skill in the art without departing from the spirit and scope of the present invention. If the database Ids correspond 508, all the drives are native to the storage system and thus, the storage system is ready to perform transactions 510.
However, if the database IDs do not correspond [0037] 508, data may be written from the new drive to be included with preexisting data on the other drives 512. Thus, the database ID enables an entire foreign database to be recognized and merged into the native system. Further, it may be desirable to then write both sets of data back to the new drives so that all the available drives correspond 514. For example, when one or more drives from a foreign system are inserted into a native system, and the data included on the drive, as well as the drives themselves, are to be included in the system.
Additionally, in some instances it may be preferable to overwrite the new drive. In this case, data may be written from the native drives to the [0038] non-native drive 516 so that data included on the drives 106, 108 & 110 (FIG. 1A) of the data storage system 102 (FIG. 1A) correspond, without utilizing data as initially included on the new drive.
Although incrementing is described in which values increase, it should be apparent to a person of ordinary skill in the art that the present invention contemplates reducing values and like methods utilized to provide a sequence. Further, an increment contemplates any value, which may be utilized to indicate a change from an initial value to a second changed value. Moreover, although the matching of the first sequence number and the second sequence number is described, it should be apparent that one to one correspondence between the values is not required, as the present invention may employ a known difference between sequence number to determine if a write operation has been performed, interrupt, corrupted, and the like, and other like methods of comparison and correspondence as contemplated by a person of ordinary skill in the art. [0039]
It is believed that the method of managing critical data in an N-Way mirrored storage device of the present invention and many of its attendant advantages will be understood by the forgoing description. It is also believed that it will be apparent that various changes may be made in the form, construction and arrangement of the components thereof without departing from the scope and spirit of the invention or without sacrificing all of its material advantages. The form herein before described being merely an explanatory embodiment thereof. It is the intention of the following claims to encompass and include such changes. [0040]

Claims

What is claimed is:

1. A method of performing a write to data storage system including a first storage device and a second storage device, comprising:

writing a first set of header information to a first storage device and a second storage device, wherein the first set of header information includes a first sequence number and a second sequence number, in which the first set of header information includes a first sequence number incremented to indicate a change from the second sequence number;

writing data to the first storage device and the second storage device; and

writing a second set of header information to a first storage device and a second storage device, wherein the second set of header information includes a first sequence number and a second sequence number, in which the second set of header information includes a second sequence number incremented to correspond to the first sequence number.

2. The method as described in claim 1, further comprising returning status of completion of a write writing the second set of header information.

3. The method as described in claim 1, wherein writing data to the first storage device and the second storage device is performed after completion of writing the first set of header information.

4. The method as described in claim 1, wherein writing the second set of header information is started after completion of writing data to the first storage device and the second storage device.

5. The method as described in claim 1, wherein data includes a database transaction.

6. The method as described in claim 1, further comprising detecting and recovering from an interruption.

7. The method as described in claim 6, wherein the interruption is detected by examining the first sequence numbers and the second sequence numbers included on the first storage device and the second storage device, so that if the first sequence number stored on the first storage device is one increment greater than the first sequence number stored on the second storage device and the second sequence numbers stored on the first storage device and the second storage device match the first sequence number stored on the second storage device, recovering includes writing header information matching the first sequence number stored on the second storage device to the first storage device.

8. The method as described in claim 6, wherein the interruption is detected by examining the first sequence numbers and the second sequence numbers included on the first storage device and the second storage device, so that if the first sequence number stored on the first storage device matches the first sequence number stored on the second storage device; and the second sequence number of the first storage device matches the first sequence numbers of the first storage device and the second storage device; and the second sequence number of the second storage device is one increment lower that the second sequence number of the first storage device, recovering includes writing header information matching the second sequence number of the second device, so that the first storage device includes the first sequence number and second sequence number of the second storage device, as well as writing header data so that the first sequence number of the second storage device matches the second sequence number of the second device.

9. The method as described in claim 6, wherein the interruption is detected by examining the first sequence numbers and the second sequence numbers included on the first storage device and the second storage device, so that if the first sequence numbers stored on the first storage device and the second storage device match, and the second sequence numbers stored on the first storage device and the second storage device match, but the first sequence numbers do not match the second sequence numbers, recovering includes performing a resynchronization.

10. The method as described in claim 1, wherein invalid data is detected.

11. The method as described in claim 10, wherein invalid data is detected by examining the first sequence numbers included on the first storage device and the second storage device, so that if the first sequence number included on the first storage device is more than one less increment of the first sequence number included on the second storage device, the header information from the second storage device is written to the first storage device.

12. The method as described in claim 10, wherein invalid data is detected by examining the first sequence numbers and the second sequence numbers included on the first storage device and the second storage device, so that if the first sequence number included on the first storage device is one less increment of the first sequence number included on the second storage device, and the second sequence number of the second storage device matches the first sequence number of the second storage device, the header information from the second storage device is written to the first storage device.

13. A data storage system suitable for performing a write of data, comprising:

a first data storage device suitable for storing electronic data;

a second data storage device suitable for storing electronic data;

a data storage controller coupled to the first data storage device and the second data storage device, the data storage controller suitable for performing a program of instructions, wherein the program of instruction configures the data storage controller to write a first set of header information to the first storage device and the second storage device, wherein the first set of header information includes a first sequence number incremented to indicate initiation of a write operation; write data to the first storage device and the second storage device; and write a second set of header information to the first storage device and the second storage device, wherein the second set of header information includes a second sequence number incremented to indicate completion of the data write.

14. The data storage system as described in claim 13, further comprising returning status of completion, after writing the second set of header information, to an application requesting a database transaction.

15. The data storage system as described in claim 13, wherein writing data to the first storage device and the second storage device is performed after completion of writing the first set of header information.

16. The data storage system as described in claim 13, wherein writing the second set of header information is started after completion of writing data to the first storage device and the second storage device.

17. The data storage system as described in claim 13, wherein data includes a database transaction.

18. The data storage system as described in claim 13, further comprising detecting and recovering from an interruption.

19. The data storage system as described in claim 18, wherein the interruption is detected by examining the first sequence numbers and the second sequence numbers included on the first storage device and the second storage device, so that if the first sequence number stored on the first storage device is one increment greater than the first sequence number stored on the second storage device and the second sequence numbers stored on the first storage device and the second storage device match the first sequence number stored on the second storage device, recovering includes writing header information matching the first sequence number stored on the second storage device to the first storage device.

20. The data storage system as described in claim 18, wherein the interruption is detected by examining the first sequence numbers and the second sequence numbers included on the first storage device and the second storage device, so that if the first sequence number stored on the first storage device matches the first sequence number stored on the second storage device; and the second sequence number of the first storage device matches the first sequence numbers of the first storage device and the second storage device; and the second sequence number of the second storage device is one increment lower that the second sequence number of the first storage device, recovering includes writing header information matching the second sequence number of the second device, so that the first storage device includes the first sequence number and second sequence number of the second storage device, as well as writing header data so that the first sequence number of the second storage device matches the second sequence number of the second device.

21. The data storage system as described in claim 18, wherein the interruption is detected by examining the first sequence numbers and the second sequence numbers included on the first storage device and the second storage device, so that if the first sequence numbers stored on the first storage device and the second storage device match, and the second sequence numbers stored on the first storage device and the second storage device match, but the first sequence numbers do not match the second sequence numbers, recovering includes performing a resynchronization.

22. The data storage system as described in claim 13, wherein invalid data is detected.

23. The data storage system as described in claim 22, wherein invalid data is detected by examining the first sequence numbers included on the first storage device and the second storage device, so that if the first sequence number included on the first storage device is more than one less increment of the first sequence number included on the second storage device, the header information from the second storage device is written to the first storage device.

24. The data storage system as described in claim 22, wherein invalid data is detected by examining the first sequence numbers and the second sequence numbers included on the first storage device and the second storage device, so that if the first sequence number included on the first storage device is one less increment of the first sequence number included on the second storage device, and the second sequence number of the second storage device matches the first sequence number of the second storage device, the header information from the second storage device is written to the first storage device.

25. A method, comprising:

obtaining a database identifier from a data storage device included in a data storage system; and

comparing the obtained database identifier with a previously stored database identifier, wherein if the obtained data identifier does not correspond with a previously stored database identifier, the data storage device is not native to a data storage system.

26. The method as described in claim 25, wherein if the obtained data identifier does correspond with a previously stored database identifier, the data storage device is native to a data storage system.

27. The method as described in claim 25, wherein the data storage device is not native to the data storage system, data included on the data storage device is written to a second data storage device included in the data storage system.

28. The method as described in claim 27, wherein after data included on the data storage device is written to the second data storage device, the data storage device is written so that the data storage device corresponds with the second data storage device.

29. The method as described in claim 25, wherein the data storage device is not native to the data storage system, data from a second data storage device native to the data storage system is written to the data storage device.

30. The method as described in claim 29, wherein the data storage device includes previously stored data, the previously stored data is overwritten by data from the second data storage device.