CN103593409A - Real-time database retrieval method and real-time database retrieval system - Google Patents
Real-time database retrieval method and real-time database retrieval system Download PDFInfo
- Publication number
- CN103593409A CN103593409A CN201310499054.2A CN201310499054A CN103593409A CN 103593409 A CN103593409 A CN 103593409A CN 201310499054 A CN201310499054 A CN 201310499054A CN 103593409 A CN103593409 A CN 103593409A
- Authority
- CN
- China
- Prior art keywords
- index
- retrieval
- node
- index node
- level
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/3331—Query processing
- G06F16/334—Query execution
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention relates to a real-time database retrieval method and a real-time database retrieval system. The real-time database retrieval method includes creating multiple levels of indexes, linking a node of each upper-level index to a node of a corresponding lower-level index according to retrieval intervals until all private indexes are linked; starting retrieval from the uppermost-level index, starting to retrieve the lower-level indexes from nodes with intersections with target intervals until data blocks of the private indexes are retrieved. The real-time database retrieval method and the real-time database retrieval system have the advantages that the integral private indexes and the integral data blocks are not required to be traversed, so that the inquiry performance can be improved; the data blocks relevant to retrieval targets are returned in retrieval results without irrelevant results, and accordingly the retrieval efficiency can be improved.
Description
Technical field
The present invention relates to Real-Time Databases System Technique, more particularly, relate to a kind of real-time data base search method and searching system.
Background technology
Real-time data base, as the basic database that is used for depositing Process History data in industrial monitoring system, has measuring point quantity many (1,000,000 ranks), memory data output large (TB rank), and writing speed is high, and inquiry request is feature frequently.In existing real-time data base, for the retrieval mode of these mass datas, normally each data file is set up to a privately owned index, for the data block in index data file; For all data files are set up the markers scope that ，Gai global index of Yi Ge global index records the data of each data file storage.While inquiring about the data of certain markers scope of certain measuring point, first by global index, retrieve storage data markers scope and inquire about the data file that markers scope has common factor, then travel through these data files, by the data block in the privately owned index ergodic data file of data file, the data of last data query piece return to user.
Such processing mode is because measuring point is a lot, and the write frequency of different measuring points is widely different, and therefore, in a data file, the markers dimensional discrepancy of the data of different measuring points storage is very large, causes the markers scope of data file extended.For example, measuring point A has stored the data in January in data file 1, and measuring point B has stored the data in May in data file 1, and the markers scope of data file is that January is to May so.In this case, by global index, inquire about the data file of returning and may not comprise required data.For example, the data in the February of measuring point C are not stored in the data file 1 in previous example, but because the markers scope of data file 1 comprises February, federation's return data file 1 while therefore inquiring about by global index.Global index has returned does not have the required data file of user in a large number, therefore causes retrieval performance low.
Summary of the invention
The object of the invention is to, low for the retrieval mode performance of existing real-time data base, the problem of the speed of impact retrieval, provides a kind of real-time data base search method and searching system, to overcome above-mentioned defect.
Real-time data base search method provided by the present invention, comprises the following steps:
S10, N other index of level of generation, wherein N is greater than 2 integer, and each index comprises a plurality of index nodes of link successively; Each index node comprises that at least one retrieval is interval;
S20, at the 2nd grade, be indexed in the index of N level, when the retrieval interval of the index node in i-1 level index belongs to the retrieval interval of the index node in i level index, the index node of foundation from i level index points to the link of the index node in i-1 level index; 2≤i≤N wherein;
S30, all data blocks are generated to privately owned index node, described privately owned node comprises that privately owned retrieval is interval; When the privately owned retrieval interval of privately owned index node belongs to the retrieval of index node of the 1st grade of index when interval, set up the link that index node from the 1st grade of index points to privately owned index node;
S40, since N level index retrieval, when there is common factor in the retrieval interval of searched targets and index node, this index node of mark is retrieve position point;
S50, when N-1 is greater than 1, from the retrieve position point of N level index, link to the index node of N-1 level index, and jump to step S60; When N-1 equals 1, jump to step S70;
S60, when the index node of searched targets and the N-1 level index linking has common factor, this index node of mark is retrieve position point, using the value of N-1 as new N value, jumps to step S50;
S70, from the index node of the 1st grade of index, link to data block.
Real-time data base search method of the present invention, wherein retrieves interval and comprises initial markers and finish markers.
Real-time data base search method of the present invention, wherein index node also comprises for linking to the link number of next index node.
Real-time data base search method of the present invention, wherein step S40 also comprises, when the interval nothing of retrieval of searched targets and index node is occured simultaneously, output Query Result is empty result for retrieval.
The present invention also provides a kind of real-time data base searching system, comprising: the index framework interlinking and privately owned index; Described index framework comprises:
N other index of level, wherein N is greater than 2 integer, and each index comprises a plurality of index nodes of link successively; Each index node comprises that at least one retrieval is interval;
At the 2nd grade, be indexed in the index of N level, when the retrieval interval of the index node in i-1 level index belongs to the retrieval interval of the index node in i level index, the index node of foundation from i level index points to the link of the index node in i-1 level index; 2≤i≤N wherein;
Described privately owned index comprises a plurality of data blocks, and each data block comprises privately owned index node; Described privately owned node comprises that privately owned retrieval is interval; When the privately owned retrieval interval of privately owned index node belongs to the retrieval of index node of the 1st grade of index when interval, set up the link that index node from the 1st grade of index points to privately owned index node;
When retrieving, since the retrieval of N level index, when there is common factor in the retrieval interval of searched targets and index node, this index node of mark is retrieve position point;
When N-1 is greater than 1, from the retrieve position point of N level index, link to the index node of N-1 level index, when the index node of searched targets and the N-1 level index linking has common factor, this index node of mark is retrieve position point, using the value of N-1 as new N value;
When N-1 equals 1, from the index node of the 1st grade of index, link to data block.
Real-time data base searching system of the present invention, wherein retrieves interval and comprises initial markers and finish markers.
Real-time data base searching system of the present invention, wherein index node also comprises for linking to the link number of next index node.
Real-time data base searching system of the present invention, when the interval nothing of retrieval of searched targets and index node is occured simultaneously, output Query Result is empty result for retrieval.
Implement real-time data base search method of the present invention and searching system, without going through all over whole privately owned index and data block, improved the performance of inquiry; Meanwhile, what result for retrieval returned is the data block relevant to searched targets, can not return and not have related result, thereby improve effectiveness of retrieval.
Accompanying drawing explanation
Below in conjunction with accompanying drawing, the present invention will be described, wherein:
Fig. 1 is the real-time data base search method of the present invention process flow diagram of preferred embodiment first;
The first index framework embodiment of Fig. 2 for setting up according to Fig. 1 process flow diagram.
Embodiment
Below in conjunction with the drawings and specific embodiments, the present invention is described in detail.
Be illustrated in figure 1 the real-time data base search method of the present invention process flow diagram of preferred embodiment first, this flow process is in the retrieving for prior art, because index granularity is too large, and the improvement that causes global index's performance lowly to be made.In the present embodiment, index granularity is reduced, from original data file, change data block into, thereby improve retrieval performance.
In the present embodiment, first perform step S10: be that real-time data base generates N other index of level, wherein N is greater than 2 integer, considers that the index itself of retrieving use also can take certain storage area, should be by its setting excessive, for example can get N=3; Each index comprises a plurality of index nodes of link successively; Each index node comprises that at least one retrieval is interval, this retrieval is interval for demarcating the target zone of retrieval, for example retrieving interval can determine by markers scope: a markers scope comprises initial markers and finishes markers, as being a markers scope January to June, be initial markers January, and June is for finishing markers.
After establishing a plurality of other index of level, to establish the link for the index node of these index, specifically by S20 step, complete: at the 2nd grade, be indexed in the index of N level, when the retrieval interval of the index node in i-1 level index belongs to the retrieval interval of the index node in i level index, the index node of foundation from i level index points to the link of the index node in i-1 level index; 2≤i≤N wherein; Take N=3 as example, for these 3 other index of level, establish the link, for example, in 3rd level index, the markers scope of some index node A is that January is to June, in the 2nd grade of index, the markers scope of some index node B be February to May, the markers scope of Node B, in the scope of node A, is set up the link of pointing to B from A.
After the sensing link completing between index nodes at different levels, the link that also needs to set up the 1st grade of index point data block, specifically completes by S30 step: all are generated to privately owned index node to all data blocks, and privately owned node comprises that privately owned retrieval is interval; When the privately owned retrieval interval of privately owned index node belongs to the retrieval of index node of the 1st grade of index when interval, set up the link that index node from the 1st grade of index points to privately owned index node.For example the markers scope of the index node C of some the 1st grade of index be September to October, the markers scope of the privately owned index node D of a certain data block be September to October, there is common factor in both, sets up the link of pointing to D from C.In order to ensure can be from correct linking to next index node of index node, for each index node adds link number, this link number be used in reference to correct index node.
So far, whole multiple index framework establishes, can retrieve, in step S40, retrieval from the most top index, that is: is retrieved since N level index, when there is common factor in the retrieval interval of searched targets and index node, this index node of mark is retrieve position point; For example get N=3, searched targets is the data block in September to October, from the 1st index node of 3rd level index, starts to retrieve common factor situation.When there is common factor, relevant index node is labeled as to retrieve position point.If in this step, do not find any and range of search to have the index node of common factor, prove in the data block of preserving less than the data that match with search condition, directly return and be retrieved as empty Query Result.
Step S50: retrieve to next stage from retrieve position point used, it is specific as follows: when N-1 is greater than 1, be that current index is not the 1st grade of index, from the retrieve position point of N level index, link to the index node of N-1 level index, and jump to step S60; Otherwise when N-1 equals 1, show currently to have arrived the 1st grade of index, jump to step S70.
At step S60, when the index node of searched targets and the N-1 level index linking has common factor, this index node of mark is retrieve position point, using the value of N-1 as new N value, jumps to step S50; , until there is not next stage index in the retrieval of all occuring simultaneously for every grade of index.
At step S70, from the index node of the 1st grade of index, link to data block.So just completed whole retrieval.
If Fig. 2 is a concrete retrieval architecture.This framework comprises 3 grades of index and the privately owned index of link with it, and the included retrieval of the node that index at different levels are included and each node is interval as following table:
Wherein A, B, C ... K is the numbering of each node, the digital scope such as " 1-8 ", " 9-16 " is markers scope, for convenient signal, use simple number designation " 1-8 ", " 9-16 " to replace the sign of comparatively tediously long " date Hour Minute Second ", those skilled in the art is to be understood that, in this retrieval architecture, the numeral in " 1-8 ", " 9-16 " means different time points., with the data instance of retrieval " 9-10 ", first from the 1st node A of 3rd level index, start retrieval now, a retrieval interval in A is that " 9-16 " occurs simultaneously with " 9-10 " existence of target.The retrieval interval of node A has comprised the node C of the 2nd grade of index and the retrieval of node D interval, according to this link, from node A, link to node C and D, in order to ensure A node, can be linked on node C and D, in node A, be provided for pointing to the link number of node C and D.Further,, there is common factor with the target " 9-10 " of retrieval in the retrieval interval " 9-12 " that node D wherein comprises, therefore accesses corresponding the 1st grade of index node H of this index.For the node H of the 1st grade of index, comprised retrieval interval " 9-10 ", there is common factor with the markers scope of inquiry, therefore access privately owned index node and corresponding data block that this index is corresponding, the data in inquiry return data piece, complete retrieval.
From the process of this retrieval, can find out, the impact that not distributed by the data of other measuring points, in query script, only access has file and the data block of the data of needs inquiry, so has improved query performance.
In above example, the included retrieval interval of each node is time range, for a person skilled in the art, can be also other the feature such as file size, access times.These features are appointed as to retrieval interval, and establish multilevel retrieval, can realize the retrieval mode being similar in Fig. 2.
In the example of Fig. 2, just only have a node H to comprise searched targets, therefore to corresponding to 1 data block, but the data block of returning in actual retrieval is to determine according to concrete searched targets and the common factor situation of multiple index.For example, use equally the retrieval architecture of Fig. 2, and the target of retrieval is " 8-9 ", now, all has common factor with two markers scopes of the node A of 3rd level index, from these two markers models, be linked to node C and the node D of the 2nd grade of index.And further from node C and node D, be linked to the node G and node H of the 1st grade of index, according to the link of the 1st grade of index, return to two data blocks as result for retrieval.
These are only the specific embodiment of the invention, can not limit scope of the present invention with this, equalization that those skilled in the art in the art do according to this creation changes, and the change known of those skilled in that art, all should still belong to the scope that the present invention is contained.
Claims (8)
1. a real-time data base search method, is characterized in that, comprises the following steps:
S10, N other index of level of generation, described N is greater than 2 integer, and every grade of described index comprises a plurality of index nodes of link successively; Described in each, index node comprises that at least one retrieval is interval;
S20, at the 2nd grade, be indexed in the index of N level, when the retrieval interval of the index node in i-1 level index belongs to the retrieval interval of the index node in i level index, the index node of foundation from i level index points to the link of the index node in i-1 level index; 2≤i≤N wherein;
S30, all data blocks are generated to privately owned index node, described privately owned index node comprises that privately owned retrieval is interval; When the privately owned retrieval interval of privately owned index node belongs to the retrieval of index node of the 1st grade of index when interval, set up the link that index node from the 1st grade of index points to privately owned index node;
S40, since N level index retrieval, when there is common factor in the retrieval interval of searched targets and index node, this index node of mark is retrieve position point;
S50, when N-1 is greater than 1, from the retrieve position point of N level index, link to the index node of N-1 level index, and jump to step S60; When N-1 equals 1, jump to step S70;
S60, when the index node of searched targets and the N-1 level index linking has common factor, this index node of mark is retrieve position point, using the value of N-1 as new N value, jumps to step S50;
S70, from the index node of the 1st grade of index, link to data block.
2. real-time data base search method according to claim 1, is characterized in that, described retrieval interval comprises initial markers and finishes markers.
3. according to right, will go the real-time data base search method described in 1 or 2, it is characterized in that, described index node also comprises for linking to the link number of next index node.
4. real-time data base search method according to claim 1, is characterized in that, described step S40 also comprises, interval when occuring simultaneously when the retrieval of searched targets and index node, output Query Result is empty result for retrieval.
5. a real-time data base searching system, is characterized in that, comprising: the index framework interlinking and privately owned index; Described index framework comprises:
N other index of level, wherein N is greater than 2 integer, and each index comprises a plurality of index nodes of link successively; Each index node comprises that at least one retrieval is interval;
At the 2nd grade, be indexed in the index of N level, when the retrieval interval of the index node in i-1 level index belongs to the retrieval interval of the index node in i level index, the index node of foundation from i level index points to the link of the index node in i-1 level index; 2≤i≤N wherein;
Described privately owned index comprises a plurality of data blocks, and each data block comprises privately owned index node; Described privately owned node comprises that privately owned retrieval is interval; When the privately owned retrieval interval of privately owned index node belongs to the retrieval of index node of the 1st grade of index when interval, set up the link that index node from the 1st grade of index points to privately owned index node;
When retrieving, since the retrieval of N level index, when there is common factor in the retrieval interval of searched targets and index node, this index node of mark is retrieve position point;
When N-1 is greater than 1, from the retrieve position point of N level index, link to the index node of N-1 level index, when the index node of searched targets and the N-1 level index linking has common factor, this index node of mark is retrieve position point, using the value of N-1 as new N value;
When N-1 equals 1, from the index node of the 1st grade of index, link to data block.
6. real-time data base searching system according to claim 5, is characterized in that, described retrieval interval comprises initial markers and finishes markers.
7. according to the real-time data base searching system described in claim 5 or 6, it is characterized in that, described index node also comprises for linking to the link number of next index node.
8. real-time data base searching system according to claim 5, is characterized in that, when the interval nothing of retrieval of searched targets and index node is occured simultaneously, output Query Result is empty result for retrieval.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310499054.2A CN103593409A (en) | 2013-10-22 | 2013-10-22 | Real-time database retrieval method and real-time database retrieval system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310499054.2A CN103593409A (en) | 2013-10-22 | 2013-10-22 | Real-time database retrieval method and real-time database retrieval system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN103593409A true CN103593409A (en) | 2014-02-19 |
Family
ID=50083550
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201310499054.2A Pending CN103593409A (en) | 2013-10-22 | 2013-10-22 | Real-time database retrieval method and real-time database retrieval system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103593409A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104765836A (en) * | 2015-04-15 | 2015-07-08 | 王磊 | Multi-level index method for data of audio fingerprint database |
| CN106227781A (en) * | 2016-07-18 | 2016-12-14 | 中国农业大学 | The method for quickly retrieving of space one point data under big data |
| CN107766452A (en) * | 2017-09-26 | 2018-03-06 | 广西电网有限责任公司电力科学研究院 | A kind of index structure and its indexing means of suitable information in power dispatching center zero access |
| CN108228865A (en) * | 2018-01-15 | 2018-06-29 | 沈阳延云云计算技术有限公司 | A kind of data query method and apparatus |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5440730A (en) * | 1990-08-09 | 1995-08-08 | Bell Communications Research, Inc. | Time index access structure for temporal databases having concurrent multiple versions |
| US20060287984A1 (en) * | 2005-06-17 | 2006-12-21 | International Business Machines Corporation | Range query methods and apparatus |
| CN101046864A (en) * | 2006-03-29 | 2007-10-03 | 国际商业机器公司 | Method and system for reproduction of date driven data |
| CN101193031A (en) * | 2006-11-24 | 2008-06-04 | 曲锐 | Data processing method and network based on distributed hash table |
-
2013
- 2013-10-22 CN CN201310499054.2A patent/CN103593409A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5440730A (en) * | 1990-08-09 | 1995-08-08 | Bell Communications Research, Inc. | Time index access structure for temporal databases having concurrent multiple versions |
| US20060287984A1 (en) * | 2005-06-17 | 2006-12-21 | International Business Machines Corporation | Range query methods and apparatus |
| CN1892656A (en) * | 2005-06-17 | 2007-01-10 | 国际商业机器公司 | Range query and data stream processing methods and apparatus |
| CN101046864A (en) * | 2006-03-29 | 2007-10-03 | 国际商业机器公司 | Method and system for reproduction of date driven data |
| CN101193031A (en) * | 2006-11-24 | 2008-06-04 | 曲锐 | Data processing method and network based on distributed hash table |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104765836A (en) * | 2015-04-15 | 2015-07-08 | 王磊 | Multi-level index method for data of audio fingerprint database |
| CN104765836B (en) * | 2015-04-15 | 2019-03-26 | 高第网络技术(北京)有限公司 | A kind of multiple index method for audio-frequency fingerprint library data |
| CN106227781A (en) * | 2016-07-18 | 2016-12-14 | 中国农业大学 | The method for quickly retrieving of space one point data under big data |
| CN106227781B (en) * | 2016-07-18 | 2019-08-23 | 中国农业大学 | The method for quickly retrieving of big data down space one point data |
| CN107766452A (en) * | 2017-09-26 | 2018-03-06 | 广西电网有限责任公司电力科学研究院 | A kind of index structure and its indexing means of suitable information in power dispatching center zero access |
| CN107766452B (en) * | 2017-09-26 | 2021-07-06 | 广西电网有限责任公司电力科学研究院 | Indexing system suitable for high-speed access of power dispatching data and indexing method thereof |
| CN108228865A (en) * | 2018-01-15 | 2018-06-29 | 沈阳延云云计算技术有限公司 | A kind of data query method and apparatus |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108256088A (en) | A kind of storage method and system of the time series data based on key value database | |
| CN100523735C (en) | Fast map matching method based on small lattice road network organization and structure | |
| CN106933833B (en) | Method for quickly querying position information based on spatial index technology | |
| CN106528787B (en) | query method and device based on multidimensional analysis of mass data | |
| CN103631910A (en) | Distributed database multi-column composite query system and method | |
| CN102890722A (en) | Indexing method applied to time sequence historical database | |
| CN102456055B (en) | Method and device for retrieving interest points | |
| CN107766433A (en) | A kind of range query method and device based on Geo BTree | |
| CN103577440A (en) | Data processing method and device in non-relational database | |
| CN104252489A (en) | Method for fast obtaining position character description information according to latitude and longitude data | |
| CN103914456A (en) | Data storage method and system | |
| CN104090962A (en) | Nested query method oriented to mass distributed-type database | |
| CN103914483B (en) | File memory method, device and file reading, device | |
| CN103294790A (en) | Spatiotemporal semantic meaning indexing and retrieving method aiming at GPS (global positioning system) trajectory data | |
| CN106649412B (en) | Data processing method and equipment | |
| CN106095863A (en) | A kind of multidimensional data query and storage system and method | |
| CN103593409A (en) | Real-time database retrieval method and real-time database retrieval system | |
| CN102890719B (en) | A kind of method that license plate number is searched for generally and device | |
| CN104636349A (en) | Method and equipment for compression and searching of index data | |
| CN112214472A (en) | Meteorological grid point data storage and query method, device and storage medium | |
| CN102193983A (en) | Relation path-based node data filtering method of graphic database | |
| CN101963993B (en) | Method for fast searching database sheet table record | |
| US20090063397A1 (en) | Method and system for disjunctive single index access | |
| CN102193988A (en) | Method and system for retrieving node data in graphic database | |
| CN103500165A (en) | High-dimensional vector quantity search method combining clustering and double key values |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20140219 |
|
| WD01 | Invention patent application deemed withdrawn after publication |