CN100394393C - Information system data consistency detection - Google Patents
Information system data consistency detection Download PDFInfo
- Publication number
- CN100394393C CN100394393C CNB2006100204574A CN200610020457A CN100394393C CN 100394393 C CN100394393 C CN 100394393C CN B2006100204574 A CNB2006100204574 A CN B2006100204574A CN 200610020457 A CN200610020457 A CN 200610020457A CN 100394393 C CN100394393 C CN 100394393C
- Authority
- CN
- China
- Prior art keywords
- data
- data block
- digest value
- backup
- block
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 34
- 238000000034 method Methods 0.000 claims description 9
- 238000009825 accumulation Methods 0.000 claims description 3
- 238000005192 partition Methods 0.000 claims description 2
- 238000005498 polishing Methods 0.000 description 3
- 230000000052 comparative effect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
Images
Abstract
The present invention provides an information system data consistency detection method which belongs to the field of information safety. The present invention determines whether source data and backup data are in a consistent state by the otherness detection of the backup data and the source data, wherein the physical position of the source data and the physical position of the backup data are not limited and have a long distance, and the source data and the backup data can be file data, databank data, disk data, etc. The present invention can have the characteristics of quick detection speed, low request to network bandwidth, transparency to an application layer, etc., be widely applied to the consistency detection of the backup of the network information system data, and have a wide application prospect.
Description
One, technical field
The present invention proposes a kind of information system data consistency detection, belong to information security field.
Two, background technology
In infosystem, for guaranteeing that commercial data backup is extremely important continuously, data consistency detects and can the detection resources data whether be in consistent state with Backup Data, and therefore, crucial effect is played in the data consistency detection in infosystem.The data consistency detection of comparative maturity has the daily record by backup software, complete data in magnetic disk relatively to wait and guarantee that source data and Backup Data are to be in consistent state at present, wherein the method by the daily record of backup software is relevant with backup software, in the daily record of carrying out depending on when data consistency detects backup software, and the problem that exists of the method for data in magnetic disk comparison is that data volume load big, network the speed big, that data consistency detects of comparison is low fully.
The present invention proposes a kind of general data consistency detection, have following characteristics:
1) detection speed is fast, network traffics are few, low to the bandwidth requirement of network;
2) this method can reduce the cost of system based on the Internet of cheapness;
3) detection of data is based on that data block operates, and is transparent to using.
This method can be widely used in confirms in the network information system whether source data is in consistent state with Backup Data, and wherein source data and Backup Data can be file data, database data, data in magnetic disk or the like, and the present invention has broad application prospects.
Three, summary of the invention
The present invention proposes a kind of information system data consistency detection, this method is by carrying out the otherness detection to Backup Data and source data, confirm whether source data is in consistent state with Backup Data, wherein source data and Backup Data position physically is not limited, can divide to be in the strange land, source data and Backup Data can be file data, database data, data in magnetic disk etc.
This method is at first carried out same deblocking to source data and Backup Data, and source data and Backup Data obtain set of data blocks separately respectively behind the piecemeal.When source data and Backup Data are carried out deblocking, the method of partition unanimity of taking, so the data block that obtains behind data block that obtains behind the source data piecemeal and the Backup Data piecemeal is one to one, promptly each data block of source data has unique data block corresponding with it in Backup Data.Secondly, each data block of source data and each data block of Backup Data are asked digest value respectively; Then, the digest value of each data block of source data is compared with the digest value of the corresponding data block of Backup Data respectively, if the digest value of the data block of source data and the Backup Data digest value of corresponding data block with it are identical, think that then the data of this data block are consistent at source data and Backup Data, otherwise, think that then the data of this data block are inconsistent in source data and Backup Data; After the data consistency detection of all data blocks was finished, the consistency detection work of source data and Backup Data had also just been finished.
Four, description of drawings
Fig. 1 is architectural framework figure.
Fig. 2 is the step of deblocking.
Fig. 3 is the step that the data block digest value calculates.
Fig. 4 is the step of variance data consistency detection.
Five, embodiment
Describe concrete grammar of the present invention in detail below in conjunction with accompanying drawing.
Fig. 1 is architectural framework figure.
Fig. 1 is an architectural framework of the present invention, and wherein source data and Backup Data position physically is not limited, can divide to be in the strange land.Data consistency detects main combination by three steps such as deblocking, the calculating of data block digest value and variance data consistency detection and realizes.Wherein the deblocking step is carried out deblocking to source data and Backup Data, sets up set of data blocks separately; Data block digest value calculation procedure is carried out digest value to each data block of source data and Backup Data and is calculated; Variance data consistency detection step compares the digest value of source end data piece and the digest value of backup end respective data blocks, realizes the data consistency of data block is detected.
Particularly, the step of the information system data consistency detection of the present invention's proposition may further comprise the steps:
1) step of deblocking;
2) step of data block digest value calculating;
3) step of variance data consistency detection.
Fig. 2 is the step of deblocking.
Fig. 2 has provided the example of data being carried out deblocking.Data are made up of data cell among the figure, each data unit size is identical, if the size of last data cell of data is less than the size of other data cell, then make its size and other data cell big or small consistent with last data cell of blank polishing.Data block is made up of continuous in logic one or more data cells, and the size of each data block can be different.Data promptly can be regarded as by data cell and form like this, also can regard as by data block and form.
The deblocking step is as follows:
1) source data piecemeal step: source data is carried out piecemeal, and concrete steps are as follows:
1. the size of definition of data unit: the data cell that data is divided into the fixed length size, if the size of last data cell of data is less than the size of definition of data unit, then making its size with last data cell of blank polishing is the size of definition of data unit;
2. the size of definition of data piece: the size of specifies data block, the number of data units that promptly comprises, the size of each data block can be different;
3. the number of computational data piece: the number that calculates the data block that constitutes whole data.
2) Backup Data piecemeal step: Backup Data is carried out piecemeal, and concrete steps are as follows:
1. the size of definition of data unit: the data cell that data is divided into the fixed length size, if the size of last data cell of data is less than the size of definition of data unit, then making its size with last data cell of blank polishing is the size of definition of data unit;
2. the size of definition of data piece: the size of specifies data block, the number of data units that promptly comprises, the size of each data block can be different;
3. the number of computational data piece: the number that calculates the data block that constitutes whole data.
By above step, source data and Backup Data are carried out the same deblocking strategy, it is all identical respectively with the size of the number of the size of Backup Data data cell when the piecemeal, data block, each data block to be source data, obtains the data block set of source data and Backup Data behind the piecemeal.
The step that Fig. 3 data block digest value calculates.
Fig. 3 has provided the example that the data block digest value calculates.Data block is made up of m data unit among the figure, H is the digest calculations function, and as MD5, SHA-1 etc., each data cell has a digest value, draw the digest value of data block among the figure by the digest value of each data cell of accumulation calculating, h is the digest value of illustrated data block among the figure.
The step that the data block digest value calculates is as follows:
1) data block digest calculations initialization: the initialization related variable makes the digest value h=sky of data block, the numbering i=1 of data cell;
2) the digest value h ' of calculating current data unit: h '=H (d
i), wherein H is an abstract function, d
iIt is the i blocks of data unit of current data block;
3) current data block digest value: h=H (h ⊙ h ') is calculated in accumulation; Wherein ⊙ represents the concatenation operation of character string, i=i+1; If i is not more than the size of current data block, then change 2), otherwise h is the digest value of current data block.
Fig. 4 is the step of variance data consistency detection.
Fig. 4 has provided the step of variance data consistency detection, as shown in the figure, at first carry out variance data consistency detection initial work, secondly try to achieve the digest value of source data and the corresponding data block of Backup Data respectively, the digest value of comparing data piece then, if the digest value of two ends data block is identical, think that then the data of respective data blocks of the data of this data block of source data and Backup Data are in consistent state, if the digest value difference of data block, think that then the data of respective data blocks of the data of this data block of source data and Backup Data are in inconsistent state, repeat above step and detect up to the otherness of all data blocks of finishing source data and Backup Data.
Particularly, the step of variance data consistency detection is as follows:
1) variance data consistency detection initialization: the initialization related variable makes the numbering i=1 of data block;
2) digest value of calculating source data end respective data blocks: the step of calling the calculating of data block digest value is calculated the digest value h of source data end i blocks of data piece;
3) digest value of calculating Backup Data end respective data blocks: the step of calling the calculating of data block digest value is calculated the digest value h ' of Backup Data end i blocks of data piece;
4) data block consistency detects: if h=h ', then current data block is in consistent state at the source data end with the Backup Data end, otherwise current data block is in inconsistent state at source data end and Backup Data end;
5) difference of next data block of cycle detection: i=i+1; If i is not more than the sum of data block. then change 2), otherwise detection of end work.
Claims (1)
1. an information system data consistency detection is characterized in that may further comprise the steps: the step of deblocking; The step that the data block digest value calculates; The step of variance data consistency detection; Wherein
(1) step of deblocking: adopt same method of partition that source data and Backup Data are carried out piecemeal, obtain the data block set separately of source data and Backup Data, data block in the set of source data data block is corresponding one by one with the data block in the set of Backup Data data block, and the step of concrete deblocking may further comprise the steps:
1) size of definition of data unit is divided into data the step of the data cell of fixed length size;
2) size of definition of data piece is stipulated the step of the number of data units that each data block comprises;
3) number of computational data piece calculates the step of number of data blocks according to the size of the size of data cell and data block;
(2) step of data block digest value calculating: the data block that data piecemeal step obtains is carried out digest value calculating, and the step that concrete data block digest value calculates may further comprise the steps:
1) the data block digest value calculates initialized step;
2) from first data cell of data block,, successively each data cell of data block is carried out following steps up to last data cell of data block:
1. calculate the step of the digest value of current data unit;
2. the step of current data block digest value is calculated in accumulation;
(3) step of variance data consistency detection: carry out data consistency according to the result of calculation of data block digest value and judge that the step of concrete variance data consistency detection may further comprise the steps:
1) the initialized step of variance data consistency detection;
2) from first data block of source data and backup end data, up to last data block of source data and backup end data, following steps are carried out in circulation:
1. calculate the step of the digest value of source data end data piece;
2. calculate the step of Backup Data end corresponding to the digest value of described source data data block;
3. the step that detects of data block consistency: the digest value of source data end data piece compares with the digest value of backup end corresponding to described source data end data piece, if both equate that then this data block is in consistent state, otherwise is in inconsistent state.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CNB2006100204574A CN100394393C (en) | 2006-03-10 | 2006-03-10 | Information system data consistency detection |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CNB2006100204574A CN100394393C (en) | 2006-03-10 | 2006-03-10 | Information system data consistency detection |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN1818878A CN1818878A (en) | 2006-08-16 |
| CN100394393C true CN100394393C (en) | 2008-06-11 |
Family
ID=36918900
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CNB2006100204574A Expired - Fee Related CN100394393C (en) | 2006-03-10 | 2006-03-10 | Information system data consistency detection |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN100394393C (en) |
Families Citing this family (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101009703B (en) * | 2007-02-07 | 2011-07-20 | 杭州华三通信技术有限公司 | A method and system for verifying the data synchronization |
| CN101452409B (en) * | 2007-12-04 | 2010-10-13 | 无锡江南计算技术研究所 | Data verification redundant method and device |
| CN101546319B (en) * | 2008-03-27 | 2012-08-15 | 北京兴宇中科科技开发股份有限公司 | Data difference analysis method |
| CN101840363B (en) * | 2009-11-10 | 2016-03-30 | 创新科存储技术有限公司 | A kind of file block comparative approach and device |
| CN102083069B (en) * | 2009-11-26 | 2015-09-16 | 中兴通讯股份有限公司 | A kind of method and apparatus detecting integrity of mobile terminal memory data |
| CN101783955B (en) * | 2010-03-24 | 2012-11-21 | 浙江宇视科技有限公司 | Data recovering method when data is abnormal and equipment thereof |
| CN102354292A (en) * | 2011-09-21 | 2012-02-15 | 国家计算机网络与信息安全管理中心 | Method and system for checking consistency of records in master and backup databases |
| CN103164523A (en) * | 2013-03-19 | 2013-06-19 | 华为技术有限公司 | Inspection method, device and system of data consistency inspection |
| CN105095300A (en) * | 2014-05-16 | 2015-11-25 | 阿里巴巴集团控股有限公司 | Method and system for database backup |
| CN105335196A (en) * | 2015-11-02 | 2016-02-17 | 深圳市新国都支付技术有限公司 | POS (Point Of Sale) terminal program incremental download method |
| CN107037978B (en) * | 2016-10-31 | 2019-11-05 | 福建亿榕信息技术有限公司 | Data Migration bearing calibration and system |
| CN109213431B (en) * | 2017-07-04 | 2022-05-13 | 阿里巴巴集团控股有限公司 | Consistency detection method and device for multi-copy data and electronic equipment |
| CN108897806A (en) * | 2018-06-15 | 2018-11-27 | 东软集团股份有限公司 | Comparison of data consistency method, apparatus, storage medium and electronic equipment |
| CN109639436A (en) * | 2019-01-04 | 2019-04-16 | 平安科技（深圳）有限公司 | The data property held verification method and terminal device based on salt figure |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5790773A (en) * | 1995-12-29 | 1998-08-04 | Symbios, Inc. | Method and apparatus for generating snapshot copies for data backup in a raid subsystem |
| US6081875A (en) * | 1997-05-19 | 2000-06-27 | Emc Corporation | Apparatus and method for backup of a disk storage system |
| US6101584A (en) * | 1996-11-05 | 2000-08-08 | Mitsubishi Denki Kabushiki Kaisha | Computer system and semiconductor device on one chip including a memory and central processing unit for making interlock access to the memory |
| WO2005103955A1 (en) * | 2004-03-31 | 2005-11-03 | Microsoft Corporation | System and method for a consistency check of a database backup |
-
2006
- 2006-03-10 CN CNB2006100204574A patent/CN100394393C/en not_active Expired - Fee Related
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5790773A (en) * | 1995-12-29 | 1998-08-04 | Symbios, Inc. | Method and apparatus for generating snapshot copies for data backup in a raid subsystem |
| US6101584A (en) * | 1996-11-05 | 2000-08-08 | Mitsubishi Denki Kabushiki Kaisha | Computer system and semiconductor device on one chip including a memory and central processing unit for making interlock access to the memory |
| US6081875A (en) * | 1997-05-19 | 2000-06-27 | Emc Corporation | Apparatus and method for backup of a disk storage system |
| WO2005103955A1 (en) * | 2004-03-31 | 2005-11-03 | Microsoft Corporation | System and method for a consistency check of a database backup |
Also Published As
| Publication number | Publication date |
|---|---|
| CN1818878A (en) | 2006-08-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN100394393C (en) | Information system data consistency detection | |
| CN108510006B (en) | Enterprise power consumption analysis and prediction method based on data mining | |
| CN107037978A (en) | Data Migration bearing calibration and system | |
| CN103336994B (en) | Transient stability evaluation in power system forecast failure collection rapid screening method | |
| CN108399268A (en) | A kind of increment type isomery figure clustering method based on game theory | |
| CN114202027A (en) | Execution configuration information generation method, model training method and device | |
| CN109740039A (en) | Dynamic network community structure recognition methods based on stack self-encoding encoder | |
| CN105138638A (en) | Database distribution method based on application layer | |
| CN103678138A (en) | Method and device for generating state conversion test samples | |
| CN106599049A (en) | Decision table data reduction method | |
| CN109582735A (en) | A kind of block chain blocks of knowledge storage system | |
| CN111612472A (en) | Block chain authoritative node authorization consensus algorithm | |
| CN105681425B (en) | Multinode restorative procedure and its system based on distributed memory system | |
| CN116301656A (en) | Data storage method, system and equipment based on log structure merging tree | |
| CN103064766A (en) | Fast data synchronization method in continuous data protection technology | |
| CN103207835A (en) | Mass data storage method through self-adaptive Range partitions | |
| CN104866687A (en) | Construction method of dynamic spatial space index for supporting STL data source | |
| CN103425787B (en) | The triangle gridding of a kind of asymptotic optimization repeats the quick minimizing technology in summit | |
| CN113821647B (en) | Construction method and system of knowledge graph in engineering machinery industry | |
| CN104182909A (en) | Multi-core parallel successive approximation method of hydropower system optimal scheduling | |
| CN106201655B (en) | Virtual machine distribution method and virtual machine distribution system | |
| CN106201784B (en) | A kind of data back up method and system based on git | |
| CN103699690A (en) | Accurate method for seeking minimal change region in process model | |
| CN100476880C (en) | Method and system for producing volume shade | |
| CN107395438B (en) | Network energy-saving method based on algebraic connectivity |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| ASS | Succession or assignment of patent right |
Owner name: SICHUAN GREAT TECHNOLOGY CO., LTD. Free format text: FORMER OWNER: SICHUAN UNIVERSITY Effective date: 20100513 |
|
| COR | Change of bibliographic data |
Free format text: CORRECT: ADDRESS; FROM: 610065 NO.24, SOUTH 1ST SECTION, 1ST RING ROAD, CHENGDU CITY, SICHUAN PROVINCE TO: 610041 2/F, COMPLEX BUILDING (FACING THE STREET), CHENGDU KEHUA MIDDLE ROAD PRIMARY SCHOOL, NO.2, CHANGSHOU ROAD, WUHOU DISTRICT, CHENGDU CITY |
|
| TR01 | Transfer of patent right |
Effective date of registration: 20100513 Address after: 610041, Wuhou District, Changshou Road, Chengdu No. 2 Chengdu KELONG Road Primary School Street building two floor Patentee after: Sichuan Gerite Technology Co., Ltd. Address before: 610065 Sichuan, Chengdu, South Ring Road, No. 1, No. 24 Patentee before: Sichuan University |
|
| ASS | Succession or assignment of patent right |
Owner name: CHENGDU GLOBAL CAPSHEAF TECHNOLOGY CO., LTD. Free format text: FORMER OWNER: SICHUAN GREAT TECHNOLOGY CO., LTD. Effective date: 20130402 |
|
| C41 | Transfer of patent application or patent right or utility model | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20130402 Address after: 610000 C-411 Asia Pacific Plaza, KELONG North Road, Sichuan, Chengdu Patentee after: Chengdu century summit Technology Co., Ltd. Address before: 610000, Sichuan, Wuhou District, Changshou Road, Chengdu No. 2 Chengdu KELONG Road Primary School Street building two floor Patentee before: Sichuan Gerite Technology Co., Ltd. |
|
| DD01 | Delivery of document by public notice |
Addressee: Wang Zhengtao Document name: Notification of Passing Examination on Formalities |
|
| CF01 | Termination of patent right due to non-payment of annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20080611 Termination date: 20180310 |