CN103188282A - Method, system and equipment for file synchronization - Google Patents
Method, system and equipment for file synchronization Download PDFInfo
- Publication number
- CN103188282A CN103188282A CN2011104441635A CN201110444163A CN103188282A CN 103188282 A CN103188282 A CN 103188282A CN 2011104441635 A CN2011104441635 A CN 2011104441635A CN 201110444163 A CN201110444163 A CN 201110444163A CN 103188282 A CN103188282 A CN 103188282A
- Authority
- CN
- China
- Prior art keywords
- data
- data source
- file
- transmission
- destination
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Abstract
The invention provides a method, a system and equipment for file synchronization. The method comprises the steps that a destination data source receiving unit receives file data transmitted by a source server, stores the file data, and generates a transmission log; a destination data source copying unit obtains a file name of the received file data from the transmission log, and copies the corresponding file data stored in the destination data source receiving unit for storage; and a data processing unit copies the file data from the destination data source copying unit for data processing. The method, the system and the equipment can avoid read-write conflict during the file synchronization.
Description
Technical field
The present invention relates to data processing technique, relate in particular to method, system and the equipment of file synchronization.
Background technology
Each source server need be transferred to destination server on a large scale with file data after producing file data, by destination server file data is focused on, and obtains the information that needs.Described file data can be specially the user to respectively showing the record of browsing of content on the webpage, and described displaying content for example is news information, sport information, advertising message on the webpage etc.
Source server is file data transmission destination server, and destination server process that the file data that writes is handled, is the process of file synchronization.Below this process is elaborated: destination server comprises destination data source receiving element and data processing unit; Source server is transferred to destination data source receiving element according to existing transmission mechanism with the file data that produces, the file data of destination data source receiving element reception sources server transmission, store file data, and generating the transmission daily record, this transmission daily record comprises the filename of the file data that writes and information such as write time point accordingly; Data processing unit reads file data from destination data source receiving element, and carry out data and handle, and the corresponding document data of storing in the receiving element of deletion destination data source.
For to file data treated in the receiving element of destination data source and also untreated file data distinguish, data processing unit is just deleted the corresponding document data of storing in the receiving element of destination data source after reading file data from destination data source receiving element.According to existing transmission mechanism, in data transmission procedure, source server compares the data of the data of storing in the receiving element of destination data source and source server self storage, judge and whether stored the All Files data in the source server in the receiving element of destination data source, if not, then the part of not storing in the receiving element of destination data source is transferred to destination data source receiving element.Because data processing unit carries out will deleting the corresponding document data of storing in the receiving element of destination data source after data are handled, this will cause source server repeatedly to write file data to destination data source receiving element, cause read/write conflict.
For to file data treated in the receiving element of destination data source and also untreated file data distinguish, also can adopt the mode of the treated file data of data processing unit being carried out real time record, but the workload of this mode is too big, and the data volume of record is very big, generally do not adopt, and the mode that data are handled the file data in the receiving element of deletion destination data source, back is carried out in more employing, but there is the defective of read/write conflict in this mode.
Summary of the invention
The invention provides a kind of method of file synchronization, the read/write conflict when this method can be avoided file synchronization.
The invention provides a kind of system of file synchronization, the read/write conflict when this system can avoid file synchronization.
The invention provides a kind of equipment of file synchronization, the read/write conflict when this equipment can be avoided file synchronization.
A kind of method of file synchronization, this method comprises:
The file data of destination data source receiving element reception sources server transmission, store file data, and generate the transmission daily record;
Destination data source copy cell is obtained the filename of the file data that has received from the transmission daily record, the corresponding document data of storing in the receiving element of copy destination data source are stored;
Data processing unit reads file data from destination data source copy cell, carries out data and handles.
A kind of system of file synchronization, this system comprises source server and destination server, described destination server comprises destination data source receiving element, destination data source copy cell and data processing unit;
Described source server is used for to described destination data source receiving element transfer files data;
Described destination data source receiving element is used for receiving the file data that described source server transmits, store file data, and generate the transmission daily record;
Described destination data source copy cell, for the filename that obtains the file data that has received from described transmission daily record, the corresponding document data of storing in the receiving element of copy destination data source are stored;
Described data processing unit is used for reading file data from destination data source copy cell, carries out data and handles.
A kind of equipment of file synchronization, this equipment comprises destination data Src StUnit, destination data source copy cell and processing unit;
Described destination data source receiving element is used for the file data that the reception sources server transmits, store file data, and generate the transmission daily record;
Described destination data source copy cell, for the filename that obtains the file data that has received from described transmission daily record, the corresponding document data of storing in the receiving element of copy destination data source are stored;
Described data processing unit is used for reading file data from destination data source copy cell, carries out data and handles.
From such scheme as can be seen, among the present invention, the file data that destination data source receiving element is received copies in the destination data data source copy cell, and data processing unit reads file data from destination data source copy cell again, handles.Like this, need not to read file data from destination data source receiving element and handle, just needn't delete the file data of destination data source receiving element storage, thereby, the read/write conflict when having avoided file synchronization.
Description of drawings
Fig. 1 is the system configuration schematic diagram of file synchronization of the present invention;
Fig. 2 is the method indicative flowchart of file synchronization of the present invention.
Embodiment
For making the purpose, technical solutions and advantages of the present invention clearer, below in conjunction with embodiment and accompanying drawing, the present invention is described in more detail.
Among the present invention, the file data that destination data source receiving element is received copies out, and the file data that copy is come out reads and handles again.Like this, need not to read file data from destination data source receiving element and handle, just needn't delete the file data of destination data source receiving element storage, thereby, the read/write conflict when having avoided file synchronization.
Referring to Fig. 1, be the system configuration schematic diagram of file synchronization of the present invention, this system comprises source server and destination server, described destination server comprises destination data source receiving element, destination data source copy cell and data processing unit;
Described source server is used for to described destination data source receiving element transfer files data;
Described destination data source receiving element is used for receiving the file data that described source server transmits, store file data, and generate the transmission daily record;
Described destination data source copy cell, for the filename that obtains the file data that has received from described transmission daily record, the corresponding document data of storing in the receiving element of copy destination data source are stored;
Described data processing unit is used for reading file data from destination data source copy cell, carries out data and handles.
For to file data treated in the copy cell of destination data source and also untreated file data distinguish, can comprise the deletion subelement in the described data processing unit, after reading data from destination data source copy cell, the corresponding document data of storing in the copy cell of deletion destination data source; Like this, being kept at the file data in the copy cell of destination data source, all is untreated file data also.
The transmission daily record is stored and generated to the file data of destination data source receiving element reception sources server transmission; File just in transmission course, end of transmission not also, because the data of this document are imperfect, will cause follow-up result mistake to occur if destination data source copy cell copies this document this moment; Therefore, the present invention arranges lock mechanism to destination data source receiving element and destination data source copy cell, to avoid this defective.Described lock mechanism arranges subelement by the sign of the lock in the receiving element of destination data source, and the inquiry subelement in the copy cell of destination data source and the realization of copy subelement; Particularly:
Described lock sign arranges subelement, is used for when generating the transmission daily record, judges whether end of transmission of file data, if, then lock to identify and be set to end of transmission, be set to transmit otherwise lock to identify;
Described inquiry subelement, be used for described lock sign is judged, if lock is designated end of transmission, then send enabled instruction to described copy subelement, if lock is designated transmit, wait until that then the lock sign is set to end of transmission and just sends enabled instruction to described copy subelement;
Described copy subelement is used for receiving described startup and specifies, and obtains the filename of the file data of end of transmission from the transmission daily record, and the corresponding document data of storing in the receiving element of copy destination data source are stored.
Described file data can be specially the user to respectively showing the record of browsing of content on the webpage, and described displaying content for example is news information, sport information, advertising message on the webpage etc.
Referring to Fig. 2, be the method indicative flowchart of file synchronization of the present invention, it may further comprise the steps:
Source server writes destination data source receiving element according to existing transmission mechanism with file data, destination data source receiving element receives and store file data, also generate the transmission daily record, this transmission daily record comprises the filename of the file data that writes and information such as write time point accordingly.
For to file data treated in the copy cell of destination data source and also untreated file data distinguish, data processing unit can be deleted the corresponding document data of storing in the copy cell of destination data source after reading file data from destination data source copy cell.
The transmission daily record is stored and generated to the file data of destination data source receiving element reception sources server transmission; File just in transmission course, end of transmission not also, because the data of this document are imperfect, will cause follow-up result mistake to occur if destination data source copy cell copies this document this moment; Therefore, the present invention arranges lock mechanism to destination data source receiving element and destination data source copy cell, to avoid this defective.Described lock mechanism specifically comprises:
In the flow process of Fig. 2, destination data source receiving element store file data and when generating the transmission daily record in the step 201 is judged whether end of transmission of file data, if, then lock to identify and be set to end of transmission, be set to transmit otherwise lock to identify; Correspondingly, before the execution in step 202, this method also comprises: destination data source copy cell is judged described lock sign, if lock is designated end of transmission, then carry out and describedly obtain the step of the filename of the file data of end of transmission from transmission the daily record, if for lock is designated and transmits, wait until that then the lock sign is set to end of transmission and just carries out and describedly obtain the step of the filename of the file data of end of transmission from the transmission daily record.
According to existing transmission mechanism, file data in the source server is write in the process of destination data source receiving element, source server compares the data of the data of storing in the receiving element of destination data source and source server self storage, judge and whether stored the All Files data in the source server in the receiving element of destination data source, if not, then the part of not storing in the receiving element of destination data source is transferred to destination data source receiving element.Among the present invention, adopt the file data that destination data source receiving element is received to copy out, the file data that copy is come out reads and handles again.Like this, need not to read file data from destination data source receiving element and handle, the file data that destination data source receiving element is received is not deleted the read/write conflict when having avoided file synchronization.
In the practical application, realize that the software of file synchronization scheme of the present invention can adopt Rsync, Rsync is file synchronization and the delivery means under a outstanding Unix system.
The above only is preferred embodiment of the present invention, and is in order to limit the present invention, within the spirit and principles in the present invention not all, any modification of making, is equal to replacement, improvement etc., all should be included within the scope of protection of the invention.
Claims (10)
1. the method for a file synchronization is characterized in that, this method comprises:
The file data of destination data source receiving element reception sources server transmission, store file data, and generate the transmission daily record;
Destination data source copy cell is obtained the filename of the file data that has received from the transmission daily record, the corresponding document data of storing in the receiving element of copy destination data source are stored;
Data processing unit reads file data from destination data source copy cell, carries out data and handles.
2. the method for claim 1 is characterized in that, after described data processing unit read file data from destination data source copy cell, this method also comprised:
The corresponding document data of storing in the copy cell of deletion destination data source.
3. the method for claim 1, it is characterized in that, when described destination data source receiving element generates the transmission daily record, this method also comprises: destination data source receiving element is judged whether end of transmission of file data, if, then lock sign and be set to end of transmission, be set to transmit otherwise lock sign;
Described destination data source copy cell is obtained before the filename of the file data that has received from the transmission daily record, and this method also comprises:
Destination data source copy cell is judged described lock sign, if lock is designated end of transmission, then carries out and describedly obtains the step of the filename of the file data of end of transmission from the transmission daily record; If for lock is designated and transmits, wait until that then the lock sign is set to end of transmission and just carries out and describedly obtain the step of the filename of the file data of end of transmission from the transmission daily record.
4. the method for claim 1 is characterized in that, described file data is that the user is to respectively showing the record of browsing of content on the webpage.
5. the system of a file synchronization is characterized in that, this system comprises source server and destination server, and described destination server comprises destination data source receiving element, destination data source copy cell and data processing unit;
Described source server is used for to described destination data source receiving element transfer files data;
Described destination data source receiving element is used for receiving the file data that described source server transmits, store file data, and generate the transmission daily record;
Described destination data source copy cell, for the filename that obtains the file data that has received from described transmission daily record, the corresponding document data of storing in the receiving element of copy destination data source are stored;
Described data processing unit is used for reading file data from destination data source copy cell, carries out data and handles.
6. system as claimed in claim 5 is characterized in that, described file data is that the user is to respectively showing the record of browsing of content on the webpage.
7. the equipment of a file synchronization is characterized in that, this equipment comprises destination data Src StUnit, destination data source copy cell and processing unit;
Described destination data source receiving element is used for the file data that the reception sources server transmits, store file data, and generate the transmission daily record;
Described destination data source copy cell, for the filename that obtains the file data that has received from described transmission daily record, the corresponding document data of storing in the receiving element of copy destination data source are stored;
Described data processing unit is used for reading file data from destination data source copy cell, carries out data and handles.
8. equipment as claimed in claim 7 is characterized in that, described data processing unit comprises the deletion subelement, after reading data from destination data source copy cell, and the corresponding document data of storing in the copy cell of deletion destination data source.
9. equipment as claimed in claim 7, it is characterized in that, described destination data source receiving element comprises that the lock sign arranges subelement, be used for when generating the transmission daily record, judge whether end of transmission of file data, if, then lock sign and be set to end of transmission, be set to transmit otherwise lock sign;
Destination data source copy cell comprises inquiry subelement and copy subelement;
Described inquiry subelement, be used for described lock sign is judged, if lock is designated end of transmission, then send enabled instruction to described copy subelement, if lock is designated transmit, wait until that then the lock sign is set to end of transmission and just sends enabled instruction to described copy subelement;
Described copy subelement is used for receiving described startup and specifies, and obtains the filename of the file data of end of transmission from the transmission daily record, and the corresponding document data of storing in the receiving element of copy destination data source are stored.
10. as claim 7,8 or 9 described equipment, it is characterized in that described file data is that the user is to respectively showing the record of browsing of content on the webpage.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201110444163.5A CN103188282B (en) | 2011-12-27 | 2011-12-27 | Method, system and the equipment of file synchronization |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201110444163.5A CN103188282B (en) | 2011-12-27 | 2011-12-27 | Method, system and the equipment of file synchronization |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN103188282A true CN103188282A (en) | 2013-07-03 |
| CN103188282B CN103188282B (en) | 2017-08-04 |
Family
ID=48679247
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201110444163.5A Active CN103188282B (en) | 2011-12-27 | 2011-12-27 | Method, system and the equipment of file synchronization |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103188282B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104794114A (en) * | 2014-01-16 | 2015-07-22 | 阿里巴巴集团控股有限公司 | Data processing method and device |
| CN110059111A (en) * | 2019-04-17 | 2019-07-26 | 珠海金山网络游戏科技有限公司 | A kind of data calling method, device, a kind of calculating equipment and storage medium |
| CN110309158A (en) * | 2018-04-13 | 2019-10-08 | 腾讯科技（深圳）有限公司 | Rolling abnormality judgment method, device and the readable medium of journal file |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5742792A (en) * | 1993-04-23 | 1998-04-21 | Emc Corporation | Remote data mirroring |
| WO2001001249A1 (en) * | 1999-06-30 | 2001-01-04 | Computer Sciences Corporation | System and method for synchronizing copies of data in a computer system |
| US20050131969A1 (en) * | 2002-10-01 | 2005-06-16 | Fujitsu Limited | Database duplicating method, database duplicating apparatus, database creating method, and database creating apparatus |
| EP1569119A2 (en) * | 1999-08-17 | 2005-08-31 | Emc Corporation | Remote mirroring system, device, and method |
| CN101640587A (en) * | 2008-08-01 | 2010-02-03 | 华为技术有限公司 | Data synchronization method and data synchronization device |
| CN101751307A (en) * | 2008-12-12 | 2010-06-23 | 比兹肯解决方法有限公司 | Data storage system, method as well as data storage and backup system |
| CN1998217B (en) * | 2004-05-28 | 2012-11-07 | 慕斯特有限公司 | System and method for replication, integration, consolidation and mobilization of data |
-
2011
- 2011-12-27 CN CN201110444163.5A patent/CN103188282B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5742792A (en) * | 1993-04-23 | 1998-04-21 | Emc Corporation | Remote data mirroring |
| WO2001001249A1 (en) * | 1999-06-30 | 2001-01-04 | Computer Sciences Corporation | System and method for synchronizing copies of data in a computer system |
| EP1569119A2 (en) * | 1999-08-17 | 2005-08-31 | Emc Corporation | Remote mirroring system, device, and method |
| US20050131969A1 (en) * | 2002-10-01 | 2005-06-16 | Fujitsu Limited | Database duplicating method, database duplicating apparatus, database creating method, and database creating apparatus |
| CN1998217B (en) * | 2004-05-28 | 2012-11-07 | 慕斯特有限公司 | System and method for replication, integration, consolidation and mobilization of data |
| CN101640587A (en) * | 2008-08-01 | 2010-02-03 | 华为技术有限公司 | Data synchronization method and data synchronization device |
| CN101751307A (en) * | 2008-12-12 | 2010-06-23 | 比兹肯解决方法有限公司 | Data storage system, method as well as data storage and backup system |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104794114A (en) * | 2014-01-16 | 2015-07-22 | 阿里巴巴集团控股有限公司 | Data processing method and device |
| CN104794114B (en) * | 2014-01-16 | 2018-04-03 | 阿里巴巴集团控股有限公司 | A kind of data processing method and device |
| CN110309158A (en) * | 2018-04-13 | 2019-10-08 | 腾讯科技（深圳）有限公司 | Rolling abnormality judgment method, device and the readable medium of journal file |
| CN110309158B (en) * | 2018-04-13 | 2023-08-01 | 腾讯科技（深圳）有限公司 | Method and device for judging rolling abnormality of log file and readable medium |
| CN110059111A (en) * | 2019-04-17 | 2019-07-26 | 珠海金山网络游戏科技有限公司 | A kind of data calling method, device, a kind of calculating equipment and storage medium |
| CN110059111B (en) * | 2019-04-17 | 2022-02-01 | 珠海金山网络游戏科技有限公司 | Data calling method and device, computing equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN103188282B (en) | 2017-08-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP6621543B2 (en) | Automatic update of hybrid applications | |
| US11074132B2 (en) | Post backup catalogs | |
| US8396938B2 (en) | Providing direct access to distributed managed content | |
| US10229007B2 (en) | Data synchronization of block-level backup | |
| US9864736B2 (en) | Information processing apparatus, control method, and recording medium | |
| US11442814B2 (en) | Variable number and size of put operations for storage backup | |
| CN107153644B (en) | Data synchronization method and device | |
| US9946609B2 (en) | Managing multi-level backups into the cloud | |
| JP2012089094A5 (en) | ||
| KR20100067976A (en) | Method for synchronizing contents files stored separately | |
| WO2017049761A1 (en) | Method, apparatus, system, device and storage medium for copying/pasting picture | |
| CN105141672A (en) | Method, device and system for data storage | |
| CN106462671A (en) | Composite document referenced resources | |
| CN103188282A (en) | Method, system and equipment for file synchronization | |
| JP6193406B2 (en) | Serialization for differential encoding | |
| US11138287B1 (en) | System for reducing size of transmitted content | |
| CN104639973A (en) | Information pushing method and device | |
| CN105653566B (en) | A kind of method and device for realizing database write access | |
| US20020087536A1 (en) | Methods for reproducing and recreating original data | |
| US20120005162A1 (en) | Managing Copies of Data Structures in File Systems | |
| JP7119324B2 (en) | Information processing device and information processing program | |
| US8082334B1 (en) | Providing direct access to managed content | |
| US20200293610A1 (en) | Document editing apparatus and non-transitory computer readable medium | |
| CN111581029A (en) | Remote backup method and system for hard link file | |
| JP7115179B2 (en) | History management device, history management program, and history management system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |