WO2008119269A1

WO2008119269A1 - Method and engine device for storing and looking up information

Info

Publication number: WO2008119269A1
Application number: PCT/CN2008/070143
Authority: WO
Inventors: Rui Hu; Shijun Shen; Meng Li; Zhenhua Xu
Original assignee: Huawei Technologies Co., Ltd.
Priority date: 2007-04-02
Filing date: 2008-01-18
Publication date: 2008-10-09
Also published as: CN101034412A; CN100470550C

Abstract

A method for storing information is disclosed, including: receive information to be stored, which has a key consisting of m bits (where m is a natural number); a first hash function is used to perform hash operation of the key of the information to be stored, so as to result in a first mapped information consisting of m bits, then n bits of information in the first mapped information is used as a first hash value (where n is a natural number, and n<m); when a storage unit of a first storage corresponding to the first hash value is free, the other (m-n) bits of information in the first mapped information is used as a first compressed key, and store the first compressed key and additional information of the information to be stored in the corresponding storage unit. A method for looking up information and an engine device are also disclosed.

Description

Information storage, search method and engine device

The present invention relates to the field of communications and database storage, and in particular, to a method for storing information, searching, and an engine device. Background technique

Nowadays, more and more systems need to quickly find the required information in a large amount of data information. For example, in the field of data communication, the core routers that need to find various forwarding tables belong to such a system. Using hardware to implement lookups is the most common choice.

Currently, there are generally two implementations for finding fixed-length information: a hash-based method and a CAM (Content Address Memory)-based method.

The hash-based search method mainly maps a long-length key (keyword) to a shorter-length space according to a certain mapping relationship (hash function), for example, using a hash function to 32bit. The (bit) IP (Internet Protocol) address is mapped to the 20-bit address space (1M). This method has a wide range of applications in fixed length lookups. When a certain information is added, the key of the information is hashed, that is, f (key)=h, where f () is a specific hash function, key is a keyword that needs to store information, and h is information. The stored hash address corresponding to the hth storage unit of the storage space, and the key value and the attached information corresponding to the key value are stored in the address. In this way, when searching, the hash value of the key value is obtained to obtain the corresponding hash address, and the corresponding subsidiary information can be obtained. However, when the two key values are hashed and the same hash value is obtained, that is, f (keyl ) = f (key2), the two pieces of information may need to be stored in an address, which creates a collision. The information corresponding to the key value forms a conflicting entry, which brings great difficulty to information storage and searching.

In order to solve the above conflict problem, there are several methods: for example, the linked list storage method, the multi-hash function method, and the CAM-based method. As shown in FIG. 1, it is a schematic diagram of a linked list storage method in the prior art. The method uses a linked list storage method to store conflicting entries in a hashed list. When the information is found, the received keyword to be searched is hashed to obtain a hash value for finding the hash table. From the hash table, a pointer to the linked list with the conflicting entry is obtained, and then the pointer is used to access the hashed list, and then the hashed list needs to be traversed to find the required information. For example, in FIG. 1, the hash table entry 0 The pointer points to a hash list, the length of the hash list is 1, indicating a conflict-free entry; and the table entry X pointer of the hash table points to a hash list, the length of the hash list is 3, indicating that There are 3 conflicting entries in the entry X. In the case of a bad case, in the case of the table item X in the figure, the search information requires multiple memory accesses, and high-speed search cannot be realized.

As shown in Fig. 2, it is a schematic diagram of the multi-hash function method used in the prior art. In the information storage process, the method uses multiple hash functions to hash the keywords to be searched, and then reads the information into the corresponding multiple hash tables for searching. When the information is stored, the first hash value is calculated using the first hash function to obtain the first hash value, if the position of the hash table corresponding to the first hash value (which may be referred to as the first hash table) is not The table item, that is, when there is no conflict, the corresponding information is stored in the position, and no other hash function is used for calculation; if there is a conflict in the position, the second hash function is used to recalculate the keyword. The second hash value. Similarly, if there is no conflict in the position in the hash table corresponding to the second hash value (which may be referred to as the second hash table), the corresponding information is stored in the location, and the information storage ends; Otherwise, you need to continue to use other hash functions for calculations until there is no conflict. In the information search, multiple hash functions are used to calculate the keywords, and at the same time, the corresponding information is obtained, and after comparing with the keywords to be searched, the corresponding information is found. In addition, the less the hash table needs to be stored, the less the table entry information needs to be stored. For example, the second hash table needs to store less entry information than the first hash table needs to store. Therefore, in order to improve the utilization of storage space, each hash table can be reduced step by step. This method uses multiple hash functions to offset the randomness of the information distribution, which can effectively reduce the probability of collisions. However, high memory bandwidth is required, which brings great difficulties to hardware design.

As shown in FIG. 3, it is a schematic diagram of a CAM-based search in the prior art, and a logic controller can Is a logic device or dedicated CPU (Central Processing Unit) provides the CAM with the keyword to be searched. The CAM searches through the hardware in parallel according to the keyword, outputs the memory address of the corresponding information, and then according to the address The attached external information is returned to the logic controller in the corresponding external memory. This allows the required information to be obtained with only one memory read operation.

Because of the rapidity of CAM lookups, CAM-based methods are used in a large number of existing communication devices, especially routers. However, due to process constraints, large-capacity CAM chips cannot be fabricated, and a large amount of data information, such as 32b i t information larger than 1M, cannot currently be stored. In addition, the cost and power consumption of the CAM chip are greatly increased as the capacity increases.

As can be seen from the various prior art solutions described above, there is an urgent need for a better performance information storage and information search scheme to realize storage of large database information and rapid search of information. Summary of the invention

Based on the foregoing analysis, an embodiment of the present invention provides a method for storing information, including: receiving information to be stored, where a keyword of the information to be stored is m bits (m is a natural number); using the first hash function to The keyword to be stored is hashed to obtain first mapping information, where the first mapping information is m bits, and the n-bit information in the first mapping information is used as the first hash value (n is a natural number, And 1 <111);

When the storage unit in the first memory corresponding to the first hash value is idle, the other (m − n ) bit information in the first mapping information is used as the first compression keyword, together with the to-be-stored The attached information of the information is stored together in a storage unit corresponding to the first hash value in the first memory.

The embodiment of the invention further provides an engine device, including:

a logic controller, configured to receive information to be stored, where the keyword of the information to be stored is m bits (m is a natural number), and the keyword of the information to be stored is hashed by using a first hash function to obtain a first a mapping information, the first mapping information is m bits, and n bits in the first mapping information Information as the first hash value (n is a natural number, and 1 <111);

a first memory, when the storage unit corresponding to the first hash value is idle, used to use other (m - n ) bit information in the first mapping information as the first compression keyword, together with the waiting The attached information storing the information is stored together in the storage unit corresponding to the first hash value.

In the meantime, an embodiment of the present invention provides a method for searching for information, including:

Receiving a keyword to be searched, the keyword is m bits (m is a natural number);

And performing the hash operation on the keyword by using the first hash function to obtain the first mapping information, where the first mapping information is m bits, and the n-bit information in the first mapping information is used as the first hash value. (n is a natural number, and n is less than m);

Searching the first memory according to the first hash value, acquiring keyword information in a storage unit corresponding to the first hash value, and comparing the keyword information with the keyword to determine the When the storage unit corresponding to the first hash value corresponds to the keyword, the information in the storage unit corresponding to the first hash value is acquired.

The embodiment of the invention further provides another method for searching for information, including:

Finding a content address memory according to the keyword;

The first hash information is obtained by hashing the key row by using a first hash function, where the first mapping information is m bits, and the n-bit information in the first mapping information is used as the first hash value. (n is a natural number, and n is less than m),

When the storage unit corresponding to the keyword is found in the content address storage, the information in the storage unit corresponding to the keyword is obtained;

When the storage unit corresponding to the keyword is not found in the content address storage, searching for the first memory according to the first hash value, acquiring a storage unit corresponding to the first hash value Keyword information, using the keyword information to compare with the keyword, and determining that the storage unit corresponding to the first hash value corresponds to the keyword, acquiring the first The information in the storage unit corresponding to the hash value. Correspondingly, an embodiment of the present invention provides an engine apparatus, including:

a first memory for storing information;

a logic controller, configured to receive a keyword to be searched, the keyword is m bits (m is a natural number), and the first hash function is used to perform hash operation on the keyword to be searched to obtain first mapping information. The first mapping information is m bits, and the n-bit information in the first mapping information is used as a first hash value (n is a natural number, and n is less than m); searching according to the first hash value. a memory, the keyword information in the storage unit corresponding to the first hash value is obtained, and the keyword information is compared with the keyword to determine the storage corresponding to the first hash value And when the unit corresponds to the keyword, acquiring information in the storage unit corresponding to the first hash value.

Another embodiment of the present invention provides an engine device, including:

a content address storage for storing information;

a first memory for storing information;

a logic controller, configured to receive a keyword to be searched, the keyword is m bits (m is a natural number), and the content address memory is searched according to the keyword, and the keyword is hashed by using a first hash function Computing, the first mapping information is obtained, the first mapping information is m bits, and the n-bit information in the first mapping information is used as the first hash value (n is a natural number, and n is less than m); Obtaining information in the storage unit corresponding to the keyword when the storage unit corresponding to the keyword is found in the content address memory; when the keyword is not found in the content address memory And searching for the first memory according to the first hash value, acquiring keyword information in the storage unit corresponding to the first hash value, using the keyword information and the keyword And comparing, when determining that the storage unit corresponding to the first hash value corresponds to the keyword, acquiring information in the storage unit corresponding to the first hash value.

The solution provided by the embodiment of the present invention does not store the complete keyword, but uses a keyword compression scheme, that is, the keyword is hashed, compressed into fewer bits, and then stored, so that the same bit width memory can be stored. The number of entries increases, which improves space utilization, reduces memory capacity requirements, and saves costs; it also reduces the probability of collisions. On the reverse side, in the phase With the same hardware configuration, with the solution of the present invention, the engine device can store more information, that is, increase the capacity of the engine device. In addition, the solution provided by the embodiment of the present invention can simplify the process of information searching and information storage, and is convenient for quick searching and information maintenance. The solution of the embodiment of the content address memory can solve the conflict problem caused by the hash operation, and the keyword compression scheme can reduce the capacity requirement of the content address memory and save costs. DRAWINGS

1 is a schematic diagram of a linked list storage method in the prior art;

2 is a schematic diagram of a multi-hash function method in the prior art;

3 is a schematic diagram of a CAM-based search in the prior art;

4 is a schematic structural view of an engine device in an embodiment of the present invention;

5 is a flow chart of information storage by an engine device in an embodiment of the present invention;

Figure 6 is a flow chart showing the information search by the engine device in one embodiment of the present invention. detailed description

The present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

An embodiment of the present invention provides an engine apparatus, including a logic controller and a first memory, where the logic controller is configured to receive information to be stored, where the key of the information to be stored is m bits (m is a natural number), The first hash information is hashed by using the first hash function to obtain the first mapping information, where the first mapping information is m bits, and the n-bit information in the first mapping information is used as the first hash value ( n is a natural number, and n is smaller than m); when the storage unit in the first memory corresponding to the first hash value is idle, the other (m - n ) bit information in the first mapping information is used as the first compression keyword The auxiliary information along with the information to be stored is stored in the storage unit corresponding to the first hash value.

In this embodiment, when performing information search, the logic controller receives the keyword to be searched, the keyword is m bits (m is a natural number), and hashes the keyword to be searched by using the first hash function. Computing, obtaining first mapping information, the first mapping information is m bits, and the n-bit information in the first mapping information is used as the first hash value (n is a natural number, and n is less than m); searching according to the first hash value The first memory obtains keyword information in a storage unit corresponding to the first hash value, and compares the keyword information with the keyword to be searched to determine a storage unit corresponding to the first hash value and the to-be-searched When the keyword is corresponding, the information in the storage unit corresponding to the first hash value is obtained. In the above process, the keyword information is compared with the keyword to be searched to determine whether the storage unit corresponding to the first hash value corresponds to the keyword to be searched, and there are multiple implementation manners, for example: The other (m - n ) bit information in the mapping information is used as the first compression keyword; comparing the keyword information with the first compression keyword, when the two are the same, determining the storage unit and the corresponding to the first hash value The keywords that are found correspond. For another example, the keyword information is combined with the first hash value to obtain a complete keyword, and compared with the keyword to be searched. When the two are the same, the storage unit corresponding to the first hash value is determined to be searched. The corresponding keyword.

The above logic controller may include: a first hash unit and a logic unit. When the information is stored, the first hash unit performs a hash operation on the keyword of the information to be stored, to obtain the first mapping information, where the first mapping information is m bits, and the n-bit information in the first mapping information is used as the first The logical unit receives the first hash value, determines whether the storage unit in the first memory corresponding to the first hash value is idle or occupied, and notifies the first when the storage unit corresponding to the first hash value is idle. The memory stores the first compressed keyword and the attached information of the information to be stored, wherein the first compressed keyword is other (m - n ) bit information in the first mapping information.

In another embodiment, the engine device may further include: a second memory, correspondingly, the logic controller further includes a second hash unit, when the storage unit in the first memory corresponding to the first hash value is occupied The second hash unit performs a hash operation on the keyword to be stored, to obtain second mapping information, where the second mapping information is m bits, and the t-bit information in the second mapping information is used as the second hash value (t Is a natural number, and t is less than m); the logic unit receives the second hash value, and determines whether the memory cell in the second memory corresponding to the second hash value is idle or occupied, when the memory cell corresponding to the second hash value When idle, the second memory is notified to store the second compressed keyword and the attached information to be stored. Generic information, wherein the second keyword is other (m - t) bit information in the second mapping information.

In still another embodiment, the engine apparatus further includes a content address memory for storing a key of the information to be stored and to be stored when a storage unit in the first memory corresponding to the first hash value is occupied The attached information of the information is stored in a free storage unit in the content address memory. In this embodiment, when performing information search, the logic controller receives the keyword to be searched, the keyword is m bits (m is a natural number), and the content address memory is searched according to the keyword; meanwhile, the first hash function is used to pair the keyword Performing a hash operation to obtain first mapping information, the first mapping information is m bits, and the n-bit information in the first mapping information is used as the first hash value (n is a natural number, and n is smaller than m). When the storage unit corresponding to the keyword is found in the content address storage, acquiring information in the storage unit corresponding to the keyword; when the storage unit corresponding to the keyword is not found in the content address storage, according to the first The hash value is searched for the first memory, and the keyword information in the storage unit corresponding to the first hash value is obtained, and the keyword information is compared with the keyword to be searched to determine the storage unit corresponding to the first hash value. When corresponding to the keyword to be searched, information in the storage unit corresponding to the first hash value is obtained. Similarly, as described above, using the keyword information to compare with the keyword to be searched, determining whether the storage unit corresponding to the first hash value corresponds to the keyword to be searched, there are multiple implementation manners, where No longer.

Of course, the solution in the above two embodiments may also be combined, that is, the engine device includes a content address memory, and further includes a second memory. Correspondingly, the logic controller further includes a second hash unit. In this way, when the storage unit in the first memory corresponding to the first hash value is occupied, the second hash unit performs a hash operation on the keyword of the information to be stored to obtain second mapping information, where the second mapping information is m bits, the t-bit information in the second mapping information is taken as the second hash value (t is a natural number, and t is less than m); the logic unit receives the second hash value, and determines the second corresponding to the second hash value The storage unit in the memory is idle or occupied. When the storage unit corresponding to the second hash value is idle, the second memory is notified to store the second compressed keyword and the auxiliary information of the information to be stored, where the second compressed keyword is Other (m - t ) bit information in the second mapping information; and when the memory cell in the second memory corresponding to the second hash value is occupied, the content address memory will be the key of the information to be stored The words and the attached information of the information to be stored are stored in an idle storage unit. The first memory and the second memory are not limited to the on-chip memory or the off-chip memory.

A more detailed description will be given below by way of example with reference to FIG. Of course, Figure 4 is only a specific example and is not intended to limit the invention. As shown in FIG. 4, it is a schematic structural diagram of an engine apparatus in an embodiment of the present invention, which comprises a logic controller 120, a CAM unit 123, and an off-chip memory 134.

The logic controller 120 is composed of a logic unit 122, a plurality of hash units 121a and 121b, and an on-chip memory 131. The CAM unit 123 includes a CAM chip 133 and a CAM memory 132. For example, the logic controller 120 can be implemented by using an ASIC (Application Specific Integrated IC) or an FPGA (Field Programmable Gatec Array); The external memory 134 can be implemented using a DRAM (Dynamic Random Accs s Memory); the CAM memory 132 can be an SRAM (Sta tic Random Acces s Memory).

In the engine device shown in Figure 4, the information lookup includes the following process:

The logic controller 120 receives the key (key) to be searched and distributes it to each hash unit and logic unit through the data bus 111;

After receiving the keyword to be searched, the logic unit sends the keyword to the CAM unit 123 for searching through the data bus 115. If the CAM unit 123 finds the information corresponding to the keyword, the CAM memory 132 returns the information to the logic unit 122. At this time, the logic unit 122 no longer uses the calculation result of each hash unit to access the corresponding memory; if the CAM unit 123 does not find the required information, the B' J logic unit 122 respectively according to the calculation result of each hash unit The corresponding keyword and the auxiliary information are read on the memory corresponding to each hash unit, and the information is returned to the logical unit 122 through the data bus 117, and the logical unit 122 compares the keyword in the information with the keyword to be searched, if If the keyword of the information is found to be the same as the keyword to be searched, the auxiliary information is obtained from the memory corresponding to the same keyword as the keyword to be searched and returned to the logical unit.

Give a very concrete example to describe the working process of the engine device: Suppose the keyword to be found It is 32 bits, the affiliate information is 20 bits, and the database to be checked contains 1M entry. Then, the data bus 1 11 is 32 bits, 112a is 20 bits, 114 is 20 bits; 112b has a number of bits less than 112a, and 1 12b can be set to 15 bits.

The hash table corresponding to the hash unit 121a is stored in the off-chip memory 134, the memory space of the off-chip memory is 1M, and the data bit width is 128 bits. Since the hash unit uses the hash function to map the keyword information of 32b it to the address space (1M) of 20b it, that is, the hash unit 121a hashes the 32-bit keyword information to obtain a 32-bit. The mapping information, the 20-bit information in the 32-bit mapping information is used as the first hash value, and the other 12-bit information is stored as the compression key in the off-chip memory 134. This process is called a keyword. compression. Thus, the keyword information in the off-chip memory 134, together with the first hash value, can be used to obtain complete keyword information. For example, the auxiliary information is 20 bits, and the 20-bit auxiliary information is stored. Then, an entry stored in the off-chip memory 134 in the memory only needs to store 12+20=32 bits of information, and a total of 32 bits is required. When the logic unit 122 retrieves the 32-bit information, it is only necessary to combine the hash function used to obtain the complete keyword, as compared with the key to be searched, as shown in Table 1.

Table 1 Table entry storage form after keyword compression

1 12 13 32

Compressed keyword

Thus, for an off-chip memory 1 34 having a bit width of 128 bits, one storage unit can store four entries, as shown in Table 2.

Table 2 Table storage structure in a storage unit with a bit width of 128 bits

1 32 33 64 65 96 97 128

The advantage of this is that a storage unit can store 4 conflicting entries, so that even if 4 conflicts occur, all the entries can be obtained in one read, and a fast search can be realized. Since the probability of occurrence of more than 4 collisions in a storage space is relatively small, the remaining unstorable rushes The number of items is relatively small, which is much less likely to be compared with the way of storing 4M entries using 4M 32-bit storage. In addition, by keyword compression, the size of information to be stored in an entry is reduced. The 1M entry can be stored using 128M space, and all key information is stored. The same storage mode requires 512M of storage space. , space utilization is much higher.

The above expression simply uses a hash function that uses a storage unit to store entries. Because the distribution of information is random, and a hash function is a fixed format, there may be a large number of conflicts in some information distribution situations. If these conflicting entries are stored in the CAM unit, the space of the CAM unit is required. Large, the cost will be 4 艮 high. To this end, the multi-hash function method can be used to offset the effects of randomness of information distribution. In order not to increase the number of off-chip memories, it is considered that the number of conflicting entries of the off-chip memory has been relatively small, so the on-chip memory 1 31 in the logic controller 120 is used to store the hash table corresponding to the hash unit 121 b. . For example, 32 Κ 32-bit on-chip memory 1 31 can be used to store a hash table, which is available for existing logic devices. Preferably, the hash function used by the hash unit 121 b is selected as far as possible from the hash function used by the hash unit 121 a to reduce the probability of collision occurring. This can effectively reduce the probability of collisions and effectively utilize the memory space. Of course, one off-chip memory and one on-chip memory shown in FIG. 4 are only an example. In the logic controller 120, there may be only one memory, not limited to on-chip or off-chip, or multiple memories. When there are multiple memories, there are multiple hash tables.

In some cases, a large amount of on-chip memory is required to implement storage of all conflicting entries, and on-chip memory resources are very limited. Therefore, for example, in the case where the number of conflicting entries is small, a CAM with a smaller capacity can be used. The unit stores the remaining conflicting entries. This effectively reduces the need for on-chip memory and resolves the problems that conflicts pose to lookups. In the engine unit shown in Fig. 4, IK X 32-bit CAM chip 1 33 and 1K χ 20-bit CAM memory 1 32 can be used to store keywords and ancillary information, respectively.

In addition, in the embodiment of the present invention, a method for information storage is provided, the process is as follows: receiving information to be stored, the keyword of the information to be stored is m bits (m is a natural number); using the first hash function to store information The keyword is hashed to get the first map Information, the first mapping information is m bits, and the n-bit information in the first mapping information is used as the first hash value (n is a natural number, and n is less than m);

When the storage unit in the first memory corresponding to the first hash value is idle, the other (m − n ) bit information in the first mapping information is stored as the first compression key together with the auxiliary information of the information to be stored. Within the first memory is in a memory location corresponding to the first hash value.

In another embodiment, when the storage unit corresponding to the first hash value in the first memory is occupied, the keyword of the information to be stored is hashed by using the second hash function, to obtain the second mapping information, The second mapping information is m bits, and the t bit information in the second mapping information is used as the second hash value (t is a natural number, and t is less than m); when the storage unit in the second memory corresponding to the second hash value When idle, the second compressed keyword and the auxiliary information of the information to be stored are stored in a storage unit corresponding to the second hash value in the second memory, where the second compressed keyword is the other of the second mapping information ( m - t ) bit information.

In still another embodiment, when a storage unit corresponding to the first hash value in the first memory is occupied, whether there is an idle storage unit in the content address memory; when there is an idle storage unit, the information to be stored The keywords and affiliate information are stored in the free storage unit. When the storage unit corresponding to the first hash value in the storage is occupied, the second hash function is used to hash the keyword to be stored, to obtain the second mapping information, and the second mapping information is m bits, The t-bit information in the second mapping information is used as the second hash value (t is a natural number, and t is less than m); when the storage unit in the second memory corresponding to the second hash value is idle, the second compressed keyword is And the auxiliary information of the information to be stored is stored in a storage unit corresponding to the second hash value in the second memory, where the second compressed keyword is other (m − t ) bit information in the second mapping information; When the storage unit in the second memory corresponding to the second hash value is occupied, whether there is an idle storage unit in the content address memory; when there is an idle storage unit, storing the keyword and the auxiliary information in the information to be stored In the free storage unit.

The following is a concrete example to describe the process of information storage. As shown in FIG. 5, it is the present invention. In one embodiment, the flow chart of the information storage by the engine device is combined with the structural diagram of the engine device shown in FIG. 4, and the information storage process is as follows:

First, starting from step 510, the stored item is less than 1M item;

In step 51 1 , the logic controller 120 receives the store instruction and the information to be stored, and then transmits it to the hash unit 121a through the data bus;

In step 512, the hash unit 121a performs an operation using the first hash function to obtain a first hash value and sends it to the logic unit 122 via the data bus 112a;

In step 51 3, the logic unit 122 correspondingly searches for the storage unit in the off-chip memory 134 according to the first hash value, and finds the storage unit corresponding to the first hash value, and takes Table 2 as an example, which is 4 tables. A total of 128 items;

In step 514, it is determined whether the storage unit corresponding to the first hash value has a free space (collision table entry <4). If there is free space, then in step 515, the logic unit 122 is based on the first hash of the hash unit 121a. The function compresses the keyword;

In step 516, the logic unit 122 puts the compressed keyword and the auxiliary information into the free space in the storage unit corresponding to the first hash value, stores it in the off-chip memory 134, and then proceeds to step 523 to wait for other instructions. . In step 514, if there is no free space (collision table entry > 4), then step 517 is entered, that is, the hash unit 121 b receives the storage instruction and the corresponding information, and the second hash value is calculated by the second hash function and sent. To the logic unit 122. In step 518, the logic unit 122 searches the on-chip memory 1 31 for the memory location (32 bits) corresponding to the second hash value according to the second hash value. In step 519, it is determined whether the storage unit has valid entry information. If not, that is, idle, then in step 520, the logic unit 122 compresses the keyword according to the second hash function of the hash unit 121b. In step 521, the logic unit 122 places the compressed keyword and the attached information into the storage unit, and then proceeds to step 523 to wait for other instructions. In step 519, if a conflict occurs, the logic unit stores the keyword information and the affiliate information in the CAM unit 123. At this point, the information storage process ends, and the flow proceeds to step 523 to wait for other instructions.

In the meantime, an embodiment of the present invention also provides a method for information searching, and the process is as follows: Receiving the keyword to be searched, the keyword is m bits (m is a natural number);

The first hash information is hashed by using the first hash function to obtain first mapping information, where the first mapping information is m bits, and the n-bit information in the first mapping information is used as the first hash value (n is a natural number, and n is less than m);

Searching the first memory according to the first hash value, acquiring keyword information in the storage unit corresponding to the first hash value, comparing the keyword information with the keyword to be searched, and determining, corresponding to the first hash value, When the storage unit corresponds to the keyword to be searched, the information in the storage unit corresponding to the first hash value is obtained.

As described above, the keyword information is compared with the keyword to be searched to determine whether the storage unit corresponding to the first hash value corresponds to the keyword to be searched. There are various solutions, and details are not described herein.

In another embodiment, the process of information lookup is slightly different, including:

Receiving the keyword to be searched, the keyword to be searched is m bits (m is a natural number);

Find the content address memory according to the keyword to be searched;

The first hash information is hashed by using the first hash function to obtain the first mapping information, the first mapping information is m bits, and the n-bit information in the first mapping information is used as the first hash value (n is a natural number) And n is less than m); when the storage unit corresponding to the keyword to be searched is found in the content address memory, the information in the storage unit corresponding to the keyword is obtained; when the search is not found in the content address memory Searching for the first storage unit according to the first hash value, obtaining keyword information in the storage unit corresponding to the first hash value, and comparing the keyword to be searched using the keyword information, When it is determined that the storage unit corresponding to the first hash value corresponds to the keyword to be searched, the information in the storage unit corresponding to the first hash value is obtained.

The following is a specific example to illustrate the specific process of information search. As shown in FIG. 6, it is a flowchart of the information search by the engine device in one embodiment of the present invention. Referring to the structural diagram of the engine device in FIG. 4, the information search process is as follows:

First, starting from step 610, at which time the engine device is in the waiting command phase; In step 611, the logic controller 120 receives the lookup instruction and the to-be-checked key and transmits them to the hash unit 121a, the hash unit 121b, and the CAM unit 123 via the data buses 111 and 115. In step 612, after the CAM unit 123 receives the to-be-checked keyword, the CAM chip 133 performs a parallel search according to the keyword. If the matching information is found, the address is output to the CAM memory 132, and the CAM memory 132 returns the auxiliary information; If the chip 133 does not find the matching information, it does not return any value to the logical unit; in steps 613 and 614, the two hash units perform a hash calculation outputting a hash value according to the to-be-checked keyword, and the hash value passes through the data bus 112a and 112b is output to the logic unit 122;

In step 615, the logic unit 122 collects the CAM search result and the operation result of the two hash units, wherein the CAM unit 123 searches for the result, that is, the return value has the highest priority. If the CAM unit 123 has a return value, the value is used as the return value. Ancillary information. If the CAM unit 123 has no return value, the hash unit is used to calculate the result to the corresponding memory to obtain the keyword information and the attached information. Each hash unit is a parallel lookup, and all hash calculations and memory accesses are done in parallel, with no priority.

In step 616, the logic unit 122 determines whether the CAM unit 123 has a return value. If so, indicating that the CAM unit finds the information to be checked, then in step 617, the CAM search result is used as the returned affiliate information. At this point, proceed to step 624, waiting for other instructions to arrive;

In step 616, if the CAM unit 123 has no return value, indicating that the CAM unit has not found the information to be checked, then in step 618, the result of the hash operation is used to obtain the keyword information and the attached information in the corresponding memory. At the same time, in step 619, logic unit 122 compresses the keywords based on the hash functions of all of the hash units. In step 620, logic unit 122 compares the computed compression key with the key information retrieved from each memory. In step 621, it is determined whether or not they are consistent. If they match, in step 622, the attached information stored in the storage unit in which the keyword matching the compressed keyword is stored is returned. At this point, the process proceeds to step 624, waiting for other commands to arrive.

In step 621, if it is inconsistent, it indicates that the keyword information to be searched for does not exist, then the "no match" information is returned in step 623, and then proceeds to step 624, waiting for other instructions to arrive, and the entire search process ends.

Corresponding software in the embodiments of the present invention may be stored in a computer readable storage medium. The solution in the embodiment of the present invention has been subjected to multiple simulation tests in a computer, and the verification operation is reliable and feasible. The solution provided by the embodiment of the present invention does not store the complete keyword, but uses a keyword compression scheme, that is, the keyword is hashed, compressed into fewer bits, and then stored, so that the same bit width memory can be stored. The number of entries increases, which improves space utilization, reduces memory capacity requirements, and saves costs; it also reduces the probability of collisions. On the contrary, in the case of the same hardware configuration, the engine device can store more information, that is, increase the capacity of the engine device. In addition, the solution provided by the embodiment of the present invention can simplify the process of information searching and information storage, and is convenient for quick searching and information maintenance. The solution of the embodiment of the content address memory can solve the conflict problem caused by the hash operation, and the keyword compression scheme can reduce the capacity requirement of the content address memory and save costs.

The above is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto, and any person skilled in the art can easily think of changes or replacements within the technical scope of the present invention. All should be covered by the scope of the present invention.

Claims

Claim

A method for storing information, characterized in that it comprises:

Receiving information to be stored, the keyword of the information to be stored is m bits (m is a natural number); performing a hash operation on the keyword of the information to be stored by using a first hash function, to obtain first mapping information, The first mapping information is m bits, and the n-bit information in the first mapping information is used as the first hash value (n is a natural number, and 1 <111);

2. The method of claim 1, further comprising:

When the storage unit in the first memory corresponding to the first hash value is occupied, the keyword of the information to be stored is hashed by using a second hash function to obtain second mapping information. The second mapping information is m bits, and the t-bit information in the second mapping information is used as a second hash value (t is a natural number, and Km);

When the storage unit in the second memory corresponding to the second hash value is idle, the other (m − t ) bit information in the second mapping information is used as the second compression keyword, together with the to-be-stored The attached information of the information is stored together in a storage unit corresponding to the second hash value in the second memory.

3. The method of claim 1, further comprising:

When the storage unit in the first memory corresponding to the first hash value is occupied, it is found whether there is a free storage unit in the content address memory;

When there is an empty storage unit in the content address memory, the keyword and the auxiliary information of the information to be stored are stored in the free storage unit.

4. The method of claim 2, further comprising: When the storage unit in the second memory corresponding to the second hash value is occupied, whether there is a free storage unit in the content address memory;

5. An engine device, comprising:

a logic controller, configured to receive information to be stored, where the keyword of the information to be stored is m bits (m is a natural number), and the keyword of the information to be stored is hashed by using a first hash function to obtain a first a mapping information, the first mapping information is m bits, and the n-bit information in the first mapping information is used as a first hash value (n is a natural number, and 1 <111);

The engine apparatus according to claim 5, wherein the logic controller comprises: a first hashing unit, configured to perform a hash operation on a keyword of the information to be stored, to obtain the first Mapping information, the first mapping information is m bits, and the n-bit information in the first mapping information is used as the first hash value;

a logic unit, configured to receive the first hash value, determine whether the storage unit corresponding to the first hash value is idle or occupied, when the storage unit corresponding to the first hash value is idle And notifying the first memory to store the first compressed keyword and the auxiliary information of the information to be stored.

7. The engine apparatus according to claim 6, further comprising a second memory; the logic controller further comprising a second hash unit, wherein the storage unit corresponding to the first hash value When being occupied, the second hashing unit is configured to perform a hash operation on the keyword of the information to be stored, to obtain second mapping information, where the second mapping information is m bits, and the second mapping information is used. The t-bit information in the second hash value (t is a natural number, and t is less than m);

The logic unit receives the second hash value, and determines the corresponding to the second hash value. The storage unit in the second memory is idle or occupied, and when the storage unit corresponding to the second hash value is idle, notifying the second memory to store the second compression keyword and the attachment of the information to be stored Information, the second keyword is other (m - t) bit information in the second mapping information.

8. The engine apparatus according to claim 5, further comprising a content address memory, said content address when said storage unit in said first memory corresponding to said first hash value is occupied The memory is configured to store the keyword of the information to be stored and the auxiliary information of the information to be stored in a storage unit that is free in the content address memory.

9. The engine apparatus according to claim 7, further comprising a content address memory, said content address when said storage unit in said second memory corresponding to said second hash value is occupied The memory is configured to store the keyword of the information to be stored and the auxiliary information of the information to be stored in a storage unit that is free in the content address memory.

10. A method for searching for information, characterized in that it comprises:

The method according to claim 10, wherein the comparing the keyword information with the keyword comprises:

Determining the other (m - n ) bit information in the first mapping information as a first compression keyword; comparing the keyword information with the first compression keyword, and when the two are the same, determining the location The storage unit corresponding to the first hash value corresponds to the keyword.

Combining the keyword information with the first hash value to obtain a complete keyword, comparing with the keyword, and when the two are the same, determining the storage unit corresponding to the first hash value Corresponding to the keyword.

1 3, a method for information searching, characterized in that:

Finding a content address memory according to the keyword;

When the storage unit corresponding to the keyword is not found in the content address storage, searching for the first memory according to the first hash value, acquiring a storage unit corresponding to the first hash value Keyword information, using the keyword information to compare with the keyword, and determining that the storage unit corresponding to the first hash value corresponds to the keyword, acquiring the first The information in the storage unit corresponding to the hash value.

The method according to claim 13, wherein the comparing the keyword information with the keyword comprises:

Combining the keyword information with the first hash value to obtain a complete keyword, The keys are compared, and when the two are the same, it is determined that the storage unit corresponding to the first hash value corresponds to the keyword.

16. An engine apparatus, comprising:

a first memory for storing information;

17. An engine apparatus, comprising:

a content address storage for storing information;

a first memory for storing information;