US9271229B2 - Methods, systems, and media for partial downloading in wireless distributed networks - Google Patents

Methods, systems, and media for partial downloading in wireless distributed networks Download PDF

Info

Publication number
US9271229B2
US9271229B2 US13/948,123 US201313948123A US9271229B2 US 9271229 B2 US9271229 B2 US 9271229B2 US 201313948123 A US201313948123 A US 201313948123A US 9271229 B2 US9271229 B2 US 9271229B2
Authority
US
United States
Prior art keywords
channel
symbols
storage node
symbol
channels
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active, expires
Application number
US13/948,123
Other versions
US20140022970A1 (en
Inventor
Chen Gong
Xiaodong Wang
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Columbia University of New York
Original Assignee
Columbia University of New York
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Columbia University of New York filed Critical Columbia University of New York
Priority to US13/948,123 priority Critical patent/US9271229B2/en
Publication of US20140022970A1 publication Critical patent/US20140022970A1/en
Assigned to THE TRUSTEES OF COLUMBIA UNIVERSITY IN THE CITY OF NEW YORK reassignment THE TRUSTEES OF COLUMBIA UNIVERSITY IN THE CITY OF NEW YORK ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: GONG, CHEN, WANG, XIAODONG
Application granted granted Critical
Publication of US9271229B2 publication Critical patent/US9271229B2/en
Assigned to NATIONAL SCIENCE FOUNDATION reassignment NATIONAL SCIENCE FOUNDATION CONFIRMATORY LICENSE (SEE DOCUMENT FOR DETAILS). Assignors: COLUMBIA UNIVERSITY
Active legal-status Critical Current
Adjusted expiration legal-status Critical

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04WWIRELESS COMMUNICATION NETWORKS
    • H04W52/00Power management, e.g. TPC [Transmission Power Control], power saving or power classes
    • H04W52/02Power saving arrangements
    • H04W52/0203Power saving arrangements in the radio access network or backbone network of wireless communication networks
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/10Protocols in which an application is distributed across nodes in the network
    • H04L67/1001Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
    • H04L67/1004Server selection for load balancing
    • H04L67/1023Server selection for load balancing based on a hash applied to IP addresses or costs
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04WWIRELESS COMMUNICATION NETWORKS
    • H04W28/00Network traffic management; Network resource management
    • H04W28/02Traffic management, e.g. flow control or congestion control
    • H04W28/08Load balancing or load distribution
    • H04W28/082Load balancing or load distribution among bearers or channels
    • H04W28/085
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02DCLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D30/00Reducing energy consumption in communication networks
    • Y02D30/70Reducing energy consumption in communication networks in wireless communication networks

Definitions

  • Distributed storage systems are generally used to store data in a distributed manner to provide reliable access to the stored data.
  • a data file of size M can be divided into k fragments, each of size M/k.
  • Each of the k fragments can be encoded and stored in a storage node of a distributed storage system.
  • the original data file can be recovered from a set of k encoded fragments.
  • conventional approaches to reconstructing data stored in a distributed storage network have limited performance, especially for wireless distributed storage networks.
  • such approaches generally include downloading all of the symbols from a subset of the storage nodes.
  • Such a full-downloading approach becomes inefficient in a wireless network, where wireless channels may not offer sufficient bandwidths for full downloading (e.g., due to channel fading).
  • full-downloading suffers from power constraints of the wireless network.
  • methods for selecting numbers of symbols to be transmitted on a plurality of channels comprising: for each of the plurality of channels, calculating using a hardware processor an increase in power that will be used by that channel if it transmits a symbol; selecting one of the plurality of channels with the smallest increase in power using the hardware processor; and allocating the symbol to the one of the plurality of channels using the hardware processor.
  • systems for selecting numbers of symbols to be transmitted on a plurality of channels comprising: at least one hardware processor that: for each of the plurality of channels, calculates an increase in power that will be used by that channel if it transmits a symbol; selects one of the plurality of channels with the smallest increase in power; and allocates the symbol to the one of the plurality of channels.
  • non-transitory computer-readable media containing computer-executable instructions that, when executed by a processor, cause the processor to perform a method for selecting numbers of symbols to be transmitted on a plurality of channels are provide, the method comprising: for each of the plurality of channels, calculating an increase in power that will be used by that channel if it transmits a symbol; selecting one of the plurality of channels with the smallest increase in power; and allocating the symbol to the one of the plurality of channels.
  • FIG. 1 is a generalized schematic diagram of an example of a distributed storage system in accordance with some embodiments of the disclosed subject matter.
  • FIG. 2 is a generalized schematic diagram of an example of a data collector and a storage node of FIG. 1 that can be used in accordance with some embodiments of the disclosed subject matter.
  • FIG. 3 is a flow chart of an example of a process for partial downloading in accordance with some embodiments of the disclosed subject matter.
  • FIGS. 4-6 are flow charts of an example of processes for determining whether a condition is sufficient for ⁇ -reconstructability for the minimum-storage regenerating (MSR) point in accordance with some embodiments of the disclosed subject matter.
  • FIG. 7 is a flow chart of an example of a recursive symbol selection process for a partial downloading scheme for the MSR point in accordance with some embodiments of the disclosed subject matter.
  • FIG. 8 is an illustrative example of data reconstruction for the coding scheme at the minimum-bandwidth regenerating (MBR) point in accordance with some embodiments of the disclosed subject matter.
  • FIG. 9 is a flow chart of an example of a process for a partial downloading scheme that includes performing a backward reconstruction in accordance with some embodiments of the disclosed subject matter.
  • FIG. 10 is a flow chart of an example of a symbol selection process for a partial downloading scheme for the MBR point in accordance with some embodiments of the disclosed subject matter.
  • FIG. 11A is a flow chart of an example of a process for minimizing power during reconstruction transmission in accordance with some embodiments of the disclosed subject matter.
  • FIG. 11B is a flow chart of an example of a process for adjusting a transmission allocation for the MSR point in accordance with some embodiments of the disclosed subject matter.
  • FIG. 11C is a flow chart of an example of a process for adjusting a transmission allocation for the MBR point in accordance with some embodiments of the disclosed subject matter.
  • FIG. 12 is a flow chart of an example of a process for minimizing power during regeneration transmission in accordance with some embodiments of the disclosed subject matter.
  • mechanisms for partial downloading in wireless distributed networks.
  • Such mechanisms can be used in a variety of applications.
  • the mechanisms can be used to retrieve and/or reconstruct data stored in a distributed storage system.
  • the mechanisms can be used to perform node regeneration in some embodiments in which one or more storage nodes in a distributed storage system fail.
  • the mechanisms can be implemented in a distributed storage network including a data collector and multiple storage nodes.
  • the data collector can communicate with one or more of the storage nodes through any suitable communication channels.
  • the storage nodes can be connected to the data collector through multiple orthogonal wireless channels.
  • the mechanisms can store data in the storage nodes in a distributed manner.
  • a data block containing M symbols can be stored in S storage nodes.
  • each of the S storage nodes can store ⁇ symbols that can be used to reconstruct the data block.
  • each of the ⁇ symbols can be generated from one or more of the M symbols based on a set of linear combination coefficients.
  • the data collector can reconstruct the data block from one or more of the storage nodes. For example, the data reconstruction can be performed based on a partial downloading scheme. In a more particular example, the data collector can download a suitable set of symbols from each storage node. In some embodiments, the data collector can determine the number of symbols to be downloaded from each channel and storage node based on suitable channel and power allocation schemes. In some embodiments, the data collector can also select the set of symbols to be downloaded based on one or more suitable symbol selection schemes. For example, the set of symbols can be selected from ⁇ symbols stored in a given storage node based on linear dependences among the symbols. As another example, the set of symbols can be selected based on a recursive algorithm. As another example, the data reconstruction can be performed based on a full downloading scheme. In a more particular example, the data collector can download ⁇ symbols for a subset of the storage nodes.
  • the mechanisms can regenerate the symbols stored in the failed node (e.g., ⁇ symbols) and create a new storage node.
  • the symbols can be regenerated by downloading a suitable number of symbols from one or more of the surviving storage nodes.
  • the new storage node can download ⁇ symbols from each of a set of d surviving storage nodes. In such an example, a total number of d ⁇ symbols can be downloaded from the surviving storage nodes for node regeneration.
  • these mechanisms can be implemented in a wireless cloud storage network, where a large number of users need to download data symbols with limited bandwidth.
  • a large amount of data that needs to be downloaded by a mobile device can be obtained by partially downloading data symbols from storage nodes for data reconstruction and node regeneration. This can be performed, for example, while conserving power and bandwidth of the wireless cloud storage network.
  • H (2) ] for ⁇ 1,2 ⁇ .
  • H 0 H can denote that H 0 is a submatrix of H by extracting columns of H.
  • H 0 ⁇ H can denote that H 0 H and H 0 ⁇ H.
  • H ⁇ H 0 can be the submatrix of H that includes all columns of H but not in H 0 .
  • span(H) can be the space spanned by the columns of H.
  • Q 0 Q can denote that Q 0 is a subspace of Q.
  • Q 0 ⁇ Q can denote that Q 0 Q and Q 0 ⁇ Q.
  • rank(H) can be the column rank of the matrix H that equals to the dimension of space span(H).
  • FIG. 1 a generalized schematic diagram of an example 100 of a distributed storage system in accordance with some embodiments of the disclosed subject matter is shown.
  • system 100 can include multiple storage nodes 110 , each of which is capable of storing a suitable amount of data.
  • system 100 can store a data block in S storage nodes 110 in a distributed manner.
  • each storage node 100 can store ⁇ symbols that are generated based on one or more of the M symbols.
  • each of the ⁇ symbols may be a packet of subsymbols in a field GF(q) and may contain B bits.
  • an encoding matrix can be defined as follows: H (i) [h 1j (i) ,h 2j (i) , . . . ,h Mj (i) ] ⁇ GF ( q ) M ⁇ .
  • system 100 can also include one or more data collectors 120 .
  • data collector 120 can perform data reconstruction by reconstructing the data stored in system 100 based on a suitable set of symbols stored in storage nodes 110 .
  • data collector 120 can download a set of symbols from one or more of storage nodes 110 .
  • Data collector 120 can then reconstruct the original data contained in the data block based on the encoding matrices associated with the storage nodes (e.g., ⁇ H (i) ⁇ i ⁇ S ).
  • data collector 120 can perform data reconstruction based on a partial downloading scheme by downloading a set of symbols from one or more of storage nodes 110 . More particularly, for example, data collector 120 can perform channel and power allocation and determine the number of symbols downloaded from each storage node 110 . Data collector 120 can then download a suitable number of symbols from each storage node 110 (e.g., downloading ⁇ i symbols from a storage node i).
  • data collector 120 can download all symbols required to perform data reconstruction from a set of storage nodes 110 . More particularly, for example, data collector 120 can download ⁇ symbols from each of K storage nodes 110 , wherein K is not greater than S. In such an example, a total number of K ⁇ symbols can be downloaded from K storage nodes 110 .
  • data collector 120 can be connected to each storage node 110 through one or more communication channels 130 that can include a command channel 132 , a data channel 134 , a feedback channel 136 , etc. Any suitable information can be transmitted through communication channels 130 to facilitate data reconstructions and/or node regeneration.
  • data collector 120 can communicate particular information with each storage node 110 through one or more communication channels.
  • a storage node 110 can transmit an encoding matrix associated with the storage node to data collector 120 through command channel 132 .
  • a storage node 110 can transmit one or more data symbols to data collector 120 through data channel 134 (e.g., an orthogonal frequency-division multiple access channel or OFDMA channel).
  • data collector 120 can transmit information about the number and/or the identities of the symbols to be downloaded from one or more storage nodes 110 through feedback channel 136 .
  • system 100 can regenerate the data stored in a failed storage node.
  • system 100 can regenerate the ⁇ symbols and create a new storage node.
  • the ⁇ symbols can be regenerated by downloading a suitable number of symbols from one or more of the surviving storage nodes.
  • data collector 120 can reconstruct the data stored in system 100 by downloading K ⁇ symbols from K storage nodes 110 as described above.
  • FIG. 2 a generalized schematic diagram of an example of a data collector and a storage node of FIG. 1 that can be used in accordance with some embodiments of the disclosed subject matter is shown.
  • storage node 110 can include one or more antennas 212 , a transceiver 214 , a hardware processor 216 , a storage device 218 , and/or any other suitable components.
  • transceiver 214 can transmit data symbols, linear combination coefficients, and/or other suitable information to data collector 120 through antennas 212 .
  • Transceiver 214 can also receive feedback signals containing information about one or more symbols to be downloaded from storage node 110 through antennas 212 .
  • transceiver 214 can pass the feedback signals to hardware processor 216 .
  • Hardware processor 216 can then process the feedback signals and identify the symbols to be transmitted to data collector.
  • storage node 110 can store suitable data in storage device 218 .
  • storage device 218 can store a data symbols, a set of linear combination coefficient (e.g., an encoding matrix) associated with the data symbols, and/or other suitable data
  • data collector 120 can include one or more antennas 222 , a transceiver 224 , a hardware processor 226 , a storage device 228 , and/or any other suitable components.
  • transceiver 224 can receive suitable data and/or commands transmitted from one or more storage nodes 110 through one or more antennas 222 . The data and/or commands can then be stored in storage device 228 and/or passed to hardware processor 226 .
  • Hardware processor 226 can perform channel estimation, wireless resource allocation, and/or other suitable functions based on the received data, commands, and/or other suitable information.
  • hardware processor 226 can generate one or more feedback signals containing information about the results of the channel estimation and/or wireless resource allocation.
  • the feedback signals can be transmitted to one or more storage nodes 110 through transceiver 224 and antenna(s) 222 .
  • storage node 110 and data collector 120 can be implemented in any suitable devices.
  • they can be implemented in mobile computers, mobile telephones, mobile access cards, wireless routers, wireless access points and/or any other suitable wireless devices.
  • each of transceivers 214 and 224 can include both a receiver and a transmitter in some embodiments.
  • each transceiver can include one or more multi-input multi-output (MIMO) transceivers where each includes multiple antennas (e.g., such as two transmit antennas and four receive antennas (some of which may also be transmit antennas)).
  • MIMO multi-input multi-output
  • each of hardware processors 216 and 226 can include any suitable hardware processor, such as a microprocessor, a micro-controller, digital signal processor, dedicated logic, and/or any other suitable circuitry.
  • each of storage devices 218 and 228 can include any suitable circuitry that is capable of storing data symbols, linear combination coefficients, feedback signals, computer readable instructions, etc.
  • each of storage devices 218 and 228 can include a hard drive, a solid state storage device, a removable storage device, etc.
  • storage node 110 and data collector 120 can include any other suitable components.
  • each of storage node 110 and data collector 120 can include a modulator, a demodulator, etc.
  • process 300 can be implemented in a data collector (e.g., data collector 120 as illustrated in FIGS. 1 and 2 ).
  • a data collector e.g., data collector 120 as illustrated in FIGS. 1 and 2 .
  • process 300 can start by receiving a set of linear combination coefficients and pilot symbols from at least one storage node at 302 .
  • the set of linear combination coefficients can contain any suitable information about one or more symbols stored in the storage node.
  • the combination coefficients can include one or more encoding matrices as defined in equation (4).
  • a storage node can transmit an encoding matrix H (i) that is associated with the storage node through a command channel. Additionally, the storage node can transmit one or more pilot symbols to the data collector through a data channel.
  • process 300 can perform a channel estimation on one or more communication channels.
  • the data collector can estimate one or more channel gains of a communication channel that connects a particular storage node to the data collector.
  • channel gains ⁇ g j (i) can be estimated for the storage node, where g j (i) denotes the complex gain of channel j from storage node i (i ⁇ S) to the data collector.
  • the channel gains can be estimated based on any suitable models.
  • the channel gains between the data collector and a particular storage node can be modeled by a complex Gaussian random variable (0, d ⁇ 2 ), where d represents the distance between the data collector and the storage node.
  • process 300 can perform a wireless resource allocation based on the results of the channel estimation.
  • the data collector can estimate the number of symbols that can be transmitted from a particular storage node to the data collector over a particular communication channel based on the estimated channels gains.
  • each of the wireless channels can have a suitable bandwidth (e.g., a bandwidth W) and a suitable duration (e.g., a duration T).
  • the number of symbols that can be transmitted from storage node i (i ⁇ S) to the data collector over channel j (j ⁇ ) can be estimated as follows:
  • c ⁇ ( ⁇ g j ( i ) ⁇ 2 ⁇ P j ) WT B ⁇ log 2 ⁇ ( 1 + k ⁇ ⁇ g j ( i ) ⁇ ⁇ P j ⁇ 2 ) , ( 6 )
  • P j denotes the transmission power of channel j
  • ⁇ 2 is the power of background noise
  • W denotes the bandwidth of channel j
  • T denotes the duration of channel j
  • k ⁇ 1 accounts for the rate loss due to the practical modulation and coding, compared with the ideal case of Gaussian signaling and infinite-length code.
  • the transition power of a communication channel can be a function of the amount of information transmitted through the communication channel.
  • data can be transmitted over a channel in unit of a symbol.
  • one or multiple symbols can be transmitted over the channel.
  • wireless resources e.g., power, bandwidth, etc.
  • process 300 can transmit the results of the wireless resource allocation to one or more of the storage nodes.
  • the results of the wireless resource allocation can be transmitted in any suitable manner.
  • the data collector can generate a feedback signal containing information about the number of the symbols and/or the identities of the symbols to be downloaded from a particular storage node.
  • the data collector can then transmit the feedback signal to the particular storage node through a suitable communication channel (e.g., a feedback channel that connects the data collector to the particular storage node).
  • process 300 can receive a set of symbols transmitted from at least one storage node.
  • a storage node can transmit a set of symbols to the data collector based on the feedback signal transmitted from the data collector.
  • the storage node in response to receiving the feedback signal, can identify the symbols chosen by the data collector by wireless resource allocation and transmit the identified symbols to the data collector through a suitable communication channel (e.g., a data channel that connects the data collector to the storage node).
  • a portion of the symbols stored in each storage node can be downloaded.
  • a data collector can use a partial downloading scheme that downloads a portion of the symbols from any suitable storage node.
  • the data collector e.g., data collector 120 as illustrated in FIGS. 1 and 2
  • the data collector can download symbols in a roughly even manner from the storage nodes such that the total number of downloaded symbols is equal to the number of data symbols to be reconstructed.
  • the reconstructability of the original data can be considered when performing such a partial downloading scheme.
  • ⁇ i be the number of symbols downloaded from storage node i. Since ⁇ i ⁇ , it can be assumed that the downloaded symbols are linear combinations of the symbols in node i given by s T H (i) A (i) , where A (i) is an ⁇ i matrix.
  • the matrix A (i) can be a full column rank, since otherwise at least one downloaded symbol can be expressed as a linear combination of other symbols downloaded from the same storage node, which means this symbol is redundant and should be removed to reduce the downloading bandwidth.
  • the search for the linear combination matrices ⁇ A (i) ⁇ i ⁇ S that satisfy the above-mentioned equation (7) may be computationally prohibitive. While it may be simpler to directly download the stored symbols from each storage node without performing such a linear combination, it can be determined whether there is a loss in optimality. That is, for some symbols ⁇ i ⁇ i ⁇ S , it can determined whether the above-mentioned equation (7) can be satisfied by performing linear combination but cannot be satisfied by simply downloading the stored symbols without linear combination. It has been determined that, in terms of the number of symbols downloaded from the storage nodes, downloading the symbols stored in the storage nodes directly and downloading their linear combinations are equivalent in some embodiments. This can be represented as follows:
  • the data can be ⁇ -reconstructable if it can be reconstructed via downloading ⁇ i symbols from storage node i for i ⁇ S, which is equivalent to that there exist M ⁇ i submatrices H (i) H (i) , i ⁇ S, such that the matrix H S [ H (i) ] i ⁇ S is of rank M.
  • a portion of the symbols in a storage node for data reconstruction can be downloaded at the minimum-storage regenerating (MSR) point or at the minimum-bandwidth regenerating (MBR) point. This can be performed, for example, by the data collector.
  • ⁇ i ⁇ i ⁇ S can be determined.
  • a partial downloading scheme given a set ⁇ i ⁇ i ⁇ S that satisfied this condition can be provided.
  • the data can be reconstructed by downloading a portion of the symbol from any suitable storage nodes at the MSR point.
  • the data collector e.g., data collector 120 as illustrated in FIGS. 1 and 2
  • the data collector can use a partial downloading scheme such that the number of downloaded symbols should be no less than the total number of data symbols to be reconstructed. This can be represented by: ⁇ i ⁇ S ⁇ i ⁇ M (8)
  • the data s can be reconstructed from the M downloaded symbols s T [H (i) ] i ⁇ S , the square matrix [H (i) ] i ⁇ R is of rank M.
  • the columns of matrix [H (i) ] i ⁇ R are linearly independent.
  • ⁇ i ⁇ S ⁇ i ⁇ M which states that the number of downloaded symbols should be no less than the total number of data symbols to be reconstructed, can also be a sufficient condition for ⁇ -reconstructability for the MSR point.
  • FIG. 4 shows an illustrative example of a process for determining whether a condition is sufficient for ⁇ -reconstructability for the MSR point in accordance with some embodiments of the disclosed subject matter. It should be noted that, in process 400 of FIG. 4 and any other process or method described herein, some steps can be added, some steps can be omitted, the order of the steps can be re-arranged, and/or some steps can be performed simultaneously.
  • Process 400 continues to select symbols that are linearly independent of the selected symbols until M symbols have been selected.
  • Process 400 begins at 410 by selecting a storage node i.
  • process 400 determines whether the selected storage node is a feasible storage node.
  • a storage node i can be considered feasible if the number of downloaded symbols from it is smaller than ⁇ i .
  • process 400 can determine whether all of the symbols in the feasible storage node are linearly independent of the selected symbols at 430 .
  • a symbol from the feasible storage node can be selected and written as a linear combination of the selected symbols at 440 . Based on the linear combination, a selected symbol can be replaced with another symbol from the same storage node such that the symbol from that feasible storage node can be selected to further increase the rank of H S at 450 .
  • Process 400 can determine whether M symbols have been selected at 460 . If M symbols have not been selected, process 400 can return to 410 and can continue to select symbols that are linearly independent of the selected symbols from one or more storage nodes. Upon selecting M symbols, process 400 can turn to process 500 of FIG. 5 .
  • process 500 begins at 510 by initializing H (i) as null matrices for i ⁇ S.
  • ⁇ i be the number of columns of H (i) (the number of symbols already selected from storage node i).
  • process 500 can turn to process 600 of FIG. 6 .
  • Process 600 begins by randomly selecting some i 0 ⁇ V and a column
  • h j 0 ( i 0 ) ⁇ i ⁇ W ⁇ ⁇ j ⁇ ?? ⁇ ⁇ j ( i ) ⁇ h j ( i ) , ⁇ ⁇ j ( i ) ⁇ 0 ( 9 )
  • i 1, 2, . . . , ⁇ , for each i ⁇ , is the set of column indices of H (i) that participate in the linear combination representation of h j 0 (i 0 ) shown above in (9).
  • process 600 determines that a column
  • process 600 removes
  • a symbol selection process for a partial downloading scheme at the MSR point can be provided.
  • FIG. 7 shows an illustrative example of a recursive symbol selection process for a partial downloading scheme at the MSR point in accordance with some embodiments of the disclosed subject matter.
  • process 700 can determine whether h is linearly independent of the columns of H S .
  • process 700 can add H to H S and, accordingly, update G S .
  • process 700 can randomly select i 0 ⁇ V and a column
  • h j 0 ( i 0 ) can be represented as:
  • h j 0 ( i 0 ) ⁇ i ⁇ W ⁇ ⁇ j ⁇ ?? ⁇ ⁇ j ( i ) ⁇ h j ( i ) , ⁇ ⁇ j ( i ) ⁇ 0.
  • process 700 can search for a column
  • process 700 only needs to update the Gaussian eliminated matrix G S for adding
  • the data can be reconstructed by downloading a portion of the symbols from any suitable storage nodes at the MBR point.
  • the data collector e.g., data collector 120 as illustrated in FIGS. 1 and 2
  • the data collector can use a partial downloading scheme that downloads a portion of the symbols from any suitable storage nodes at the MBR point.
  • the M symbols can be represented using the following d ⁇ d symmetric matrix:
  • B [ B ( 1 ) B ( 2 ) ( B ( 2 ) ) T 0 ]
  • B (1) is a K ⁇ K symmetric matrix storing
  • K ⁇ ( K + 1 ) 2 symbols and B (2) is a K ⁇ (d ⁇ K) matrix storing K(d ⁇ K) symbols.
  • the matrix B is premultiplied by an S ⁇ d Vandermonde matrix given by ⁇ , and each node i ⁇ S stores the d symbols corresponding to the i th row of ⁇ i B, where ⁇ i is the i th row of ⁇ .
  • a partial downloading scheme for reconstructing data at the MBR point based on the above coding scheme is provided.
  • the matrix B (1) is symmetric, the data symbols in B (2) and in the upper triangular part of B (1) can be decoded.
  • the data symbol can be divided into d columns ⁇ b j ⁇ 1 ⁇ j ⁇ d , where b j for 1 ⁇ j ⁇ K are in the upper triangular part of B (1) , and b j for K+1 ⁇ j ⁇ d are the columns of B (2) .
  • the data can be reconstructed in the backward order of b d , b d-1 , . . . , b 1 .
  • a partial downloading scheme that performs a backward reconstruction process 900 is provided in accordance with some embodiments of the disclosed subject matter.
  • process 900 can begin by reconstructing B (2) of matrix B.
  • the data collector can download the symbols ( ⁇ i B) j for i belonging to a subset S, here the size
  • ⁇ K ⁇ j . Since ⁇ is a Vandermonde matrix, b j can be reconstructed with K downloaded symbols.
  • process 900 can then reconstruct B (1) of matrix B. This can be done, for example, by reconstructing b j for 1 ⁇ j ⁇ K in the order of b k , b k-1 , . . . , b 1 .
  • B (1) is symmetric
  • the part of B (1) shown in area 802 is known from the previous reconstruction.
  • B (2) has been reconstructed in 910 and thus is known, reconstructing b j amounts to downloading the symbols ( ⁇ i B) j for i ⁇ j S, where the size
  • ⁇ j ⁇ j .
  • is a Vandermonde matrix
  • a sufficient condition in terms of ⁇ i ⁇ i ⁇ S is provided for the two above-mentioned equations to hold.
  • the sufficiency of the condition can be tested via an illustrative partial downloading scheme shown in FIG. 10 . More particularly, given ⁇ i ⁇ i ⁇ S satisfying the two above-mentioned equations.
  • process 1000 can begin by initializing at 1010 .
  • process 1000 can then determine
  • process 1000 can include ranking ⁇ j ⁇ 1 ⁇ j ⁇ d in decreasing order ⁇ j 1 ⁇ j 2 ⁇ . . . ⁇ j d .
  • process 1000 can then let
  • ⁇ j p ( i k ) 1 for 1 ⁇ p ⁇ i k and letting
  • process 1000 can subtract
  • the data collector can determine the number of symbols to be downloaded from each channel and storage node based on suitable channel and power allocation schemes.
  • P j be the transmission power for channel j.
  • a minimum total power used during such a reconstruction can be represented in some embodiments as:
  • each channel if used, may be restricted to being used only to transmit symbols from one node.
  • the data collector can run a channel and power allocation process to determine the number of symbols to be downloaded from each storage node. Any suitable channel and power allocation process can be used in some embodiments.
  • each channel j ⁇ N can then be assigned to a node i as follows:
  • ; and ⁇ j (i) 0, otherwise.
  • a greedy algorithm can be used to find this minimum power.
  • the minimum power can be found as follows:
  • process 1100 can be implemented in a data collector (e.g., data collector 120 as illustrated in FIGS. 1 and 2 ).
  • a data collector e.g., data collector 120 as illustrated in FIGS. 1 and 2 .
  • process 1100 can initialize the number of symbols for each channel to zero at 1104 .
  • Process 1100 can then select the channel with the minimum increase in power at 1108 .
  • process 1100 can allocate a symbol to be transmitted on the selected channel.
  • process 1100 can determine whether all symbols have been allocated. If not, process 1100 can loop back to 1106 . Otherwise, at 1114 , process 1100 can provide the total numbers of symbols to be transmitted on each channel to a suitable process that provides this data to the storage nodes via a feedback channel.
  • an adjustment can be performed by identifying each storage node i with ⁇ i > ⁇ , and reassigning symbols to be downloaded on one of the storage nodes channels to another storage node to decrease ⁇ i , until ⁇ i ⁇ for all i ⁇ S.
  • This adjustment can be performed in any suitable manner in some embodiments. For example, in some embodiments, the adjustment can be performed as follows:
  • process 1101 for performing an adjustment when the data collector is operating at the MSR point is illustrated.
  • process 1101 can be implemented in a data collector (e.g., data collector 120 as illustrated in FIGS. 1 and 2 ).
  • process 1101 may be a continuation of process 1100 of FIG. 11A .
  • process 1101 can be skipped or omitted.
  • process 1101 After process 1101 begins, the process can set an index i equal to one at 1114 .
  • the process can determine if the number of symbols to be downloaded from node i is greater than the number of symbols stored at that node. If so, process 1101 can then calculate the power increase to each other node that would result from reassigning the symbols on a channel assigned to that node to that channel on another node at 1118 .
  • This calculation can be made in any suitable manner. For example, in some embodiments, the calculation can be made using: p j (i′) (X j ) ⁇ p j (i) (X j ).
  • process 1101 can select as node i the node that (1) will have the minimum power increase if the symbols to be transmitted on that channel are reassigned to that node to and (2) that will not have more symbols to be transmitted from that node that are present in that node after reassignment.
  • process 1101 can the reassign the symbols to the channel on the new node i.
  • process 1101 can branch to 1124 at which i can be incremented. Then, at 1126 , process 1101 can determine whether i is greater than M, the number of symbols to be downloaded. If not, process 1101 can loop back to 1116 . Otherwise, process 1101 can complete.
  • the following constraint can be applied against ⁇ i ⁇ i ⁇ S :
  • this constraint can require that ⁇ (1) ⁇ d, ⁇ (1) + ⁇ (2) ⁇ 2d ⁇ 1, ⁇ (1) + ⁇ (2) + ⁇ (3) ⁇ 3d ⁇ 3, etc.
  • an adjustment can be performed as follows:
  • process 1102 can be implemented in a data collector (e.g., data collector 120 as illustrated in FIGS. 1 and 2 ). As shown, process 1102 may be a continuation of process 1101 of FIG. 11B . In some embodiments, and in some instances, such as when the data collector is not operating at the MBR point, process 1102 can be skipped or omitted.
  • a data collector e.g., data collector 120 as illustrated in FIGS. 1 and 2
  • process 1102 may be a continuation of process 1101 of FIG. 11B .
  • process 1102 can be skipped or omitted.
  • process 1102 can select as i the storage node with the maximum ⁇ i at 1128 .
  • process 1102 can select as i′ the storage node with the minimum ⁇ i .
  • the process can reassign one symbol to be downloaded from storage node i so that the symbol will be downloaded from storage node i′.
  • process 1102 can recalculate ⁇ i ⁇ i ⁇ S as described above.
  • the process can determine if the constraint is still violated. If so, the process can loop back to 1128 . Otherwise, the process can end.
  • the data collector can regenerate data of a failed storage node by downloading d data symbols from other storage nodes. These downloaded data symbols can then be loaded onto a new storage node.
  • the wireless resource allocation for both the MSR and MBR points can be similarly formulated as described above, i.e., to minimize the power consumption of downloading d symbols from d storage nodes.
  • the minimum power can be found as follows:
  • process 1200 can be implemented in a data collector (e.g., data collector 120 as illustrated in FIGS. 1 and 2 ).
  • a data collector e.g., data collector 120 as illustrated in FIGS. 1 and 2 .
  • process 1200 can initialize the number of symbols for each channel to zero at 1204 .
  • process 1200 can, for each unused channel, calculate the power required for transmitting a symbol on that channel. This calculation can be performed in any suitable manner.
  • Process 1200 can then select the channel with the minimum power at 1208 .
  • process 1100 can allocate a symbol to be transmitted on the selected channel.
  • process 1100 can determine whether all symbols have been allocated. If not, process 1200 can loop back to 1206 . Otherwise, at 1214 , process 1200 can provide the total numbers of symbols to be transmitted on each channel to any suitable process that needs this data.
  • a reassignment method similar to that described above in connection with FIG. 11B can be used to reassign one or more of the allocated symbols for regeneration.
  • processes of FIGS. 3 , 5 - 7 , 9 , 10 , 11 A, 11 B, 11 C, and 12 can be performed concurrently in some embodiments. It should also be noted that the above steps of the flow diagrams of FIGS. 3 , 5 - 7 , 9 , 10 , 11 A, 11 B, 11 C, and 12 may be executed or performed in any order or sequence not limited to the order and sequence shown and described in the figures. Furthermore, it should be noted, some of the above steps of the flow diagrams of FIGS. 3 , 5 - 7 , 9 , 10 , 11 A, 11 B, 11 C, and 12 may be executed or performed substantially simultaneously where appropriate or in parallel to reduce latency and processing times. And still furthermore, it should be noted, some of the above steps of the flow diagrams of FIGS. 3 , 5 - 7 , 9 , 10 , 11 A, 11 B, 11 C, and 12 may be omitted.
  • any suitable computer readable media can be used for storing instructions for performing the processes described herein.
  • computer readable media can be transitory or non-transitory.
  • non-transitory computer readable media can include media such as magnetic media (such as hard disks, floppy disks, etc.), optical media (such as compact discs, digital video discs, Blu-ray discs, etc.), semiconductor media (such as flash memory, electrically programmable read only memory (EPROM), electrically erasable programmable read only memory (EEPROM), etc.), any suitable media that is not fleeting or devoid of any semblance of permanence during transmission, and/or any suitable tangible media.
  • transitory computer readable media can include signals on networks, in wires, conductors, optical fibers, circuits, any suitable media that is fleeting and devoid of any semblance of permanence during transmission, and/or any suitable intangible media.

Abstract

Methods, systems, and media for partial downloading in wireless distributed networks are provided. In some embodiments, methods for selecting numbers of symbols to be transmitted on a plurality of channels are provided, the methods comprising: for each of the plurality of channels, calculating using a hardware processor an increase in power that will be used by that channel if it transmits a symbol; selecting one of the plurality of channels with the smallest increase in power using the hardware processor; and allocating the symbol to the one of the plurality of channels using the hardware processor.

Description

CROSS REFERENCE TO RELATED APPLICATION
This application claims the benefit of U.S. Provisional Patent Application No. 61/674,264, filed Jul. 20, 2012, which is hereby incorporated by reference herein in its entirety.
STATEMENT REGARDING GOVERNMENT FUNDED RESEARCH
This invention was made with government support under Grant No. CCF-0726480 awarded by the National Science Foundation (NSF) and Grant No. N00014-08-1-0318 awarded by the Office of Naval Research (ONR). The government has certain rights in the invention.
TECHNICAL FIELD
Methods, systems, and media for partial downloading in wireless distributed networks are provided.
BACKGROUND
Distributed storage systems are generally used to store data in a distributed manner to provide reliable access to the stored data. For example, a data file of size M can be divided into k fragments, each of size M/k. Each of the k fragments can be encoded and stored in a storage node of a distributed storage system. In such an example, the original data file can be recovered from a set of k encoded fragments. However, conventional approaches to reconstructing data stored in a distributed storage network have limited performance, especially for wireless distributed storage networks. For example, such approaches generally include downloading all of the symbols from a subset of the storage nodes. Such a full-downloading approach becomes inefficient in a wireless network, where wireless channels may not offer sufficient bandwidths for full downloading (e.g., due to channel fading). Moreover, full-downloading suffers from power constraints of the wireless network.
SUMMARY
In accordance with some embodiments of the disclosed subject matter, methods, systems, and media for partial downloading in wireless distributed networks are provided.
In some embodiments, methods for selecting numbers of symbols to be transmitted on a plurality of channels are provided, the methods comprising: for each of the plurality of channels, calculating using a hardware processor an increase in power that will be used by that channel if it transmits a symbol; selecting one of the plurality of channels with the smallest increase in power using the hardware processor; and allocating the symbol to the one of the plurality of channels using the hardware processor.
In some embodiments, systems for selecting numbers of symbols to be transmitted on a plurality of channels are provided, the systems comprising: at least one hardware processor that: for each of the plurality of channels, calculates an increase in power that will be used by that channel if it transmits a symbol; selects one of the plurality of channels with the smallest increase in power; and allocates the symbol to the one of the plurality of channels.
In some embodiments, non-transitory computer-readable media containing computer-executable instructions that, when executed by a processor, cause the processor to perform a method for selecting numbers of symbols to be transmitted on a plurality of channels are provide, the method comprising: for each of the plurality of channels, calculating an increase in power that will be used by that channel if it transmits a symbol; selecting one of the plurality of channels with the smallest increase in power; and allocating the symbol to the one of the plurality of channels.
BRIEF DESCRIPTION OF THE DRAWINGS
Various objects, features, and advantages of the disclosed subject matter can be more fully appreciated with reference to the following detailed description of the disclosed subject matter when considered in connection with the following drawings, in which like reference numerals identify like elements.
FIG. 1 is a generalized schematic diagram of an example of a distributed storage system in accordance with some embodiments of the disclosed subject matter.
FIG. 2 is a generalized schematic diagram of an example of a data collector and a storage node of FIG. 1 that can be used in accordance with some embodiments of the disclosed subject matter.
FIG. 3 is a flow chart of an example of a process for partial downloading in accordance with some embodiments of the disclosed subject matter.
FIGS. 4-6 are flow charts of an example of processes for determining whether a condition is sufficient for μ-reconstructability for the minimum-storage regenerating (MSR) point in accordance with some embodiments of the disclosed subject matter.
FIG. 7 is a flow chart of an example of a recursive symbol selection process for a partial downloading scheme for the MSR point in accordance with some embodiments of the disclosed subject matter.
FIG. 8 is an illustrative example of data reconstruction for the coding scheme at the minimum-bandwidth regenerating (MBR) point in accordance with some embodiments of the disclosed subject matter.
FIG. 9 is a flow chart of an example of a process for a partial downloading scheme that includes performing a backward reconstruction in accordance with some embodiments of the disclosed subject matter.
FIG. 10 is a flow chart of an example of a symbol selection process for a partial downloading scheme for the MBR point in accordance with some embodiments of the disclosed subject matter.
FIG. 11A is a flow chart of an example of a process for minimizing power during reconstruction transmission in accordance with some embodiments of the disclosed subject matter.
FIG. 11B is a flow chart of an example of a process for adjusting a transmission allocation for the MSR point in accordance with some embodiments of the disclosed subject matter.
FIG. 11C is a flow chart of an example of a process for adjusting a transmission allocation for the MBR point in accordance with some embodiments of the disclosed subject matter.
FIG. 12 is a flow chart of an example of a process for minimizing power during regeneration transmission in accordance with some embodiments of the disclosed subject matter.
DETAILED DESCRIPTION
In accordance with various embodiments, as described in more detail below, mechanisms (e.g., systems, methods, media, etc.) for partial downloading in wireless distributed networks are provided. Such mechanisms can be used in a variety of applications. For example, the mechanisms can be used to retrieve and/or reconstruct data stored in a distributed storage system. As another example, the mechanisms can be used to perform node regeneration in some embodiments in which one or more storage nodes in a distributed storage system fail.
In some embodiments, the mechanisms can be implemented in a distributed storage network including a data collector and multiple storage nodes. The data collector can communicate with one or more of the storage nodes through any suitable communication channels. For example, in some embodiments, the storage nodes can be connected to the data collector through multiple orthogonal wireless channels.
In some embodiments, the mechanisms can store data in the storage nodes in a distributed manner. For example, a data block containing M symbols can be stored in S storage nodes. In a more particular example, each of the S storage nodes can store α symbols that can be used to reconstruct the data block. In some embodiments, each of the α symbols can be generated from one or more of the M symbols based on a set of linear combination coefficients.
In some embodiments, the data collector can reconstruct the data block from one or more of the storage nodes. For example, the data reconstruction can be performed based on a partial downloading scheme. In a more particular example, the data collector can download a suitable set of symbols from each storage node. In some embodiments, the data collector can determine the number of symbols to be downloaded from each channel and storage node based on suitable channel and power allocation schemes. In some embodiments, the data collector can also select the set of symbols to be downloaded based on one or more suitable symbol selection schemes. For example, the set of symbols can be selected from α symbols stored in a given storage node based on linear dependences among the symbols. As another example, the set of symbols can be selected based on a recursive algorithm. As another example, the data reconstruction can be performed based on a full downloading scheme. In a more particular example, the data collector can download α symbols for a subset of the storage nodes.
In some embodiments, when a storage node fails or leaves the distributed storage system, the mechanisms can regenerate the symbols stored in the failed node (e.g., α symbols) and create a new storage node. For example, the symbols can be regenerated by downloading a suitable number of symbols from one or more of the surviving storage nodes. In some embodiments, for example, the new storage node can download β symbols from each of a set of d surviving storage nodes. In such an example, a total number of dβ symbols can be downloaded from the surviving storage nodes for node regeneration.
In some embodiments, these mechanisms can be implemented in a wireless cloud storage network, where a large number of users need to download data symbols with limited bandwidth. In such a network, a large amount of data that needs to be downloaded by a mobile device can be obtained by partially downloading data symbols from storage nodes for data reconstruction and node regeneration. This can be performed, for example, while conserving power and bandwidth of the wireless cloud storage network.
It should be noted that, as used herein, for a family of matrices {H(i)
Figure US09271229-20160223-P00001
with the same number of rows,
Figure US09271229-20160223-P00002
Figure US09271229-20160223-P00003
[H(i), iε
Figure US09271229-20160223-P00004
] can be the matrix obtained by horizontally concatenating H(i) for iε
Figure US09271229-20160223-P00004
, e.g.,
Figure US09271229-20160223-P00005
=[H(1)|H(2)] for
Figure US09271229-20160223-P00004
={1,2}. In some embodiments, H0
Figure US09271229-20160223-P00006
H can denote that H0 is a submatrix of H by extracting columns of H. In some embodiments, H0⊂H can denote that H0
Figure US09271229-20160223-P00006
H and H0≠H. In some embodiments, for H0
Figure US09271229-20160223-P00006
H, H\H0 can be the submatrix of H that includes all columns of H but not in H0. In some embodiments, span(H) can be the space spanned by the columns of H. In some embodiments, for two spaces Q0 and Q, Q0
Figure US09271229-20160223-P00006
Q can denote that Q0 is a subspace of Q. In some embodiments, Q0⊂Q can denote that Q0
Figure US09271229-20160223-P00006
Q and Q0≠Q. In some embodiments, if H0
Figure US09271229-20160223-P00006
H, span(H0)
Figure US09271229-20160223-P00006
span(H). In some embodiments, rank(H) can be the column rank of the matrix H that equals to the dimension of space span(H).
Turning to FIG. 1, a generalized schematic diagram of an example 100 of a distributed storage system in accordance with some embodiments of the disclosed subject matter is shown.
As illustrated, system 100 can include multiple storage nodes 110, each of which is capable of storing a suitable amount of data. For example, system 100 can store a data block in S storage nodes 110 in a distributed manner. In a more particular example, the data block can contain M symbols that can be denoted as:
s=[s 1 ,s 2 , . . . ,s M]T  (1)
In such an example, each storage node 100 can store α symbols that are generated based on one or more of the M symbols. In some embodiments, each of the α symbols may be a packet of subsymbols in a field GF(q) and may contain B bits. More particularly, for example, each storage node 110 (e.g., storage node i) can store a linear combination of data symbols that can be denoted as:
m (i) =[m 1 (i) , . . . ,m α (i)],  (2)
where i denotes the index of the storage nodes. In some embodiments, each of the symbols given in Equation 2 can be obtained based on a set of the M symbols as follows:
m j (i)k=1 M h kj (i) s k =s T h j (i),1≦j≦α,  (3)
where the coefficients hj (i)=[h1j (i), h2j (i), . . . , hMj (i)]TεGF(q)M, for 1≦j≦α. In some embodiments, an encoding matrix can be defined as follows:
H (i)
Figure US09271229-20160223-P00003
[h1j (i) ,h 2j (i) , . . . ,h Mj (i) ]εGF(q)M×α.  (4)
In some embodiments, equation (2) can be converted into the following format based on equations (3) and (4):
(m (i))T =s T H (i) for iεS.  (5)
As shown in FIG. 1, system 100 can also include one or more data collectors 120. In some embodiments, data collector 120 can perform data reconstruction by reconstructing the data stored in system 100 based on a suitable set of symbols stored in storage nodes 110. For example, to reconstruct the data block defined in equation (1), data collector 120 can download a set of symbols from one or more of storage nodes 110. Data collector 120 can then reconstruct the original data contained in the data block based on the encoding matrices associated with the storage nodes (e.g., {H(i)}iεS).
In a more particular example, data collector 120 can perform data reconstruction based on a partial downloading scheme by downloading a set of symbols from one or more of storage nodes 110. More particularly, for example, data collector 120 can perform channel and power allocation and determine the number of symbols downloaded from each storage node 110. Data collector 120 can then download a suitable number of symbols from each storage node 110 (e.g., downloading μi symbols from a storage node i).
In another more particular example, data collector 120 can download all symbols required to perform data reconstruction from a set of storage nodes 110. More particularly, for example, data collector 120 can download α symbols from each of K storage nodes 110, wherein K is not greater than S. In such an example, a total number of Kα symbols can be downloaded from K storage nodes 110.
In some embodiments, data collector 120 can be connected to each storage node 110 through one or more communication channels 130 that can include a command channel 132, a data channel 134, a feedback channel 136, etc. Any suitable information can be transmitted through communication channels 130 to facilitate data reconstructions and/or node regeneration. For example, in some embodiments, data collector 120 can communicate particular information with each storage node 110 through one or more communication channels. In a more particular example, a storage node 110 can transmit an encoding matrix associated with the storage node to data collector 120 through command channel 132. In another more particular example, a storage node 110 can transmit one or more data symbols to data collector 120 through data channel 134 (e.g., an orthogonal frequency-division multiple access channel or OFDMA channel). In yet another more particular example, data collector 120 can transmit information about the number and/or the identities of the symbols to be downloaded from one or more storage nodes 110 through feedback channel 136.
In some embodiments, system 100 can regenerate the data stored in a failed storage node. For example, in some embodiments in which a storage node 110 that stores a symbols fails or leaves system 100, system 100 can regenerate the α symbols and create a new storage node. In a more particular example, the α symbols can be regenerated by downloading a suitable number of symbols from one or more of the surviving storage nodes. In some embodiments, for example, the new storage node can download β symbols from each of a set of d surviving storage nodes. In such an example, a total number of γ=dβ symbols can be downloaded from the surviving storage nodes for node regeneration. In some embodiments, upon the creation of the new storage node, data collector 120 can reconstruct the data stored in system 100 by downloading Kα symbols from K storage nodes 110 as described above.
In some embodiments, the amount of symbols downloaded for data reconstruction (e.g., Kα) and the amount of symbols downloaded for node regeneration (e.g. γ=dβ) can be described using an optimal tradeoff curve. For example, the optimal tradeoff curve may have a minimum-storage regenerating (MSR) point corresponding to coding schemes with the best efficiency for data reconstruction (e.g., Kα=M). As another example, the optimal tradeoff curve may have a minimum-bandwidth regenerating (MBR) point corresponding to coding schemes with the best efficiency for node regeneration (e.g., dβ=α).
Turning to FIG. 2, a generalized schematic diagram of an example of a data collector and a storage node of FIG. 1 that can be used in accordance with some embodiments of the disclosed subject matter is shown.
As illustrated, storage node 110 can include one or more antennas 212, a transceiver 214, a hardware processor 216, a storage device 218, and/or any other suitable components. In some embodiments, transceiver 214 can transmit data symbols, linear combination coefficients, and/or other suitable information to data collector 120 through antennas 212. Transceiver 214 can also receive feedback signals containing information about one or more symbols to be downloaded from storage node 110 through antennas 212. In some embodiments, transceiver 214 can pass the feedback signals to hardware processor 216. Hardware processor 216 can then process the feedback signals and identify the symbols to be transmitted to data collector. In some embodiments, storage node 110 can store suitable data in storage device 218. For example, storage device 218 can store a data symbols, a set of linear combination coefficient (e.g., an encoding matrix) associated with the data symbols, and/or other suitable data
As shown, data collector 120 can include one or more antennas 222, a transceiver 224, a hardware processor 226, a storage device 228, and/or any other suitable components. In some embodiments, transceiver 224 can receive suitable data and/or commands transmitted from one or more storage nodes 110 through one or more antennas 222. The data and/or commands can then be stored in storage device 228 and/or passed to hardware processor 226. Hardware processor 226 can perform channel estimation, wireless resource allocation, and/or other suitable functions based on the received data, commands, and/or other suitable information. In some embodiments, hardware processor 226 can generate one or more feedback signals containing information about the results of the channel estimation and/or wireless resource allocation. In some embodiments, the feedback signals can be transmitted to one or more storage nodes 110 through transceiver 224 and antenna(s) 222.
In some embodiments, storage node 110 and data collector 120 can be implemented in any suitable devices. For example, they can be implemented in mobile computers, mobile telephones, mobile access cards, wireless routers, wireless access points and/or any other suitable wireless devices.
In some embodiments, each of transceivers 214 and 224 can include both a receiver and a transmitter in some embodiments. In some embodiments, each transceiver can include one or more multi-input multi-output (MIMO) transceivers where each includes multiple antennas (e.g., such as two transmit antennas and four receive antennas (some of which may also be transmit antennas)).
In some embodiments, each of hardware processors 216 and 226 can include any suitable hardware processor, such as a microprocessor, a micro-controller, digital signal processor, dedicated logic, and/or any other suitable circuitry.
In some embodiments, each of storage devices 218 and 228 can include any suitable circuitry that is capable of storing data symbols, linear combination coefficients, feedback signals, computer readable instructions, etc. For example, each of storage devices 218 and 228 can include a hard drive, a solid state storage device, a removable storage device, etc.
It should be noted that storage node 110 and data collector 120 can include any other suitable components. For example, in some embodiments, each of storage node 110 and data collector 120 can include a modulator, a demodulator, etc.
Turning to FIG. 3, a flow chart of an example 300 of a process for partial downloading for wireless distributed storage networks in accordance with some embodiments of the disclosed subject matter is shown. In some embodiments, process 300 can be implemented in a data collector (e.g., data collector 120 as illustrated in FIGS. 1 and 2).
As illustrated, process 300 can start by receiving a set of linear combination coefficients and pilot symbols from at least one storage node at 302. The set of linear combination coefficients can contain any suitable information about one or more symbols stored in the storage node. For example, the combination coefficients can include one or more encoding matrices as defined in equation (4). In a more particular example, a storage node can transmit an encoding matrix H(i) that is associated with the storage node through a command channel. Additionally, the storage node can transmit one or more pilot symbols to the data collector through a data channel.
Next, at 304, process 300 can perform a channel estimation on one or more communication channels. For example, the data collector can estimate one or more channel gains of a communication channel that connects a particular storage node to the data collector. In a more particular example, channel gains {gj (i)
Figure US09271229-20160223-P00007
can be estimated for the storage node, where gj (i) denotes the complex gain of channel j from storage node i (iεS) to the data collector. In some embodiments, the channel gains can be estimated based on any suitable models. For example, the channel gains between the data collector and a particular storage node can be modeled by a complex Gaussian random variable
Figure US09271229-20160223-P00008
(0, d−2), where d represents the distance between the data collector and the storage node.
At 306, process 300 can perform a wireless resource allocation based on the results of the channel estimation. For example, the data collector can estimate the number of symbols that can be transmitted from a particular storage node to the data collector over a particular communication channel based on the estimated channels gains. In a more particular example, the data collector can be connected to S storage nodes via N orthogonal wireless channels
Figure US09271229-20160223-P00009
={1, 2, . . . , N}. In some embodiments, each of the wireless channels can have a suitable bandwidth (e.g., a bandwidth W) and a suitable duration (e.g., a duration T). In such an example, the number of symbols that can be transmitted from storage node i (iεS) to the data collector over channel j (jε
Figure US09271229-20160223-P00009
) can be estimated as follows:
c ( g j ( i ) 2 P j ) = WT B log 2 ( 1 + k g j ( i ) P j σ 2 ) , ( 6 )
where Pj denotes the transmission power of channel j; σ2 is the power of background noise; W denotes the bandwidth of channel j; T denotes the duration of channel j; and k<1 accounts for the rate loss due to the practical modulation and coding, compared with the ideal case of Gaussian signaling and infinite-length code. In some embodiments, the transition power of a communication channel can be a function of the amount of information transmitted through the communication channel.
In some embodiments, data can be transmitted over a channel in unit of a symbol. For example, one or multiple symbols can be transmitted over the channel. In such an example, wireless resources (e.g., power, bandwidth, etc.) can be allocated to minimize the total transmission power of N channels while achieving successful data reconstruction.
At 308, process 300 can transmit the results of the wireless resource allocation to one or more of the storage nodes. In some embodiments, the results of the wireless resource allocation can be transmitted in any suitable manner. For example, the data collector can generate a feedback signal containing information about the number of the symbols and/or the identities of the symbols to be downloaded from a particular storage node. The data collector can then transmit the feedback signal to the particular storage node through a suitable communication channel (e.g., a feedback channel that connects the data collector to the particular storage node).
At 310, process 300 can receive a set of symbols transmitted from at least one storage node. For example, a storage node can transmit a set of symbols to the data collector based on the feedback signal transmitted from the data collector. In a more particular example, in response to receiving the feedback signal, the storage node can identify the symbols chosen by the data collector by wireless resource allocation and transmit the identified symbols to the data collector through a suitable communication channel (e.g., a data channel that connects the data collector to the storage node).
As described herein, a portion of the symbols stored in each storage node can be downloaded. For example, a data collector can use a partial downloading scheme that downloads a portion of the symbols from any suitable storage node. In a more particular example, for storage nodes and a data collector linked by wireless channels, the data collector (e.g., data collector 120 as illustrated in FIGS. 1 and 2) can download symbols in a roughly even manner from the storage nodes such that the total number of downloaded symbols is equal to the number of data symbols to be reconstructed.
It should be noted that the reconstructability of the original data can be considered when performing such a partial downloading scheme. For example, for iεS, let μi be the number of symbols downloaded from storage node i. Since μi≦∝, it can be assumed that the downloaded symbols are linear combinations of the symbols in node i given by sTH(i)A(i), where A(i) is an α×μi matrix. The data s can then be reconstructed from the downloaded symbols sT[H(i)A(i)]iεS if:
rank([H (i) A (i)]iεS)=M  (7)
For each iεS, the matrix A(i) can be a full column rank, since otherwise at least one downloaded symbol can be expressed as a linear combination of other symbols downloaded from the same storage node, which means this symbol is redundant and should be removed to reduce the downloading bandwidth.
However, it should be noted that the search for the linear combination matrices {A(i)}iεS that satisfy the above-mentioned equation (7) may be computationally prohibitive. While it may be simpler to directly download the stored symbols from each storage node without performing such a linear combination, it can be determined whether there is a loss in optimality. That is, for some symbols {μi}iεS, it can determined whether the above-mentioned equation (7) can be satisfied by performing linear combination but cannot be satisfied by simply downloading the stored symbols without linear combination. It has been determined that, in terms of the number of symbols downloaded from the storage nodes, downloading the symbols stored in the storage nodes directly and downloading their linear combinations are equivalent in some embodiments. This can be represented as follows:
    • If there exists α×μi matrices A(i) for iεS such that the above-mentioned equation (7) is satisfied, then there exist M×μi submatrices H (i)
      Figure US09271229-20160223-P00006
      H(i), iεS, such that the matrix H S
      Figure US09271229-20160223-P00006
      [ H (i)]iεS is of rank M.
Accordingly, given μ=[μ1, μ2, . . . , μS], μiε{0, 1, . . . , α} for 1≦i≦S, the data can be μ-reconstructable if it can be reconstructed via downloading μi symbols from storage node i for iεS, which is equivalent to that there exist M×μi submatrices H (i)
Figure US09271229-20160223-P00006
H(i), iεS, such that the matrix H S
Figure US09271229-20160223-P00006
[ H (i)]iεS is of rank M.
In some embodiments, a portion of the symbols in a storage node for data reconstruction can be downloaded at the minimum-storage regenerating (MSR) point or at the minimum-bandwidth regenerating (MBR) point. This can be performed, for example, by the data collector.
Generally speaking, to analyze the μ-reconstructability at either the MSR or the MBR point, one or more sufficient conditions on {μi}iεS such that data can be reconstructed (e.g., by the data collector 120 illustrated in FIGS. 1 and 2) via downloading μi symbols from storage node i, iεS can be determined. In response, a partial downloading scheme given a set {μi}iεS that satisfied this condition can be provided.
In some embodiments, the data can be reconstructed by downloading a portion of the symbol from any suitable storage nodes at the MSR point. In this example, the data collector (e.g., data collector 120 as illustrated in FIGS. 1 and 2) can use a partial downloading scheme such that the number of downloaded symbols should be no less than the total number of data symbols to be reconstructed. This can be represented by:
ΣiεSμi ≧M  (8)
It should be noted that the μ-reconstructability can be considered for all coding schemes satisfying the constraint of the MSR point that Kα=M in some embodiments. In some embodiments, since, for any size-K subset
Figure US09271229-20160223-P00010
Figure US09271229-20160223-P00006
S, the data s can be reconstructed from the M downloaded symbols sT[H(i)]iεS, the square matrix [H(i)]iεR is of rank M. Thus, in such embodiments, for any subset
Figure US09271229-20160223-P00010
Figure US09271229-20160223-P00006
S with |
Figure US09271229-20160223-P00010
|≦K, the columns of matrix [H(i)]iεR are linearly independent.
Accordingly, ΣiεSμi≧M, which states that the number of downloaded symbols should be no less than the total number of data symbols to be reconstructed, can also be a sufficient condition for μ-reconstructability for the MSR point.
In some embodiments, it can be determined or tested whether a condition is sufficient for μ-reconstructability for the MSR point. FIG. 4 shows an illustrative example of a process for determining whether a condition is sufficient for μ-reconstructability for the MSR point in accordance with some embodiments of the disclosed subject matter. It should be noted that, in process 400 of FIG. 4 and any other process or method described herein, some steps can be added, some steps can be omitted, the order of the steps can be re-arranged, and/or some steps can be performed simultaneously.
Generally speaking, to determine whether ΣiεSμi≧M is a sufficient condition for μ-reconstructability for the MSR point, process 400 selects μi symbols sT H (i) from storage node i for any {μi}iεS satisfying ΣiεSμi=M, where H (i) is an M×μi submatrix of H(i), iεS such that H S=[H(i)]iεS is of rank M. Process 400 continues to select symbols that are linearly independent of the selected symbols until M symbols have been selected.
Process 400 begins at 410 by selecting a storage node i. At 420, process 400 determines whether the selected storage node is a feasible storage node. In some embodiments, a storage node i can be considered feasible if the number of downloaded symbols from it is smaller than μi.
Upon determining that the storage node is a feasible storage node at 420, process 400 can determine whether all of the symbols in the feasible storage node are linearly independent of the selected symbols at 430. Upon determining that the symbols in the feasible storage node are linearly independent of the selected symbols, a symbol from the feasible storage node can be selected and written as a linear combination of the selected symbols at 440. Based on the linear combination, a selected symbol can be replaced with another symbol from the same storage node such that the symbol from that feasible storage node can be selected to further increase the rank of H S at 450.
Process 400 can determine whether M symbols have been selected at 460. If M symbols have not been selected, process 400 can return to 410 and can continue to select symbols that are linearly independent of the selected symbols from one or more storage nodes. Upon selecting M symbols, process 400 can turn to process 500 of FIG. 5.
As shown in FIG. 5, process 500 begins at 510 by initializing H (i) as null matrices for iεS. It should be noted that, in the selection process, let λi be the number of columns of H (i) (the number of symbols already selected from storage node i). It should also be noted that V={iεS:λii} is the set of storage nodes that does not satisfy the downloading requirement, V0={i:λii}, and {tilde over (H)}(i)=H(i)\ H (i) for iεS.
At 520, process 500 can extract a column of {tilde over (H)}(i)
Figure US09271229-20160223-P00003
H(i)\ H (i) and add the extracted column to H (i) for some iεV
Figure US09271229-20160223-P00003
{iεS:λii}. This extraction and addition can continue until H S=[H(i)]iεS is of rank M.
When it is determined that rank ( H S)<M at 530, process 500 can check each column of {tilde over (H)}V=[{tilde over (H)}(i)]iεV to determine whether it is linearly independent of the columns of H S at 540. Upon determining that the column is linearly independent, process 500 can add the column to {tilde over (H)}V and, thus, increase the rank by one at 550.
Otherwise, process 500 can turn to process 600 of FIG. 6.
Process 600 begins by randomly selecting some i0εV and a column
h j 0 ( i 0 )
of {tilde over (H)}(i 0 ). This can, for example, be expressed as a linear combination of the columns in H S. For example:
h j 0 ( i 0 ) = i W j ?? γ j ( i ) h j ( i ) , γ j ( i ) 0 ( 9 )
where
Figure US09271229-20160223-P00011
is the set of storage nodes, and
Figure US09271229-20160223-P00012
i
Figure US09271229-20160223-P00006
{1, 2, . . . , α}, for each iε
Figure US09271229-20160223-P00011
, is the set of column indices of H (i) that participate in the linear combination representation of hj 0 (i 0 ) shown above in (9). It should be noted that
Figure US09271229-20160223-P00011
0=
Figure US09271229-20160223-P00011
\{i0} is the set of storage nodes other than i0 that participates in the above-mentioned linear combination of (9).
At 620, process 600 determines that a column
h j 1 ( i 1 )
of [{tilde over (H)}(i)
Figure US09271229-20160223-P00013
exists that is linearly independent from the columns of H S. Again, it should be noted that a property of the MSR point is that the columns of
Figure US09271229-20160223-P00014
are linearly independent for any |
Figure US09271229-20160223-P00015
|≦K and rank (
Figure US09271229-20160223-P00014
)=M. Accordingly, span(
Figure US09271229-20160223-P00014
)=Q=GF(q)M for any |
Figure US09271229-20160223-P00015
|≧K. Since the above-mentioned linear combination (9) involves the columns from matrices H (i) for iε
Figure US09271229-20160223-P00011
∪{i0}, |
Figure US09271229-20160223-P00011
∪{i0}|≧K+1 and |
Figure US09271229-20160223-P00011
0|=|
Figure US09271229-20160223-P00011
\{i0}|≧K, and thus:
Q=span(H W 0 )
Figure US09271229-20160223-P00006
span([ H S\W 0 |H W 0 ])=span([ H S |{tilde over (H)} W 0 ])
Accordingly, rank([ H S|{tilde over (H)}W 0 ])=M. It should be noted that, by assuming that rank ( H S)<M, it follows that there exists a column
h j 1 ( i 1 )
of {tilde over (H)}W 0 that is linearly independent from the columns of H S.
It should be noted that, for the linear combination shown above in (9) in some embodiments, replacing a column
h j 1 ( i 1 )
for a jε
Figure US09271229-20160223-P00012
1 with
h j 0 ( i 0 )
does not change span ( H S), but provides space for
h j 1 ( i 1 ) ,
which increases the rank of H S by one. Accordingly, process 600 removes
h j ( i 1 )
from H S and then adds
h j 0 ( i 0 ) and h j 1 ( i 1 )
to H S, thereby increasing the rank of H S by one.
In some embodiments, a symbol selection process for a partial downloading scheme at the MSR point can be provided. For example, FIG. 7 shows an illustrative example of a recursive symbol selection process for a partial downloading scheme at the MSR point in accordance with some embodiments of the disclosed subject matter. It should be noted that Λ can be used to represent the number of columns of H S and G S can be used to represent the Gaussian elimination representation of H S via column transformation ( G S= H ST).
At 710, process 700 can determine whether h is linearly independent of the columns of H S. At 720, upon determining that h is linearly independent of the columns of H S, process 700 can add H to H S and, accordingly, update G S.
Alternatively, upon determining that h is not linearly independent of the columns of H S, process 700 can randomly select i0εV and a column
h j 0 ( i 0 )
of {tilde over (H)}(i 0 ), which can be expressed as a linear combination of the vectors in G S. For example:
h j 0 ( i 0 ) = G _ S t 0 .
Allowing H S= G ST0 to represent the Gaussian elimination procedure for H S,
h j 0 ( i 0 )
can be represented as:
h j 0 ( i 0 ) = H _ S T 0 - 1 t 0
It should be noted that this is an explicit representation of the equation
h j 0 ( i 0 ) = i W j ?? γ j ( i ) h j ( i ) , γ j ( i ) 0.
Referring back to FIG. 7, process 700 can search for a column
h j 1 ( i 1 )
this is linearly independent of H S based on the Gaussian eliminated matrix G S. It should be noted that, since replacing
h j ( i 1 ) with h j 0 ( i 0 )
does not change the spanned space and thus the Gaussian eliminated matrix G S, process 700 only needs to update the Gaussian eliminated matrix G S for adding
h j 1 ( i 1 )
to H S.
In some embodiments, the data can be reconstructed by downloading a portion of the symbols from any suitable storage nodes at the MBR point. For example, the data collector (e.g., data collector 120 as illustrated in FIGS. 1 and 2) can use a partial downloading scheme that downloads a portion of the symbols from any suitable storage nodes at the MBR point.
A data block of:
M = K ( K + 1 ) 2 + K ( d - K )
symbols can be stored among S storage nodes, where each node stores α=d symbols. The M symbols can be represented using the following d×d symmetric matrix:
B = [ B ( 1 ) B ( 2 ) ( B ( 2 ) ) T 0 ]
where B(1) is a K×K symmetric matrix storing
K ( K + 1 ) 2
symbols and B(2) is a K×(d−K) matrix storing K(d−K) symbols. For encoding, the matrix B is premultiplied by an S×d Vandermonde matrix given by Ψ, and each node iεS stores the d symbols corresponding to the ith row of ψiB, where ψi is the ith row of Ψ.
In some embodiments, a partial downloading scheme for reconstructing data at the MBR point based on the above coding scheme is provided. As shown in FIG. 8, since the matrix B(1) is symmetric, the data symbols in B(2) and in the upper triangular part of B(1) can be decoded. As also shown in FIG. 8, the data symbol can be divided into d columns {bj}1≦j≦d, where bj for 1≦j≦K are in the upper triangular part of B(1), and bj for K+1≦j≦d are the columns of B(2). The data can be reconstructed in the backward order of bd, bd-1, . . . , b1. It should be noted that the number of symbols in bj, 1≦j≦d, can be given by min {j, K}
Figure US09271229-20160223-P00003
θj. Let (ψiB)j be the jth symbol in storage node i which is the product of ψi and the jth column of B.
As shown in the example flow diagram of FIG. 9, a partial downloading scheme that performs a backward reconstruction process 900 is provided in accordance with some embodiments of the disclosed subject matter.
At 910, process 900 can begin by reconstructing B(2) of matrix B. Referring back to FIG. 8, to reconstruct bj, the data collector can download the symbols (ψiB)j for i belonging to a subset
Figure US09271229-20160223-P00010
Figure US09271229-20160223-P00006
S, here the size |
Figure US09271229-20160223-P00010
j|≧K=θj. Since Ψ is a Vandermonde matrix, bj can be reconstructed with K downloaded symbols.
At 920, process 900 can then reconstruct B(1) of matrix B. This can be done, for example, by reconstructing bj for 1≦j≦K in the order of bk, bk-1, . . . , b1. Referring back to FIG. 8, when reconstructing bj since bl for j<l≦K have been reconstructed and B(1) is symmetric, the part of B(1) shown in area 802 is known from the previous reconstruction. Then, as B(2) has been reconstructed in 910 and thus is known, reconstructing bj amounts to downloading the symbols (ψiB)j for iε
Figure US09271229-20160223-P00010
j
Figure US09271229-20160223-P00006
S, where the size |
Figure US09271229-20160223-P00010
|≧j=θj. As mentioned above, since Ψ is a Vandermonde matrix, bj can be reconstructed with j downloaded symbols.
Based on the above partial downloading scheme, let ηj (i)=1 if the data collector downloads (ψiB)j to reconstruct bj. Otherwise, let ηj (i)=0. Accordingly, the minimum requirement for data construction can be represented as:
i = 1 S η j ( i ) = θ j , 1 j d
The data is μ-reconstructable if there exists ηj (i)ε{0,1} for 1≦i≦S and 1≦j≦d such that:
μ i = i = 1 d η j ( i ) , i S
and the above-equations are satisfied.
In some embodiments, a sufficient condition in terms of {μi}iεS is provided for the two above-mentioned equations to hold.
One condition for the two above-mentioned equations is that, for any subset A
Figure US09271229-20160223-P00006
S:
i A μ i = i A i = 1 d η j ( i ) = i = 1 d i A η j ( i ) j = 1 d min { θ j , A }
Since ΣiεAηj (i)≦|A| and ΣiεAηj (i)≦ΣiεSηj (i)j for all 1≦j≦d. Denoting the sorted {μi}iεS in decreasing order as μ(1)≧μ(2)≧ . . . ≧μ(S) provides:
i = 1 l μ ( i ) j = 1 d min { θ j , l } for 1 l d , and i = 1 S μ ( i ) = j = 1 d θ j .
Since θj=min{j, K}, the above-mentioned equations can be represented as:
i = 1 l μ ( i ) dl - l ( l - 1 ) 2 for 1 l d , and l = 1 S μ ( l ) = M .
In some embodiments, the sufficiency of the condition can be tested via an illustrative partial downloading scheme shown in FIG. 10. More particularly, given {μi}iεS satisfying the two above-mentioned equations.
{ η j p ( i k ) } i S , 1 j d
can be determined.
Turning to FIG. 10, process 1000 can begin by initializing at 1010. Such an initialization can include ranking {μi}iεS in decreasing order μi 1 ≧μi 2 ≧ . . . ≧μi S and letting θj=min{j, K}, j=1, . . . , K.
At 1020, process 1000 can then determine
{ η j p ( i k ) } 1 j d
for k=1, 2, . . . S. At 1022, process 1000 can include ranking {θj}1≦j≦d in decreasing order θj 1 ≧θj 2 ≧ . . . ≧θj d . At 1024, process 1000 can then let
η j p ( i k ) = 1
for 1≦p≦μi k and letting
η j p ( i k ) = 0
for μi k +1≦p≦d. At 1026, process 1000 can subtract
{ η j p ( i k ) } 1 j d
from θj p and update θj p j p −1 for 1≦p≦μi k .
It should be noted that the above symbol selection scheme of FIG. 10 can be used to obtain ηj (i), iεS, 1≦j≦d to reconstruct the data.
As described above, in some embodiments, the data collector can determine the number of symbols to be downloaded from each channel and storage node based on suitable channel and power allocation schemes.
In accordance with some embodiments, the number of symbols to be downloaded over channel j can be given by:
X j =c(P jΣiεSβj (i) |g j (i)|2)  (10)
where: iεS;
Figure US09271229-20160223-P00009
;
c ( · ) is c ( P j i S β j ( i ) g j ( i ) 2 ) = WT B log 2 ( 1 + P j i S β j ( i ) g j ( i ) 2 σ 2 )
as described above in connection with equation (6);
βj (i)=1 if the data collector (e.g., data collector 120 in FIGS. 1 and 2) downloads symbols from storage node i using channel j and βi (i)=0 otherwise; and
Pj be the transmission power for channel j.
Based on equation (10), the number of symbols μi to be downloaded from a storage node i can then be calculated as:
μij=1 Nβj (i) X j ,iεS  (11)
In some embodiments, it may be desirable to minimize the total power used across all N channels during data reconstruction. A minimum total power used during such a reconstruction can be represented in some embodiments as:
min { β j ( i ) , P j } i S , j ?? j ?? P j
such that: the data is μ-reconstructable as described above;
Xj=c (PjΣiεSβj (i)|gj (i)|2);
μij=1 Nβj (i)Xj, iεS;
Xjε{0, 1, 2, . . . , α} for jε
Figure US09271229-20160223-P00009
; and
ΣiεSβj (i)≦1, jε
Figure US09271229-20160223-P00009
βj (i)ε{0,1}.
As reflected by ΣiεSβj (i)≦1, jε
Figure US09271229-20160223-P00009
; βj (i)ε{0,1}, in some embodiments, each channel, if used, may be restricted to being used only to transmit symbols from one node.
In accordance with some embodiments, based on channel estimation results, the data collector can run a channel and power allocation process to determine the number of symbols to be downloaded from each storage node. Any suitable channel and power allocation process can be used in some embodiments.
According to equation (10), letting p(·) be the inverse function c(·) of the capacity function in equation (6), the transmission power of a channel j can be represented as:
P j = p ( X j ) i S β j ( i ) g j ( i ) 2 ( 12 )
Based on equation (12), the minimum total power used during a reconstruction, represented above by
min { β j ( i ) , P j } i S , j ?? j ?? P j ,
can alternatively be represented as
min { β j ( i ) , X j } i S , j ?? j ?? p ( X j ) β j ( i ) i S g j ( i ) 2 such that : j = 1 N X j = M ; X j + { 0 , 1 } ; i S β j ( i ) 1 , j ?? ; β j ( i ) { 0 , 1 } ( 13 )
In some embodiments, each channel jεN can then be assigned to a node i as follows:
βj (i j )=1, for ij=arg max|gj (i)|; and βj (i)=0, otherwise.
Letting
p j ( X j ) = 1 g j ( i j ) 2 p ( X j ) ,
the power allocation problem then becomes:
min { X i } i ?? j = 1 N p j ( X j ) such that : j = 1 N X j = M ; X j + { 0 } . ( 14 )
In some embodiments, a greedy algorithm can be used to find this minimum power. For example, in some embodiments, the minimum power can be found as follows:
1) Initialize Xj=0 for jε
Figure US09271229-20160223-P00009
;
2) While ΣjεNXj<M, do the following:
    • a) For jε
      Figure US09271229-20160223-P00016
      , let ΔPj=pj(Xj+1)−pj(Xj) be the power increment for channel j; and
    • b) Find the channel j0=arg min
      Figure US09271229-20160223-P00009
      ΔPj with the minimum power increment, and update Xj 0 ←Xj 0 +1; and
3) Output Xj, for jε
Figure US09271229-20160223-P00009
.
Turning to FIG. 11A, an example of a process 1100 for finding a minimum power is illustrated. In some embodiments, process 1100 can be implemented in a data collector (e.g., data collector 120 as illustrated in FIGS. 1 and 2).
As shown, after process 1100 begins, the process can initialize the number of symbols for each channel to zero at 1104. Next, at 1106, process 1100 can, for each channel, calculate the increase in power required for transmitting another symbol on that channel. This calculation can be performed in any suitable manner. For example, as described above, this calculation can be performed using the following equation: ΔPj=pj(Xj+1)−pj(Xj). Process 1100 can then select the channel with the minimum increase in power at 1108. Next, at 1110, process 1100 can allocate a symbol to be transmitted on the selected channel. At 1112, process 1100 can determine whether all symbols have been allocated. If not, process 1100 can loop back to 1106. Otherwise, at 1114, process 1100 can provide the total numbers of symbols to be transmitted on each channel to a suitable process that provides this data to the storage nodes via a feedback channel.
As described above, based on the solution {Xj
Figure US09271229-20160223-P00017
obtained by a greedy algorithm, the number of symbols assigned to each storage node can be given by μi=
Figure US09271229-20160223-P00018
βj (i)Xj for iεS, in some embodiments.
In some embodiments, if {μi}iεS violates a constraint that μi≦α, iεS and the data collector is operating at the MSR point, an adjustment can be performed by identifying each storage node i with μi>α, and reassigning symbols to be downloaded on one of the storage nodes channels to another storage node to decrease μi, until μi<α for all iεS. This adjustment can be performed in any suitable manner in some embodiments. For example, in some embodiments, the adjustment can be performed as follows:
    • 1) While μi>α for some iεS, do the following:
      • a) Find a storage node i with μi>α and the set of assigned channels denoted as
        Figure US09271229-20160223-P00009
        i={j: βj (i)=1};
      • b) Find the storage node i′εS\{i}
        Figure US09271229-20160223-P00003
        S′ and the channel jε
        Figure US09271229-20160223-P00009
        i that minimizes the power increment of reassigning the Xj symbols in channel j to node i′ such that μi′+Xj≦α, i.e.,
( i 0 , j 0 ) = argmin ( i , j ) S × ?? i : μ i + X j α p j ( i ) ( X j ) - p j ( i ) ( X j ) ( 15 )
    • 2) Reassign the Xj 0 symbols in channel j0 to storage node i0, by letting βj 0 (i 0 )=1 and βj 0 (i)=0.
Turning to FIG. 11B, an example of a process 1101 for performing an adjustment when the data collector is operating at the MSR point is illustrated. In some embodiments, process 1101 can be implemented in a data collector (e.g., data collector 120 as illustrated in FIGS. 1 and 2). As shown, process 1101 may be a continuation of process 1100 of FIG. 11A. In some embodiments, and in some instances, such as when the data collector is not operating at the MSR point, process 1101 can be skipped or omitted.
After process 1101 begins, the process can set an index i equal to one at 1114. Next, at 1116, the process can determine if the number of symbols to be downloaded from node i is greater than the number of symbols stored at that node. If so, process 1101 can then calculate the power increase to each other node that would result from reassigning the symbols on a channel assigned to that node to that channel on another node at 1118. This calculation can be made in any suitable manner. For example, in some embodiments, the calculation can be made using: pj (i′)(Xj)−pj (i)(Xj). Next, at 1120, process 1101 can select as node i the node that (1) will have the minimum power increase if the symbols to be transmitted on that channel are reassigned to that node to and (2) that will not have more symbols to be transmitted from that node that are present in that node after reassignment. At 1122, process 1101 can the reassign the symbols to the channel on the new node i.
After performing 1122 or determining at 1116 that the number of symbols to be downloaded from node i is not greater than the number of symbols stored at that node, process 1101 can branch to 1124 at which i can be incremented. Then, at 1126, process 1101 can determine whether i is greater than M, the number of symbols to be downloaded. If not, process 1101 can loop back to 1116. Otherwise, process 1101 can complete.
In some embodiments, if the data collector is operating at the MBR point, the following constraint can be applied against {μi}iεS:
i = 1 l μ ( i ) dl - l ( l - 1 ) 2 , for 1 l d ; and l = 1 S μ ( l ) = M
As described above, this constraint can require that μ(1)≦d, μ(1)(2)≦2d−1, μ(1)(2)(3)≦3d−3, etc.
In some embodiments, if {μi}iεS violates this constraint and the data collector is operating at the MBR point, an adjustment can be performed as follows:
    • 1) Select as i the storage node with the maximum μi;
    • 2) Select as i′ the storage node with the minimum μi;
    • 3) Reassign one symbol to be downloaded from storage node i so that the symbol will be downloaded from storage node i′;
    • 4) Recalculate {μi}iεS;
    • 5) Determine if the constraint is still violated; and
    • 6) If so, loop back to 1), otherwise end.
Turning to FIG. 11C, an example of a process 1102 for performing an adjustment when the data collector is operating at the MBR point is illustrated. In some embodiments, process 1102 can be implemented in a data collector (e.g., data collector 120 as illustrated in FIGS. 1 and 2). As shown, process 1102 may be a continuation of process 1101 of FIG. 11B. In some embodiments, and in some instances, such as when the data collector is not operating at the MBR point, process 1102 can be skipped or omitted.
After process 1102 begins, the process can select as i the storage node with the maximum μi at 1128. Next, at 1130, process 1102 can select as i′ the storage node with the minimum μi. Then, at 1132, the process can reassign one symbol to be downloaded from storage node i so that the symbol will be downloaded from storage node i′. At 1134, process 1102 can recalculate {μi}iεS as described above. Then, at 1136, the process can determine if the constraint is still violated. If so, the process can loop back to 1128. Otherwise, the process can end.
As described above, in some embodiments, the data collector can regenerate data of a failed storage node by downloading d data symbols from other storage nodes. These downloaded data symbols can then be loaded onto a new storage node.
In some embodiments, when the data collector is operating at the MSR point, this can be accomplished by downloading one symbol from each of any α+K−1 storage nodes. More particularly, for example, in some embodiments, when the data collector is operating at the MSR point and α≧K−1, this can be accomplished by choosing any d=α+K−1 storage nodes and downloading one symbol from each of the d nodes.
In some embodiments, when the data collector is operating at the MBR point, this can be accomplished by downloading one symbol from each of any α storage nodes. More particularly, for example, in some embodiments, when the data collector is operating at the MBR point, this can be accomplished by selecting any d=α nodes and downloading one symbol from each of them.
Then, the wireless resource allocation for both the MSR and MBR points can be similarly formulated as described above, i.e., to minimize the power consumption of downloading d symbols from d storage nodes.
In some embodiments, the data collector can attempt to minimize the total power used to download these symbols subject to the constraints of: ΣiεSμi=d; and μiε{0,1} for iεS.
This attempt to minimize the total power can be performed in any suitable manner. For example, in some embodiments, a greedy algorithm can be used to find this minimum power. For example, in some embodiments, the minimum power can be found as follows:
    • 1) Initialize Xj=0 for jε
      Figure US09271229-20160223-P00009
      ;
    • 2) While
      Figure US09271229-20160223-P00019
      Xj<d, do the following:
      • a) For jε
        Figure US09271229-20160223-P00009
        where Xj equals 0, calculate the power for transmitting one symbol pj(1) on channel j; and
      • b) Find the channel j0=arg
        Figure US09271229-20160223-P00020
        pj(1) with the minimum power, and select that channel j as to be used to download a symbol: Xj 0 ←1; and
    • 3) Output Xj for jε
      Figure US09271229-20160223-P00009
      .
Turning to FIG. 12, an example of a process 1200 for finding a minimum power is illustrated. In some embodiments, process 1200 can be implemented in a data collector (e.g., data collector 120 as illustrated in FIGS. 1 and 2).
As shown, after process 1200 begins, the process can initialize the number of symbols for each channel to zero at 1204. Next, at 1206, process 1200 can, for each unused channel, calculate the power required for transmitting a symbol on that channel. This calculation can be performed in any suitable manner. Process 1200 can then select the channel with the minimum power at 1208. Next, at 1210, process 1100 can allocate a symbol to be transmitted on the selected channel. At 1212, process 1100 can determine whether all symbols have been allocated. If not, process 1200 can loop back to 1206. Otherwise, at 1214, process 1200 can provide the total numbers of symbols to be transmitted on each channel to any suitable process that needs this data. In some embodiments, in the event that the allocated symbols {μi}iεS violates a constraint that μi≦1, iεS, a reassignment method similar to that described above in connection with FIG. 11B can be used to reassign one or more of the allocated symbols for regeneration.
Accordingly, methods, systems, and media for partial downloading in wireless distributed networks are provided.
It should be noted that processes of FIGS. 3, 5-7, 9, 10, 11A, 11B, 11C, and 12 can be performed concurrently in some embodiments. It should also be noted that the above steps of the flow diagrams of FIGS. 3, 5-7, 9, 10, 11A, 11B, 11C, and 12 may be executed or performed in any order or sequence not limited to the order and sequence shown and described in the figures. Furthermore, it should be noted, some of the above steps of the flow diagrams of FIGS. 3, 5-7, 9, 10, 11A, 11B, 11C, and 12 may be executed or performed substantially simultaneously where appropriate or in parallel to reduce latency and processing times. And still furthermore, it should be noted, some of the above steps of the flow diagrams of FIGS. 3, 5-7, 9, 10, 11A, 11B, 11C, and 12 may be omitted.
In some embodiments, any suitable computer readable media can be used for storing instructions for performing the processes described herein. For example, in some embodiments, computer readable media can be transitory or non-transitory. For example, non-transitory computer readable media can include media such as magnetic media (such as hard disks, floppy disks, etc.), optical media (such as compact discs, digital video discs, Blu-ray discs, etc.), semiconductor media (such as flash memory, electrically programmable read only memory (EPROM), electrically erasable programmable read only memory (EEPROM), etc.), any suitable media that is not fleeting or devoid of any semblance of permanence during transmission, and/or any suitable tangible media. As another example, transitory computer readable media can include signals on networks, in wires, conductors, optical fibers, circuits, any suitable media that is fleeting and devoid of any semblance of permanence during transmission, and/or any suitable intangible media.
Although the invention has been described and illustrated in the foregoing illustrative embodiments, it is understood that the present disclosure has been made only by way of example, and that numerous changes in the details of embodiment of the invention can be made without departing from the spirit and scope of the invention, which is only limited by the claims which follow. Features of the disclosed embodiments can be combined and rearranged in various ways.

Claims (19)

What is claimed is:
1. A method for selecting numbers of symbols to be transmitted on a plurality of channels, comprising:
receiving, from each of a plurality of storage nodes, information about one or more symbols stored in the storage node, wherein each of the one or more symbols represents a portion of a data block and wherein each of the plurality of storage nodes is connected via a channel of the plurality of channels;
for each of the plurality of channels, calculating using a hardware processor at least an increase in transmission power of that channel if it transmits a first symbol that represents a first portion of the data block from a corresponding storage node of the plurality of storage nodes;
selecting, using the hardware processor, a first channel of the plurality of channels to transmit the first symbol and a second channel of the plurality of channels to transmit a second symbol that represents a second portion of the data block based at least in part on the calculation indicating that the first channel will have the smallest increase in transmission power from transmitting the first symbol and based at least in part on the received information about one or more symbols stored in a first storage node, wherein the first storage node of the plurality of storage nodes is connected via the first channel and a second storage node of the plurality of storage nodes is connected via the second channel; and
indicating that the first symbol is to be transmitted by the first channel and that the second symbol is to be transmitted by the second channel using the hardware processor.
2. The method of claim 1, further comprising determining a reduction in the number of symbols to be transmitted on a channel of the plurality of channels.
3. The method of claim 2, further comprising:
determining that the number of symbols to be transmitted from a third storage node of the plurality of storage nodes exceeds the number of symbols stored at the third storage node, wherein the third storage node is connected via a third channel of the plurality of channels;
calculating a power increase that would occur by reassigning a symbol from the third channel to each of a subset of the plurality of channels wherein each storage node of a subset of the plurality of storage nodes is connected via a corresponding channel of the subset of the plurality of channels; and
selecting a fourth storage node of the subset of the plurality of storage nodes to which the symbol is to be reassigned based on the calculating.
4. The method of claim 3, further comprising transmitting symbols at a minimum-storage regenerating (MSR) point.
5. The method of claim 2, further comprising:
selecting the first storage node as having a maximum number of symbols to be transmitted;
selecting the second storage node as having a minimum number of symbols to be transmitted; and
reassigning a symbol as to be transmitted from the second storage node instead of the first storage node.
6. The method of claim 5, further comprising transmitting symbols at a minimum-bandwidth regenerating (MBR) point.
7. A system for selecting numbers of symbols to be transmitted on a plurality of channels, comprising:
at least one hardware processor; and
memory containing computer-executable instructions that, when executed by the hardware processor, cause the hardware processor to:
receive, from each of a plurality of storage nodes, information about one or more symbols stored in the storage node, wherein each of the one or more symbols represents a portion of a data block and wherein each of the plurality of storage nodes is connected via a channel of the plurality of channels;
for each of the plurality of channels, calculate at least an increase in transmission power of that channel if it transmits a first symbol that represents a first portion of the data block from a corresponding storage node of the plurality of storage nodes;
select a first channel of the plurality of channels to transmit the first symbol and a second channel of the plurality of channels to transmit a second symbol that represents a second portion of the data block based at least in part on the calculation indicating that the first channel will have the smallest increase in transmission power from transmitting the first symbol and based at least in part on the received information about one or more symbols stored in a first storage node, wherein the first storage node of the plurality of storage nodes is connected via the first channel and a second storage node of the plurality of storage nodes is connected via the second channel; and
indicate that the first symbol is to be transmitted by the first channel and that the second symbol is to be transmitted by the second channel.
8. The system of claim 7, wherein the instructions further cause the at least one hardware processor to determine a reduction in the number of symbols to be transmitted on a channel of the plurality of channels.
9. The system of claim 8, wherein the instructions further cause the at least one hardware processor to:
determine that the number of symbols to be transmitted from a third storage node of the plurality of storage nodes exceeds the number of symbols stored at the third storage node, wherein the third storage node is connected via a third channel of the plurality of channels;
calculate a power increase that would occur by reassigning a symbol from the third channel to each of a subset of the plurality of channels wherein each storage node of a subset of the plurality of storage nodes is connected via a corresponding channel of the subset of the plurality of channels; and
select a fourth storage node of the subset of the plurality of storage nodes to which the symbol is to be reassigned based on the calculating.
10. The system of claim 9, wherein the instructions further cause the at least one hardware processor to cause symbols to be transmitted a minimum-storage regenerating (MSR) point.
11. The system of claim 8, wherein the instructions further cause the at least one hardware processor to:
select the first storage node as having a maximum number of symbols to be transmitted;
select the second storage node as having a minimum number of symbols to be transmitted; and
reassign a symbol as to be transmitted from the second storage node instead of the first storage node.
12. The system of claim 11, wherein the instructions further cause the at least one hardware processor to cause symbols to be transmitted at a minimum-bandwidth regenerating (MBR) point.
13. A non-transitory computer-readable medium containing computer-executable instructions that, when executed by a processor, cause the processor to perform a method for selecting numbers of symbols to be transmitted on a plurality of channels, the method comprising:
receiving, from each of a plurality of storage nodes, information about one or more symbols stored in the storage node, wherein each of the one or more symbols represents a portion of a data block and wherein each of the plurality of storage nodes is connected via a channel of the plurality of channels;
for each of the plurality of channels, calculating at least an increase in transmission power of that channel if it transmits a first symbol that represents a first portion of the data block from a corresponding storage node of the plurality of storage nodes;
selecting a first channel of the plurality of channels to transmit the first symbol and a second channel of the plurality of channels to transmit a second symbol that represents a second portion of the data block based at least in part on the calculation indicating that the first channel will have the smallest increase in transmission power from transmitting the first symbol and based at least in part on the received information about one or more symbols stored in a first storage node, wherein the first storage node of the plurality of storage nodes is connected via the first channel and a second storage node of the plurality of storage nodes is connected via the second channel; and
indicating that the first symbol is to be transmitted by the first channel and that the second symbol is to be transmitted by the second channel.
14. The non-transitory computer-readable medium of claim 13, wherein the method further comprises determining a reduction in the number of symbols to be transmitted on a channel of the plurality of channels.
15. The non-transitory computer-readable medium of claim 14, wherein the method further comprises:
determining that the number of symbols to be transmitted from a third storage node of the plurality of storage nodes exceeds the number of symbols stored at the third storage node, wherein the third storage node is connected via a third channel of the plurality of channels;
calculating a power increase that would occur by reassigning a symbol from the third channel to each of a subset of the plurality of channels, wherein each storage node of a subset of the plurality of storage nodes is connected via a corresponding channel of the subset of the plurality of channels; and
selecting a fourth storage node of the subset of the plurality of storage nodes to which the symbol is to be reassigned based on the calculating.
16. The non-transitory computer-readable medium of claim 15, wherein the method further comprises transmitting symbols at a minimum-storage regenerating (MSR) point.
17. The non-transitory computer-readable medium of claim 14, wherein the method further comprises:
selecting the first storage node as having a maximum number of symbols to be transmitted;
selecting the second storage node as having a minimum number of symbols to be transmitted; and
reassigning a symbol as to be transmitted from the second storage node instead of the first storage node.
18. The non-transitory computer-readable medium of claim 17, wherein the method further comprises transmitting symbols at a minimum-bandwidth regenerating (MBR) point.
19. A method for selecting numbers of symbols to be transmitted on a plurality of channels, comprising:
receiving, using a hardware processor of a data collection device, from each of a plurality of storage nodes connected to a data collection device via a channel of the plurality of channels, information about one or more symbols stored in that storage node, wherein each of the one or more symbols represents part of a data block;
calculating, using the hardware processor, for each of the plurality of channels, an increase in transmission power of that channel if it transmits a first symbol from a corresponding storage node of the plurality of storage nodes;
selecting, using the hardware processor, a first channel of the plurality of channels based at least in part on the calculation indicating that the first channel will have the smallest increase in power from transmitting the first symbol and based at least in part on the information about one or more symbols stored in a first storage node of the plurality of storage nodes;
in response to selecting the first channel for transmitting the first symbol, calculating, using the hardware processor, for each of the plurality of channels, an increase in transmission power of that channel if it transmits a second symbol from a corresponding storage node of the plurality of storage nodes, wherein the calculation of the increase in transmission power of the first channel if it transmits the second symbol is based on the calculation of the increase in transmission power of the first channel if it transmits the first symbol;
selecting, using the hardware processor, a second channel of the plurality of channels based on the calculation indicating that the second channel will have the smallest increase in power from transmitting the second symbol and based at least in part on the information about one or more symbols stored in a second storage node of the plurality of storage nodes; and
indicating, using the hardware processor, that the first symbol is to be transmitted by the first storage node using the first channel and that the second symbol is to be transmitted by the second storage node using the second channel.
US13/948,123 2012-07-20 2013-07-22 Methods, systems, and media for partial downloading in wireless distributed networks Active 2033-09-29 US9271229B2 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US13/948,123 US9271229B2 (en) 2012-07-20 2013-07-22 Methods, systems, and media for partial downloading in wireless distributed networks

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US201261674264P 2012-07-20 2012-07-20
US13/948,123 US9271229B2 (en) 2012-07-20 2013-07-22 Methods, systems, and media for partial downloading in wireless distributed networks

Publications (2)

Publication Number Publication Date
US20140022970A1 US20140022970A1 (en) 2014-01-23
US9271229B2 true US9271229B2 (en) 2016-02-23

Family

ID=49946486

Family Applications (1)

Application Number Title Priority Date Filing Date
US13/948,123 Active 2033-09-29 US9271229B2 (en) 2012-07-20 2013-07-22 Methods, systems, and media for partial downloading in wireless distributed networks

Country Status (1)

Country Link
US (1) US9271229B2 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11714715B2 (en) * 2014-06-04 2023-08-01 Pure Storage, Inc. Storage system accommodating varying storage capacities

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2014059651A1 (en) * 2012-10-19 2014-04-24 北京大学深圳研究生院 Method for encoding, data-restructuring and repairing projective self-repairing codes
CN103688515B (en) * 2013-03-26 2016-10-05 北京大学深圳研究生院 The coding of a kind of minimum bandwidth regeneration code and memory node restorative procedure
US10235736B2 (en) * 2017-04-21 2019-03-19 Intel Corporation Intelligent graphics dispatching mechanism

Citations (33)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4271480A (en) * 1975-12-31 1981-06-02 Compagnie Internationale Pour L'informatique Cii-Honeywell Bull Apparatus enabling the transfer of data blocks of variable lengths between two memory interfaces of different widths
US5355372A (en) * 1992-08-19 1994-10-11 Nec Usa, Inc. Threshold-based load balancing in ATM switches with parallel switch planes related applications
US5598435A (en) * 1993-12-23 1997-01-28 British Telecommunications Public Limited Company Digital modulation using QAM with multiple signal point constellations not equal to a power of two
US5685010A (en) * 1995-02-22 1997-11-04 Nec Corporation Data transfer control device for controlling data transfer between shared memories of network clusters
US6073189A (en) * 1995-07-24 2000-06-06 Adaptec, Inc. System and method for incremental packet recording of recordable compact discs
US20020059439A1 (en) * 1999-02-26 2002-05-16 Arroyo Keith M. Streaming method and system for fibre channel network devices
US20020112102A1 (en) * 2001-01-24 2002-08-15 Hitachi, Ltd. Computer forming logical partitions
US20030065808A1 (en) * 2001-10-02 2003-04-03 Dawson Christopher Byron System and method for distribution of software
US20030115275A1 (en) * 1996-12-31 2003-06-19 Toga James E. Method and apparatus for delivering data
US20040100941A1 (en) * 2002-11-20 2004-05-27 Kwang Jae Lim Adaptive packet transmission method for transmitting packets in multibeam satellite communication system
US20040120332A1 (en) * 2002-12-24 2004-06-24 Ariel Hendel System and method for sharing a resource among multiple queues
US20040249956A1 (en) * 2003-02-24 2004-12-09 Seiko Epson Corporation Printing method and device
US20050018702A1 (en) * 2003-07-09 2005-01-27 Industrial Technology Research Institute PAPR reduction method using bit reallocation
US20050132078A1 (en) * 2003-12-12 2005-06-16 Alok Kumar Facilitating transmission of a packet in accordance with a number of transmit buffers to be associated with the packet
US20050268125A1 (en) * 2004-05-17 2005-12-01 Kabushiki Kaisha Toshiba Logic circuit apparatus
US20060013179A1 (en) * 2004-07-13 2006-01-19 Iwatsu Electric Co., Ltd. Channel decision system for access point
US20070014397A1 (en) * 2005-07-06 2007-01-18 Masaharu Ukeda Storage device and information processing device
US20080014981A1 (en) * 2006-07-17 2008-01-17 Muthaiah Venkatachalam Methods and apparatus for providing idle mode operations for a platform with a plurality of wireless communication devices
US20080039058A1 (en) * 2006-08-10 2008-02-14 Qualcomm Incorporated Systems and methods for temporary media file storage on a wireless communication device
US20080248752A1 (en) * 2007-04-09 2008-10-09 Nec Access Technica, Ltd. Radio transmission system, transmitter and radio transmission method
US20090005109A1 (en) * 2004-07-30 2009-01-01 Matsushita Electric Industrial Co., Ltd. Wireless Transmitter and Wireless Transmitting Method
US7576884B2 (en) * 2003-07-11 2009-08-18 Seiko Epson Corporation Image output system, client terminal device, image output device, and image output method using e-mail to inform client of output status
US20090279847A1 (en) * 2008-04-30 2009-11-12 Sony Corporation Program accumulation apparatus, program accumulation reservation method, and program
US7640357B2 (en) * 2004-04-30 2009-12-29 Sap Ag Transmitting enterprise messages based on buffer sizes
US20100011145A1 (en) * 2008-07-10 2010-01-14 Blackwave Inc. Dynamic Storage Resources
US20100325339A1 (en) * 2008-10-10 2010-12-23 Junji Ogawa Storage system and method for controlling the same
US20110161712A1 (en) * 2009-12-30 2011-06-30 International Business Machines Corporation Cooling appliance rating aware data placement
US20110197092A1 (en) * 2008-10-10 2011-08-11 Thomson Licensing, LLC A method for requesting a file repair distribution mode
US20110205898A1 (en) * 2010-02-24 2011-08-25 Fujitsu Limited Router, management apparatus, and routing control program
US20110289351A1 (en) * 2010-05-21 2011-11-24 Indian Institute Of Science Distributed storage system and a method thereof
US20120030356A1 (en) * 2010-07-30 2012-02-02 International Business Machines Corporation Maximizing efficiency in a cloud computing environment
US20130024561A1 (en) * 2011-07-19 2013-01-24 Fujitsu Limited Apparatus and method for determining route, computer-readable recording medium having stored therein management program, and management device
US20130182575A1 (en) * 2012-01-16 2013-07-18 Ingo McLean Systems And Methods For Automatically Selecting A Communication Channel

Patent Citations (33)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4271480A (en) * 1975-12-31 1981-06-02 Compagnie Internationale Pour L'informatique Cii-Honeywell Bull Apparatus enabling the transfer of data blocks of variable lengths between two memory interfaces of different widths
US5355372A (en) * 1992-08-19 1994-10-11 Nec Usa, Inc. Threshold-based load balancing in ATM switches with parallel switch planes related applications
US5598435A (en) * 1993-12-23 1997-01-28 British Telecommunications Public Limited Company Digital modulation using QAM with multiple signal point constellations not equal to a power of two
US5685010A (en) * 1995-02-22 1997-11-04 Nec Corporation Data transfer control device for controlling data transfer between shared memories of network clusters
US6073189A (en) * 1995-07-24 2000-06-06 Adaptec, Inc. System and method for incremental packet recording of recordable compact discs
US20030115275A1 (en) * 1996-12-31 2003-06-19 Toga James E. Method and apparatus for delivering data
US20020059439A1 (en) * 1999-02-26 2002-05-16 Arroyo Keith M. Streaming method and system for fibre channel network devices
US20020112102A1 (en) * 2001-01-24 2002-08-15 Hitachi, Ltd. Computer forming logical partitions
US20030065808A1 (en) * 2001-10-02 2003-04-03 Dawson Christopher Byron System and method for distribution of software
US20040100941A1 (en) * 2002-11-20 2004-05-27 Kwang Jae Lim Adaptive packet transmission method for transmitting packets in multibeam satellite communication system
US20040120332A1 (en) * 2002-12-24 2004-06-24 Ariel Hendel System and method for sharing a resource among multiple queues
US20040249956A1 (en) * 2003-02-24 2004-12-09 Seiko Epson Corporation Printing method and device
US20050018702A1 (en) * 2003-07-09 2005-01-27 Industrial Technology Research Institute PAPR reduction method using bit reallocation
US7576884B2 (en) * 2003-07-11 2009-08-18 Seiko Epson Corporation Image output system, client terminal device, image output device, and image output method using e-mail to inform client of output status
US20050132078A1 (en) * 2003-12-12 2005-06-16 Alok Kumar Facilitating transmission of a packet in accordance with a number of transmit buffers to be associated with the packet
US7640357B2 (en) * 2004-04-30 2009-12-29 Sap Ag Transmitting enterprise messages based on buffer sizes
US20050268125A1 (en) * 2004-05-17 2005-12-01 Kabushiki Kaisha Toshiba Logic circuit apparatus
US20060013179A1 (en) * 2004-07-13 2006-01-19 Iwatsu Electric Co., Ltd. Channel decision system for access point
US20090005109A1 (en) * 2004-07-30 2009-01-01 Matsushita Electric Industrial Co., Ltd. Wireless Transmitter and Wireless Transmitting Method
US20070014397A1 (en) * 2005-07-06 2007-01-18 Masaharu Ukeda Storage device and information processing device
US20080014981A1 (en) * 2006-07-17 2008-01-17 Muthaiah Venkatachalam Methods and apparatus for providing idle mode operations for a platform with a plurality of wireless communication devices
US20080039058A1 (en) * 2006-08-10 2008-02-14 Qualcomm Incorporated Systems and methods for temporary media file storage on a wireless communication device
US20080248752A1 (en) * 2007-04-09 2008-10-09 Nec Access Technica, Ltd. Radio transmission system, transmitter and radio transmission method
US20090279847A1 (en) * 2008-04-30 2009-11-12 Sony Corporation Program accumulation apparatus, program accumulation reservation method, and program
US20100011145A1 (en) * 2008-07-10 2010-01-14 Blackwave Inc. Dynamic Storage Resources
US20110197092A1 (en) * 2008-10-10 2011-08-11 Thomson Licensing, LLC A method for requesting a file repair distribution mode
US20100325339A1 (en) * 2008-10-10 2010-12-23 Junji Ogawa Storage system and method for controlling the same
US20110161712A1 (en) * 2009-12-30 2011-06-30 International Business Machines Corporation Cooling appliance rating aware data placement
US20110205898A1 (en) * 2010-02-24 2011-08-25 Fujitsu Limited Router, management apparatus, and routing control program
US20110289351A1 (en) * 2010-05-21 2011-11-24 Indian Institute Of Science Distributed storage system and a method thereof
US20120030356A1 (en) * 2010-07-30 2012-02-02 International Business Machines Corporation Maximizing efficiency in a cloud computing environment
US20130024561A1 (en) * 2011-07-19 2013-01-24 Fujitsu Limited Apparatus and method for determining route, computer-readable recording medium having stored therein management program, and management device
US20130182575A1 (en) * 2012-01-16 2013-07-18 Ingo McLean Systems And Methods For Automatically Selecting A Communication Channel

Non-Patent Citations (12)

* Cited by examiner, † Cited by third party
Title
Cadambe, V.R., et al, "Minimum Repair Bandwidth for Exact Regeneration in Distributed Storage", In Proceedings of the 2010 IEEE Wireless Network Coding Conference (WiNC), Boston, MA, US, Jun. 21, 2010, pp. 1-6.
Dimakis, A.G., et al., "Network Coding for Distributed Storage Systems", In IEEE Transactions on Information Theory, vol. 56, No. 9, Sep. 2010, pp. 4539-4551.
Koetter, R., et al., "On a Theory of Network Equivalence", In Proceedings of the IEEE Information Theory Workshop on Networking and Information Theory (ITW '09), Volos, GR, Jun. 10-12, 2009, pp. 326-330.
Rashmi, K.V., et al, "Explicit Construction of Optimal Exact Regenerating Codes for Distributed Storage", Oct. 6, 2009, pp. 1-7, available at: http://arxiv.org/pdf/0906.4913.pdf.
Rashmi, K.V., et al., "Explicit and Optimal Exact-Regenerating Codes for the Minimum-BandWidth in Distributed Storage", Technical Report TR-PME-2010-5, Indian Institute of Science, Bangalore, IN, Feb. 8, 2010, pp. 1-25.
Rashmi, K.V., et al., "Optimal Exact-Regerating Codes for Distributed Storage at the MSR and MBR Points via a Product-Matrix Construction", Jan. 20, 2011, pp. 1-20, available at: http://arxiv.org/pdf/1005.4178v2.pdf.
Shah, N.B., et al., "Explicit Codes Minimizing Repair Bandwidth for Distributed Storage", Sep. 5, 2009, pp. 1-11, available at: http://arxiv.org/pdf/0908.2984.pdf.
Shah, N.B., et al., "Flexible Class of Regenerating Codes for Distributed Storage", In Proceedings of the 2010 IEEE International Symposium on Information Theory (ISIT '10), Austin, TX, US, Jun. 13-18, 2010, pp. 1943-1947.
Suh, C. and Ramachandran, K., "Exact-Repair MDS Code Construction Using Interference Alignment", In IEEE Transactions on Information Theory, vol. 57, No. 3, Mar. 2011, pp. 1425-1442.
Suh, C. and Ramachandran, K., "Exact-Repair MDS Codes for Distributed Storage Using Interference Alignment", Apr. 15, 2010, pp. 1-35, available at: http://arxiv.org/pdf/1001.0107.pdf.
Wu, Y. and Dimakis, A.G., "Reducing Repair Traffic for Erasure Coding-Based Storage via Interference Alignment", In Proceedings of the 2009 IEEE International Conference on Symposium on Information Theory (ISIT '09), Seoul, KR, Jun. 28-Jul. 3, 2009, pp. 2276-2280.
Wu, Y., "Existence and Construction of Capacity-Achieving Network Codes for Distributed Storage", In IEEE Journal on Selected Areas in Communications, vol. 28, No. 2, Feb. 2010, pp. 277-288.

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11714715B2 (en) * 2014-06-04 2023-08-01 Pure Storage, Inc. Storage system accommodating varying storage capacities
US20230376379A1 (en) * 2014-06-04 2023-11-23 Pure Storage, Inc. Storage system accommodating differing types of storage

Also Published As

Publication number Publication date
US20140022970A1 (en) 2014-01-23

Similar Documents

Publication Publication Date Title
US20190007165A1 (en) Polar code encoding method and device
US10833706B2 (en) Polar encoding and decoding using predefined information
US9271229B2 (en) Methods, systems, and media for partial downloading in wireless distributed networks
US7979777B2 (en) Apparatus, method and program for decoding
EP2136474B1 (en) Encoding and decoding of low density parity check (LDPC) codes for frequency selective channels
US11296723B2 (en) Methods and apparatuses for data processing in communication system
CN108809482B (en) Rate matching method and device for Polar codes
CN109547034B (en) Decoding method and device, decoder
CN108494527B (en) Data sending and receiving method based on LoRa
CN108430040B (en) Joint user grouping and power allocation method and base station using same
US8855028B2 (en) Apparatus and method for bidirectional communication between multi-nodes using relay node
US8385462B2 (en) Subband indexing methods and systems
CN110247730B (en) Blind detection method and device based on polarization code
US8503346B2 (en) Wireless network using network coding scheme based on overhearing channel
KR100953567B1 (en) Data reception acknowledge signal transmission/reception apparatus and method in mobile communication system
WO2018210216A1 (en) Data transmission method, chip, transceiver, and computer readable storage medium
KR102455652B1 (en) Method and apparatus for decoding using soft decision
US9692616B1 (en) Apparatus for and method of channel estimation buffer compression via decimation, prediction, and error encoding
US20200067635A1 (en) Apparatuses, computer-readable media, and methods for single dimension maximal likelihood symbol detection
CN108429602B (en) Data processing method and device and transmitting terminal
KR20200017587A (en) Method and apparatus for design of punctured polar codes
Sheikh et al. Energy efficient image transmission through orthogonal frequency division multiplexing (OFDM) based multiple input multiple output (MIMO) systems
US20090168911A1 (en) Apparatus and method for channel information feedback in multiple antenna system
CN114915376B (en) Decoding method, encoding method, apparatus, device, and storage medium
US8605805B2 (en) Receiver, channel state information compressing method, and computer program

Legal Events

Date Code Title Description
AS Assignment

Owner name: THE TRUSTEES OF COLUMBIA UNIVERSITY IN THE CITY OF

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:GONG, CHEN;WANG, XIAODONG;SIGNING DATES FROM 20151001 TO 20151003;REEL/FRAME:036739/0692

STCF Information on status: patent grant

Free format text: PATENTED CASE

AS Assignment

Owner name: NATIONAL SCIENCE FOUNDATION, VIRGINIA

Free format text: CONFIRMATORY LICENSE;ASSIGNOR:COLUMBIA UNIVERSITY;REEL/FRAME:039521/0427

Effective date: 20160516

FEPP Fee payment procedure

Free format text: MAINTENANCE FEE REMINDER MAILED (ORIGINAL EVENT CODE: REM.); ENTITY STATUS OF PATENT OWNER: MICROENTITY

FEPP Fee payment procedure

Free format text: SURCHARGE FOR LATE PAYMENT, MICRO ENTITY (ORIGINAL EVENT CODE: M3554); ENTITY STATUS OF PATENT OWNER: MICROENTITY

MAFP Maintenance fee payment

Free format text: PAYMENT OF MAINTENANCE FEE, 4TH YEAR, MICRO ENTITY (ORIGINAL EVENT CODE: M3551); ENTITY STATUS OF PATENT OWNER: MICROENTITY

Year of fee payment: 4

FEPP Fee payment procedure

Free format text: MAINTENANCE FEE REMINDER MAILED (ORIGINAL EVENT CODE: REM.); ENTITY STATUS OF PATENT OWNER: MICROENTITY

FEPP Fee payment procedure

Free format text: SURCHARGE FOR LATE PAYMENT, MICRO ENTITY (ORIGINAL EVENT CODE: M3555); ENTITY STATUS OF PATENT OWNER: MICROENTITY

MAFP Maintenance fee payment

Free format text: PAYMENT OF MAINTENANCE FEE, 8TH YEAR, MICRO ENTITY (ORIGINAL EVENT CODE: M3552); ENTITY STATUS OF PATENT OWNER: MICROENTITY

Year of fee payment: 8