US20110078343A1 - Distributed storage network including memory diversity - Google Patents

Distributed storage network including memory diversity Download PDF

Info

Publication number
US20110078343A1
US20110078343A1 US12/777,850 US77785010A US2011078343A1 US 20110078343 A1 US20110078343 A1 US 20110078343A1 US 77785010 A US77785010 A US 77785010A US 2011078343 A1 US2011078343 A1 US 2011078343A1
Authority
US
United States
Prior art keywords
storage
memory
data
unit
memories
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US12/777,850
Inventor
Jason K. Resch
S. Christopher Gladwin
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Pure Storage Inc
Original Assignee
Cleversafe Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Cleversafe Inc filed Critical Cleversafe Inc
Priority to US12/777,850 priority Critical patent/US20110078343A1/en
Assigned to CLEVERSAFE, INC. reassignment CLEVERSAFE, INC. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: GLADWIN, S. CHRISTOPHER, RESCH, JASON K.
Publication of US20110078343A1 publication Critical patent/US20110078343A1/en
Priority to US13/529,052 priority patent/US8862800B2/en
Priority to US13/925,270 priority patent/US9274890B2/en
Assigned to INTERNATIONAL BUSINESS MACHINES CORPORATION reassignment INTERNATIONAL BUSINESS MACHINES CORPORATION ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: CLEVERSAFE, INC.
Assigned to PURE STORAGE, INC. reassignment PURE STORAGE, INC. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: INTERNATIONAL BUSINESS MACHINES CORPORATION
Assigned to PURE STORAGE, INC. reassignment PURE STORAGE, INC. CORRECTIVE ASSIGNMENT TO CORRECT THE DELETE 15/174/279 AND 15/174/596 PROPERTY NUMBERS PREVIOUSLY RECORDED AT REEL: 49555 FRAME: 530. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Assignors: INTERNATIONAL BUSINESS MACHINES CORPORATION
Abandoned legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/07Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/08Error detection or correction by redundancy in data representation, e.g. by using checking codes
    • G06F11/10Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's
    • G06F11/1076Parity data used in redundant arrays of independent storages, e.g. in RAID systems
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0602Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/0614Improving the reliability of storage systems
    • G06F3/0617Improving the reliability of storage systems in relation to availability
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10File systems; File servers
    • G06F16/18File system types
    • G06F16/182Distributed file systems
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0628Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0655Vertical data movement, i.e. input-output transfer; data movement between one or more hosts and one or more storage devices
    • G06F3/0656Data buffering arrangements
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0628Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0655Vertical data movement, i.e. input-output transfer; data movement between one or more hosts and one or more storage devices
    • G06F3/0659Command handling arrangements, e.g. command buffers, queues, command scheduling
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0668Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/067Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0668Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/0671In-line storage system
    • G06F3/0683Plurality of storage devices
    • G06F3/0689Disk arrays, e.g. RAID, JBOD
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/10Protocols in which an application is distributed across nodes in the network
    • H04L67/1001Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
    • H04L67/1004Server selection for load balancing
    • H04L67/1008Server selection for load balancing based on parameters of servers, e.g. available memory or workload
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/10Protocols in which an application is distributed across nodes in the network
    • H04L67/1001Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
    • H04L67/1004Server selection for load balancing
    • H04L67/101Server selection for load balancing based on network conditions
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/10Protocols in which an application is distributed across nodes in the network
    • H04L67/1097Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/231Content storage operation, e.g. caching movies for short term storage, replicating data over plural servers, prioritizing data for deletion
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/24Monitoring of processes or resources, e.g. monitoring of server load, available bandwidth, upstream requests
    • H04N21/2405Monitoring of the internal components or processes of the server, e.g. server load
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/442Monitoring of processes or resources, e.g. detecting the failure of a recording device, monitoring the downstream bandwidth, the number of times a movie has been viewed, the storage space available from the internal hard disk
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F12/00Accessing, addressing or allocating within memory systems or architectures
    • G06F12/02Addressing or allocation; Relocation
    • G06F12/08Addressing or allocation; Relocation in hierarchically structured memory systems, e.g. virtual memory systems
    • G06F12/0802Addressing of a memory level in which the access to the desired data or data block requires associative addressing means, e.g. caches
    • G06F12/0806Multiuser, multiprocessor or multiprocessing cache systems
    • G06F12/0813Multiuser, multiprocessor or multiprocessing cache systems with a network or matrix configuration
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F2211/00Indexing scheme relating to details of data-processing equipment not covered by groups G06F3/00 - G06F13/00
    • G06F2211/10Indexing scheme relating to G06F11/10
    • G06F2211/1002Indexing scheme relating to G06F11/1076
    • G06F2211/1011Clustered RAID, i.e. clustered or de-clustered RAID where data and parity are spread over more disks than blocks in a parity group

Definitions

  • This invention relates generally to computing and more particularly to storage of information.
  • Computing systems are known to communicate, process, and store data. Such computing systems range from wireless smart phones to data centers that support millions of web searches, stock trades, or on-line purchases every day.
  • Computing processing is known to manipulate data from one form into another. For instance, raw picture data from an image sensor may be compressed, or manipulated, in accordance with a picture compression standard to produce a standardized compressed picture that can be saved or shared with others.
  • Computer processing capability continues to advance as processing speed advances and software applications that perform the manipulation become more sophisticated.
  • Each type of computing system is constructed, and hence operates, in accordance with one or more communication, processing, and storage standards.
  • communication, processing, and storage standards With such standards, and with advances in technology, more and more of the global information content is being converted into electronic formats. For example, more digital cameras are now being sold than film cameras, thus producing more digital pictures.
  • Digital content standards such as used in pictures, papers, books, video entertainment, home video, all enable this global transformation to a digital format. Electronic content pervasiveness is producing increasing demands on the storage function of computing systems.
  • a typical computer storage function includes one or more memory devices to match the needs of the various operational aspects of the processing and communication functions.
  • a memory device may include solid-state NAND flash, random access memory (RAM), read only memory (ROM), a mechanical hard disk drive.
  • RAM random access memory
  • ROM read only memory
  • Each type of memory device has a particular performance range and normalized cost.
  • the computing system architecture optimizes the use of one or more types of memory devices to achieve the desired functional and performance goals of the computing system.
  • the immediacy of access dictates what type of memory device is used.
  • RAM memory can be accessed in any random order with a constant response time.
  • memory device technologies that require physical movement such as magnetic discs, tapes, and optical discs, have a variable responses time as the physical movement can take longer than the data transfer.
  • Each type of computer storage system is constructed, and hence operates, in accordance with one or more storage standards.
  • computer storage systems may operate in accordance with one or more standards including, but not limited to network file system (NFS), flash file system (FFS), disk file system (DFS), small computer system interface (SCSI), internet small computer system interface (iSCSI), file transfer protocol (FTP), and web-based distributed authoring and versioning (WebDAV).
  • An operating systems (OS) and storage standard may specify the data storage format and interface between the processing subsystem and the memory devices.
  • the interface may specify a structure such as directories and files.
  • a memory controller provides an interface function between the processing function and memory devices. As new storage systems are developed, the memory controller functional requirements may change to adapt to new standards.
  • Memory devices may fail, especially those that utilize technologies that require physical movement like a disc drive. For example, it is not uncommon for a disc drive to suffer from bit level corruption on a regular basis, or complete drive failure after an average of three years of use.
  • One common solution is to utilize more costly disc drives that have higher quality internal components.
  • Another solution is to utilize multiple levels of redundant disc drives to abate these issues by replicating the data into two or more copies.
  • One such redundant drive approach is called redundant array of independent discs (RAID).
  • RAID redundant array of independent discs
  • Multiple physical discs comprise an array where parity data is added to the original data before storing across the array. The parity is calculated such that the failure of one or more discs will not result in the loss of the original data. The original data can be reconstructed from the other discs.
  • RAID 5 uses three or more discs to protect data from the failure of any one disc.
  • RAID 6 can recover from a loss of two discs and requires a minimum of four discs with an efficiency of n-2.
  • Typical RAID systems utilize a RAID control to encode and decode the data across the array.
  • Drawbacks of the RAID approach include effectiveness, efficiency and security. As more discs are added, the probability of one or two discs failing rises and is not negligible, especially if more desired less costly discs are used. When one disc fails, it should be immediately replaced and the data reconstructed before a second drive fails. To provide high reliability over a long time period, and if the RAID array is part of a national level computing system with occasional site outages, it is also common to mirror RAID arrays at different physical locations. Unauthorized file access becomes a more acute problem when whole copies of the same file are replicated, either on just one storage system site or at two or more sites. In light of the effectiveness, the efficiency of dedicating 1 to 2 discs per array for the RAID overhead is an issue.
  • FIG. 1 is a schematic block diagram of an embodiment of a computing system in accordance with the invention.
  • FIG. 2 is a schematic block diagram of an embodiment of a computing core in accordance with the invention.
  • FIG. 3 is a schematic block diagram of an embodiment of a distributed storage processing unit in accordance with the invention.
  • FIG. 4 is a schematic block diagram of an embodiment of a distributed storage unit in accordance with the invention.
  • FIG. 5 is a flowchart illustrating the reading and writing of memory
  • FIG. 6 is a state transition diagram illustrating the reading and writing of memory
  • FIG. 7 is a flowchart illustrating the writing of memory
  • FIG. 8A is a schematic block diagram of an embodiment of a distributed storage system in accordance with the invention.
  • FIG. 8B is another flowchart illustrating the writing of memory
  • FIG. 9A is a schematic block diagram of another embodiment of a distributed storage system in accordance with the invention.
  • FIG. 9B is another flowchart illustrating the writing of memory
  • FIG. 10 is a schematic block diagram of another embodiment of a distributed storage system in accordance with the invention.
  • FIG. 11 is another flowchart illustrating the writing of memory.
  • FIG. 1 is a schematic block diagram of a computing system 10 that includes one or more of a first type of user devices 12 , one or more of a second type of user devices 14 , at least one distributed storage (DS) processing unit 16 , at least one DS managing unit 18 , at least one storage integrity processing unit 20 , and a distributed storage network (DSN) memory 22 coupled via a network 24 .
  • the network 24 may include one or more wireless and/or wire lined communication systems; one or more private intranet systems and/or public internet systems; and/or one or more local area networks (LAN) and/or wide area networks (WAN).
  • the DSN memory 22 includes a plurality of distributed storage (DS) units 36 for storing data of the system.
  • Each of the DS units 36 includes a processing module and memory and may be located at a geographically different site than the other DS units (e.g., one in Chicago, one in Milwaukee, etc.).
  • the processing module may be a single processing device or a plurality of processing devices.
  • Such a processing device may be a microprocessor, micro-controller, digital signal processor, microcomputer, central processing unit, field programmable gate array, programmable logic device, state machine, logic circuitry, analog circuitry, digital circuitry, and/or any device that manipulates signals (analog and/or digital) based on hard coding of the circuitry and/or operational instructions.
  • the processing module may have an associated memory and/or memory element, which may be a single memory device, a plurality of memory devices, and/or embedded circuitry of the processing module.
  • a memory device may be a read-only memory, random access memory, volatile memory, non-volatile memory, static memory, dynamic memory, flash memory, cache memory, and/or any device that stores digital information.
  • the processing module includes more than one processing device, the processing devices may be centrally located (e.g., directly coupled together via a wired and/or wireless bus structure) or may be distributedly located (e.g., cloud computing via indirect coupling via a local area network and/or a wide area network).
  • the processing module implements one or more of its functions via a state machine, analog circuitry, digital circuitry, and/or logic circuitry
  • the memory and/or memory element storing the corresponding operational instructions may be embedded within, or external to, the circuitry comprising the state machine, analog circuitry, digital circuitry, and/or logic circuitry.
  • the memory element stores, and the processing module executes, hard coded and/or operational instructions corresponding to at least some of the steps and/or functions illustrated in FIGS. 1-11 .
  • Each of the user devices 12 - 14 , the DS processing unit 16 , the DS managing unit 18 , and the storage integrity processing unit 20 may be a portable computing device (e.g., a social networking device, a gaming device, a cell phone, a smart phone, a personal digital assistant, a digital music player, a digital video player, a laptop computer, a handheld computer, a video game controller, and/or any other portable device that includes a computing core) and/or a fixed computing device (e.g., a personal computer, a computer server, a cable set-top box, a satellite receiver, a television set, a printer, a fax machine, home entertainment equipment, a video game console, and/or any type of home or office computing equipment).
  • a portable or fixed computing device includes a computing core 26 and one or more interfaces 30 , 32 , and/or 33 . An embodiment of the computing core 26 will be described with reference to FIG. 2 .
  • each of the interfaces 30 , 32 , and 33 includes software and/or hardware to support one or more communication links via the network 24 and/or directly.
  • interfaces 30 support a communication link (wired, wireless, direct, via a LAN, via the network 24 , etc.) between the first type of user device 14 and the DS processing unit 16 .
  • DSN interface 32 supports a plurality of communication links via the network 24 between the DSN memory 22 and the DS processing unit 16 , the first type of user device 12 , and/or the storage integrity processing unit 20 .
  • interface 33 supports a communication link between the DS managing unit 18 and any one of the other devices and/or units 12 , 14 , 16 , 20 , and/or 22 via the network 24 .
  • the system 10 supports three primary functions: distributed network data storage management, distributed data storage and retrieval, and data storage integrity verification.
  • data can be distributedly stored in a plurality of physically different locations and subsequently retrieved in a reliable and secure manner regardless of failures of individual storage devices, failures of network equipment, the duration of storage, the amount of data being stored, attempts at hacking the data, etc.
  • the DS managing unit 18 performs the distributed network data storage management functions, which include establishing distributed data storage parameters, performing network operations, performing network administration, and/or performing network maintenance.
  • the DS managing unit 18 establishes the distributed data storage parameters (e.g., allocation of virtual DSN memory space, distributed storage parameters, security parameters, billing information, user profile information, etc.) for one or more of the user devices 12 - 14 (e.g., established for individual devices, established for a user group of devices, established for public access by the user devices, etc.).
  • the DS managing unit 18 coordinates the creation of a vault (e.g., a virtual memory block) within the DSN memory 22 for a user device (for a group of devices, or for public access).
  • a vault e.g., a virtual memory block
  • the DS managing unit 18 also determines the distributed data storage parameters for the vault. In particular, the DS managing unit 18 determines a number of slices (e.g., the number that a data segment of a data file and/or data block is partitioned into for distributed storage) and a threshold value (e.g., the minimum number of slices required to reconstruct the data segment).
  • a number of slices e.g., the number that a data segment of a data file and/or data block is partitioned into for distributed storage
  • a threshold value e.g., the minimum number of slices required to reconstruct the data segment.
  • the DS managing module 18 may create and store locally or within the DSN memory 22 user profile information.
  • the user profile information includes one or more of authentication information, permissions, and/or the security parameters.
  • the security parameters may include one or more of encryption/decryption scheme, one or more encryption keys, key generation scheme, and data encoding/decoding scheme.
  • the DS managing unit 18 may create billing information for a particular user, user group, vault access, public vault access, etc. For instance, the DS managing unit 18 may track the number of times user accesses a private vault and/or public vaults, which can be used to generate a per-access bill. In another instance, the DS managing unit 18 tracks the amount of data stored and/or retrieved by a user device and/or a user group, which can be used to generate a per-data-amount bill.
  • the DS managing unit 18 also performs network operations, network administration, and/or network maintenance. As at least part of performing the network operations and/or administration, the DS managing unit 18 monitors performance of the devices and/or units of the system 10 for potential failures, determines the devices and/or unit's activation status, determines the devices' and/or units' loading, and any other system level operation that affects the performance level of the system 10 . For example, the DS managing unit 18 may receive and aggregate network management alarms, alerts, errors, status information, performance information, and messages from the devices 12 - 14 and/or the units 16 , 20 , 22 . For example, the DS managing unit 18 may receive a simple network management protocol (SNMP) message regarding the status of the DS processing unit 16 .
  • SNMP simple network management protocol
  • the DS managing unit 18 performs the network maintenance by identifying equipment within the system 10 that needs replacing, upgrading, repairing, and/or expanding. For example, the DS managing unit 18 may determine that the DSN memory 22 needs more DS units 36 or that one or more of the DS units 36 needs updating.
  • the second primary function of distributed data storage and retrieval function begins and ends with a user device 12 - 14 .
  • a second type of user device 14 has a data file 38 and/or data block 40 to store in the DSN memory 22 , it send the data file 38 and/or data block 40 to the DS processing unit 16 via its interface 30 .
  • the interface 30 functions to mimic a conventional operating system (OS) file system interface (e.g., network file system (NFS), flash file system (FFS), disk file system (DFS), file transfer protocol (FTP), web-based distributed authoring and versioning (WebDAV), etc.) and/or a block memory interface (e.g., small computer system interface (SCSI), internet small computer system interface (iSCSI), etc.).
  • OS operating system
  • NFS network file system
  • FFS flash file system
  • DFS disk file system
  • FTP file transfer protocol
  • WebDAV web-based distributed authoring and versioning
  • the interface 30 may attach a user identification code (ID) to the data file 38 and/or data block 40 .
  • ID user identification code
  • the DS processing unit 16 receives the data file 38 and/or data block 40 via its interface 30 and performs a distributed storage (DS) process 34 thereon.
  • the DS processing 34 begins by partitioning the data file 38 and/or data block 40 into one or more data segments, which is represented as Y data segments.
  • the DS processing 34 error encodes (e.g., forward error correction (FEC), information dispersal algorithm, or error correction coding) and slices (or slices then error encodes) the data segment into a plurality of error coded (EC) data slices 42 - 48 , which is represented as X slices per data segment.
  • FEC forward error correction
  • EC error coded
  • n/k For example, if a Reed-Solomon (or other FEC scheme) is used in an n/k system, then a data segment is divided into n slices, where k number of slices is needed to reconstruct the original data (i.e., k is the threshold).
  • k is the threshold
  • the n/k factor may be 5/3; 6/4; 8/6; 8/5; 16/10.
  • the DS processing unit 16 For each slice 42 - 48 , the DS processing unit 16 creates a unique slice name and appends it to the corresponding slice 42 - 48 .
  • the slice name includes universal DSN memory addressing routing information (e.g., virtual memory addresses in the DSN memory 22 ) and user-specific information (e.g., user ID, file name, data block identifier, etc.).
  • the DS processing unit 16 transmits the plurality of EC slices 42 - 48 to a plurality of DS units 36 of the DSN memory 22 via the DSN interface 32 and the network 24 .
  • the DSN interface 32 formats each of the slices for transmission via the network 24 .
  • the DSN interface 32 may utilize an internet protocol (e.g., TCP/IP, etc.) to packetize the slices 42 - 48 for transmission via the network 24 .
  • the number of DS units 36 receiving the slices 42 - 48 is dependent on the distributed data storage parameters established by the DS managing unit 18 .
  • the DS managing unit 18 may indicate that each slice is to be stored in a different DS unit 36 .
  • the DS managing unit 18 may indicate that like slice numbers of different data segments are to be stored in the same DS unit 36 .
  • the first slice of each of the data segments is to be stored in a first DS unit 36
  • the second slice of each of the data segments is to be stored in a second DS unit 36 , etc.
  • the data is encoded and distributedly stored at physically diverse locations to improved data storage integrity and security. Further examples of encoding the data segments will be provided with reference to one or more of FIGS. 2-11 .
  • Each DS unit 36 that receives a slice 42 - 48 for storage translates the virtual DSN memory address of the slice into a local physical address for storage. Accordingly, each DS unit 36 maintains a virtual to physical memory mapping to assist in the storage and retrieval of data.
  • the first type of user device 12 performs a similar function to store data in the DSN memory 22 with the exception that it includes the DS processing. As such, the device 12 encoded and slices the data file and/or data block it has to store. The device then transmits the slices 35 to the DSN memory via its DSN interface 32 and the network 24 .
  • a second type of user device 14 For a second type of user device 14 to retrieve a data file or data block from memory, it issues a read command via its interface 30 to the DS processing unit 16 .
  • the DS processing unit 16 performs the DS processing 34 to identify the DS units 36 storing the slices of the data file and/or data block based on the read command.
  • the DS processing unit 16 may also communicate with the DS managing unit 18 to verify that the user device 14 is authorized to access the requested data.
  • the DS processing unit 16 issues slice read commands to at least a threshold number of the DS units 36 storing the requested data (e.g., to at least 10 DS units for a 16/10 error coding scheme).
  • Each of the DS units 36 receiving the slice read command verifies the command, accesses its virtual to physical memory mapping, retrieves the requested slice, or slices, and transmits it to the DS processing unit 16 .
  • the DS processing unit 16 After the DS processing unit 16 has received a threshold number of slices for a data segment, it performs an error decoding function and de-slicing to reconstruct the data segment. When Y number of data segments has been reconstructed, the DS processing unit 16 provides the data file 38 and/or data block 40 to the user device 14 . Note that the first type of user device 12 performs a similar process to retrieve a data file and/or data block.
  • the storage integrity processing unit 20 performs the third primary function of data storage integrity verification.
  • the storage integrity processing unit 20 periodically retrieves slices 45 of a data file or data block of a user device to verify that one or more slices has not been corrupted or lost (e.g., the DS unit failed).
  • the retrieval process mimics the read process previously described.
  • the storage integrity processing unit 20 determines that one or more slices is corrupted or lost, it rebuilds the corrupted or lost slice(s) in accordance with the error coding scheme.
  • the storage integrity processing unit 20 stores the rebuild slice, or slices, in the appropriate DS unit(s) 36 in a manner that mimics the write process previously described.
  • FIG. 2 is a schematic block diagram of an embodiment of a computing core 26 that includes a processing module 50 , a memory controller 52 , main memory 54 , a video graphics processing unit 55 , an input/output (TO) controller 56 , a peripheral component interconnect (PCI) interface 58 , at least one IO device interface module 62 , a read only memory (ROM) basic input output system (BIOS) 64 , and one or more memory interface modules.
  • a processing module 50 a memory controller 52 , main memory 54 , a video graphics processing unit 55 , an input/output (TO) controller 56 , a peripheral component interconnect (PCI) interface 58 , at least one IO device interface module 62 , a read only memory (ROM) basic input output system (BIOS) 64 , and one or more memory interface modules.
  • a processing module 50 includes a processor 50 , a memory controller 52 , main memory 54 , a video graphics processing unit 55 , an input/output (TO) controller 56
  • the memory interface module(s) includes one or more of a universal serial bus (USB) interface module 66 , a host bus adapter (HBA) interface module 68 , a network interface module 70 , a flash interface module 72 , a hard drive interface module 74 , and a DSN interface module 76 .
  • USB universal serial bus
  • HBA host bus adapter
  • network interface module 70 may function as the interface 30 of the user device 14 of FIG. 1 .
  • the IO device interface module 62 and/or the memory interface modules may be collectively or individually referred to as IO ports.
  • the processing module 50 may be a single processing device or a plurality of processing devices.
  • a processing device may be a microprocessor, micro-controller, digital signal processor, microcomputer, central processing unit, field programmable gate array, programmable logic device, state machine, logic circuitry, analog circuitry, digital circuitry, and/or any device that manipulates signals (analog and/or digital) based on hard coding of the circuitry and/or operational instructions.
  • the processing module may have an associated memory and/or memory element, which may be a single memory device, a plurality of memory devices, and/or embedded circuitry of the processing module.
  • Such a memory device may be a read-only memory, random access memory, volatile memory, non-volatile memory, static memory, dynamic memory, flash memory, cache memory, and/or any device that stores digital information.
  • the processing module includes more than one processing device, the processing devices may be centrally located (e.g., directly coupled together via a wired and/or wireless bus structure) or may be distributedly located (e.g., cloud computing via indirect coupling via a local area network and/or a wide area network).
  • the processing module implements one or more of its functions via a state machine, analog circuitry, digital circuitry, and/or logic circuitry
  • the memory and/or memory element storing the corresponding operational instructions may be embedded within, or external to, the circuitry comprising the state machine, analog circuitry, digital circuitry, and/or logic circuitry.
  • the memory element stores, and the processing module executes, hard coded and/or operational instructions corresponding to at least some of the steps and/or functions illustrated in FIGS. 1-11 .
  • FIG. 3 is a schematic block diagram of an embodiment of a dispersed storage (DS) processing unit 16 and/or of the DS processing module 34 of user device 12 (see FIG. 1 ).
  • the DS processing unit 16 includes a gateway module 107 , an access module 109 , a grid module 84 , a storage module 113 , and a bypass/feedback path.
  • the DS processing unit 16 may also include an interface 30 and the DSnet interface 32 .
  • the gateway module 107 of the DS processing unit 16 receives an incoming data object (e.g., a data file, a data block, an EC data slice, etc.), authenticates the user associated with the data object, obtains user information of the authenticated user, and assigns a source name to the data object in accordance with the user information.
  • the gateway module 107 verifies the user ID 119 with the managing unit 18 (see FIG. 1 ) and/or another authenticating unit. If the user ID is verified, the gateway module 107 retrieves the user information from the managing unit 18 (see FIG. 1 ), the user device 14 , and/or the other authenticating unit based on the user ID.
  • the user information includes a vault identifier, operational parameters, and user attributes (e.g., user data, billing information, etc.).
  • a vault identifier identifies a vault, which is a virtual memory space that maps to a set of DS storage units 36 .
  • vault 1 i.e., user 1 's DSN memory space
  • vault 2 i.e., user 2 's DSN memory space
  • the operational parameters may include an error coding algorithm, the width n (number of pillars X or slices per segment for this vault), a read threshold T, an encryption algorithm, a slicing parameter, a compression algorithm, an integrity check method, caching settings, parallelism settings, and/or other parameters that may be used to access the DSN memory layer.
  • the gateway module 107 determines the source name to associate with the data object based on the vault identifier and the data object.
  • the source name may contain a data name (block number or a file number), the vault generation number, a reserved field, and a vault identifier.
  • the data name may be randomly assigned but is associated with the user data object.
  • the gateway module 107 may utilize the bypass/feedback path to transfer an incoming EC data slice to another DS storage unit 36 (see FIG. 1 ) when the DS processing module 34 determines that the EC data should be transferred.
  • the gateway module 60 may use the bypass/feedback path to feedback an EC slice for sub-slicing.
  • the access module 109 receives the data object and creates a series of data segments 1 through Y therefrom.
  • the grid module 84 may pre-manipulate (e.g., compression, encryption, cyclic redundancy check (CRC), etc.) the data segment before creating X error coded data slices for each data segment.
  • the grid module 84 creates XY error coded data slices for the Y data segments of the data object.
  • the grid module 84 adds forward error correction bits to the data segment bits in accordance with an error coding algorithm (e.g., Reed-Solomon, Convolution encoding, Trellis encoding, etc.) to produce an encoded data segment.
  • the grid module 84 determines the slice name and attaches the unique slice name to each EC data slice.
  • the DS processing module may utilize different error coding parameters for EC data slices and EC data sub-slices based on guidance from one or more of a user vault (e.g., stored parameters for this user), a command from the DS managing unit or other system element, priority of the EC data slice, type of data in the EC data slice, and/or retrieval speed requirements.
  • the grid module 84 receives each data segment 1 -Y and, for each data segment generates X number of error coded (EC) slices using an error coding function.
  • the grid module 84 also determines the DS storage units 36 for storing the EC data slices based on a dispersed storage memory mapping associated with the user's vault and/or DS storage unit 36 attributes, which include availability, self-selection, performance history, link speed, link latency, ownership, available DSN memory, domain, cost, a prioritization scheme, a centralized selection message from another source, a lookup table, data ownership, and/or any other factor to optimize the operation of the computing system.
  • the storage module 113 may perform integrity checks on the EC data slices and then transmit the EC data slices 1 through X of each segment 1 through Y to the DS storage units.
  • the DS storage units 36 may store the EC data slices and locally keep a table to convert virtual DSN addresses into physical storage addresses. Note that the number of DS storage units 36 is equal to or greater than the number of pillars (slices X per segment) so that no more than one error coded data slice of the same data segment is stored on the same DS storage unit 36 . Further note that EC data slices of the same pillar number but of different segments (e.g., EC data slice 1 of data segment 1 and EC data slice 1 of data segment 2 ) may be stored on the same or different DS storage units 36 (see FIG. 1 ).
  • the user device 10 or 12 sends a read request to the DS processing unit 14 , which authenticates the request.
  • the DS processing unit 14 sends a read message to each of the DS storage units 36 storing slices of the data object being read.
  • the slices are received via the DSnet interface 34 and processed by the storage module 113 , which performs a parity check and provides the slices to the grid module 84 .
  • the grid module 84 de-slices and decodes the slices of a data segment to reconstruct the data segment.
  • the access module reconstructs the data object from the data segments and the gateway module 107 formats the data object for transmission to the user device.
  • FIG. 4 is a schematic block diagram of an embodiment of a distributed storage unit 36 that includes a storage unit control module 402 , a plurality of memories 403 , 404 , 405 , and 406 , a plurality of parity memories 408 and 409 , and a cache memory 415 .
  • a storage unit control module 402 a plurality of memories 403 , 404 , 405 , and 406 , a plurality of parity memories 408 and 409 , and a cache memory 415 .
  • the storage unit control module 402 may be implemented with the computing core of FIG. 2 .
  • the memories 403 - 406 may be one or more of a magnetic hard disk, NAND flash, read only memory, optical disk, and/or any other type of read-only, or read/write memory.
  • the memories may be implemented as part of or outside of the DS storage unit.
  • memory 1 may be implemented in the DS unit and memory 4 may be implemented in a remote server (e.g., a different DS unit operably coupled to the DS unit via the network).
  • memories 403 - 406 and parity memories 408 - 409 are implemented with the magnetic hard disk technology and the cache memory 415 is implemented with the NAND flash technology.
  • a DS unit includes cache memory 415 implemented using a single solid state drive (SSD).
  • all of the memories are implemented using the same type of device, and one or more of the memories is temporarily selected for use as “cache memory” for purposes of temporarily storing data to be written.
  • the temporarily selected memory can serve as a cache memory until the DS unit shifts responsibility for caching writes to another memory.
  • the storage unit control module 402 includes the DSnet interface 32 and a processing module.
  • the storage unit control module 402 may be operably coupled to the computing system via the DSnet interface 32 via the network.
  • the storage unit control module 402 may receive EC data slices to store via the DSnet interface 32 .
  • the storage unit control module 402 determines where (e.g., which address on which of the memories) to store the received EC data slice.
  • the determination may be based on one or more of the metadata, a command (e.g., from the DS processing unit indicating which memory type to use), a type of data indicator, a priority indicator, a memory state indicator, available memory, memory performance data, memory cost data, the memory characteristics, and/or any other parameter to facilitate desired levels of efficiency and performance.
  • the memory state may indicate whether the memory is in a write only state, a read only state, a write with read priority state, or some other state to indicate the availability.
  • the storage unit control module 402 creates and maintains a local virtual DSN address to physical memory table.
  • the storage unit control module 402 determines where previously stored EC data slices are located based on the local virtual DSN address to physical memory table upon receiving a retrieve command via the network.
  • the storage unit control module 402 may save activity records (e.g., memory utilization, errors, stores, retrievals, etc.) as logs in any of the memories.
  • the storage unit control module 402 may utilize the parity memories 408 - 409 to store and retrieve parity across the data stored in memories 403 - 406 .
  • the storage unit control module 402 may immediately recreate a slice that is stored in a memory in the write only state based on reading the other memories in the read only state, reading the parity memory 1 and/or parity memory 2 , and calculating the desired slice.
  • the storage unit control module 402 may temporarily pair a write only state memory 403 - 406 with a write only state parity memory 408 - 409 to enable rapid writes of new slices (e.g., write a slice to memory 1 and write the parity to parity memory 1 ), while another parity memory in the read only state may be available to provide the needed parity to reconstruct slices that are stored on the write only state memory.
  • the storage unit control module 402 may choose memory 1 (e.g., a magnetic hard disk drive) to store the received EC data slice since memory 1 is in a write only state (e.g., available immediately), the memories 2 - 4 are in the read only state, parity memory 1 is paired with memory 1 in the write only state, parity memory 2 is in the ready only state, and the memory 1 memory characteristics favorably match the metadata of the EC data slice, including performance, efficiency, cost, and response time.
  • the storage unit control module 402 queues a read request in the cache memory when the requested slice is in the memory 1 (but in the write state).
  • the storage unit control module 402 may process the queued read request for memory 1 by retrieving the request from the cache memory, reading the memories 2 - 4 (e.g., the same memory stripe or common address range across each), reading the party memory 2 , and calculating the desired slice.
  • the storage unit control module 402 may queue write requests and slices when a desired memory 403 - 406 is in the read only state.
  • the storage unit control module may subsequently change the state of memory 1 from write only to the read only state, or the write with read priority state to enable processing of the queued read request.
  • the DS unit 36 can immediately retrieve slices where the slices are stored in memories in the read only state, or in the write with read priority state (e.g., memories 2 - 4 ). Further note that the DS unit 36 may rotate the write only state amongst the memories 1 - 4 and the parity memories 1 - 2 from time to time to even out the cumulative storage and optimize performance. A method to choose the memories and change the memory state will be discussed in greater detail with reference to FIGS. 5-11 .
  • FIG. 5 is a flowchart illustrating a method 500 of reading and writing to memory where the DS unit 36 (see FIG. 4 ) may control the DS unit memory state and memory utilization to optimize the performance of the memory.
  • the method begins where the storage unit control module 402 (see FIG. 4 ) checks for a received request.
  • the DS unit may receive the request from one or more of the DS processing unit 16 , the user device 12 , the storage integrity processing unit 20 , and/or the DS managing unit 18 (see FIG. 1 ).
  • the storage unit control module determines the request type based on the request when the request is received.
  • the method branches to block 532 , which illustrates receiving a slice to store when the storage unit control module determines the request type is a write request.
  • the storage unit control module determines the slice location and state when the request type is a read request. As illustrated by block 511 , the determination is based in part on accessing the local virtual DSN address to physical location table to identify the memory, the address, and the memory state. As illustrated by block 513 , the storage unit control module retrieves the slice based on the memory and address when the memory state is the read state. The storage unit control module sends the slice to the requester and the method branches back to look for more requests.
  • the storage unit control module determines the method to read the slice when the memory state is the write state. Note that in this state the memory is only writing at this time to optimize the throughput performance of the memory requiring the requested slice to be obtained in another way other than reading it directly from the memory where the slice was initially stored (e.g., which may disrupt the write state performance when the memory is a hard disk drive). As illustrated by block 519 , the determination of the method to read the slice is based on one or more of a predetermination, a command, a DS unit status indicator, a loading indicator for the memories in the read state, a priority indicator, and/or any other indicator to optimize the memory performance.
  • the storage unit control module may send a read request response message to the requester where the response denies the request when the storage unit control module determines the method to be to utilize another DS unit.
  • the DS unit does not return the requested slice to the requester but instead informs the requester that no slice will be returned.
  • the requester must rely on reconstructing the original data object based on the retrieving the slices from the other pillars and performing the de-slicing and decoding steps.
  • the requester may repeat the read request to the DS unit with a priority indicator set when the process to reconstruct the data object fails since a read threshold of k good slices are not retrieved from the DS units.
  • the DS unit In various embodiments, including embodiments in which a DS unit uses an SSD cache or where responsibility for caching writes is delegated to various different memories within a DS unit, the DS unit always responds to read requests, and implementation of block 517 is not required.
  • the storage unit control module may reconstruct the slice from a reverse parity operation based on reading a portion of the memories (e.g., a logical stripe across the memories) and parity memory in the read state when the storage unit control module determines the method to be to utilize the DS unit now.
  • the storage unit control module sends the slice to the requester and returns to the step to look for received requests.
  • Handling the write request begins, as illustrated by block 532 , with the storage unit control module receiving the slice to store in the write request.
  • the storage unit control module determines the present write state memory based on the local virtual DSN address to physical address table.
  • the storage unit control module stores the slice in the write state memory and updates the write parity memory by reading a corresponding portion of the read state memories (e.g., same logical stripe across the memories) and calculating the parity across the slice just written to the write state memory and the read state memories.
  • the storage unit control module stores the parity to the write state parity memory, as shown by block 538 .
  • the storage unit control module determines if it is time to rotate the write state memory and write state parity memory to different memories. The determination may be based on one or more of a timer expiration since the last rotation, a command, a memory utilization indicator (e.g., the present write state memory is filling up), a read request history indicator (e.g., many read requests for slices in the write state memory), and/or any other indicator to optimize the memory performance. As illustrated by block 542 , the method branches back to look for received requests when the storage unit control module determines it is not time to rotate the write state memory.
  • the storage unit control module determines the next write state memory and write state parity memory when the storage unit control module determines it is time to rotate the write state memory.
  • the determination may be based on one or more of identifying which memory was in the write state least recently, a predetermination, a rotation order indicator, a command, a memory utilization indicator (e.g., choose a memory with the most available unused space), a read request history indicator (e.g., avoid a memory with a higher read request frequency than other memories), and/or any other indicator to optimize the memory performance.
  • the storage unit control module updates the local virtual DSN address to physical location table with the chosen write state memory and write state parity memory.
  • the storage unit control module updates the local virtual DSN address to physical location table to modify the state of the previous write state memory and write state parity memory from write state to the read state. Additionally, slices can be moved back to their proper drives. The method branches back to look for received requests.
  • the number of write state memories may be two or more to further improve the write performance of the DS unit.
  • the storage unit control module may only rotate one memory at a time from the write state to the read state or the storage unit control module may rotate more than one memory at a time from the write state to the read state.
  • FIG. 6 is a state transition diagram 600 illustrating the reading and writing of memory where the DS unit may control the DS unit memory state 601 and memory utilization to optimize the performance of the memory.
  • the DS unit may control the DS unit memory state 601 and memory utilization to optimize the performance of the memory.
  • the storage unit control module determines the memory state and processes received read and write requests based on the memory state to optimize the memory performance. For example, when the memory is in the read only state 607 , the storage unit control module processes only read requests, unless too many write requests are pending (e.g., the number write requests is greater than a high threshold). In another example, when the memory is in the write only state 603 , the storage unit control module processes only write requests until the pending write requests are reduced to a low threshold level. In another example, when the memory is in the write state with read priority 605 , the storage unit control module opportunistically processes any pending write requests unless there are pending read requests.
  • the DS unit always responds to read requests.
  • a particular piece of memory being in write only mode 603 means that a read will be delayed, and data will always be stored immediately in read cache memory.
  • the storage unit control module queues received read requests into a read queue and received write requests into a write queue by storing the request (and slice in the case of a write request) in the cache memory as indicated by the upper right portion of FIG. 6 .
  • the requests may be subsequently de-queued and processed as discussed below.
  • the storage unit control module determines if the read queue is not empty and de-queues the read request, determines the memory location, retrieves the slice, and sends the slice to the requester when the storage unit control module determines the read queue is not empty.
  • the storage unit control module determines if the write queue is above the high threshold of write requests while the memory is in the read only state.
  • the storage unit control module changes the state of the memory from the read only state to the write only state when the storage unit control module determines that the write queue is above the high threshold of write requests.
  • the storage unit control module determines if the read queue is empty while the memory is in the read only state.
  • the storage unit control module changes the state of the memory from the read only state to the write state with read priority when the storage unit control module determines that the read queue is empty.
  • While in the write only state (e.g., the second state of three states) the storage unit control module determines if the write queue is not empty and de-queues the write request with slice from the cache memory, determines the memory location, stores the slice, and updates the local virtual DSN address to physical storage table when the storage unit control module determines the write queue is not empty.
  • the storage unit control module determines if the write queue is below the low threshold of write requests while the memory is in the write only state.
  • the storage unit control module changes the state of the memory from the write only state to the read only state when the storage unit control module determines that the write queue is below the low threshold of write requests.
  • the storage unit control module determines if the write queue is not empty and de-queues the write request with slice from the cache memory, determines the memory location, stores the slice, and updates the local virtual DSN address to physical storage table when the storage unit control module determines the write queue is not empty.
  • the storage unit control module determines if the read queue is not empty while the memory is in the write state with read priority.
  • the storage unit control module changes the state of the memory from the write state with read priority to the read only state when the storage unit control module determines that the read queue is not empty.
  • FIG. 7 is a flowchart illustrating a method 700 of writing memory where the DS processing unit (or DS unit) may employ a memory diversity scheme to choose memories to store slices such that the overall system reliability is improved.
  • the memory diversity scheme may ensure that a read threshold of k slices are stored in pillar memories that are each of a different model to avoid unrecoverable data due to a potentially common memory design defect.
  • the DS processing unit creates the slices for distributed storage.
  • the DS processing unit determines the slice metadata based on one or more of a file type, file size, priority, a security index, estimated storage time, estimated time between retrievals and more.
  • the DS processing unit determines the similarity requirements and difference requirements, sometimes referred to as diversity preferences, based on the metadata. Similarity requirements drive similar attributes of the pillar memory choices and difference requirements drive difference attributes of the pillar memory choices. For example, a preference or requirement for a relatively short estimated time between retrievals may drive pillar memory choices that all share a similar fast retrieval characteristic to speed frequent retrievals. Other examples of similarity preferences and requirements may include similar cost and similar capacity. In another example, a preference or requirement for very high reliability may drive pillar memory choices that all have a different memory model to improve the reliability of retrievals. Other examples of difference requirements and preferences may include different operating systems and different installation sites.
  • the DS processing unit determines the DS unit memory characteristics for one or more candidate DS units. The determination may be via a table lookup or a real time request to each DS unit to query for the memory characteristics.
  • the memory characteristics may include one or more of memory model, memory type, total capacity, available capacity, access speed, error history, estimated mean time between failures, actual mean time between failures, and/or hours of operation.
  • the DS processing unit sorts the DS units that favorably match the similarity requirements and difference requirements based on comparing the requirements to the memory characteristics. For example, DS units with memory that has a fast access memory characteristic may be sorted to favorably match the fast access similarity requirement. In another example, DS units with memory that has a different model memory characteristic may be sorted to favorably match the reliability-driven different-model requirement or preference.
  • the DS processing unit determines the best match of DS unit memories to the diversity preferences or requirements based on the sort if possible, or at least a favorable match. For example, the DS processing unit may choose at most n-k DS unit memories with the same model, similar error histories, or similar total hours to improve the reliability of data object retrieval. In other words, the DS unit may choose the read threshold k of DS unit memories that has the most different models, error histories, and total hours as the memory diversity scheme.
  • the DS processing unit sends the slices to the chosen DS units with the best match of memory characteristics to requirements and updates the virtual DSN address to physical location table with the locations of the slices.
  • the DS unit may implement similar functionality to that discussed above to select available memory units that favorably match the diversity preferences determined from the slice metadata.
  • FIG. 8A is a schematic block diagram of an embodiment of a distributed storage system that includes the DS processing unit 16 , a temporary memory 802 , and a plurality of DS units 36 .
  • the DS processing unit 16 may temporarily store new pillar 4 slices in the temporary memory, and/or yet another DS unit, for subsequent storage in DS unit 4 .
  • cache memory refers to a memory that can be used temporarily store information and includes but is not limited to, cache memories such as those included in various processor architectures, memory specifically designated as cache memory, and the like.
  • cache memory is also used in a less rigorous sense to refer to any type of memories used for substantially non-permanent information storage. The method of operation to determine where to temporarily store the slices will be discussed in greater detail with reference to FIGS. 8B and 9B .
  • FIG. 8B is another flowchart illustrating a method 800 of writing to memory where the DS processing unit 16 determines where to store newly created slices when at least one primary DS unit 36 is not available.
  • the method 800 begins as illustrated by block 803 , where the DS processing unit creates the n slices for each data segment for storage. As illustrated by block 805 , the DS processing unit determines the desired primary DS units in which to store the slices based in part on a predetermination of the slice name in the user vault, or in the virtual DSN address to physical location table.
  • the DS processing unit determines the status of the chosen primary DS units based on one or more of a status table lookup and/or a real time query to the DS unit. For example, the status indicates not available if the network is down to the DS unit, or if the DS unit is down. As illustrated by block 810 , the DS processing unit determines the number of primary DS units that are in the ready status. As illustrated by block 809 , the DS processing unit tries other DS units and returns to the step to determine which DS units when the number of ready primary DS units is less than the read threshold k. Note that the threshold for this scenario may be k+1, k+2, or etc. in another embodiment to further improve the probability of subsequent data object recreation.
  • the DS processing unit sends the n slices to the chosen primary DS units when the DS processing unit determines that the number of ready primary DS units is all n (e.g., all pillars ready). The method then continues to the step to create more slices.
  • the DS processing unit sends slices to the available chosen primary DS units when the DS processing unit determines that the number of ready primary DS units is greater than or equal to the read threshold k but is less than all n.
  • the DS processing unit temporarily stores slices by storing slices in temporary memory for any chosen primary DS units that are not available.
  • the DS processing unit determines if the status of any unavailable chosen primary DS units has changed to ready. As illustrated by blocks 819 and 821 , the DS processing unit retrieves the slices from temporary memory and sends the slices to the ready DS unit when the DS processing unit determines that the status of the unavailable chosen primary DS unit has changed to ready. As illustrated by block 823 , the DS processing unit determines if all the temporarily cached slices have been stored in the chosen DS unit and continues to the step of determining if the status has changed when all the cached slices have not been stored in the chosen DS units.
  • a timeout may occur where the DS processing unit gives up on waiting for the ready status to change in which case the DS processing unit may try another DS unit or just not store a pillar of slices (e.g., deleting them from the temporary memory).
  • the DS processing unit method goes back to the step of creating slices when all the cached slices have been stored in the chosen DS units.
  • some or all slices stored in temporary memory may be discarded according to a discard policy.
  • the discard policy may specify that slices are to be discarded after a threshold period of time, based on an amount of available storage, or based on reliability of the data. For example, a data slice may be discarded only when it is no longer possible to use the data slice, when the data slice is no longer needed, or when the data slice is deemed unreliable. Some data slices may be given retention preference over other data slices, so that very data slices associated with reliable data slices already in long term storage may be discarded in favor of data slices that may be needed to correct unreliable data slices.
  • FIG. 9A is a schematic block diagram of another embodiment of a distributed storage system that includes the DS processing unit 16 , the plurality of DS units 36 , and a plurality of associated temporary memories 904 .
  • the DS unit 4 may not be available due to a site outage, a DS unit failure, and/or the network is not available at DS unit 4 site.
  • the DS processing unit 16 may temporarily store new pillar 4 slices in one of the temporary memories 904 , and/or yet another DS unit, for subsequent storage in DS unit 4 .
  • the method of operation to determine where to temporarily store the slices will be discussed in greater detail with reference to FIG. 9B .
  • FIG. 9B is another flowchart illustrating a method 900 of writing to memory where the DS processing unit determines where to store newly created slices when at least one primary DS unit is not available.
  • the method begins as illustrated by block 903 , where the DS processing unit creates the n slices for each data segment for storage. As illustrated by block 905 , the DS processing unit determines the desired primary DS units in which to store the slices based in part on a predetermination of the slice name in the user vault, or in the virtual DSN address to physical location table.
  • the DS processing unit determines the status of the chosen primary DS units based on one or more of a status table lookup and/or a real time query to the DS unit. For example, the status indicates not available if the network is down to the DS unit or if the DS unit is down.
  • the DS processing unit determines the number of primary DS units that are in the ready status.
  • the DS processing unit tries other DS units and returns to the step to determine which DS units when the number of ready primary DS units is less than the read threshold k. Note that the threshold for this scenario may be k+1 or k+2, etc. in another embodiment to further improve the probability of subsequent data object recreation.
  • the DS processing unit sends the n slices to the chosen primary DS units when the DS processing unit determines that the number of ready primary DS units is all n (e.g., all pillars ready).
  • the method 900 then continues to create more slices, as illustrated by block 903 .
  • the DS processing unit sends slices to the available chosen primary DS units when the DS processing unit determines that the number of ready primary DS units is greater than or equal to the read threshold k but is less than all n.
  • the DS processing unit determines which temporary memory 1 - 3 to utilize to temporarily store the slices for the DS unit 4 that is not ready. The determination may be based on one or more of an even rotation across the ready DS unit temporary memories (e.g., temporary/cache memory 1 , then 2 , then 3 , then 1 etc.), one pillar high or low from the DS unit that is not ready, a list, a command, and/or the performance of the temporary memory.
  • the DS processing unit caches slices by storing slices in the chosen temporary memory for any chosen primary DS units that are not available.
  • the DS processing unit determines if the status of any unavailable chosen primary DS units 36 has changed to ready. As illustrated by blocks 919 and 921 , the DS processing unit retrieves the slices from the temporary memory and sends the slices to the ready DS unit when the DS processing unit determines that the status of the unavailable chosen primary DS unit has changed to ready. As illustrated by block 923 , the DS processing unit determines if all the temporarily cached slices have been stored in the chosen DS unit and continues to the step of determining if the status has changed when all the cached slices have not been stored in the chosen DS units.
  • a timeout may occur where the DS processing unit gives up on waiting for the ready status to change in which case the DS processing unit may try another DS unit or just not store a pillar of slices (e.g., deleting them from the temporary memory).
  • the DS processing unit method goes back to the step of creating slices when all the cached slices have been stored in the chosen DS units.
  • FIG. 10 is a schematic block diagram of another embodiment of a distributed storage system that includes the DS processing unit 16 , and a plurality of DS units 36 .
  • the DS units 1 - 4 may each include a matching number of memories 1 - 4 in some embodiments. In another embodiment, the number of memories per DS unit may be 8 , 16 or more.
  • the DS units can include a matching number of memories to facilitate organizing memories across the DS units 1 - 4 as storage groups or stripes 1 - 4 .
  • the stripes 1 - 4 may be physical as shown or logical such that the stripe boundaries are within the memory ranges of the memories.
  • the DS processing unit 16 and/or the DS units determine which memories across the DS units to utilize to store slices of the same data object. Note that the overall system reliability can be improved when the number of logical stripes is minimized such that same data segment slices are contained within the same stripe.
  • a logical stripe may include memory 1 of DS unit 1 , memory 4 of DS unit 2 , memory 2 of DS unit 3 , and memory 3 of DS unit 4 . This embodiment may be undesired as it can lead to lower system reliability since a memory failure can affect many data sets.
  • a logical stripe may include memory 2 of DS unit 1 , memory 2 of DS unit 2 , memory 2 of DS unit 3 , and memory 2 of DS unit 4 .
  • This embodiment may be more desired as it can lead to improved system reliability, since a memory failure can affect a more limited number of data sets.
  • n there are n choose m possible logical stripes where m is the number of memories per DS unit and n is the pillar width of the vault, and “choose” refers to the combinatorial operation for determining the number of distinct k-combinations.
  • the system mean time to data loss (stripe mean time to data loss)/(number of logical stripes). Minimizing the number of logical stripes may improve the system reliability.
  • the DS processing unit and/or DS unit may determine the provisioning and utilization of the memories into logical stripes such as to minimize the number of logical stripes.
  • the DS processing unit and/or DS managing unit provision memory 1 of each of DS unit 1 - 4 to be stripe 1 , memory 2 of each of DS unit 1 - 4 to be stripe 2 , memory 3 of each of DS unit 1 - 4 to be stripe 3 , and memory 4 of each of DS unit 1 - 4 to be stripe 4 .
  • the DS processing unit and/or DS unit determines to store a pillar 1 slice of data segment A at stripe 1 of DS unit 1 (slice Al at memory 1 of DS unit 1 ), slice A 2 at memory 1 of DS unit 2 , slice A 3 at memory 1 of DS unit 3 , and slice A 4 at memory 1 of DS unit 4 .
  • the DS processing unit and/or DS unit determines to store the slices of data segment E in stripe 1 (E 1 -E 4 ), B 1 -B 4 and F 1 -F 4 in stripe 2 , C 1 -C 4 and G 1 -G 4 in stripe 3 , and D 1 -D 4 and H 1 -H 4 in stripe 4 .
  • a method of determining which stripe to utilize is discussed in greater detail with reference to FIG. 11 .
  • every DS unit receives slices from a contiguous set of segments of a data source. So, as illustrated in FIG. 10 , DS unit 1 would receive, in order, A 1 , B 1 , C 1 , D 1 , E 1 , and so on.
  • the striping algorithm can be used to even the load, such that no one memory has to handle all the input/output traffic.
  • FIG. 10 if slices from segments A-D come in at once, all 4 disks may begin storage operations, since each of the 4 memories gets something to store.
  • some embodiments apply a random-like (but deterministic), or round-robin process to select which memory the slice will go to based on its name. It should be a deterministic process so that when reading, the DS unit knows which memory to access to find the source. For example, if the store had 8 disks, it might look at the 3 least significant bits of the segment's name (which would represent any number from 0-7 in binary). This result would determine which of the 8 disks a slice would be stored in.
  • the least significant bits of the input source name are not used, because they are not guaranteed to have a uniform enough distribution.
  • the hash of the source name is used to create something with an even distribution, and, the least significant bits of the hash are examined.
  • Other implementations use the result of taking the remainder when dividing the hash result by a smaller number.
  • FIG. 11 is another flowchart illustrating method 1100 of writing to memory where the DS processing unit and/or DS unit determine which stripe to utilize.
  • the DS unit receives a slice to store from one of the DS processing unit, the user device, the DS managing unit, or the storage integrity processing unit.
  • the slice is accompanied by one or more of the command/request to store it, the slice name, the source name, and or the slice metadata.
  • the DS unit determines the source name either by receiving the source name or deriving it from the slice name.
  • the DS unit calculates a reduced length source name.
  • the reduced length source name can be calculated, for example, using a hash (e.g., CRC) function of the source name which will always be the same number for the same source name (e.g., vault ID, vault gen, resv, and file ID).
  • the reduced length source name can be calculated using other suitable functions, for example, a modulo function.
  • any reduction function that can be used to reduce the original source name to a smaller number that can be used to uniquely identify a particular memory can be used.
  • a reduction function can be chosen to maintain a random distribution among the various memories of a DS unit. The randomness of the file ID ensures that the hash will have desired distancing properties to spread out the slices of data objects evenly across the stripes.
  • the DS unit determines the memory device based on the hash of the source name by truncating the hash to the number of bits required to specify the stripe range. For example, the least two significant bits of the hash may be utilized to specify the memory number.
  • the DS unit updates the local virtual DSN address to physical location table with the memory number before storing the slice in the chosen memory, as illustrated by block 1115
  • the terms “substantially” and “approximately” provides an industry-accepted tolerance for its corresponding term and/or relativity between items. Such an industry-accepted tolerance ranges from less than one percent to fifty percent and corresponds to, but is not limited to, component values, integrated circuit process variations, temperature variations, rise and fall times, and/or thermal noise. Such relativity between items ranges from a difference of a few percent to magnitude differences.
  • the term(s) “coupled to” and/or “coupling” and/or includes direct coupling between items and/or indirect coupling between items via an intervening item (e.g., an item includes, but is not limited to, a component, an element, a circuit, and/or a module) where, for indirect coupling, the intervening item does not modify the information of a signal but may adjust its current level, voltage level, and/or power level.
  • an intervening item e.g., an item includes, but is not limited to, a component, an element, a circuit, and/or a module
  • inferred coupling i.e., where one element is coupled to another element by inference
  • the term “operable to” indicates that an item includes one or more of power connections, input(s), output(s), etc., to perform one or more its corresponding functions and may further include inferred coupling to one or more other items.
  • the term “associated with”, includes direct and/or indirect coupling of separate items and/or one item being embedded within another item.
  • the term “compares favorably”, indicates that a comparison between two or more items, signals, etc., provides a desired relationship. For example, when the desired relationship is that signal 1 has a greater magnitude than signal 2 , a favorable comparison may be achieved when the magnitude of signal 1 is greater than that of signal 2 or when the magnitude of signal 2 is less than that of signal 1 .

Abstract

A distributed storage processing unit can generate data slices and determine metadata for each of the data slices. The metadata includes information that can be used to determine storage diversity preferences, which can include requirements that data slices generated from a common data segment each be stored in memories of the same (or different) type and model, memories with the same (or different) failure rates, memories having fast read (or write) characteristics, and so on. Decisions about which memory units to use for storing data slices can be made based on how closely the characteristics of the memories match the storage diversity preferences. The decision can be made at a distributed storage processing unit that generates the data slices, at a distributed storage unit receiving the data slices for storage, or elsewhere.

Description

    CROSS REFERENCE To RELATED PATENTS
  • This application claims the benefit of U.S. Provisional Application No. 61/246,876, filed Sep. 29, 2009, and entitled “DISTRIBUTED STORAGE NETWORK MEMORY UTILIZATION OPTIMIZATION,” which is incorporated herein in its entirety by reference for all purposes.
  • The present application is related to the following co-pending applications:
      • 1. Utility application Ser. No. 12/______ filed on even date herewith, and entitled “DISTRIBUTED STORAGE NETWORK MEMORY ACCESS BASED ON MEMORY STATE”;
      • 2. Utility application Ser. No. 12/______ filed on even date herewith, and entitled “HANDLING UNAVAILABLE MEMORIES IN DISTRIBUTED STORAGE NETWORK,” and
      • 3. Utility application Ser. No. 12/______ filed on even date herewith, and entitled “DISTRIBUTED STORAGE NETWORK UTILIZING MEMORY STRIPES,”
        all of which are incorporated herein for all purposes.
    STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT—Not Applicable INCORPORATION-BY-REFERENCE OF MATERIAL SUBMITTED ON A COMPACT DISC—Not Applicable BACKGROUND OF THE INVENTION
  • 1. Technical Field of the Invention
  • This invention relates generally to computing and more particularly to storage of information.
  • 2. Description of Related Art
  • Computing systems are known to communicate, process, and store data. Such computing systems range from wireless smart phones to data centers that support millions of web searches, stock trades, or on-line purchases every day. Computing processing is known to manipulate data from one form into another. For instance, raw picture data from an image sensor may be compressed, or manipulated, in accordance with a picture compression standard to produce a standardized compressed picture that can be saved or shared with others. Computer processing capability continues to advance as processing speed advances and software applications that perform the manipulation become more sophisticated.
  • With the advances in computing processing speed and communication speed, computers manipulate real time media from voice to streaming high definition video. Purpose-built communications devices, like the phone, are being replaced by more general-purpose information appliances. For example, smart phones can support telephony communications but they are also capable of text messaging, and accessing the internet to perform functions including email, web browsing, remote applications access, and media communications. Media communications includes telephony voice, image transfer, music files, video files, real time video streaming and more.
  • Each type of computing system is constructed, and hence operates, in accordance with one or more communication, processing, and storage standards. With such standards, and with advances in technology, more and more of the global information content is being converted into electronic formats. For example, more digital cameras are now being sold than film cameras, thus producing more digital pictures. High growth rates exist for web based programming that until recently was all broadcast by just a few over the air television stations and cable television providers. Digital content standards, such as used in pictures, papers, books, video entertainment, home video, all enable this global transformation to a digital format. Electronic content pervasiveness is producing increasing demands on the storage function of computing systems.
  • A typical computer storage function includes one or more memory devices to match the needs of the various operational aspects of the processing and communication functions. For example, a memory device may include solid-state NAND flash, random access memory (RAM), read only memory (ROM), a mechanical hard disk drive. Each type of memory device has a particular performance range and normalized cost. The computing system architecture optimizes the use of one or more types of memory devices to achieve the desired functional and performance goals of the computing system. Generally, the immediacy of access dictates what type of memory device is used. For example, RAM memory can be accessed in any random order with a constant response time. By contrast, memory device technologies that require physical movement such as magnetic discs, tapes, and optical discs, have a variable responses time as the physical movement can take longer than the data transfer.
  • Each type of computer storage system is constructed, and hence operates, in accordance with one or more storage standards. For instance, computer storage systems may operate in accordance with one or more standards including, but not limited to network file system (NFS), flash file system (FFS), disk file system (DFS), small computer system interface (SCSI), internet small computer system interface (iSCSI), file transfer protocol (FTP), and web-based distributed authoring and versioning (WebDAV). An operating systems (OS) and storage standard may specify the data storage format and interface between the processing subsystem and the memory devices. The interface may specify a structure such as directories and files. Typically a memory controller provides an interface function between the processing function and memory devices. As new storage systems are developed, the memory controller functional requirements may change to adapt to new standards.
  • Memory devices may fail, especially those that utilize technologies that require physical movement like a disc drive. For example, it is not uncommon for a disc drive to suffer from bit level corruption on a regular basis, or complete drive failure after an average of three years of use. One common solution is to utilize more costly disc drives that have higher quality internal components. Another solution is to utilize multiple levels of redundant disc drives to abate these issues by replicating the data into two or more copies. One such redundant drive approach is called redundant array of independent discs (RAID). Multiple physical discs comprise an array where parity data is added to the original data before storing across the array. The parity is calculated such that the failure of one or more discs will not result in the loss of the original data. The original data can be reconstructed from the other discs. RAID 5 uses three or more discs to protect data from the failure of any one disc. The parity and redundancy overhead reduces the capacity of what three independent discs can store by one third (n-1=3-2=2 discs of capacity using 3 discs). RAID 6 can recover from a loss of two discs and requires a minimum of four discs with an efficiency of n-2. Typical RAID systems utilize a RAID control to encode and decode the data across the array.
  • Drawbacks of the RAID approach include effectiveness, efficiency and security. As more discs are added, the probability of one or two discs failing rises and is not negligible, especially if more desired less costly discs are used. When one disc fails, it should be immediately replaced and the data reconstructed before a second drive fails. To provide high reliability over a long time period, and if the RAID array is part of a national level computing system with occasional site outages, it is also common to mirror RAID arrays at different physical locations. Unauthorized file access becomes a more acute problem when whole copies of the same file are replicated, either on just one storage system site or at two or more sites. In light of the effectiveness, the efficiency of dedicating 1 to 2 discs per array for the RAID overhead is an issue.
  • Therefore, a need exists to provide a data storage solution that provides more effective timeless continuity of data, minimizes adverse affects of multiple memory elements failures, provides improved security, can be adapted to a wide variety storage system standards and is compatible with computing and communications systems.
  • BRIEF SUMMARY OF THE INVENTION
  • The present invention is directed to apparatus and methods of operation that are further described in the following Brief Description of the Drawings, the Detailed Description of the Invention, and the claims. Various features and advantages of the present invention will become apparent from the following detailed description of the invention made with reference to the accompanying drawings.
  • BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWING(S)
  • FIG. 1 is a schematic block diagram of an embodiment of a computing system in accordance with the invention;
  • FIG. 2 is a schematic block diagram of an embodiment of a computing core in accordance with the invention;
  • FIG. 3 is a schematic block diagram of an embodiment of a distributed storage processing unit in accordance with the invention;
  • FIG. 4 is a schematic block diagram of an embodiment of a distributed storage unit in accordance with the invention;
  • FIG. 5 is a flowchart illustrating the reading and writing of memory;
  • FIG. 6 is a state transition diagram illustrating the reading and writing of memory;
  • FIG. 7 is a flowchart illustrating the writing of memory;
  • FIG. 8A is a schematic block diagram of an embodiment of a distributed storage system in accordance with the invention;
  • FIG. 8B is another flowchart illustrating the writing of memory;
  • FIG. 9A is a schematic block diagram of another embodiment of a distributed storage system in accordance with the invention;
  • FIG. 9B is another flowchart illustrating the writing of memory;
  • FIG. 10 is a schematic block diagram of another embodiment of a distributed storage system in accordance with the invention; and
  • FIG. 11 is another flowchart illustrating the writing of memory.
  • DETAILED DESCRIPTION OF THE INVENTION
  • FIG. 1 is a schematic block diagram of a computing system 10 that includes one or more of a first type of user devices 12, one or more of a second type of user devices 14, at least one distributed storage (DS) processing unit 16, at least one DS managing unit 18, at least one storage integrity processing unit 20, and a distributed storage network (DSN) memory 22 coupled via a network 24. The network 24 may include one or more wireless and/or wire lined communication systems; one or more private intranet systems and/or public internet systems; and/or one or more local area networks (LAN) and/or wide area networks (WAN).
  • The DSN memory 22 includes a plurality of distributed storage (DS) units 36 for storing data of the system. Each of the DS units 36 includes a processing module and memory and may be located at a geographically different site than the other DS units (e.g., one in Chicago, one in Milwaukee, etc.). The processing module may be a single processing device or a plurality of processing devices. Such a processing device may be a microprocessor, micro-controller, digital signal processor, microcomputer, central processing unit, field programmable gate array, programmable logic device, state machine, logic circuitry, analog circuitry, digital circuitry, and/or any device that manipulates signals (analog and/or digital) based on hard coding of the circuitry and/or operational instructions. The processing module may have an associated memory and/or memory element, which may be a single memory device, a plurality of memory devices, and/or embedded circuitry of the processing module. Such a memory device may be a read-only memory, random access memory, volatile memory, non-volatile memory, static memory, dynamic memory, flash memory, cache memory, and/or any device that stores digital information. Note that if the processing module includes more than one processing device, the processing devices may be centrally located (e.g., directly coupled together via a wired and/or wireless bus structure) or may be distributedly located (e.g., cloud computing via indirect coupling via a local area network and/or a wide area network). Further note that when the processing module implements one or more of its functions via a state machine, analog circuitry, digital circuitry, and/or logic circuitry, the memory and/or memory element storing the corresponding operational instructions may be embedded within, or external to, the circuitry comprising the state machine, analog circuitry, digital circuitry, and/or logic circuitry. Still further note that, the memory element stores, and the processing module executes, hard coded and/or operational instructions corresponding to at least some of the steps and/or functions illustrated in FIGS. 1-11.
  • Each of the user devices 12-14, the DS processing unit 16, the DS managing unit 18, and the storage integrity processing unit 20 may be a portable computing device (e.g., a social networking device, a gaming device, a cell phone, a smart phone, a personal digital assistant, a digital music player, a digital video player, a laptop computer, a handheld computer, a video game controller, and/or any other portable device that includes a computing core) and/or a fixed computing device (e.g., a personal computer, a computer server, a cable set-top box, a satellite receiver, a television set, a printer, a fax machine, home entertainment equipment, a video game console, and/or any type of home or office computing equipment). Such a portable or fixed computing device includes a computing core 26 and one or more interfaces 30, 32, and/or 33. An embodiment of the computing core 26 will be described with reference to FIG. 2.
  • With respect to the interfaces, each of the interfaces 30, 32, and 33 includes software and/or hardware to support one or more communication links via the network 24 and/or directly. For example, interfaces 30 support a communication link (wired, wireless, direct, via a LAN, via the network 24, etc.) between the first type of user device 14 and the DS processing unit 16. As another example, DSN interface 32 supports a plurality of communication links via the network 24 between the DSN memory 22 and the DS processing unit 16, the first type of user device 12, and/or the storage integrity processing unit 20. As yet another example, interface 33 supports a communication link between the DS managing unit 18 and any one of the other devices and/or units 12, 14, 16, 20, and/or 22 via the network 24.
  • In general, the system 10 supports three primary functions: distributed network data storage management, distributed data storage and retrieval, and data storage integrity verification. In accordance with these three primary functions, data can be distributedly stored in a plurality of physically different locations and subsequently retrieved in a reliable and secure manner regardless of failures of individual storage devices, failures of network equipment, the duration of storage, the amount of data being stored, attempts at hacking the data, etc.
  • The DS managing unit 18 performs the distributed network data storage management functions, which include establishing distributed data storage parameters, performing network operations, performing network administration, and/or performing network maintenance. The DS managing unit 18 establishes the distributed data storage parameters (e.g., allocation of virtual DSN memory space, distributed storage parameters, security parameters, billing information, user profile information, etc.) for one or more of the user devices 12-14 (e.g., established for individual devices, established for a user group of devices, established for public access by the user devices, etc.). For example, the DS managing unit 18 coordinates the creation of a vault (e.g., a virtual memory block) within the DSN memory 22 for a user device (for a group of devices, or for public access). The DS managing unit 18 also determines the distributed data storage parameters for the vault. In particular, the DS managing unit 18 determines a number of slices (e.g., the number that a data segment of a data file and/or data block is partitioned into for distributed storage) and a threshold value (e.g., the minimum number of slices required to reconstruct the data segment).
  • As another example, the DS managing module 18 may create and store locally or within the DSN memory 22 user profile information. The user profile information includes one or more of authentication information, permissions, and/or the security parameters. The security parameters may include one or more of encryption/decryption scheme, one or more encryption keys, key generation scheme, and data encoding/decoding scheme.
  • As yet another example, the DS managing unit 18 may create billing information for a particular user, user group, vault access, public vault access, etc. For instance, the DS managing unit 18 may track the number of times user accesses a private vault and/or public vaults, which can be used to generate a per-access bill. In another instance, the DS managing unit 18 tracks the amount of data stored and/or retrieved by a user device and/or a user group, which can be used to generate a per-data-amount bill.
  • The DS managing unit 18 also performs network operations, network administration, and/or network maintenance. As at least part of performing the network operations and/or administration, the DS managing unit 18 monitors performance of the devices and/or units of the system 10 for potential failures, determines the devices and/or unit's activation status, determines the devices' and/or units' loading, and any other system level operation that affects the performance level of the system 10. For example, the DS managing unit 18 may receive and aggregate network management alarms, alerts, errors, status information, performance information, and messages from the devices 12-14 and/or the units 16, 20, 22. For example, the DS managing unit 18 may receive a simple network management protocol (SNMP) message regarding the status of the DS processing unit 16.
  • The DS managing unit 18 performs the network maintenance by identifying equipment within the system 10 that needs replacing, upgrading, repairing, and/or expanding. For example, the DS managing unit 18 may determine that the DSN memory 22 needs more DS units 36 or that one or more of the DS units 36 needs updating.
  • The second primary function of distributed data storage and retrieval function begins and ends with a user device 12-14. For instance, if a second type of user device 14 has a data file 38 and/or data block 40 to store in the DSN memory 22, it send the data file 38 and/or data block 40 to the DS processing unit 16 via its interface 30. As will be described in greater detail with reference to FIG. 2, the interface 30 functions to mimic a conventional operating system (OS) file system interface (e.g., network file system (NFS), flash file system (FFS), disk file system (DFS), file transfer protocol (FTP), web-based distributed authoring and versioning (WebDAV), etc.) and/or a block memory interface (e.g., small computer system interface (SCSI), internet small computer system interface (iSCSI), etc.). In addition, the interface 30 may attach a user identification code (ID) to the data file 38 and/or data block 40.
  • The DS processing unit 16 receives the data file 38 and/or data block 40 via its interface 30 and performs a distributed storage (DS) process 34 thereon. The DS processing 34 begins by partitioning the data file 38 and/or data block 40 into one or more data segments, which is represented as Y data segments. For example, the DS processing 34 may partition the data file 38 and/or data block 40 into a fixed byte size segment (e.g., 21 to 2n bytes, where n=>2) or a variable byte size (e.g., change byte size from segment to segment, or from groups of segments to groups of segments, etc.).
  • For each of the Y data segments, the DS processing 34 error encodes (e.g., forward error correction (FEC), information dispersal algorithm, or error correction coding) and slices (or slices then error encodes) the data segment into a plurality of error coded (EC) data slices 42-48, which is represented as X slices per data segment. The number of slices (X) per segment, which corresponds to a number of pillars n, is set in accordance with the distributed data storage parameters and the error coding scheme. For example, if a Reed-Solomon (or other FEC scheme) is used in an n/k system, then a data segment is divided into n slices, where k number of slices is needed to reconstruct the original data (i.e., k is the threshold). As a few specific examples, the n/k factor may be 5/3; 6/4; 8/6; 8/5; 16/10.
  • For each slice 42-48, the DS processing unit 16 creates a unique slice name and appends it to the corresponding slice 42-48. The slice name includes universal DSN memory addressing routing information (e.g., virtual memory addresses in the DSN memory 22) and user-specific information (e.g., user ID, file name, data block identifier, etc.).
  • The DS processing unit 16 transmits the plurality of EC slices 42-48 to a plurality of DS units 36 of the DSN memory 22 via the DSN interface 32 and the network 24. The DSN interface 32 formats each of the slices for transmission via the network 24. For example, the DSN interface 32 may utilize an internet protocol (e.g., TCP/IP, etc.) to packetize the slices 42-48 for transmission via the network 24.
  • The number of DS units 36 receiving the slices 42-48 is dependent on the distributed data storage parameters established by the DS managing unit 18. For example, the DS managing unit 18 may indicate that each slice is to be stored in a different DS unit 36. As another example, the DS managing unit 18 may indicate that like slice numbers of different data segments are to be stored in the same DS unit 36. For example, the first slice of each of the data segments is to be stored in a first DS unit 36, the second slice of each of the data segments is to be stored in a second DS unit 36, etc. In this manner, the data is encoded and distributedly stored at physically diverse locations to improved data storage integrity and security. Further examples of encoding the data segments will be provided with reference to one or more of FIGS. 2-11.
  • Each DS unit 36 that receives a slice 42-48 for storage translates the virtual DSN memory address of the slice into a local physical address for storage. Accordingly, each DS unit 36 maintains a virtual to physical memory mapping to assist in the storage and retrieval of data.
  • The first type of user device 12 performs a similar function to store data in the DSN memory 22 with the exception that it includes the DS processing. As such, the device 12 encoded and slices the data file and/or data block it has to store. The device then transmits the slices 35 to the DSN memory via its DSN interface 32 and the network 24.
  • For a second type of user device 14 to retrieve a data file or data block from memory, it issues a read command via its interface 30 to the DS processing unit 16. The DS processing unit 16 performs the DS processing 34 to identify the DS units 36 storing the slices of the data file and/or data block based on the read command. The DS processing unit 16 may also communicate with the DS managing unit 18 to verify that the user device 14 is authorized to access the requested data.
  • Assuming that the user device is authorized to access the requested data, the DS processing unit 16 issues slice read commands to at least a threshold number of the DS units 36 storing the requested data (e.g., to at least 10 DS units for a 16/10 error coding scheme). Each of the DS units 36 receiving the slice read command, verifies the command, accesses its virtual to physical memory mapping, retrieves the requested slice, or slices, and transmits it to the DS processing unit 16.
  • Once the DS processing unit 16 has received a threshold number of slices for a data segment, it performs an error decoding function and de-slicing to reconstruct the data segment. When Y number of data segments has been reconstructed, the DS processing unit 16 provides the data file 38 and/or data block 40 to the user device 14. Note that the first type of user device 12 performs a similar process to retrieve a data file and/or data block.
  • The storage integrity processing unit 20 performs the third primary function of data storage integrity verification. In general, the storage integrity processing unit 20 periodically retrieves slices 45 of a data file or data block of a user device to verify that one or more slices has not been corrupted or lost (e.g., the DS unit failed). The retrieval process mimics the read process previously described.
  • If the storage integrity processing unit 20 determines that one or more slices is corrupted or lost, it rebuilds the corrupted or lost slice(s) in accordance with the error coding scheme. The storage integrity processing unit 20 stores the rebuild slice, or slices, in the appropriate DS unit(s) 36 in a manner that mimics the write process previously described.
  • FIG. 2 is a schematic block diagram of an embodiment of a computing core 26 that includes a processing module 50, a memory controller 52, main memory 54, a video graphics processing unit 55, an input/output (TO) controller 56, a peripheral component interconnect (PCI) interface 58, at least one IO device interface module 62, a read only memory (ROM) basic input output system (BIOS) 64, and one or more memory interface modules. The memory interface module(s) includes one or more of a universal serial bus (USB) interface module 66, a host bus adapter (HBA) interface module 68, a network interface module 70, a flash interface module 72, a hard drive interface module 74, and a DSN interface module 76. Note the DSN interface module 76 and/or the network interface module 70 may function as the interface 30 of the user device 14 of FIG. 1. Further note that the IO device interface module 62 and/or the memory interface modules may be collectively or individually referred to as IO ports.
  • The processing module 50 may be a single processing device or a plurality of processing devices. Such a processing device may be a microprocessor, micro-controller, digital signal processor, microcomputer, central processing unit, field programmable gate array, programmable logic device, state machine, logic circuitry, analog circuitry, digital circuitry, and/or any device that manipulates signals (analog and/or digital) based on hard coding of the circuitry and/or operational instructions. The processing module may have an associated memory and/or memory element, which may be a single memory device, a plurality of memory devices, and/or embedded circuitry of the processing module. Such a memory device may be a read-only memory, random access memory, volatile memory, non-volatile memory, static memory, dynamic memory, flash memory, cache memory, and/or any device that stores digital information. Note that if the processing module includes more than one processing device, the processing devices may be centrally located (e.g., directly coupled together via a wired and/or wireless bus structure) or may be distributedly located (e.g., cloud computing via indirect coupling via a local area network and/or a wide area network). Further note that when the processing module implements one or more of its functions via a state machine, analog circuitry, digital circuitry, and/or logic circuitry, the memory and/or memory element storing the corresponding operational instructions may be embedded within, or external to, the circuitry comprising the state machine, analog circuitry, digital circuitry, and/or logic circuitry. Still further note that, the memory element stores, and the processing module executes, hard coded and/or operational instructions corresponding to at least some of the steps and/or functions illustrated in FIGS. 1-11.
  • FIG. 3 is a schematic block diagram of an embodiment of a dispersed storage (DS) processing unit 16 and/or of the DS processing module 34 of user device 12 (see FIG. 1). The DS processing unit 16 includes a gateway module 107, an access module 109, a grid module 84, a storage module 113, and a bypass/feedback path. The DS processing unit 16 may also include an interface 30 and the DSnet interface 32.
  • In an example of storing data, the gateway module 107 of the DS processing unit 16 receives an incoming data object (e.g., a data file, a data block, an EC data slice, etc.), authenticates the user associated with the data object, obtains user information of the authenticated user, and assigns a source name to the data object in accordance with the user information. To authenticate the user, the gateway module 107 verifies the user ID 119 with the managing unit 18 (see FIG. 1) and/or another authenticating unit. If the user ID is verified, the gateway module 107 retrieves the user information from the managing unit 18 (see FIG. 1), the user device 14, and/or the other authenticating unit based on the user ID.
  • The user information includes a vault identifier, operational parameters, and user attributes (e.g., user data, billing information, etc.). A vault identifier identifies a vault, which is a virtual memory space that maps to a set of DS storage units 36. For example, vault 1 (i.e., user 1's DSN memory space) includes eight DS storage units (X=8 wide) and vault 2 (i.e., user 2's DSN memory space) includes sixteen DS storage units (X=16 wide). The operational parameters may include an error coding algorithm, the width n (number of pillars X or slices per segment for this vault), a read threshold T, an encryption algorithm, a slicing parameter, a compression algorithm, an integrity check method, caching settings, parallelism settings, and/or other parameters that may be used to access the DSN memory layer.
  • The gateway module 107 determines the source name to associate with the data object based on the vault identifier and the data object. For example, the source name may contain a data name (block number or a file number), the vault generation number, a reserved field, and a vault identifier. The data name may be randomly assigned but is associated with the user data object.
  • The gateway module 107 may utilize the bypass/feedback path to transfer an incoming EC data slice to another DS storage unit 36 (see FIG. 1) when the DS processing module 34 determines that the EC data should be transferred. Alternatively, or in addition to, the gateway module 60 may use the bypass/feedback path to feedback an EC slice for sub-slicing.
  • The access module 109 receives the data object and creates a series of data segments 1 through Y therefrom. The number of segments Y may be chosen or random based on a selected segment size and the size of the data object. For example, if the number of segments is chosen to be a fixed number, then the size of the segments varies as a function of the size of the data object. For instance, if the data object is an image file of 4,194,304 eight bit bytes (e.g., 33,554,432 bits) and the number of segments Y=131,072, then each segment is 256 bits or 32 bytes. As another example, if segment sized is fixed, then the number of segments Y varies based on the size of data object. For instance, if the data object is an image file of 4,194,304 bytes and the fixed size of each segment is 4,096 bytes, the then number of segments Y=1,024. Note that each segment is associated with the source name.
  • The grid module 84, as previously discussed, may pre-manipulate (e.g., compression, encryption, cyclic redundancy check (CRC), etc.) the data segment before creating X error coded data slices for each data segment. The grid module 84 creates XY error coded data slices for the Y data segments of the data object. The grid module 84 adds forward error correction bits to the data segment bits in accordance with an error coding algorithm (e.g., Reed-Solomon, Convolution encoding, Trellis encoding, etc.) to produce an encoded data segment. The grid module 84 determines the slice name and attaches the unique slice name to each EC data slice.
  • The number of pillars, or slices X per data segment (e.g., X=16) is chosen as a function of the error coding objectives. The DS processing module may utilize different error coding parameters for EC data slices and EC data sub-slices based on guidance from one or more of a user vault (e.g., stored parameters for this user), a command from the DS managing unit or other system element, priority of the EC data slice, type of data in the EC data slice, and/or retrieval speed requirements. A read threshold T (e.g., T=10) of the error coding algorithm is the minimum number of error-free error coded data slices required to be able to reconstruct a data segment. The DS processing unit can compensate for X-T (e.g., 16-10=6) missing, out-of-date, and/or corrupted error coded data slices per data segment.
  • The grid module 84 receives each data segment 1-Y and, for each data segment generates X number of error coded (EC) slices using an error coding function. The grid module 84 also determines the DS storage units 36 for storing the EC data slices based on a dispersed storage memory mapping associated with the user's vault and/or DS storage unit 36 attributes, which include availability, self-selection, performance history, link speed, link latency, ownership, available DSN memory, domain, cost, a prioritization scheme, a centralized selection message from another source, a lookup table, data ownership, and/or any other factor to optimize the operation of the computing system.
  • The storage module 113 may perform integrity checks on the EC data slices and then transmit the EC data slices 1 through X of each segment 1 through Y to the DS storage units. The DS storage units 36 may store the EC data slices and locally keep a table to convert virtual DSN addresses into physical storage addresses. Note that the number of DS storage units 36 is equal to or greater than the number of pillars (slices X per segment) so that no more than one error coded data slice of the same data segment is stored on the same DS storage unit 36. Further note that EC data slices of the same pillar number but of different segments (e.g., EC data slice 1 of data segment 1 and EC data slice 1 of data segment 2) may be stored on the same or different DS storage units 36 (see FIG. 1).
  • In an example of a read operation, the user device 10 or 12 sends a read request to the DS processing unit 14, which authenticates the request. When the request is authentic, the DS processing unit 14 sends a read message to each of the DS storage units 36 storing slices of the data object being read. The slices are received via the DSnet interface 34 and processed by the storage module 113, which performs a parity check and provides the slices to the grid module 84. The grid module 84 de-slices and decodes the slices of a data segment to reconstruct the data segment. The access module reconstructs the data object from the data segments and the gateway module 107 formats the data object for transmission to the user device.
  • FIG. 4 is a schematic block diagram of an embodiment of a distributed storage unit 36 that includes a storage unit control module 402, a plurality of memories 403, 404, 405, and 406, a plurality of parity memories 408 and 409, and a cache memory 415. In another embodiment, there may be 8, 16, or more memories.
  • The storage unit control module 402 may be implemented with the computing core of FIG. 2. The memories 403-406 may be one or more of a magnetic hard disk, NAND flash, read only memory, optical disk, and/or any other type of read-only, or read/write memory. The memories may be implemented as part of or outside of the DS storage unit. For example, memory 1 may be implemented in the DS unit and memory 4 may be implemented in a remote server (e.g., a different DS unit operably coupled to the DS unit via the network). In an example, memories 403-406 and parity memories 408-409 are implemented with the magnetic hard disk technology and the cache memory 415 is implemented with the NAND flash technology.
  • In some embodiments, a DS unit includes cache memory 415 implemented using a single solid state drive (SSD). In other embodiments, all of the memories are implemented using the same type of device, and one or more of the memories is temporarily selected for use as “cache memory” for purposes of temporarily storing data to be written. The temporarily selected memory can serve as a cache memory until the DS unit shifts responsibility for caching writes to another memory.
  • The storage unit control module 402 includes the DSnet interface 32 and a processing module. The storage unit control module 402 may be operably coupled to the computing system via the DSnet interface 32 via the network. The storage unit control module 402 may receive EC data slices to store via the DSnet interface 32. In an embodiment, the storage unit control module 402 determines where (e.g., which address on which of the memories) to store the received EC data slice. The determination may be based on one or more of the metadata, a command (e.g., from the DS processing unit indicating which memory type to use), a type of data indicator, a priority indicator, a memory state indicator, available memory, memory performance data, memory cost data, the memory characteristics, and/or any other parameter to facilitate desired levels of efficiency and performance. The memory state may indicate whether the memory is in a write only state, a read only state, a write with read priority state, or some other state to indicate the availability.
  • The storage unit control module 402 creates and maintains a local virtual DSN address to physical memory table. The storage unit control module 402 determines where previously stored EC data slices are located based on the local virtual DSN address to physical memory table upon receiving a retrieve command via the network. The storage unit control module 402 may save activity records (e.g., memory utilization, errors, stores, retrievals, etc.) as logs in any of the memories.
  • The storage unit control module 402 may utilize the parity memories 408-409 to store and retrieve parity across the data stored in memories 403-406. The storage unit control module 402 may immediately recreate a slice that is stored in a memory in the write only state based on reading the other memories in the read only state, reading the parity memory 1 and/or parity memory 2, and calculating the desired slice. The storage unit control module 402 may temporarily pair a write only state memory 403-406 with a write only state parity memory 408-409 to enable rapid writes of new slices (e.g., write a slice to memory 1 and write the parity to parity memory 1), while another parity memory in the read only state may be available to provide the needed parity to reconstruct slices that are stored on the write only state memory.
  • In an example, the storage unit control module 402 may choose memory 1 (e.g., a magnetic hard disk drive) to store the received EC data slice since memory 1 is in a write only state (e.g., available immediately), the memories 2-4 are in the read only state, parity memory 1 is paired with memory 1 in the write only state, parity memory 2 is in the ready only state, and the memory 1 memory characteristics favorably match the metadata of the EC data slice, including performance, efficiency, cost, and response time. The storage unit control module 402 queues a read request in the cache memory when the requested slice is in the memory 1 (but in the write state). The storage unit control module 402 may process the queued read request for memory 1 by retrieving the request from the cache memory, reading the memories 2-4 (e.g., the same memory stripe or common address range across each), reading the party memory 2, and calculating the desired slice.
  • Note that the storage unit control module 402 may queue write requests and slices when a desired memory 403-406 is in the read only state. The storage unit control module may subsequently change the state of memory 1 from write only to the read only state, or the write with read priority state to enable processing of the queued read request. Note that the DS unit 36 can immediately retrieve slices where the slices are stored in memories in the read only state, or in the write with read priority state (e.g., memories 2-4). Further note that the DS unit 36 may rotate the write only state amongst the memories 1-4 and the parity memories 1-2 from time to time to even out the cumulative storage and optimize performance. A method to choose the memories and change the memory state will be discussed in greater detail with reference to FIGS. 5-11.
  • FIG. 5 is a flowchart illustrating a method 500 of reading and writing to memory where the DS unit 36 (see FIG. 4) may control the DS unit memory state and memory utilization to optimize the performance of the memory.
  • The method begins where the storage unit control module 402 (see FIG. 4) checks for a received request. As illustrated by block 505, the DS unit may receive the request from one or more of the DS processing unit 16, the user device 12, the storage integrity processing unit 20, and/or the DS managing unit 18 (see FIG. 1). As illustrated by block 507, the storage unit control module determines the request type based on the request when the request is received. The method branches to block 532, which illustrates receiving a slice to store when the storage unit control module determines the request type is a write request.
  • As illustrated by block 509, the storage unit control module determines the slice location and state when the request type is a read request. As illustrated by block 511, the determination is based in part on accessing the local virtual DSN address to physical location table to identify the memory, the address, and the memory state. As illustrated by block 513, the storage unit control module retrieves the slice based on the memory and address when the memory state is the read state. The storage unit control module sends the slice to the requester and the method branches back to look for more requests.
  • As illustrated by block 515, the storage unit control module determines the method to read the slice when the memory state is the write state. Note that in this state the memory is only writing at this time to optimize the throughput performance of the memory requiring the requested slice to be obtained in another way other than reading it directly from the memory where the slice was initially stored (e.g., which may disrupt the write state performance when the memory is a hard disk drive). As illustrated by block 519, the determination of the method to read the slice is based on one or more of a predetermination, a command, a DS unit status indicator, a loading indicator for the memories in the read state, a priority indicator, and/or any other indicator to optimize the memory performance. As illustrated by block 517, the storage unit control module may send a read request response message to the requester where the response denies the request when the storage unit control module determines the method to be to utilize another DS unit. Note that in this scenario the DS unit does not return the requested slice to the requester but instead informs the requester that no slice will be returned. The requester must rely on reconstructing the original data object based on the retrieving the slices from the other pillars and performing the de-slicing and decoding steps. In another embodiment, the requester may repeat the read request to the DS unit with a priority indicator set when the process to reconstruct the data object fails since a read threshold of k good slices are not retrieved from the DS units.
  • In various embodiments, including embodiments in which a DS unit uses an SSD cache or where responsibility for caching writes is delegated to various different memories within a DS unit, the DS unit always responds to read requests, and implementation of block 517 is not required.
  • As illustrated by block 521, the storage unit control module may reconstruct the slice from a reverse parity operation based on reading a portion of the memories (e.g., a logical stripe across the memories) and parity memory in the read state when the storage unit control module determines the method to be to utilize the DS unit now. As illustrated by block 523, the storage unit control module sends the slice to the requester and returns to the step to look for received requests.
  • Handling the write request begins, as illustrated by block 532, with the storage unit control module receiving the slice to store in the write request. As illustrated by block 534, the storage unit control module determines the present write state memory based on the local virtual DSN address to physical address table. As illustrated by block 536, the storage unit control module stores the slice in the write state memory and updates the write parity memory by reading a corresponding portion of the read state memories (e.g., same logical stripe across the memories) and calculating the parity across the slice just written to the write state memory and the read state memories. The storage unit control module stores the parity to the write state parity memory, as shown by block 538.
  • As illustrated by block 540, the storage unit control module determines if it is time to rotate the write state memory and write state parity memory to different memories. The determination may be based on one or more of a timer expiration since the last rotation, a command, a memory utilization indicator (e.g., the present write state memory is filling up), a read request history indicator (e.g., many read requests for slices in the write state memory), and/or any other indicator to optimize the memory performance. As illustrated by block 542, the method branches back to look for received requests when the storage unit control module determines it is not time to rotate the write state memory.
  • As illustrated by block 544, the storage unit control module determines the next write state memory and write state parity memory when the storage unit control module determines it is time to rotate the write state memory. The determination may be based on one or more of identifying which memory was in the write state least recently, a predetermination, a rotation order indicator, a command, a memory utilization indicator (e.g., choose a memory with the most available unused space), a read request history indicator (e.g., avoid a memory with a higher read request frequency than other memories), and/or any other indicator to optimize the memory performance. The storage unit control module updates the local virtual DSN address to physical location table with the chosen write state memory and write state parity memory. As illustrated by block 546, the storage unit control module updates the local virtual DSN address to physical location table to modify the state of the previous write state memory and write state parity memory from write state to the read state. Additionally, slices can be moved back to their proper drives. The method branches back to look for received requests.
  • In another embodiment, the number of write state memories may be two or more to further improve the write performance of the DS unit. The storage unit control module may only rotate one memory at a time from the write state to the read state or the storage unit control module may rotate more than one memory at a time from the write state to the read state.
  • FIG. 6 is a state transition diagram 600 illustrating the reading and writing of memory where the DS unit may control the DS unit memory state 601 and memory utilization to optimize the performance of the memory. There are three states of the memory: the read only state 607, the write only state 603, and the write state with read priority 605.
  • The storage unit control module determines the memory state and processes received read and write requests based on the memory state to optimize the memory performance. For example, when the memory is in the read only state 607, the storage unit control module processes only read requests, unless too many write requests are pending (e.g., the number write requests is greater than a high threshold). In another example, when the memory is in the write only state 603, the storage unit control module processes only write requests until the pending write requests are reduced to a low threshold level. In another example, when the memory is in the write state with read priority 605, the storage unit control module opportunistically processes any pending write requests unless there are pending read requests.
  • In various embodiments, including embodiments in which a DS unit uses an SSD cache or where responsibility for caching writes is delegated to various different memories within a DS unit, the DS unit always responds to read requests. In such embodiments, a particular piece of memory being in write only mode 603 means that a read will be delayed, and data will always be stored immediately in read cache memory.
  • Note that in all memory states 601, the storage unit control module queues received read requests into a read queue and received write requests into a write queue by storing the request (and slice in the case of a write request) in the cache memory as indicated by the upper right portion of FIG. 6. The requests may be subsequently de-queued and processed as discussed below.
  • Starting with the read only state, the storage unit control module determines if the read queue is not empty and de-queues the read request, determines the memory location, retrieves the slice, and sends the slice to the requester when the storage unit control module determines the read queue is not empty. The storage unit control module determines if the write queue is above the high threshold of write requests while the memory is in the read only state. The storage unit control module changes the state of the memory from the read only state to the write only state when the storage unit control module determines that the write queue is above the high threshold of write requests. The storage unit control module determines if the read queue is empty while the memory is in the read only state. The storage unit control module changes the state of the memory from the read only state to the write state with read priority when the storage unit control module determines that the read queue is empty.
  • While in the write only state (e.g., the second state of three states) the storage unit control module determines if the write queue is not empty and de-queues the write request with slice from the cache memory, determines the memory location, stores the slice, and updates the local virtual DSN address to physical storage table when the storage unit control module determines the write queue is not empty. The storage unit control module determines if the write queue is below the low threshold of write requests while the memory is in the write only state. The storage unit control module changes the state of the memory from the write only state to the read only state when the storage unit control module determines that the write queue is below the low threshold of write requests.
  • While in the write state with read priority (e.g., the third state of three states) the storage unit control module determines if the write queue is not empty and de-queues the write request with slice from the cache memory, determines the memory location, stores the slice, and updates the local virtual DSN address to physical storage table when the storage unit control module determines the write queue is not empty. The storage unit control module determines if the read queue is not empty while the memory is in the write state with read priority. The storage unit control module changes the state of the memory from the write state with read priority to the read only state when the storage unit control module determines that the read queue is not empty.
  • FIG. 7 is a flowchart illustrating a method 700 of writing memory where the DS processing unit (or DS unit) may employ a memory diversity scheme to choose memories to store slices such that the overall system reliability is improved. For example, the memory diversity scheme may ensure that a read threshold of k slices are stored in pillar memories that are each of a different model to avoid unrecoverable data due to a potentially common memory design defect.
  • As illustrated by block 701, the DS processing unit creates the slices for distributed storage. As illustrated by block 703, the DS processing unit determines the slice metadata based on one or more of a file type, file size, priority, a security index, estimated storage time, estimated time between retrievals and more. As illustrated by block 705, the DS processing unit determines the similarity requirements and difference requirements, sometimes referred to as diversity preferences, based on the metadata. Similarity requirements drive similar attributes of the pillar memory choices and difference requirements drive difference attributes of the pillar memory choices. For example, a preference or requirement for a relatively short estimated time between retrievals may drive pillar memory choices that all share a similar fast retrieval characteristic to speed frequent retrievals. Other examples of similarity preferences and requirements may include similar cost and similar capacity. In another example, a preference or requirement for very high reliability may drive pillar memory choices that all have a different memory model to improve the reliability of retrievals. Other examples of difference requirements and preferences may include different operating systems and different installation sites.
  • As illustrated by block 709, the DS processing unit determines the DS unit memory characteristics for one or more candidate DS units. The determination may be via a table lookup or a real time request to each DS unit to query for the memory characteristics. The memory characteristics may include one or more of memory model, memory type, total capacity, available capacity, access speed, error history, estimated mean time between failures, actual mean time between failures, and/or hours of operation.
  • As illustrated by block 711, the DS processing unit sorts the DS units that favorably match the similarity requirements and difference requirements based on comparing the requirements to the memory characteristics. For example, DS units with memory that has a fast access memory characteristic may be sorted to favorably match the fast access similarity requirement. In another example, DS units with memory that has a different model memory characteristic may be sorted to favorably match the reliability-driven different-model requirement or preference.
  • As illustrated by block 713, the DS processing unit determines the best match of DS unit memories to the diversity preferences or requirements based on the sort if possible, or at least a favorable match. For example, the DS processing unit may choose at most n-k DS unit memories with the same model, similar error histories, or similar total hours to improve the reliability of data object retrieval. In other words, the DS unit may choose the read threshold k of DS unit memories that has the most different models, error histories, and total hours as the memory diversity scheme.
  • As illustrated by block 715, the DS processing unit sends the slices to the chosen DS units with the best match of memory characteristics to requirements and updates the virtual DSN address to physical location table with the locations of the slices. In at least some embodiments where a DS unit includes multiple memory devices, the DS unit may implement similar functionality to that discussed above to select available memory units that favorably match the diversity preferences determined from the slice metadata.
  • FIG. 8A is a schematic block diagram of an embodiment of a distributed storage system that includes the DS processing unit 16, a temporary memory 802, and a plurality of DS units 36. Consider an example in which DS unit 4 may not be available due to a site outage, a DS unit failure, and/or the network is not available at DS unit 4 site. The DS processing unit 16 may temporarily store new pillar 4 slices in the temporary memory, and/or yet another DS unit, for subsequent storage in DS unit 4. As used herein, the term “cache memory” refers to a memory that can be used temporarily store information and includes but is not limited to, cache memories such as those included in various processor architectures, memory specifically designated as cache memory, and the like. The term “cache memory” is also used in a less rigorous sense to refer to any type of memories used for substantially non-permanent information storage. The method of operation to determine where to temporarily store the slices will be discussed in greater detail with reference to FIGS. 8B and 9B.
  • FIG. 8B is another flowchart illustrating a method 800 of writing to memory where the DS processing unit 16 determines where to store newly created slices when at least one primary DS unit 36 is not available.
  • The method 800 begins as illustrated by block 803, where the DS processing unit creates the n slices for each data segment for storage. As illustrated by block 805, the DS processing unit determines the desired primary DS units in which to store the slices based in part on a predetermination of the slice name in the user vault, or in the virtual DSN address to physical location table.
  • As illustrated by block 807, the DS processing unit determines the status of the chosen primary DS units based on one or more of a status table lookup and/or a real time query to the DS unit. For example, the status indicates not available if the network is down to the DS unit, or if the DS unit is down. As illustrated by block 810, the DS processing unit determines the number of primary DS units that are in the ready status. As illustrated by block 809, the DS processing unit tries other DS units and returns to the step to determine which DS units when the number of ready primary DS units is less than the read threshold k. Note that the threshold for this scenario may be k+1, k+2, or etc. in another embodiment to further improve the probability of subsequent data object recreation.
  • As illustrated by block 811, the DS processing unit sends the n slices to the chosen primary DS units when the DS processing unit determines that the number of ready primary DS units is all n (e.g., all pillars ready). The method then continues to the step to create more slices.
  • As illustrated by block 813, the DS processing unit sends slices to the available chosen primary DS units when the DS processing unit determines that the number of ready primary DS units is greater than or equal to the read threshold k but is less than all n. As illustrated by block 815, the DS processing unit temporarily stores slices by storing slices in temporary memory for any chosen primary DS units that are not available.
  • As illustrated by block 817, the DS processing unit determines if the status of any unavailable chosen primary DS units has changed to ready. As illustrated by blocks 819 and 821, the DS processing unit retrieves the slices from temporary memory and sends the slices to the ready DS unit when the DS processing unit determines that the status of the unavailable chosen primary DS unit has changed to ready. As illustrated by block 823, the DS processing unit determines if all the temporarily cached slices have been stored in the chosen DS unit and continues to the step of determining if the status has changed when all the cached slices have not been stored in the chosen DS units. In another embodiment, a timeout may occur where the DS processing unit gives up on waiting for the ready status to change in which case the DS processing unit may try another DS unit or just not store a pillar of slices (e.g., deleting them from the temporary memory). The DS processing unit method goes back to the step of creating slices when all the cached slices have been stored in the chosen DS units.
  • In some embodiments, some or all slices stored in temporary memory may be discarded according to a discard policy. The discard policy may specify that slices are to be discarded after a threshold period of time, based on an amount of available storage, or based on reliability of the data. For example, a data slice may be discarded only when it is no longer possible to use the data slice, when the data slice is no longer needed, or when the data slice is deemed unreliable. Some data slices may be given retention preference over other data slices, so that very data slices associated with reliable data slices already in long term storage may be discarded in favor of data slices that may be needed to correct unreliable data slices.
  • FIG. 9A is a schematic block diagram of another embodiment of a distributed storage system that includes the DS processing unit 16, the plurality of DS units 36, and a plurality of associated temporary memories 904. In one example of operation, the DS unit 4 may not be available due to a site outage, a DS unit failure, and/or the network is not available at DS unit 4 site. The DS processing unit 16 may temporarily store new pillar 4 slices in one of the temporary memories 904, and/or yet another DS unit, for subsequent storage in DS unit 4. The method of operation to determine where to temporarily store the slices will be discussed in greater detail with reference to FIG. 9B.
  • FIG. 9B is another flowchart illustrating a method 900 of writing to memory where the DS processing unit determines where to store newly created slices when at least one primary DS unit is not available.
  • The method begins as illustrated by block 903, where the DS processing unit creates the n slices for each data segment for storage. As illustrated by block 905, the DS processing unit determines the desired primary DS units in which to store the slices based in part on a predetermination of the slice name in the user vault, or in the virtual DSN address to physical location table.
  • As illustrated by block 907, the DS processing unit determines the status of the chosen primary DS units based on one or more of a status table lookup and/or a real time query to the DS unit. For example, the status indicates not available if the network is down to the DS unit or if the DS unit is down. As illustrated by block 910, the DS processing unit determines the number of primary DS units that are in the ready status. As illustrated by block 909, the DS processing unit tries other DS units and returns to the step to determine which DS units when the number of ready primary DS units is less than the read threshold k. Note that the threshold for this scenario may be k+1 or k+2, etc. in another embodiment to further improve the probability of subsequent data object recreation.
  • As illustrated by block 911, the DS processing unit sends the n slices to the chosen primary DS units when the DS processing unit determines that the number of ready primary DS units is all n (e.g., all pillars ready). The method 900 then continues to create more slices, as illustrated by block 903.
  • As illustrated by block 913, the DS processing unit sends slices to the available chosen primary DS units when the DS processing unit determines that the number of ready primary DS units is greater than or equal to the read threshold k but is less than all n.
  • As illustrated by block 915, the DS processing unit determines which temporary memory 1-3 to utilize to temporarily store the slices for the DS unit 4 that is not ready. The determination may be based on one or more of an even rotation across the ready DS unit temporary memories (e.g., temporary/cache memory 1, then 2, then 3, then 1 etc.), one pillar high or low from the DS unit that is not ready, a list, a command, and/or the performance of the temporary memory. The DS processing unit caches slices by storing slices in the chosen temporary memory for any chosen primary DS units that are not available.
  • As illustrated by block 917, the DS processing unit determines if the status of any unavailable chosen primary DS units 36 has changed to ready. As illustrated by blocks 919 and 921, the DS processing unit retrieves the slices from the temporary memory and sends the slices to the ready DS unit when the DS processing unit determines that the status of the unavailable chosen primary DS unit has changed to ready. As illustrated by block 923, the DS processing unit determines if all the temporarily cached slices have been stored in the chosen DS unit and continues to the step of determining if the status has changed when all the cached slices have not been stored in the chosen DS units. In another embodiment, a timeout may occur where the DS processing unit gives up on waiting for the ready status to change in which case the DS processing unit may try another DS unit or just not store a pillar of slices (e.g., deleting them from the temporary memory). The DS processing unit method goes back to the step of creating slices when all the cached slices have been stored in the chosen DS units.
  • FIG. 10 is a schematic block diagram of another embodiment of a distributed storage system that includes the DS processing unit 16, and a plurality of DS units 36. The DS units 1-4 may each include a matching number of memories 1-4 in some embodiments. In another embodiment, the number of memories per DS unit may be 8, 16 or more.
  • The DS units can include a matching number of memories to facilitate organizing memories across the DS units 1-4 as storage groups or stripes 1-4. The stripes 1-4 may be physical as shown or logical such that the stripe boundaries are within the memory ranges of the memories.
  • The DS processing unit 16 and/or the DS units determine which memories across the DS units to utilize to store slices of the same data object. Note that the overall system reliability can be improved when the number of logical stripes is minimized such that same data segment slices are contained within the same stripe. In an embodiment (not illustrated), a logical stripe may include memory 1 of DS unit 1, memory 4 of DS unit 2, memory 2 of DS unit 3, and memory 3 of DS unit 4. This embodiment may be undesired as it can lead to lower system reliability since a memory failure can affect many data sets.
  • In another embodiment, a logical stripe may include memory 2 of DS unit 1, memory 2 of DS unit 2, memory 2 of DS unit 3, and memory 2 of DS unit 4. This embodiment may be more desired as it can lead to improved system reliability, since a memory failure can affect a more limited number of data sets.
  • In general, there are n choose m possible logical stripes where m is the number of memories per DS unit and n is the pillar width of the vault, and “choose” refers to the combinatorial operation for determining the number of distinct k-combinations. The system mean time to data loss=(stripe mean time to data loss)/(number of logical stripes). Minimizing the number of logical stripes may improve the system reliability. The DS processing unit and/or DS unit may determine the provisioning and utilization of the memories into logical stripes such as to minimize the number of logical stripes.
  • In an example of operation, the DS processing unit and/or DS managing unit provision memory 1 of each of DS unit 1-4 to be stripe 1, memory 2 of each of DS unit 1-4 to be stripe 2, memory 3 of each of DS unit 1-4 to be stripe 3, and memory 4 of each of DS unit 1-4 to be stripe 4. The DS processing unit and/or DS unit determines to store a pillar 1 slice of data segment A at stripe 1 of DS unit 1 (slice Al at memory 1 of DS unit 1), slice A2 at memory 1 of DS unit 2, slice A3 at memory 1 of DS unit 3, and slice A4 at memory 1 of DS unit 4. In a similar fashion the DS processing unit and/or DS unit determines to store the slices of data segment E in stripe 1 (E1-E4), B1-B4 and F1-F4 in stripe 2, C1-C4 and G1-G4 in stripe 3, and D1-D4 and H1-H4 in stripe 4. A method of determining which stripe to utilize is discussed in greater detail with reference to FIG. 11.
  • In some embodiments, every DS unit receives slices from a contiguous set of segments of a data source. So, as illustrated in FIG. 10, DS unit 1 would receive, in order, A1, B1, C1, D1, E1, and so on. The striping algorithm can be used to even the load, such that no one memory has to handle all the input/output traffic. In an embodiment illustrated by FIG. 10, if slices from segments A-D come in at once, all 4 disks may begin storage operations, since each of the 4 memories gets something to store.
  • To achieve load balancing, some embodiments apply a random-like (but deterministic), or round-robin process to select which memory the slice will go to based on its name. It should be a deterministic process so that when reading, the DS unit knows which memory to access to find the source. For example, if the store had 8 disks, it might look at the 3 least significant bits of the segment's name (which would represent any number from 0-7 in binary). This result would determine which of the 8 disks a slice would be stored in.
  • In other embodiments, the least significant bits of the input source name are not used, because they are not guaranteed to have a uniform enough distribution. In some cases, the hash of the source name is used to create something with an even distribution, and, the least significant bits of the hash are examined. Other implementations use the result of taking the remainder when dividing the hash result by a smaller number.
  • FIG. 11 is another flowchart illustrating method 1100 of writing to memory where the DS processing unit and/or DS unit determine which stripe to utilize.
  • As illustrated by block 1103, the DS unit receives a slice to store from one of the DS processing unit, the user device, the DS managing unit, or the storage integrity processing unit. The slice is accompanied by one or more of the command/request to store it, the slice name, the source name, and or the slice metadata. As illustrated by block 1105, the DS unit determines the source name either by receiving the source name or deriving it from the slice name.
  • As illustrated by block 1107, the DS unit calculates a reduced length source name. The reduced length source name can be calculated, for example, using a hash (e.g., CRC) function of the source name which will always be the same number for the same source name (e.g., vault ID, vault gen, resv, and file ID). In other instances, the reduced length source name can be calculated using other suitable functions, for example, a modulo function. Generally, any reduction function that can be used to reduce the original source name to a smaller number that can be used to uniquely identify a particular memory can be used. In most cases, a reduction function can be chosen to maintain a random distribution among the various memories of a DS unit. The randomness of the file ID ensures that the hash will have desired distancing properties to spread out the slices of data objects evenly across the stripes.
  • As illustrated by block 1109, the DS unit determines the memory device based on the hash of the source name by truncating the hash to the number of bits required to specify the stripe range. For example, the least two significant bits of the hash may be utilized to specify the memory number.
  • As illustrated by block 1113, the DS unit updates the local virtual DSN address to physical location table with the memory number before storing the slice in the chosen memory, as illustrated by block 1115
  • In various embodiments employing a deterministic technique to find the memory device based on the hash, as discussed for example with reference to block 1109, there a physical location table for each element is not maintained, because the name itself is all the information needed for the DS unit to determine the memory location. However, such a table can be maintained for a DS processing unit to determine which DS unit keeps a particular slice. Additionally rather than using an algorithm to determine which memory to use, an individual DS unit can further subdivide its namespace range so that one memory is responsible for some contiguous range of the namespace, with that range being a subset of the DS units entire assigned range. This technique may not allow for I/O load balancing to the same degree as other methods, since contiguous segments for the same source would likely all fall to one or a few memories, rather than most or all of them.
  • As may be used herein, the terms “substantially” and “approximately” provides an industry-accepted tolerance for its corresponding term and/or relativity between items. Such an industry-accepted tolerance ranges from less than one percent to fifty percent and corresponds to, but is not limited to, component values, integrated circuit process variations, temperature variations, rise and fall times, and/or thermal noise. Such relativity between items ranges from a difference of a few percent to magnitude differences. As may also be used herein, the term(s) “coupled to” and/or “coupling” and/or includes direct coupling between items and/or indirect coupling between items via an intervening item (e.g., an item includes, but is not limited to, a component, an element, a circuit, and/or a module) where, for indirect coupling, the intervening item does not modify the information of a signal but may adjust its current level, voltage level, and/or power level. As may further be used herein, inferred coupling (i.e., where one element is coupled to another element by inference) includes direct and indirect coupling between two items in the same manner as “coupled to”. As may even further be used herein, the term “operable to” indicates that an item includes one or more of power connections, input(s), output(s), etc., to perform one or more its corresponding functions and may further include inferred coupling to one or more other items. As may still further be used herein, the term “associated with”, includes direct and/or indirect coupling of separate items and/or one item being embedded within another item. As may be used herein, the term “compares favorably”, indicates that a comparison between two or more items, signals, etc., provides a desired relationship. For example, when the desired relationship is that signal 1 has a greater magnitude than signal 2, a favorable comparison may be achieved when the magnitude of signal 1 is greater than that of signal 2 or when the magnitude of signal 2 is less than that of signal 1.
  • The present invention has also been described above with the aid of method steps illustrating the performance of specified functions and relationships thereof. The boundaries and sequence of these functional building blocks and method steps have been arbitrarily defined herein for convenience of description. Alternate boundaries and sequences can be defined so long as the specified functions and relationships are appropriately performed. Any such alternate boundaries or sequences are thus within the scope and spirit of the claimed invention.
  • The present invention has been described above with the aid of functional building blocks illustrating the performance of certain significant functions. The boundaries of these functional building blocks have been arbitrarily defined for convenience of description. Alternate boundaries could be defined as long as the certain significant functions are appropriately performed. Similarly, flow diagram blocks may also have been arbitrarily defined herein to illustrate certain significant functionality. To the extent used, the flow diagram block boundaries and sequence could have been defined otherwise and still perform the certain significant functionality. Such alternate definitions of both functional building blocks and flow diagram blocks and sequences are thus within the scope and spirit of the claimed invention. One of average skill in the art will also recognize that the functional building blocks, and other illustrative blocks, modules and components herein, can be implemented as illustrated or by discrete components, application specific integrated circuits, processors executing appropriate software and the like or any combination thereof.

Claims (21)

1. A method comprising:
generating a plurality of data slices in a distributed storage processing unit;
determining metadata associated with the plurality of data slices, the metadata including information that can be used to determine storage diversity preferences;
determining characteristics of a plurality of storage units included in a distributed storage network;
selecting particular storage units of the plurality of storage units based on a favorable match between the storage diversity preferences and the characteristics of the particular storage units; and
delivering the plurality of data slices to the particular storage units.
2. The method of claim 1, wherein selecting particular storage units further comprising:
identifying candidate storage units having characteristics that favorably match different storage diversity preferences;
sorting the candidate storage units based on the different storage diversity preferences; and
selecting particular candidate storage units based on a favorable match of the different storage diversity preferences.
3. The method of claim 1, wherein the storage diversity preferences include similarity and difference requirements.
4. The method of claim 1, wherein the storage diversity preferences include a requirement that a stripe used to store data slices associated with a common data segment include no more than a threshold number of memories of the same model.
5. The method of claim 1, wherein the metadata includes an indication that the plurality of data slices should be stored in storage units selected to optimize frequent access of stored data.
6. The method of claim 1, wherein the metadata includes an indication that the plurality of data slices should be stored in storage units selected to optimize failure rates.
7. A method comprising:
receiving a data slice to be stored in one of a plurality of memories at a dispersed storage unit;
obtaining metadata associated with the plurality data slice, the metadata including an indication of storage diversity preferences;
determining characteristics of the plurality of memories;
selecting particular memories of the plurality of memories based on a favorable match between the storage diversity preferences and the characteristics of the particular memories; and
storing the data slices to the particular memory.
8. The method of claim 7, wherein selecting particular memories further comprising:
identifying candidate memories having characteristics that favorably match storage different diversity preferences;
sorting the candidate memories based on the storage different diversity preferences; and
selecting particular candidate memories based on a favorable match of the storage different diversity preferences.
9. The method of claim 7, wherein the storage diversity preferences include similarity and difference requirements.
10. The method of claim 7, wherein the metadata includes an indication that the plurality of data slices should be stored in memories optimized for cost.
11. The method of claim 7, wherein the metadata includes an indication that the plurality of data slices should be stored in storage units optimized for capacity.
12. A dispersed storage processing unit comprising:
a processor to generate a plurality of data slices, each of the plurality of data slices including metadata;
the processor further to select candidate storage units based on a correlation between the metadata and characteristics of the candidate storage units; and
a communications interface to deliver the plurality of data slices to the candidate storage units.
13. The dispersed storage processing unit of claim 12, wherein the processor is further to:
identify a plurality of candidate storage units having a plurality of characteristics;
sort the plurality of candidate storage units based on the plurality of characteristics; and
select particular candidate storage units based on a favorable correlation of the metadata and the plurality of characteristics of the plurality of candidate storage units.
14. The dispersed storage processing unit of claim 12, wherein the metadata includes information used to determine storage unit similarity and difference requirements.
15. The dispersed storage processing unit of claim 12, wherein the dispersed storage processing unit ensures that a read threshold number of data slices are delivered to storage units employing different types of memories.
16. The dispersed storage processing unit of claim 12, wherein the metadata includes an indication that the plurality of data slices should be stored in storage units optimized for location diversity.
17. A dispersed storage unit comprising:
a communications interface to receive a data slice for storage, the data slice including metadata;
a plurality of memory units;
a processor to select at least one of the memory units to store the data slice, the selection based on a correlation between the metadata and characteristics of the plurality of memory units;
the processor further to direct the data slices to the at least of the memory units to be stored.
18. The dispersed storage unit of claim 17, wherein the processor is further to:
identify a plurality of candidate memory units having a plurality of characteristics;
sort the plurality of candidate memory units based on the plurality of characteristics; and
select particular candidate memory units based on a favorable correlation of the metadata and the plurality of characteristics of the plurality of candidate memory units.
19. The dispersed storage unit of claim 17, wherein the metadata includes information used to determine memory unit similarity and difference requirements.
20. The dispersed storage unit of claim 17, wherein the metadata includes an indication that the data slice is to be stored in a memory unit selected to meet access speed requirements.
21. The dispersed storage unit of claim 17, wherein the metadata includes an indication that the data slice is to be stored in a memory unit selected based on a reliability requirement.
US12/777,850 2009-09-29 2010-05-11 Distributed storage network including memory diversity Abandoned US20110078343A1 (en)

Priority Applications (3)

Application Number Priority Date Filing Date Title
US12/777,850 US20110078343A1 (en) 2009-09-29 2010-05-11 Distributed storage network including memory diversity
US13/529,052 US8862800B2 (en) 2009-09-29 2012-06-21 Distributed storage network including memory diversity
US13/925,270 US9274890B2 (en) 2009-09-29 2013-06-24 Distributed storage network memory access based on memory state

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US24687609P 2009-09-29 2009-09-29
US12/777,850 US20110078343A1 (en) 2009-09-29 2010-05-11 Distributed storage network including memory diversity

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US13/529,052 Continuation US8862800B2 (en) 2009-09-29 2012-06-21 Distributed storage network including memory diversity

Publications (1)

Publication Number Publication Date
US20110078343A1 true US20110078343A1 (en) 2011-03-31

Family

ID=43781516

Family Applications (6)

Application Number Title Priority Date Filing Date
US12/777,887 Expired - Fee Related US8554994B2 (en) 2009-09-29 2010-05-11 Distributed storage network utilizing memory stripes
US12/777,904 Expired - Fee Related US8473677B2 (en) 2009-09-29 2010-05-11 Distributed storage network memory access based on memory state
US12/777,850 Abandoned US20110078343A1 (en) 2009-09-29 2010-05-11 Distributed storage network including memory diversity
US12/777,864 Active 2031-06-08 US8918534B2 (en) 2009-09-29 2010-05-11 Writing data slices to ready and non-ready distributed storage units in a distributed storage network
US13/529,052 Active 2030-07-27 US8862800B2 (en) 2009-09-29 2012-06-21 Distributed storage network including memory diversity
US13/925,270 Active 2031-02-18 US9274890B2 (en) 2009-09-29 2013-06-24 Distributed storage network memory access based on memory state

Family Applications Before (2)

Application Number Title Priority Date Filing Date
US12/777,887 Expired - Fee Related US8554994B2 (en) 2009-09-29 2010-05-11 Distributed storage network utilizing memory stripes
US12/777,904 Expired - Fee Related US8473677B2 (en) 2009-09-29 2010-05-11 Distributed storage network memory access based on memory state

Family Applications After (3)

Application Number Title Priority Date Filing Date
US12/777,864 Active 2031-06-08 US8918534B2 (en) 2009-09-29 2010-05-11 Writing data slices to ready and non-ready distributed storage units in a distributed storage network
US13/529,052 Active 2030-07-27 US8862800B2 (en) 2009-09-29 2012-06-21 Distributed storage network including memory diversity
US13/925,270 Active 2031-02-18 US9274890B2 (en) 2009-09-29 2013-06-24 Distributed storage network memory access based on memory state

Country Status (1)

Country Link
US (6) US8554994B2 (en)

Cited By (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110078277A1 (en) * 2009-09-29 2011-03-31 Cleversafe, Inc. Handling unavailable memories in distributed storage network
US20120163394A1 (en) * 2009-09-02 2012-06-28 Zte Corporation Route Switching Device and Data Cashing Method Thereof
US8327141B2 (en) 2009-02-05 2012-12-04 Wwpass Corporation Centralized authentication system with safe private data storage and method
US8555079B2 (en) 2011-12-06 2013-10-08 Wwpass Corporation Token management
US8656180B2 (en) 2011-12-06 2014-02-18 Wwpass Corporation Token activation
US8713661B2 (en) 2009-02-05 2014-04-29 Wwpass Corporation Authentication service
US8751829B2 (en) 2009-02-05 2014-06-10 Wwpass Corporation Dispersed secure data storage and retrieval
US8752153B2 (en) 2009-02-05 2014-06-10 Wwpass Corporation Accessing data based on authenticated user, provider and system
US8839391B2 (en) 2009-02-05 2014-09-16 Wwpass Corporation Single token authentication
US8972719B2 (en) 2011-12-06 2015-03-03 Wwpass Corporation Passcode restoration
US20160212454A1 (en) * 2010-08-22 2016-07-21 Qwilt, Inc. System and method for live service content handling with content storing servers caching popular content therein
US20170277453A1 (en) * 2016-03-24 2017-09-28 Microsoft Technology Licensing, Llc Distributed metadata management in a distributed storage system
US10044802B2 (en) 2010-08-22 2018-08-07 Qwilt, Inc. System for detection of content servers and caching popular content therein
US10097428B2 (en) 2010-08-22 2018-10-09 Qwilt, Inc. System and method for caching popular content respective of a content strong server in an asymmetrical routing topology
US10127335B2 (en) 2010-08-22 2018-11-13 Qwilt, Inc System and method of performing analytics with respect to content storing servers caching popular content
US11032583B2 (en) 2010-08-22 2021-06-08 QWLT, Inc. Method and system for improving high availability for live content
US11240335B2 (en) 2014-04-22 2022-02-01 Qwilt, Inc. System and methods thereof for delivery of popular content using a multimedia broadcast multicast service
US20220308962A1 (en) * 2012-06-25 2022-09-29 Pure Storage, Inc. Selection of Memory for Data Storage in a Storage Network

Families Citing this family (110)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11221917B1 (en) * 2005-09-30 2022-01-11 Pure Storage, Inc. Integrity processing in a dispersed storage network
US11620185B2 (en) 2005-09-30 2023-04-04 Pure Storage, Inc. Integrity processing in a dispersed storage network
US8527838B2 (en) * 2009-07-31 2013-09-03 Cleversafe, Inc. Memory controller utilizing an error coding dispersal function
US8819452B2 (en) * 2009-11-25 2014-08-26 Cleversafe, Inc. Efficient storage of encrypted data in a dispersed storage network
US9413393B2 (en) * 2009-12-29 2016-08-09 International Business Machines Corporation Encoding multi-media content for a centralized digital video storage system
US9672108B2 (en) 2009-12-29 2017-06-06 International Business Machines Corporation Dispersed storage network (DSN) and system with improved security
US11614893B2 (en) 2010-09-15 2023-03-28 Pure Storage, Inc. Optimizing storage device access based on latency
KR101035302B1 (en) * 2010-10-11 2011-05-19 (주)이스트소프트 A cloud system and a method of compressing and transmtting files in a cloud system
JP2013061847A (en) * 2011-09-14 2013-04-04 Sony Corp Information processor and information processing method
JP5744244B2 (en) * 2011-10-19 2015-07-08 株式会社日立製作所 Storage system
US8751746B2 (en) * 2011-12-15 2014-06-10 Apple Inc. QoS management in the L2 cache
US8868647B2 (en) * 2012-01-11 2014-10-21 Alcatel Lucent Reducing latency and cost in resilient cloud file systems
US9891995B2 (en) * 2012-01-31 2018-02-13 International Business Machines Corporation Cooperative decentralized rebuild scanning
CN102662609B (en) * 2012-04-16 2016-03-30 华为软件技术有限公司 The method of video access and device
US20180321872A1 (en) * 2012-06-05 2018-11-08 International Business Machines Corporation Fragmented storage maps
US20130343377A1 (en) * 2012-06-21 2013-12-26 Jonathan Stroud Hash-based packet distribution in a computer system
US20130343181A1 (en) * 2012-06-21 2013-12-26 Jonathan Stroud Systems and methods of data processing using an fpga-implemented hash function
US9537609B2 (en) * 2012-08-02 2017-01-03 International Business Machines Corporation Storing a stream of data in a dispersed storage network
US10651975B2 (en) 2012-08-02 2020-05-12 Pure Storage, Inc. Forwarding data amongst cooperative DSTN processing units of a massive data ingestion system
US20170322734A1 (en) * 2012-12-05 2017-11-09 International Business Machines Corporation Using locks to prevent multiple rebuilds of the same source
US9053058B2 (en) 2012-12-20 2015-06-09 Apple Inc. QoS inband upgrade
US9372642B2 (en) * 2013-07-22 2016-06-21 International Business Machines Corporation RAID 10 reads optimized for solid state drives
CN104425018B (en) 2013-08-23 2019-07-23 慧荣科技股份有限公司 Access the method for storage element and the device using this method in flash memory
CN104424127A (en) * 2013-08-23 2015-03-18 慧荣科技股份有限公司 Methods for accessing a storage unit of a flash memory and apparatuses using the same
US9323615B2 (en) 2014-01-31 2016-04-26 Google Inc. Efficient data reads from distributed storage systems
US10802732B2 (en) 2014-04-30 2020-10-13 Pure Storage, Inc. Multi-level stage locality selection on a large system
US9735967B2 (en) * 2014-04-30 2017-08-15 International Business Machines Corporation Self-validating request message structure and operation
US10394476B2 (en) * 2014-04-30 2019-08-27 Pure Storage, Inc. Multi-level stage locality selection on a large system
US10303795B2 (en) * 2014-09-10 2019-05-28 Amazon Technologies, Inc. Read descriptors at heterogeneous storage systems
WO2016051512A1 (en) * 2014-09-30 2016-04-07 株式会社日立製作所 Distributed storage system
US10740180B2 (en) * 2015-01-30 2020-08-11 Pure Storage, Inc. Storing and retrieving data using proxies
US11188665B2 (en) 2015-02-27 2021-11-30 Pure Storage, Inc. Using internal sensors to detect adverse interference and take defensive actions
US10440115B2 (en) * 2015-02-27 2019-10-08 Pure Storage, Inc. Write intent messaging in a dispersed storage network
US10503445B2 (en) * 2015-03-23 2019-12-10 Netapp, Inc. Data structure store and data management
WO2016164033A1 (en) * 2015-04-10 2016-10-13 Hewlett Packard Enterprise Development Lp Partial backup during runtime for memory modules with volatile memory and non-volatile memory
US10613798B2 (en) * 2015-05-29 2020-04-07 Pure Storage, Inc. Slice fanout write request
CN107844268B (en) * 2015-06-04 2021-09-14 华为技术有限公司 Data distribution method, data storage method, related device and system
US10270476B1 (en) 2015-06-16 2019-04-23 Amazon Technologies, Inc. Failure mode-sensitive layered redundancy coding techniques
US10270475B1 (en) 2015-06-16 2019-04-23 Amazon Technologies, Inc. Layered redundancy coding for encoded parity data
US9998150B1 (en) * 2015-06-16 2018-06-12 Amazon Technologies, Inc. Layered data redundancy coding techniques for layer-local data recovery
US10298259B1 (en) 2015-06-16 2019-05-21 Amazon Technologies, Inc. Multi-layered data redundancy coding techniques
US10977128B1 (en) * 2015-06-16 2021-04-13 Amazon Technologies, Inc. Adaptive data loss mitigation for redundancy coding systems
US10311020B1 (en) 2015-06-17 2019-06-04 Amazon Technologies, Inc. Locality-sensitive data retrieval for redundancy coded data storage systems
US9838041B1 (en) * 2015-06-17 2017-12-05 Amazon Technologies, Inc. Device type differentiation for redundancy coded data storage systems
US9838042B1 (en) 2015-06-17 2017-12-05 Amazon Technologies, Inc. Data retrieval optimization for redundancy coded data storage systems with static redundancy ratios
US10009044B1 (en) * 2015-06-17 2018-06-26 Amazon Technologies, Inc. Device type differentiation for redundancy coded data storage systems
US9825652B1 (en) 2015-06-17 2017-11-21 Amazon Technologies, Inc. Inter-facility network traffic optimization for redundancy coded data storage systems
US9866242B1 (en) 2015-06-17 2018-01-09 Amazon Technologies, Inc. Throughput optimization for redundancy coded data storage systems
US9853662B1 (en) 2015-06-17 2017-12-26 Amazon Technologies, Inc. Random access optimization for redundancy coded data storage systems
US10055291B2 (en) * 2015-06-30 2018-08-21 International Business Machines Corporation Method and system for processing data access requests during data transfers
US9959167B1 (en) 2015-07-01 2018-05-01 Amazon Technologies, Inc. Rebundling grid encoded data storage systems
US10089176B1 (en) 2015-07-01 2018-10-02 Amazon Technologies, Inc. Incremental updates of grid encoded data storage systems
US10198311B1 (en) 2015-07-01 2019-02-05 Amazon Technologies, Inc. Cross-datacenter validation of grid encoded data storage systems
US10162704B1 (en) 2015-07-01 2018-12-25 Amazon Technologies, Inc. Grid encoded data storage systems for efficient data repair
US10394762B1 (en) 2015-07-01 2019-08-27 Amazon Technologies, Inc. Determining data redundancy in grid encoded data storage systems
US10108819B1 (en) 2015-07-01 2018-10-23 Amazon Technologies, Inc. Cross-datacenter extension of grid encoded data storage systems
US9998539B1 (en) 2015-07-01 2018-06-12 Amazon Technologies, Inc. Non-parity in grid encoded data storage systems
US9904589B1 (en) 2015-07-01 2018-02-27 Amazon Technologies, Inc. Incremental media size extension for grid encoded data storage systems
US9928141B1 (en) 2015-09-21 2018-03-27 Amazon Technologies, Inc. Exploiting variable media size in grid encoded data storage systems
US11386060B1 (en) 2015-09-23 2022-07-12 Amazon Technologies, Inc. Techniques for verifiably processing data in distributed computing systems
US9940474B1 (en) 2015-09-29 2018-04-10 Amazon Technologies, Inc. Techniques and systems for data segregation in data storage systems
US9785364B2 (en) * 2015-10-17 2017-10-10 International Business Machines Corporation Increasing data storage reliability in a heterogeneous storage system
US10394789B1 (en) 2015-12-07 2019-08-27 Amazon Technologies, Inc. Techniques and systems for scalable request handling in data processing systems
US10642813B1 (en) 2015-12-14 2020-05-05 Amazon Technologies, Inc. Techniques and systems for storage and processing of operational data
US9785495B1 (en) 2015-12-14 2017-10-10 Amazon Technologies, Inc. Techniques and systems for detecting anomalous operational data
US10248793B1 (en) 2015-12-16 2019-04-02 Amazon Technologies, Inc. Techniques and systems for durable encryption and deletion in data storage systems
US10102065B1 (en) 2015-12-17 2018-10-16 Amazon Technologies, Inc. Localized failure mode decorrelation in redundancy encoded data storage systems
US10180912B1 (en) 2015-12-17 2019-01-15 Amazon Technologies, Inc. Techniques and systems for data segregation in redundancy coded data storage systems
US10324790B1 (en) 2015-12-17 2019-06-18 Amazon Technologies, Inc. Flexible data storage device mapping for data storage systems
US10235402B1 (en) 2015-12-17 2019-03-19 Amazon Technologies, Inc. Techniques for combining grid-encoded data storage systems
US10127105B1 (en) 2015-12-17 2018-11-13 Amazon Technologies, Inc. Techniques for extending grids in data storage systems
US10095413B2 (en) 2016-01-28 2018-10-09 Toshiba Memory Corporation Memory system with address translation between a logical address and a physical address
EP3417376A4 (en) * 2016-02-16 2019-03-20 Technion Research & Development Foundation Limited Optimized data distribution system
KR102114256B1 (en) 2016-02-19 2020-05-22 후아웨이 테크놀러지 컴퍼니 리미티드 Flash memory device access method and apparatus
US10592336B1 (en) 2016-03-24 2020-03-17 Amazon Technologies, Inc. Layered indexing for asynchronous retrieval of redundancy coded data
US10678664B1 (en) 2016-03-28 2020-06-09 Amazon Technologies, Inc. Hybridized storage operation for redundancy coded data storage systems
US10366062B1 (en) 2016-03-28 2019-07-30 Amazon Technologies, Inc. Cycled clustering for redundancy coded data storage systems
US10061668B1 (en) 2016-03-28 2018-08-28 Amazon Technologies, Inc. Local storage clustering for redundancy coded data storage system
US20170293490A1 (en) * 2016-04-11 2017-10-12 Aqua Products, Inc. Method for modifying an onboard control system of a pool cleaner, and power source for a pool cleaner
US20170364275A1 (en) * 2016-06-20 2017-12-21 Intel Corporation Technologies for managing end of life behavior for storage devices
US10180875B2 (en) * 2016-07-08 2019-01-15 Toshiba Memory Corporation Pool-level solid state drive error correction
US11137980B1 (en) 2016-09-27 2021-10-05 Amazon Technologies, Inc. Monotonic time-based data storage
US10810157B1 (en) 2016-09-28 2020-10-20 Amazon Technologies, Inc. Command aggregation for data storage operations
US10496327B1 (en) 2016-09-28 2019-12-03 Amazon Technologies, Inc. Command parallelization for data storage systems
US11281624B1 (en) 2016-09-28 2022-03-22 Amazon Technologies, Inc. Client-based batching of data payload
US10437790B1 (en) 2016-09-28 2019-10-08 Amazon Technologies, Inc. Contextual optimization for data storage systems
US11204895B1 (en) 2016-09-28 2021-12-21 Amazon Technologies, Inc. Data payload clustering for data storage systems
US10657097B1 (en) 2016-09-28 2020-05-19 Amazon Technologies, Inc. Data payload aggregation for data storage systems
US10614239B2 (en) 2016-09-30 2020-04-07 Amazon Technologies, Inc. Immutable cryptographically secured ledger-backed databases
JP6767691B2 (en) * 2016-10-18 2020-10-14 パナソニックIpマネジメント株式会社 Image coding method, transmission method and image coding device
US10296764B1 (en) 2016-11-18 2019-05-21 Amazon Technologies, Inc. Verifiable cryptographically secured ledgers for human resource systems
US11269888B1 (en) 2016-11-28 2022-03-08 Amazon Technologies, Inc. Archival data storage for structured data
US10530752B2 (en) 2017-03-28 2020-01-07 Amazon Technologies, Inc. Efficient device provision
US11356445B2 (en) 2017-03-28 2022-06-07 Amazon Technologies, Inc. Data access interface for clustered devices
US10621055B2 (en) 2017-03-28 2020-04-14 Amazon Technologies, Inc. Adaptive data recovery for clustered data devices
KR102293069B1 (en) * 2017-09-08 2021-08-27 삼성전자주식회사 Storage device including nonvolatile memory device and controller, controller and operating method of nonvolatile memory device
US10430117B2 (en) * 2017-10-23 2019-10-01 Micron Technology, Inc. Host accelerated operations in managed NAND devices
US11663094B2 (en) * 2017-11-30 2023-05-30 Hewlett Packard Enterprise Development Lp Reducing recovery time of an application
US10715184B2 (en) * 2017-12-11 2020-07-14 Rubrik, Inc. Techniques for fast IO and low memory consumption while using erasure codes
US10866893B2 (en) * 2018-01-23 2020-12-15 Home Depot Product Authority, Llc Cache coherency engine
US10592340B2 (en) * 2018-02-28 2020-03-17 International Business Machines Corporation Dynamic authorization batching in a dispersed storage network
US11132145B2 (en) * 2018-03-14 2021-09-28 Apple Inc. Techniques for reducing write amplification on solid state storage devices (SSDs)
US10831378B2 (en) 2018-09-24 2020-11-10 International Business Machines Corporation Optimizing data access in a DSN memory for wear leveling
US11005825B2 (en) 2018-11-13 2021-05-11 Seagate Technology Llc Sensor nodes and host forming a tiered ecosystem that uses public and private data for duplication
US11200168B2 (en) 2018-12-10 2021-12-14 International Business Machines Corporation Caching data from remote memories
US11281790B2 (en) 2019-05-22 2022-03-22 Myota, Inc. Method and system for distributed data storage with enhanced security, resilience, and control
US11153173B1 (en) * 2019-09-10 2021-10-19 Juniper Networks, Inc. Dynamically updating compute node location information in a distributed computing environment
US11487443B2 (en) 2020-05-22 2022-11-01 Mastercard International Incorporated Systems and methods for use in segregating data blocks to distributed storage
US11562102B2 (en) 2020-05-22 2023-01-24 Mastercard International Incorporated Systems and methods for use in segregating data blocks to distributed storage
US20230236758A1 (en) * 2022-01-27 2023-07-27 Micron Technology, Inc. Control of back pressure based on a total number of buffered read and write entries

Citations (76)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4092732A (en) * 1977-05-31 1978-05-30 International Business Machines Corporation System for recovering data stored in failed memory unit
US5454101A (en) * 1992-09-15 1995-09-26 Universal Firmware Industries, Ltd. Data storage system with set lists which contain elements associated with parents for defining a logical hierarchy and general record pointers identifying specific data sets
US5485474A (en) * 1988-02-25 1996-01-16 The President And Fellows Of Harvard College Scheme for information dispersal and reconstruction
US5774643A (en) * 1995-10-13 1998-06-30 Digital Equipment Corporation Enhanced raid write hole protection and recovery
US5802364A (en) * 1996-04-15 1998-09-01 Sun Microsystems, Inc. Metadevice driver rename/exchange technique for a computer system incorporating a plurality of independent device drivers
US5809285A (en) * 1995-12-21 1998-09-15 Compaq Computer Corporation Computer system having a virtual drive array controller
US5890156A (en) * 1996-05-02 1999-03-30 Alcatel Usa, Inc. Distributed redundant database
US5987622A (en) * 1993-12-10 1999-11-16 Tm Patents, Lp Parallel computer system including parallel storage subsystem including facility for correction of data in the event of failure of a storage device in parallel storage subsystem
US5991414A (en) * 1997-09-12 1999-11-23 International Business Machines Corporation Method and apparatus for the secure distributed storage and retrieval of information
US6012159A (en) * 1996-01-17 2000-01-04 Kencast, Inc. Method and system for error-free data transfer
US6058454A (en) * 1997-06-09 2000-05-02 International Business Machines Corporation Method and system for automatically configuring redundant arrays of disk memory devices
US6128277A (en) * 1997-10-01 2000-10-03 California Inst Of Techn Reliable array of distributed computing nodes
US6175571B1 (en) * 1994-07-22 2001-01-16 Network Peripherals, Inc. Distributed memory switching hub
US6256688B1 (en) * 1997-12-02 2001-07-03 Casio Computer Co., Ltd. Interface apparatus operable by using floppy disk drive
US6272658B1 (en) * 1997-10-27 2001-08-07 Kencast, Inc. Method and system for reliable broadcasting of data files and streams
US6301604B1 (en) * 1997-12-01 2001-10-09 Matsushita Electric Industrial Co., Ltd. Multimedia server
US6356949B1 (en) * 1999-01-29 2002-03-12 Intermec Ip Corp. Automatic data collection device that receives data output instruction from data consumer
US6366995B1 (en) * 1998-08-19 2002-04-02 Acuid Corporation Limited System and a method for defining transforms of memory device addresses
US6374336B1 (en) * 1997-12-24 2002-04-16 Avid Technology, Inc. Computer system and process for transferring multiple high bandwidth streams of data between multiple storage units and multiple applications in a scalable and reliable manner
US20020062422A1 (en) * 2000-11-18 2002-05-23 International Business Machines Corporation Method for rebuilding meta-data in a data storage system and a data storage system
US6415373B1 (en) * 1997-12-24 2002-07-02 Avid Technology, Inc. Computer system and process for transferring multiple high bandwidth streams of data between multiple storage units and multiple applications in a scalable and reliable manner
US6418539B1 (en) * 1995-05-25 2002-07-09 Compaq Computer Corporation Continuously available computer memory systems
US20020166079A1 (en) * 2001-01-29 2002-11-07 Ulrich Thomas R. Dynamic data recovery
US20020184463A1 (en) * 2000-07-06 2002-12-05 Hitachi, Ltd. Computer system
US20030018927A1 (en) * 2001-07-23 2003-01-23 Gadir Omar M.A. High-availability cluster virtual server system
US20030037261A1 (en) * 2001-03-26 2003-02-20 Ilumin Corporation Secured content delivery system and method
US20030065617A1 (en) * 2001-06-30 2003-04-03 Watkins Mark Robert Method of billing for utilization of a data storage array, and an array controller therefor
US6571282B1 (en) * 1999-08-31 2003-05-27 Accenture Llp Block-based communication in a communication services patterns environment
US6609223B1 (en) * 1999-04-06 2003-08-19 Kencast, Inc. Method for packet-level fec encoding, in which on a source packet-by-source packet basis, the error correction contributions of a source packet to a plurality of wildcard packets are computed, and the source packet is transmitted thereafter
US20040024963A1 (en) * 2002-08-05 2004-02-05 Nisha Talagala Method and system for striping data to accommodate integrity metadata
US6718361B1 (en) * 2000-04-07 2004-04-06 Network Appliance Inc. Method and apparatus for reliable and scalable distribution of data files in distributed networks
US20040122917A1 (en) * 2002-12-18 2004-06-24 Menon Jaishankar Moothedath Distributed storage system for data-sharing among client computers running defferent operating system types
US20040153479A1 (en) * 2002-11-14 2004-08-05 Mikesell Paul A. Systems and methods for restriping files in a distributed file system
US6785783B2 (en) * 2000-11-30 2004-08-31 International Business Machines Corporation NUMA system with redundant main memory architecture
US20040199566A1 (en) * 2003-03-14 2004-10-07 International Business Machines Corporation System, method, and apparatus for policy-based data management
US20040215998A1 (en) * 2003-04-10 2004-10-28 International Business Machines Corporation Recovery from failures within data processing systems
US20040228493A1 (en) * 2003-05-14 2004-11-18 Kenneth Ma Method and system for disaster recovery of data from a storage device
US6826711B2 (en) * 2000-02-18 2004-11-30 Avamar Technologies, Inc. System and method for data protection with multidimensional parity
US20050027735A1 (en) * 2000-08-24 2005-02-03 Microsoft Corporation Method and system for relocating files that are partially stored in remote storage
US6879596B1 (en) * 2001-04-11 2005-04-12 Applied Micro Circuits Corporation System and method for systolic array sorting of information segments
US20050100022A1 (en) * 2003-11-12 2005-05-12 Ramprashad Sean A. Media delivery using quality of service differentiation within a media stream
US20050114594A1 (en) * 2003-11-24 2005-05-26 Corbett Peter F. Semi-static distribution technique
US20050125593A1 (en) * 2001-01-11 2005-06-09 Yotta Yotta, Inc. Storage virtualization system and methods
US20050132070A1 (en) * 2000-11-13 2005-06-16 Redlich Ron M. Data security system and method with editor
US20050131993A1 (en) * 2003-12-15 2005-06-16 Fatula Joseph J.Jr. Apparatus, system, and method for autonomic control of grid system resources
US20050144382A1 (en) * 2003-12-29 2005-06-30 Schmisseur Mark A. Method, system, and program for managing data organization
US20050229069A1 (en) * 2004-04-02 2005-10-13 Hitachi Global Storage Technologies Netherlands, B.V. Techniques for detecting and correcting errors using multiple interleave erasure pointers
US7003688B1 (en) * 2001-11-15 2006-02-21 Xiotech Corporation System and method for a reserved memory area shared by all redundant storage controllers
US20060047907A1 (en) * 2004-08-30 2006-03-02 Hitachi, Ltd. Storage system and a storage management system
US7024451B2 (en) * 2001-11-05 2006-04-04 Hewlett-Packard Development Company, L.P. System and method for maintaining consistent independent server-side state among collaborating servers
US7024609B2 (en) * 2001-04-20 2006-04-04 Kencast, Inc. System for protecting the transmission of live data streams, and upon reception, for reconstructing the live data streams and recording them into files
US7054927B2 (en) * 2001-01-29 2006-05-30 Adaptec, Inc. File system metadata describing server directory information
US20060136448A1 (en) * 2004-12-20 2006-06-22 Enzo Cialini Apparatus, system, and method for database provisioning
US20060156059A1 (en) * 2005-01-13 2006-07-13 Manabu Kitamura Method and apparatus for reconstructing data in object-based storage arrays
US7080101B1 (en) * 2000-12-01 2006-07-18 Ncr Corp. Method and apparatus for partitioning data for storage in a database
US7103824B2 (en) * 2002-07-29 2006-09-05 Robert Halford Multi-dimensional data protection and mirroring method for micro level data
US7103915B2 (en) * 2000-11-13 2006-09-05 Digital Doors, Inc. Data security system and method
US20060224603A1 (en) * 2005-04-05 2006-10-05 Wal-Mart Stores, Inc. System and methods for facilitating a linear grid database with data organization by dimension
US7140044B2 (en) * 2000-11-13 2006-11-21 Digital Doors, Inc. Data security system and method for separation of user communities
US7146644B2 (en) * 2000-11-13 2006-12-05 Digital Doors, Inc. Data security system and method responsive to electronic attacks
US7171493B2 (en) * 2001-12-19 2007-01-30 The Charles Stark Draper Laboratory Camouflage of network traffic to resist attack
US20070079083A1 (en) * 2005-09-30 2007-04-05 Gladwin S Christopher Metadata management system for an information dispersed storage system
US20070079082A1 (en) * 2005-09-30 2007-04-05 Gladwin S C System for rebuilding dispersed data
US20070079081A1 (en) * 2005-09-30 2007-04-05 Cleversafe, Llc Digital data storage system
US20070088970A1 (en) * 2003-04-10 2007-04-19 Lenovo (Singapore) Pte.Ltd Recovery from failures within data processing systems
US7222133B1 (en) * 2004-02-05 2007-05-22 Unisys Corporation Method for reducing database recovery time
US7240236B2 (en) * 2004-03-23 2007-07-03 Archivas, Inc. Fixed content distributed data storage using permutation ring encoding
US20070174192A1 (en) * 2005-09-30 2007-07-26 Gladwin S C Billing system for information dispersal system
US20070214285A1 (en) * 2006-03-08 2007-09-13 Omneon Video Networks Gateway server
US7272613B2 (en) * 2000-10-26 2007-09-18 Intel Corporation Method and system for managing distributed content and related metadata
US20070234110A1 (en) * 2003-08-14 2007-10-04 Soran Philip E Virtual Disk Drive System and Method
US20070283167A1 (en) * 2003-03-13 2007-12-06 Venters Carl V Iii Secure streaming container
US20080068899A1 (en) * 2006-09-15 2008-03-20 Fujitsu Limited Storage management process, storage management apparatus, and computer-readable medium storing storage management program
US20090094318A1 (en) * 2005-09-30 2009-04-09 Gladwin S Christopher Smart access to a dispersed data storage network
US20090094251A1 (en) * 2007-10-09 2009-04-09 Gladwin S Christopher Virtualized data storage vaults on a dispersed data storage network
US20100023524A1 (en) * 2007-10-09 2010-01-28 Gladwin S Christopher Block based access to a dispersed data storage network

Family Cites Families (74)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5327570A (en) * 1991-07-22 1994-07-05 International Business Machines Corporation Multiprocessor system having local write cache within each data processor node
US5634060A (en) * 1994-08-09 1997-05-27 Unisys Corporation Method and apparatus for high-speed efficient bi-directional communication between multiple processor over a common bus
US5822550A (en) * 1994-12-22 1998-10-13 Texas Instruments Incorporated Split data path fast at-bus on chip circuits systems and methods
US6016526A (en) * 1997-09-19 2000-01-18 International Business Machines Corporation Method and apparatus for transferring data between buses having differing ordering policies via the use of autonomous units
US6374241B1 (en) * 1999-03-31 2002-04-16 Verizon Laboratories Inc. Data merging techniques
US6338126B1 (en) * 1999-12-06 2002-01-08 Legato Systems, Inc. Crash recovery without complete remirror
JP3992427B2 (en) * 2000-08-01 2007-10-17 株式会社日立製作所 File system
US7707319B2 (en) * 2000-09-18 2010-04-27 Rockwell Automation Technologies, Inc. Noninvasive testing of network interface error codes for safety network
US20020120874A1 (en) 2000-12-22 2002-08-29 Li Shu Method and system for secure exchange of messages
US6757768B1 (en) * 2001-05-17 2004-06-29 Cisco Technology, Inc. Apparatus and technique for maintaining order among requests issued over an external bus of an intermediate network node
US20030014523A1 (en) * 2001-07-13 2003-01-16 John Teloh Storage network data replicator
US7231486B2 (en) * 2001-08-24 2007-06-12 Intel Corporation General input/output architecture, protocol and related methods to support legacy interrupts
US7636724B2 (en) 2001-08-31 2009-12-22 Peerify Technologies LLC Data storage system and method by shredding and deshredding
US6938133B2 (en) * 2001-09-28 2005-08-30 Hewlett-Packard Development Company, L.P. Memory latency and bandwidth optimizations
US7644136B2 (en) * 2001-11-28 2010-01-05 Interactive Content Engines, Llc. Virtual file system
US7036040B2 (en) * 2002-11-26 2006-04-25 Microsoft Corporation Reliability of diskless network-bootable computers using non-volatile memory cache
US7065619B1 (en) * 2002-12-20 2006-06-20 Data Domain, Inc. Efficient data storage system
US20060202186A1 (en) * 2003-01-21 2006-09-14 On The Fence Technologies, Llc. Corporation Methods and apparatus for fencing and other outdoor structures
US7472334B1 (en) * 2003-10-15 2008-12-30 Scott Thomas P Efficient method for the reconstruction of digital information
US7349999B2 (en) * 2003-12-29 2008-03-25 Intel Corporation Method, system, and program for managing data read operations on network controller with offloading functions
US7111092B1 (en) * 2004-04-16 2006-09-19 Cisco Technology, Inc. Buffer management technique for a hypertransport data path protocol
US8688934B1 (en) * 2004-06-30 2014-04-01 Emc Corporation Method for caching data
US7296180B1 (en) * 2004-06-30 2007-11-13 Sun Microsystems, Inc. Method for recovery of data
US8677086B2 (en) * 2004-06-30 2014-03-18 Emc Corporation System for caching data
US7631023B1 (en) * 2004-11-24 2009-12-08 Symantec Operating Corporation Performance-adjusted data allocation in a multi-device file system
US7475167B2 (en) * 2005-04-15 2009-01-06 Intel Corporation Offloading data path functions
US8200887B2 (en) * 2007-03-29 2012-06-12 Violin Memory, Inc. Memory management system and method
US7617370B2 (en) * 2005-04-29 2009-11-10 Netapp, Inc. Data allocation within a storage system architecture
KR100666612B1 (en) * 2005-05-27 2007-01-09 삼성전자주식회사 Semiconductor memory device having a redundancy code check function and memory system having the same
US7376792B2 (en) * 2005-08-17 2008-05-20 International Business Machines Corporation Variable cache data retention system
US7441212B1 (en) * 2005-09-07 2008-10-21 Altera Corporation State machine recognition and optimization
US8880799B2 (en) * 2005-09-30 2014-11-04 Cleversafe, Inc. Rebuilding data on a dispersed storage network
JP4718340B2 (en) * 2006-02-02 2011-07-06 富士通株式会社 Storage system, control method and program
US20070208760A1 (en) * 2006-03-06 2007-09-06 Reuter James M Data-state-describing data structures
US8095699B2 (en) * 2006-09-29 2012-01-10 Mediatek Inc. Methods and apparatus for interfacing between a host processor and a coprocessor
US20080155191A1 (en) * 2006-12-21 2008-06-26 Anderson Robert J Systems and methods for providing heterogeneous storage systems
US8037266B2 (en) * 2007-01-02 2011-10-11 Sandisk Il Ltd. Apparatus and method for archiving digital content
US8533256B2 (en) * 2007-10-09 2013-09-10 Cleversafe, Inc. Object interface to a dispersed data storage network
US7970881B2 (en) * 2008-02-19 2011-06-28 Microsoft Corporation Bypassing uploading of data from a wireless device using outbound attachment caching
US8797178B2 (en) * 2008-03-10 2014-08-05 Microsoft Corporation Efficient stream sharing for multi-user sensor data collection
US8019728B2 (en) * 2008-04-17 2011-09-13 Nec Laboratories America, Inc. Dynamically quantifying and improving the reliability of distributed data storage systems
US8086797B2 (en) * 2008-06-06 2011-12-27 Pivot3 Method and system for distributing commands to targets
EP2342661A4 (en) * 2008-09-16 2013-02-20 File System Labs Llc Matrix-based error correction and erasure code methods and apparatus and applications thereof
US20100174676A1 (en) * 2009-01-06 2010-07-08 International Business Machines Corporation Determining modified data in cache for use during a recovery operation
US20100191907A1 (en) * 2009-01-26 2010-07-29 Lsi Corporation RAID Converter and Methods for Transforming a First RAID Array to a Second RAID Array Without Creating a Backup Copy
US9483656B2 (en) * 2009-04-20 2016-11-01 International Business Machines Corporation Efficient and secure data storage utilizing a dispersed data storage system
US8504847B2 (en) * 2009-04-20 2013-08-06 Cleversafe, Inc. Securing data in a dispersed storage network using shared secret slices
US8527838B2 (en) * 2009-07-31 2013-09-03 Cleversafe, Inc. Memory controller utilizing an error coding dispersal function
US8782086B2 (en) * 2009-08-27 2014-07-15 Cleversafe, Inc. Updating dispersed storage network access control information
US8554994B2 (en) * 2009-09-29 2013-10-08 Cleversafe, Inc. Distributed storage network utilizing memory stripes
US8438456B2 (en) * 2009-10-05 2013-05-07 Cleversafe, Inc. Method and apparatus for dispersed storage of streaming data
US9900150B2 (en) * 2009-10-30 2018-02-20 International Business Machines Corporation Dispersed storage camera device and method of operation
US10509709B2 (en) * 2009-10-30 2019-12-17 Pure Storage, Inc. Indirect storage of data in a dispersed storage system
US8819452B2 (en) * 2009-11-25 2014-08-26 Cleversafe, Inc. Efficient storage of encrypted data in a dispersed storage network
US8370600B2 (en) * 2010-03-12 2013-02-05 Cleversafe, Inc. Dispersed storage unit and method for configuration thereof
US8625635B2 (en) * 2010-04-26 2014-01-07 Cleversafe, Inc. Dispersed storage network frame protocol header
US10447767B2 (en) * 2010-04-26 2019-10-15 Pure Storage, Inc. Resolving a performance issue within a dispersed storage network
US8683259B2 (en) * 2010-05-19 2014-03-25 Cleversafe, Inc. Accessing data in multiple dispersed storage networks
US8621580B2 (en) * 2010-05-19 2013-12-31 Cleversafe, Inc. Retrieving access information in a dispersed storage network
US9842222B2 (en) * 2010-08-25 2017-12-12 International Business Machines Corporation Securely rebuilding an encoded data slice
US10157002B2 (en) * 2010-08-26 2018-12-18 International Business Machines Corporation Migrating an encoded data slice based on an end-of-life memory level of a memory device
US9590838B2 (en) * 2010-11-09 2017-03-07 International Business Machines Corporation Transferring data of a dispersed storage network
US8762479B2 (en) * 2011-06-06 2014-06-24 Cleversafe, Inc. Distributing multi-media content to a plurality of potential accessing devices
US9400714B2 (en) * 2011-06-06 2016-07-26 International Business Machines Corporation Wirelessly communicating a data file
US9460148B2 (en) * 2011-07-06 2016-10-04 International Business Machines Corporation Completing distribution of multi-media content to an accessing device
US8914667B2 (en) * 2011-07-27 2014-12-16 Cleversafe, Inc. Identifying a slice error in a dispersed storage network
US8555130B2 (en) * 2011-10-04 2013-10-08 Cleversafe, Inc. Storing encoded data slices in a dispersed storage unit
US8856549B2 (en) * 2011-11-28 2014-10-07 Cleversafe, Inc. Deleting encoded data slices in a dispersed storage network
US9584359B2 (en) * 2011-12-12 2017-02-28 International Business Machines Corporation Distributed storage and computing of interim data
US9304857B2 (en) * 2011-12-12 2016-04-05 Cleversafe, Inc. Retrieving data from a distributed storage network
US9203902B2 (en) * 2012-01-31 2015-12-01 Cleversafe, Inc. Securely and reliably storing data in a dispersed storage network
US9195684B2 (en) * 2012-03-02 2015-11-24 Cleversafe, Inc. Redundant task execution in a distributed storage and task network
US20130290482A1 (en) * 2012-04-25 2013-10-31 Cleversafe, Inc. Retrieving data in a dispersed storage network
US9537609B2 (en) * 2012-08-02 2017-01-03 International Business Machines Corporation Storing a stream of data in a dispersed storage network

Patent Citations (82)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4092732A (en) * 1977-05-31 1978-05-30 International Business Machines Corporation System for recovering data stored in failed memory unit
US5485474A (en) * 1988-02-25 1996-01-16 The President And Fellows Of Harvard College Scheme for information dispersal and reconstruction
US5454101A (en) * 1992-09-15 1995-09-26 Universal Firmware Industries, Ltd. Data storage system with set lists which contain elements associated with parents for defining a logical hierarchy and general record pointers identifying specific data sets
US5987622A (en) * 1993-12-10 1999-11-16 Tm Patents, Lp Parallel computer system including parallel storage subsystem including facility for correction of data in the event of failure of a storage device in parallel storage subsystem
US6175571B1 (en) * 1994-07-22 2001-01-16 Network Peripherals, Inc. Distributed memory switching hub
US6418539B1 (en) * 1995-05-25 2002-07-09 Compaq Computer Corporation Continuously available computer memory systems
US5774643A (en) * 1995-10-13 1998-06-30 Digital Equipment Corporation Enhanced raid write hole protection and recovery
US5809285A (en) * 1995-12-21 1998-09-15 Compaq Computer Corporation Computer system having a virtual drive array controller
US6012159A (en) * 1996-01-17 2000-01-04 Kencast, Inc. Method and system for error-free data transfer
US5802364A (en) * 1996-04-15 1998-09-01 Sun Microsystems, Inc. Metadevice driver rename/exchange technique for a computer system incorporating a plurality of independent device drivers
US5890156A (en) * 1996-05-02 1999-03-30 Alcatel Usa, Inc. Distributed redundant database
US6058454A (en) * 1997-06-09 2000-05-02 International Business Machines Corporation Method and system for automatically configuring redundant arrays of disk memory devices
US5991414A (en) * 1997-09-12 1999-11-23 International Business Machines Corporation Method and apparatus for the secure distributed storage and retrieval of information
US6192472B1 (en) * 1997-09-12 2001-02-20 International Business Machines Corporation Method and apparatus for the secure distributed storage and retrieval of information
US6128277A (en) * 1997-10-01 2000-10-03 California Inst Of Techn Reliable array of distributed computing nodes
US6567948B2 (en) * 1997-10-27 2003-05-20 Kencast, Inc. Method and system for reliable broadcasting of data files and streams
US6272658B1 (en) * 1997-10-27 2001-08-07 Kencast, Inc. Method and system for reliable broadcasting of data files and streams
US6301604B1 (en) * 1997-12-01 2001-10-09 Matsushita Electric Industrial Co., Ltd. Multimedia server
US6256688B1 (en) * 1997-12-02 2001-07-03 Casio Computer Co., Ltd. Interface apparatus operable by using floppy disk drive
US6374336B1 (en) * 1997-12-24 2002-04-16 Avid Technology, Inc. Computer system and process for transferring multiple high bandwidth streams of data between multiple storage units and multiple applications in a scalable and reliable manner
US6785768B2 (en) * 1997-12-24 2004-08-31 Avid Technology, Inc. Computer system and process for transferring streams of data between multiple storage units and multiple applications in a scalable and reliable manner
US6415373B1 (en) * 1997-12-24 2002-07-02 Avid Technology, Inc. Computer system and process for transferring multiple high bandwidth streams of data between multiple storage units and multiple applications in a scalable and reliable manner
US6449688B1 (en) * 1997-12-24 2002-09-10 Avid Technology, Inc. Computer system and process for transferring streams of data between multiple storage units and multiple applications in a scalable and reliable manner
US6760808B2 (en) * 1997-12-24 2004-07-06 Avid Technology, Inc. Computer system and process for transferring multiple high bandwidth streams of data between multiple storage units and multiple applications in a scalable and reliable manner
US7111115B2 (en) * 1997-12-24 2006-09-19 Avid Technology, Inc. Computer system and process for transferring multiple high bandwidth streams of data between multiple storage units and multiple applications in a scalable and reliable manner
US6366995B1 (en) * 1998-08-19 2002-04-02 Acuid Corporation Limited System and a method for defining transforms of memory device addresses
US6356949B1 (en) * 1999-01-29 2002-03-12 Intermec Ip Corp. Automatic data collection device that receives data output instruction from data consumer
US6609223B1 (en) * 1999-04-06 2003-08-19 Kencast, Inc. Method for packet-level fec encoding, in which on a source packet-by-source packet basis, the error correction contributions of a source packet to a plurality of wildcard packets are computed, and the source packet is transmitted thereafter
US6571282B1 (en) * 1999-08-31 2003-05-27 Accenture Llp Block-based communication in a communication services patterns environment
US6826711B2 (en) * 2000-02-18 2004-11-30 Avamar Technologies, Inc. System and method for data protection with multidimensional parity
US6718361B1 (en) * 2000-04-07 2004-04-06 Network Appliance Inc. Method and apparatus for reliable and scalable distribution of data files in distributed networks
US20020184463A1 (en) * 2000-07-06 2002-12-05 Hitachi, Ltd. Computer system
US20050027735A1 (en) * 2000-08-24 2005-02-03 Microsoft Corporation Method and system for relocating files that are partially stored in remote storage
US7272613B2 (en) * 2000-10-26 2007-09-18 Intel Corporation Method and system for managing distributed content and related metadata
US7140044B2 (en) * 2000-11-13 2006-11-21 Digital Doors, Inc. Data security system and method for separation of user communities
US7146644B2 (en) * 2000-11-13 2006-12-05 Digital Doors, Inc. Data security system and method responsive to electronic attacks
US7103915B2 (en) * 2000-11-13 2006-09-05 Digital Doors, Inc. Data security system and method
US20050132070A1 (en) * 2000-11-13 2005-06-16 Redlich Ron M. Data security system and method with editor
US20020062422A1 (en) * 2000-11-18 2002-05-23 International Business Machines Corporation Method for rebuilding meta-data in a data storage system and a data storage system
US6785783B2 (en) * 2000-11-30 2004-08-31 International Business Machines Corporation NUMA system with redundant main memory architecture
US7080101B1 (en) * 2000-12-01 2006-07-18 Ncr Corp. Method and apparatus for partitioning data for storage in a database
US20050125593A1 (en) * 2001-01-11 2005-06-09 Yotta Yotta, Inc. Storage virtualization system and methods
US7054927B2 (en) * 2001-01-29 2006-05-30 Adaptec, Inc. File system metadata describing server directory information
US20020166079A1 (en) * 2001-01-29 2002-11-07 Ulrich Thomas R. Dynamic data recovery
US20030037261A1 (en) * 2001-03-26 2003-02-20 Ilumin Corporation Secured content delivery system and method
US6879596B1 (en) * 2001-04-11 2005-04-12 Applied Micro Circuits Corporation System and method for systolic array sorting of information segments
US7024609B2 (en) * 2001-04-20 2006-04-04 Kencast, Inc. System for protecting the transmission of live data streams, and upon reception, for reconstructing the live data streams and recording them into files
US20030065617A1 (en) * 2001-06-30 2003-04-03 Watkins Mark Robert Method of billing for utilization of a data storage array, and an array controller therefor
US20030018927A1 (en) * 2001-07-23 2003-01-23 Gadir Omar M.A. High-availability cluster virtual server system
US7024451B2 (en) * 2001-11-05 2006-04-04 Hewlett-Packard Development Company, L.P. System and method for maintaining consistent independent server-side state among collaborating servers
US7003688B1 (en) * 2001-11-15 2006-02-21 Xiotech Corporation System and method for a reserved memory area shared by all redundant storage controllers
US7171493B2 (en) * 2001-12-19 2007-01-30 The Charles Stark Draper Laboratory Camouflage of network traffic to resist attack
US7103824B2 (en) * 2002-07-29 2006-09-05 Robert Halford Multi-dimensional data protection and mirroring method for micro level data
US20040024963A1 (en) * 2002-08-05 2004-02-05 Nisha Talagala Method and system for striping data to accommodate integrity metadata
US20040153479A1 (en) * 2002-11-14 2004-08-05 Mikesell Paul A. Systems and methods for restriping files in a distributed file system
US20040122917A1 (en) * 2002-12-18 2004-06-24 Menon Jaishankar Moothedath Distributed storage system for data-sharing among client computers running defferent operating system types
US20070283167A1 (en) * 2003-03-13 2007-12-06 Venters Carl V Iii Secure streaming container
US20040199566A1 (en) * 2003-03-14 2004-10-07 International Business Machines Corporation System, method, and apparatus for policy-based data management
US20070088970A1 (en) * 2003-04-10 2007-04-19 Lenovo (Singapore) Pte.Ltd Recovery from failures within data processing systems
US20040215998A1 (en) * 2003-04-10 2004-10-28 International Business Machines Corporation Recovery from failures within data processing systems
US20040228493A1 (en) * 2003-05-14 2004-11-18 Kenneth Ma Method and system for disaster recovery of data from a storage device
US20070234110A1 (en) * 2003-08-14 2007-10-04 Soran Philip E Virtual Disk Drive System and Method
US20050100022A1 (en) * 2003-11-12 2005-05-12 Ramprashad Sean A. Media delivery using quality of service differentiation within a media stream
US20050114594A1 (en) * 2003-11-24 2005-05-26 Corbett Peter F. Semi-static distribution technique
US20050131993A1 (en) * 2003-12-15 2005-06-16 Fatula Joseph J.Jr. Apparatus, system, and method for autonomic control of grid system resources
US20050144382A1 (en) * 2003-12-29 2005-06-30 Schmisseur Mark A. Method, system, and program for managing data organization
US7222133B1 (en) * 2004-02-05 2007-05-22 Unisys Corporation Method for reducing database recovery time
US7240236B2 (en) * 2004-03-23 2007-07-03 Archivas, Inc. Fixed content distributed data storage using permutation ring encoding
US20050229069A1 (en) * 2004-04-02 2005-10-13 Hitachi Global Storage Technologies Netherlands, B.V. Techniques for detecting and correcting errors using multiple interleave erasure pointers
US20060047907A1 (en) * 2004-08-30 2006-03-02 Hitachi, Ltd. Storage system and a storage management system
US20060136448A1 (en) * 2004-12-20 2006-06-22 Enzo Cialini Apparatus, system, and method for database provisioning
US20060156059A1 (en) * 2005-01-13 2006-07-13 Manabu Kitamura Method and apparatus for reconstructing data in object-based storage arrays
US20060224603A1 (en) * 2005-04-05 2006-10-05 Wal-Mart Stores, Inc. System and methods for facilitating a linear grid database with data organization by dimension
US20070174192A1 (en) * 2005-09-30 2007-07-26 Gladwin S C Billing system for information dispersal system
US20070079083A1 (en) * 2005-09-30 2007-04-05 Gladwin S Christopher Metadata management system for an information dispersed storage system
US20070079081A1 (en) * 2005-09-30 2007-04-05 Cleversafe, Llc Digital data storage system
US20070079082A1 (en) * 2005-09-30 2007-04-05 Gladwin S C System for rebuilding dispersed data
US20090094318A1 (en) * 2005-09-30 2009-04-09 Gladwin S Christopher Smart access to a dispersed data storage network
US20070214285A1 (en) * 2006-03-08 2007-09-13 Omneon Video Networks Gateway server
US20080068899A1 (en) * 2006-09-15 2008-03-20 Fujitsu Limited Storage management process, storage management apparatus, and computer-readable medium storing storage management program
US20090094251A1 (en) * 2007-10-09 2009-04-09 Gladwin S Christopher Virtualized data storage vaults on a dispersed data storage network
US20100023524A1 (en) * 2007-10-09 2010-01-28 Gladwin S Christopher Block based access to a dispersed data storage network

Cited By (24)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8751829B2 (en) 2009-02-05 2014-06-10 Wwpass Corporation Dispersed secure data storage and retrieval
US8839391B2 (en) 2009-02-05 2014-09-16 Wwpass Corporation Single token authentication
US8327141B2 (en) 2009-02-05 2012-12-04 Wwpass Corporation Centralized authentication system with safe private data storage and method
US8826019B2 (en) 2009-02-05 2014-09-02 Wwpass Corporation Centralized authentication system with safe private data storage and method
US8752153B2 (en) 2009-02-05 2014-06-10 Wwpass Corporation Accessing data based on authenticated user, provider and system
US8713661B2 (en) 2009-02-05 2014-04-29 Wwpass Corporation Authentication service
US9025608B2 (en) * 2009-09-02 2015-05-05 Zte Corporation Route switching device and data cashing method thereof
US20120163394A1 (en) * 2009-09-02 2012-06-28 Zte Corporation Route Switching Device and Data Cashing Method Thereof
US8918534B2 (en) * 2009-09-29 2014-12-23 Cleversafe, Inc. Writing data slices to ready and non-ready distributed storage units in a distributed storage network
US20110078277A1 (en) * 2009-09-29 2011-03-31 Cleversafe, Inc. Handling unavailable memories in distributed storage network
US10097863B2 (en) * 2010-08-22 2018-10-09 Qwilt, Inc. System and method for live service content handling with content storing servers caching popular content therein
US11032583B2 (en) 2010-08-22 2021-06-08 QWLT, Inc. Method and system for improving high availability for live content
US10812837B2 (en) 2010-08-22 2020-10-20 Qwilt, Inc System and method for live service content handling with content storing servers caching popular content therein
US20160212454A1 (en) * 2010-08-22 2016-07-21 Qwilt, Inc. System and method for live service content handling with content storing servers caching popular content therein
US10127335B2 (en) 2010-08-22 2018-11-13 Qwilt, Inc System and method of performing analytics with respect to content storing servers caching popular content
US10044802B2 (en) 2010-08-22 2018-08-07 Qwilt, Inc. System for detection of content servers and caching popular content therein
US10097428B2 (en) 2010-08-22 2018-10-09 Qwilt, Inc. System and method for caching popular content respective of a content strong server in an asymmetrical routing topology
US8972719B2 (en) 2011-12-06 2015-03-03 Wwpass Corporation Passcode restoration
US8656180B2 (en) 2011-12-06 2014-02-18 Wwpass Corporation Token activation
US8555079B2 (en) 2011-12-06 2013-10-08 Wwpass Corporation Token management
US20220308962A1 (en) * 2012-06-25 2022-09-29 Pure Storage, Inc. Selection of Memory for Data Storage in a Storage Network
US11240335B2 (en) 2014-04-22 2022-02-01 Qwilt, Inc. System and methods thereof for delivery of popular content using a multimedia broadcast multicast service
US20170277453A1 (en) * 2016-03-24 2017-09-28 Microsoft Technology Licensing, Llc Distributed metadata management in a distributed storage system
US10585627B2 (en) * 2016-03-24 2020-03-10 Microsoft Technology Licensing, Llc Distributed metadata management in a distributed storage system

Also Published As

Publication number Publication date
US8473677B2 (en) 2013-06-25
US8862800B2 (en) 2014-10-14
US20110078277A1 (en) 2011-03-31
US20120265937A1 (en) 2012-10-18
US8554994B2 (en) 2013-10-08
US20130283125A1 (en) 2013-10-24
US9274890B2 (en) 2016-03-01
US20110078371A1 (en) 2011-03-31
US8918534B2 (en) 2014-12-23
US20110078372A1 (en) 2011-03-31

Similar Documents

Publication Publication Date Title
US9274890B2 (en) Distributed storage network memory access based on memory state
US11675502B1 (en) Transferring encoded data slices stored in flash memory of a storage network
US11194672B1 (en) Storage network with connection security and methods for use therewith
US10997136B2 (en) Method and apparatus for identifying data inconsistency in a dispersed storage network
US9870795B2 (en) Localized dispersed storage memory system
US9774680B2 (en) Distributed rebuilding of data in a dispersed storage network
US9122596B2 (en) Updating a set of memory devices in a dispersed storage network
US9836352B2 (en) Detecting a utilization imbalance between dispersed storage network storage units
US9626248B2 (en) Likelihood based rebuilding of missing encoded data slices
US9489264B2 (en) Storing an encoded data slice as a set of sub-slices

Legal Events

Date Code Title Description
AS Assignment

Owner name: CLEVERSAFE, INC., ILLINOIS

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:RESCH, JASON K.;GLADWIN, S. CHRISTOPHER;SIGNING DATES FROM 20100505 TO 20100510;REEL/FRAME:024368/0484

STCB Information on status: application discontinuation

Free format text: ABANDONED -- AFTER EXAMINER'S ANSWER OR BOARD OF APPEALS DECISION

AS Assignment

Owner name: INTERNATIONAL BUSINESS MACHINES CORPORATION, NEW YORK

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:CLEVERSAFE, INC.;REEL/FRAME:038629/0015

Effective date: 20160405

Owner name: INTERNATIONAL BUSINESS MACHINES CORPORATION, NEW Y

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:CLEVERSAFE, INC.;REEL/FRAME:038629/0015

Effective date: 20160405

AS Assignment

Owner name: PURE STORAGE, INC., CALIFORNIA

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:INTERNATIONAL BUSINESS MACHINES CORPORATION;REEL/FRAME:049555/0530

Effective date: 20190611

AS Assignment

Owner name: PURE STORAGE, INC., CALIFORNIA

Free format text: CORRECTIVE ASSIGNMENT TO CORRECT THE DELETE 15/174/279 AND 15/174/596 PROPERTY NUMBERS PREVIOUSLY RECORDED AT REEL: 49555 FRAME: 530. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT;ASSIGNOR:INTERNATIONAL BUSINESS MACHINES CORPORATION;REEL/FRAME:051495/0831

Effective date: 20190611