US8082547B1 - Reallocating hardware resources among workloads in accordance with license rights - Google Patents
Reallocating hardware resources among workloads in accordance with license rights Download PDFInfo
- Publication number
- US8082547B1 US8082547B1 US11/590,584 US59058406A US8082547B1 US 8082547 B1 US8082547 B1 US 8082547B1 US 59058406 A US59058406 A US 59058406A US 8082547 B1 US8082547 B1 US 8082547B1
- Authority
- US
- United States
- Prior art keywords
- workload
- resource units
- workloads
- partition
- hardware resource
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related, expires
Links
- 238000000034 method Methods 0.000 claims abstract description 42
- 238000005192 partition Methods 0.000 claims description 51
- 238000007726 management method Methods 0.000 claims description 9
- 238000013439 planning Methods 0.000 claims description 3
- 230000000415 inactivating effect Effects 0.000 claims 1
- 230000006870 function Effects 0.000 description 3
- 238000004590 computer program Methods 0.000 description 2
- 230000001419 dependent effect Effects 0.000 description 2
- 238000013468 resource allocation Methods 0.000 description 2
- 230000003190 augmentative effect Effects 0.000 description 1
- 239000012141 concentrate Substances 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5061—Partitioning or combining of resources
- G06F9/5077—Logical partitioning of resources; Management or configuration of virtualized resources
Definitions
- Servers e.g., web servers, database servers, are computers that provide services to other computers. License fees for server software are often based on the hardware resources available to run the software. Thus, the fees for running software restricted to an 8-CPU partition of a 32-CPU server can be much less than software permitted to run on the full system.
- Virtualization and other technologies provide for software-controlled reallocation of hardware resources to partitions. This means that restricting software to a partition does not restrict it to a fixed amount of resources. Accordingly, software licenses may have to provide for the maximum number of resources that can be allocated to a partition, which can lead to wasteful over-provisioning on the licensee's part. As is apparent from the detailed description below with reference to the following drawings, the present invention addresses the problem of license over-provisioning in servers that allow software-controlled reallocation of resources to partitions.
- FIG. 1 is a schematic diagram of a server and associated method in accordance with an embodiment of the invention.
- FIG. 2 is a more detailed flow chart of the method of FIG. 1 .
- the present invention provides a server workload management function that iteratively allocates software license rights along with hardware resources to workloads in a manner analogous to the allocation of hardware resources to workloads.
- a resource (hardware or software license right or a combination) is allocated to the highest priority workload as defined by the policies and taking into account resources assigned during previous iterations.
- both hardware resources and software license rights are distributed optimally as defined by management policies.
- a licensor can securely offer more limited and, thus, more economical, licenses to customers, reducing the need for a customer to over-provision licenses.
- a server AP 1 includes partitions P 1 -P 3 . Each partition has processor, memory, and input/output resources assigned to it. As shown in FIG. 1 , processors (also known as “central processing units” (CPUs) C 01 -C 04 are assigned to partition P 1 , CPUs C 05 -C 08 are assigned to partition P 2 , and CPUs C 09 -C 12 are assigned to partition P 3 .
- processors also known as “central processing units” (CPUs) C 01 -C 04 are assigned to partition P 1
- CPUs C 05 -C 08 are assigned to partition P 2
- CPUs C 09 -C 12 are assigned to partition P 3 .
- Partitions P 1 -P 3 are running respective workloads A 1 , A 2 , and B 1 .
- Each workload can include an operating system and one or more applications.
- Workloads A 1 and A 2 are two instances of the same application, e.g., the same database application, while workload B 1 is an instance of another application.
- the database application of workload A 1 can be a database for one department of a company, while the database application of workload A 2 can be running a second database for a different department of the company.
- Workload B 1 can be a web server application.
- the workload management function WMF is implemented by workload managers WL 1 -WL 3 , which are software agents running respectively in partitions P 1 -P 3 .
- Each workload manager WL 1 -WL 3 has access to a respective copy of workload management policies MP 1 -MP 3 and a respective copy of license data LD 1 -LD 3 .
- workload managers WL 1 -WL 3 provide for automated reallocation of resources. The reallocations are made based on policies MP 1 -MP 3 and license data LD 1 -LD 3 .
- workload managers WL 1 -WL 3 collectively implement a method ME 1 flow-charted in FIG. 1 and shown tangibly embodied in computer readable storage media 15 .
- workload manager WL 1 recognizes a need for more hardware resources to be allocated to its respective workload A 1 running in partition P 1 . More specifically, workload manager WL 1 has detected that 90% of the available CPU resources for partition P 1 are being utilized. Policies MP 1 indicated that CPU resources should be added to a partition at this level of utilization to provide adequate headroom for a potential spike in demand. As indicated in the alternative representation of method ME 1 in FIG.
- recognition of a need to reallocation can involve 1) a request from a workload manager for more resources, e.g., because of high utilization; 2) an offer by a workload manager to relinquish idle resources; 3) a scheduled reallocation (e.g., day versus night allocations); or 4) user initiated reallocations, as shown at method segment M 1 A.
- Method segment M 2 Whether in response to a resource utilization problem or a scheduled event, a new allocation of resources to workloads is determined at method segment M 2 .
- Optimal resource allocation can be a complex problem.
- Method segment M 2 breaks the problem down into iterations in which a single resource or a bundle of resources is assigned to a workload based on priorities determined by management policies MP 1 -MP 3 . Policy considerations can include the importance of the workload, the expected utilization of the workload, and the resources already allocated to the workload in previous iterations.
- planning a reallocation can involve assigning, at method segment M 2 A, a first minimum granularity resource unit to the highest priority workload as determined by management policies. Then, dependent resource allocations are made at method segment M 2 B. For example, if a partition runs two workloads, e.g., two virtual machines running on an OS, then a fraction of a CPU might be assigned to one of the workloads. However, the partition might have a minimum granularity of a whole CPU, so method segment M 2 A would involve assigning a fraction of a CPU to the workload and method segment M 2 B would involve assigning a whole CPU to the partition (if the resources previously assigned to that partition did not provide for the fraction).
- method ME 1 returns to method segment M 2 A to allocate the next resource unit.
- method ME 1 Each time a resource is allocated at method segment M 2 A, its dependencies are assigned at method segment M 2 B unless all dependent resource assignments have already been made. When all resources have been assigned, method ME 1 continues with method segment M 3 A.
- the resource is a software license right
- it can only be assigned to qualified workloads, e.g., one of perhaps plural instances of an operating system or application to which the right applies.
- license data LD 1 -LD 3 may indicate that a database program is licensed for up to four instances and a total of eight CPUs.
- Server AP 1 can have two instances of the database, with one having a higher priority than the other. The first license right would be assigned to the higher priority instance.
- a later license right might be assigned to the same instance or to the lower priority instance (e.g., because the “needs” of the originally higher priority instance had been relatively satisfied).
- Hardware resources could then be assigned to workloads as license rights permit.
- the new allocation is implemented.
- reallocation can involve determining a license-compatible least-disruptive series of steps to implement the new allocation and then implementation of those steps.
- Method segment M 3 of FIG. 1 can be broken down into method segments M 3 A-M 3 D as shown in FIG. 2 to ensure that license terms are complied with during the actual reallocation.
- source hardware is deactivated.
- license rights are removed from source workloads.
- license rights are added to the target workload.
- the inactivated hardware resources are transferred to the target workload and reactivated.
- the new allocation is the same as the old, no changes are required.
- method ME 1 provides for enforcing licensing rights at method segment M 4 . If no workload has been assigned unlicensed resources, this enforcement is trivial. However there may be cases where a workload manager commands an operating system to limit access to hardware resources, as indicated at method segment 4 A in FIG. 2 . For example, the operating system might be licensed to use five cores, but the cores come four to a processor, the lowest unit that can be assigned to a partition. In that case, two processors and eight cores may be assigned to a partition, while the workload manager limits the operating system to using five cores.
- the licenses for the database applications running in partitions P 1 and P 2 each permit four CPUs, but can be pooled so that licensing restrictions are met as long as the total number of CPUs for both partitions is no more than eight.
- “one CPU” of the license for partition P 2 is transferred to partition P 1 at method segment M 4 .
- one CPU e.g., CPU C 05 is transferred from partition P 2 to partition P 1 , relieving the high utilization level of P 1 . (This is the transfer indicated by the arrow from CPU C 05 to partition P 1 in FIG. 1 .)
- Moving a CPU to a different partition can involve several steps, including allowing threads running on it to terminate, preventing new threads from starting on that CPU, and then transferring the CPU.
- policies can specify conditions under which additional hardware and software licenses can be purchased.
- a server may have reserved processors that can be “instantly activated” under a pre-arranged fee provision.
- a software license might have provisions for instant expansion under a pre-arranged fee provision.
- the amount of resources and license rights to be allocated can be varied by the workload management function.
- “software agents” are computer programs, and a computer “workload” is a program or set of programs.
- a “computer program” or more simply a “program” is an ordered set of instructions tangibly embodied in computer-readable storage media and interpretable and executable by a central processing unit.
- program does not encompass purely abstract ideas, natural phenomena, or laws of nature.
- a “program set” is a set of one or more programs. All programs described herein effect changes in state in computer-readable memory.
- the invention is illustrated for a system with three hard partitions, it is applicable to systems with different numbers of partitions, and for systems with virtual instead of hard partitions.
- the invention can be applied to un-partitioned systems provided a workload manager controls the allocations of resources to workloads.
- the policies can involve utilization levels, usage predictions, and business priorities, among other considerations.
- the policies can seek to evenly distribute workloads or concentrate them so that some resources can be powered down.
- the license terms can vary and provide various means for augmenting a license to permit a reallocation, including automatic payment for an additional resource.
- the resources can be CPUs, memory, I/O devices, and various combinations and types of those classes of computer components.
Abstract
Description
Claims (15)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US11/590,584 US8082547B1 (en) | 2006-10-31 | 2006-10-31 | Reallocating hardware resources among workloads in accordance with license rights |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US11/590,584 US8082547B1 (en) | 2006-10-31 | 2006-10-31 | Reallocating hardware resources among workloads in accordance with license rights |
Publications (1)
Publication Number | Publication Date |
---|---|
US8082547B1 true US8082547B1 (en) | 2011-12-20 |
Family
ID=45219313
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US11/590,584 Expired - Fee Related US8082547B1 (en) | 2006-10-31 | 2006-10-31 | Reallocating hardware resources among workloads in accordance with license rights |
Country Status (1)
Country | Link |
---|---|
US (1) | US8082547B1 (en) |
Cited By (11)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20110258633A1 (en) * | 2008-10-21 | 2011-10-20 | Fujitsu Limited | Information processing system and use right collective management method |
US8650296B1 (en) * | 2006-10-31 | 2014-02-11 | Hewlett-Packard Development Company, L.P. | Workload reallocation involving inter-server transfer of software license rights and intra-server transfer of hardware resources |
US9110731B1 (en) * | 2012-08-15 | 2015-08-18 | Xiotech Corporation | Hard allocation of resources partitioning |
WO2016133998A1 (en) * | 2015-02-17 | 2016-08-25 | Amazon Technologies, Inc. | System on a chip comprising an i/o steering engine |
US20180007127A1 (en) * | 2016-06-30 | 2018-01-04 | International Business Machines Corporation | Managing software licenses in a disaggregated environment |
US20180300176A1 (en) * | 2017-04-17 | 2018-10-18 | Red Hat, Inc. | Self-programmable and self-tunable resource scheduler for jobs in cloud computing |
US10394731B2 (en) | 2014-12-19 | 2019-08-27 | Amazon Technologies, Inc. | System on a chip comprising reconfigurable resources for multiple compute sub-systems |
US10523585B2 (en) | 2014-12-19 | 2019-12-31 | Amazon Technologies, Inc. | System on a chip comprising multiple compute sub-systems |
US11200192B2 (en) | 2015-02-13 | 2021-12-14 | Amazon Technologies. lac. | Multi-mode system on a chip |
US20220092154A1 (en) * | 2019-09-27 | 2022-03-24 | Intel Corporation | Software defined silicon implementation and management |
US11579897B2 (en) | 2019-09-27 | 2023-02-14 | Intel Corporation | Systems, methods, and apparatus for software defined silicon security |
Citations (15)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5579222A (en) * | 1991-11-27 | 1996-11-26 | Intergraph Corporation | Distributed license administration system using a local policy server to communicate with a license server and control execution of computer programs |
US5745879A (en) * | 1991-05-08 | 1998-04-28 | Digital Equipment Corporation | Method and system for managing execution of licensed programs |
US20020016892A1 (en) * | 1997-11-04 | 2002-02-07 | Stephen H. Zalewski | Multiprocessor computer architecture with multiple operating system instances and software controlled resource allocation |
US20020087611A1 (en) * | 2000-12-28 | 2002-07-04 | Tsuyoshi Tanaka | Virtual computer system with dynamic resource reallocation |
US20020169625A1 (en) * | 2001-05-11 | 2002-11-14 | Eric Yang | Software licensing management system |
US6859793B1 (en) * | 2002-12-19 | 2005-02-22 | Networks Associates Technology, Inc. | Software license reporting and control system and method |
US20050132347A1 (en) * | 2003-12-15 | 2005-06-16 | Harper Eric D. | System for controlling the use of a software application on a plurality of computers |
US20050216716A1 (en) * | 2004-03-29 | 2005-09-29 | Hoffman Philip M | System and method for licensing and distribution of I/O in partitioned computer systems |
US6957435B2 (en) * | 2001-04-19 | 2005-10-18 | International Business Machines Corporation | Method and apparatus for allocating processor resources in a logically partitioned computer system |
US6978374B1 (en) * | 2000-09-29 | 2005-12-20 | Unisys Corporation | Authorization key system for selectively controlling the performance of a data processing system |
US6988134B2 (en) | 2002-04-18 | 2006-01-17 | Bdna Corporation | Apparatus and method to automatically collect data regarding assets of a business entity |
US20060053215A1 (en) * | 2004-09-07 | 2006-03-09 | Metamachinix, Inc. | Systems and methods for providing users with access to computer resources |
US7035918B1 (en) * | 1999-09-03 | 2006-04-25 | Safenet Canada. Inc. | License management system and method with multiple license servers |
US7089306B2 (en) | 2002-04-18 | 2006-08-08 | Bdna Corporation | Apparatus and method to automatically collect data regarding assets of a business entity |
US7096469B1 (en) * | 2000-10-02 | 2006-08-22 | International Business Machines Corporation | Method and apparatus for enforcing capacity limitations in a logically partitioned system |
-
2006
- 2006-10-31 US US11/590,584 patent/US8082547B1/en not_active Expired - Fee Related
Patent Citations (15)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5745879A (en) * | 1991-05-08 | 1998-04-28 | Digital Equipment Corporation | Method and system for managing execution of licensed programs |
US5579222A (en) * | 1991-11-27 | 1996-11-26 | Intergraph Corporation | Distributed license administration system using a local policy server to communicate with a license server and control execution of computer programs |
US20020016892A1 (en) * | 1997-11-04 | 2002-02-07 | Stephen H. Zalewski | Multiprocessor computer architecture with multiple operating system instances and software controlled resource allocation |
US7035918B1 (en) * | 1999-09-03 | 2006-04-25 | Safenet Canada. Inc. | License management system and method with multiple license servers |
US6978374B1 (en) * | 2000-09-29 | 2005-12-20 | Unisys Corporation | Authorization key system for selectively controlling the performance of a data processing system |
US7096469B1 (en) * | 2000-10-02 | 2006-08-22 | International Business Machines Corporation | Method and apparatus for enforcing capacity limitations in a logically partitioned system |
US20020087611A1 (en) * | 2000-12-28 | 2002-07-04 | Tsuyoshi Tanaka | Virtual computer system with dynamic resource reallocation |
US6957435B2 (en) * | 2001-04-19 | 2005-10-18 | International Business Machines Corporation | Method and apparatus for allocating processor resources in a logically partitioned computer system |
US20020169625A1 (en) * | 2001-05-11 | 2002-11-14 | Eric Yang | Software licensing management system |
US6988134B2 (en) | 2002-04-18 | 2006-01-17 | Bdna Corporation | Apparatus and method to automatically collect data regarding assets of a business entity |
US7089306B2 (en) | 2002-04-18 | 2006-08-08 | Bdna Corporation | Apparatus and method to automatically collect data regarding assets of a business entity |
US6859793B1 (en) * | 2002-12-19 | 2005-02-22 | Networks Associates Technology, Inc. | Software license reporting and control system and method |
US20050132347A1 (en) * | 2003-12-15 | 2005-06-16 | Harper Eric D. | System for controlling the use of a software application on a plurality of computers |
US20050216716A1 (en) * | 2004-03-29 | 2005-09-29 | Hoffman Philip M | System and method for licensing and distribution of I/O in partitioned computer systems |
US20060053215A1 (en) * | 2004-09-07 | 2006-03-09 | Metamachinix, Inc. | Systems and methods for providing users with access to computer resources |
Non-Patent Citations (3)
Title |
---|
Chandra et al., Dynamic Resource Allocation for Shared Data Centers Using Online Measurements, Springer-Verlag Berlin Heidelberg, pp. 381-398, 2003. * |
FLEXIm Reference Manual, Macrovision, Version 9.5, Aug. 2004. * |
FLEXnet Licensing End User Guide, Macrovision, Version 10.8, Jul. 2005. * |
Cited By (17)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8650296B1 (en) * | 2006-10-31 | 2014-02-11 | Hewlett-Packard Development Company, L.P. | Workload reallocation involving inter-server transfer of software license rights and intra-server transfer of hardware resources |
US8533725B2 (en) * | 2008-10-21 | 2013-09-10 | Fujitsu Limited | Information processing system and use right collective management method |
US20110258633A1 (en) * | 2008-10-21 | 2011-10-20 | Fujitsu Limited | Information processing system and use right collective management method |
US9110731B1 (en) * | 2012-08-15 | 2015-08-18 | Xiotech Corporation | Hard allocation of resources partitioning |
US10394731B2 (en) | 2014-12-19 | 2019-08-27 | Amazon Technologies, Inc. | System on a chip comprising reconfigurable resources for multiple compute sub-systems |
US10523585B2 (en) | 2014-12-19 | 2019-12-31 | Amazon Technologies, Inc. | System on a chip comprising multiple compute sub-systems |
US11200192B2 (en) | 2015-02-13 | 2021-12-14 | Amazon Technologies. lac. | Multi-mode system on a chip |
WO2016133998A1 (en) * | 2015-02-17 | 2016-08-25 | Amazon Technologies, Inc. | System on a chip comprising an i/o steering engine |
US9588921B2 (en) | 2015-02-17 | 2017-03-07 | Amazon Technologies, Inc. | System on a chip comprising an I/O steering engine |
US20180007127A1 (en) * | 2016-06-30 | 2018-01-04 | International Business Machines Corporation | Managing software licenses in a disaggregated environment |
US10701141B2 (en) * | 2016-06-30 | 2020-06-30 | International Business Machines Corporation | Managing software licenses in a disaggregated environment |
US20180300176A1 (en) * | 2017-04-17 | 2018-10-18 | Red Hat, Inc. | Self-programmable and self-tunable resource scheduler for jobs in cloud computing |
US11334391B2 (en) * | 2017-04-17 | 2022-05-17 | Red Hat, Inc. | Self-programmable and self-tunable resource scheduler for jobs in cloud computing |
US20220092154A1 (en) * | 2019-09-27 | 2022-03-24 | Intel Corporation | Software defined silicon implementation and management |
US11573830B2 (en) * | 2019-09-27 | 2023-02-07 | Intel Corporation | Software defined silicon implementation and management |
US11579897B2 (en) | 2019-09-27 | 2023-02-14 | Intel Corporation | Systems, methods, and apparatus for software defined silicon security |
US11599368B2 (en) | 2019-09-27 | 2023-03-07 | Intel Corporation | Device enhancements for software defined silicon implementations |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US8082547B1 (en) | Reallocating hardware resources among workloads in accordance with license rights | |
US11681562B2 (en) | Resource manager for managing the sharing of resources among multiple workloads in a distributed computing environment | |
US8650296B1 (en) | Workload reallocation involving inter-server transfer of software license rights and intra-server transfer of hardware resources | |
US8185907B2 (en) | Method and system for assigning logical partitions to multiple shared processor pools | |
Delimitrou et al. | Hcloud: Resource-efficient provisioning in shared cloud systems | |
CN108701059B (en) | Multi-tenant resource allocation method and system | |
US9886322B2 (en) | System and method for providing advanced reservations in a compute environment | |
US10191771B2 (en) | System and method for resource management | |
US9298514B2 (en) | System and method for enforcing future policies in a compute environment | |
Ananthanarayanan et al. | True elasticity in multi-tenant data-intensive compute clusters | |
US9465663B2 (en) | Allocating resources in a compute farm to increase resource utilization by using a priority-based allocation layer to allocate job slots to projects | |
US9021490B2 (en) | Optimizing allocation of computer resources by tracking job status and resource availability profiles | |
US8468530B2 (en) | Determining and describing available resources and capabilities to match jobs to endpoints | |
US7711822B1 (en) | Resource management in application servers | |
US9596148B2 (en) | Minimizing service restart by optimally resizing service pools | |
US11150951B2 (en) | Releasable resource based preemptive scheduling | |
US11573946B2 (en) | Management of memory usage using usage analytics | |
JP2016133964A (en) | Computation resource allocation method and system | |
Iorgulescu et al. | Don't cry over spilled records: Memory elasticity of data-parallel applications and its application to cluster scheduling | |
US20140143496A1 (en) | Self-Sizing Dynamic Cache for Virtualized Environments | |
Xiong et al. | Energy Efficient Data Intensive Distributed Computing. | |
Parida et al. | Truthful resource allocation detection mechanism for cloud computing | |
JP2004192400A (en) | Job scheduling method and system | |
US10412193B2 (en) | Usage-aware standby service in a grid environment |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: HEWLETT-PACKARD DEVELOPMENT COMPANY, L.P., TEXAS Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:HERINGTON, DANIEL EDWARD;BLANDING, WILLIAM H.;SIGNING DATES FROM 20061024 TO 20061026;REEL/FRAME:018487/0165 |
|
ZAAA | Notice of allowance and fees due |
Free format text: ORIGINAL CODE: NOA |
|
ZAAB | Notice of allowance mailed |
Free format text: ORIGINAL CODE: MN/=. |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |
|
FPAY | Fee payment |
Year of fee payment: 4 |
|
AS | Assignment |
Owner name: HEWLETT PACKARD ENTERPRISE DEVELOPMENT LP, TEXAS Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:HEWLETT-PACKARD DEVELOPMENT COMPANY, L.P.;REEL/FRAME:037079/0001 Effective date: 20151027 |
|
MAFP | Maintenance fee payment |
Free format text: PAYMENT OF MAINTENANCE FEE, 8TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1552); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY Year of fee payment: 8 |
|
FEPP | Fee payment procedure |
Free format text: MAINTENANCE FEE REMINDER MAILED (ORIGINAL EVENT CODE: REM.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
LAPS | Lapse for failure to pay maintenance fees |
Free format text: PATENT EXPIRED FOR FAILURE TO PAY MAINTENANCE FEES (ORIGINAL EVENT CODE: EXP.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
STCH | Information on status: patent discontinuation |
Free format text: PATENT EXPIRED DUE TO NONPAYMENT OF MAINTENANCE FEES UNDER 37 CFR 1.362 |
|
FP | Lapsed due to failure to pay maintenance fee |
Effective date: 20231220 |