US10608433B1 — Methods and systems for adjusting power consumption based on a fixed-duration power option agreement (Part 1 of 2)
Document text
Research, not advice. Part of the Bitcoin research archive (October 2026). Claims labelled unverified, contested or fringe are reported, not endorsed; statuses of bills and rules are as of the date checked. Government, court and patent records are public domain; the research notes are CC BY 4.0.
US010608433B1
(12)McNamara
United States Patent ( 10 ) Patent No.: US 10,608,433 B1
et al. (45 ) Date of Patent: Mar. 31 , 2020
(54 ) METHODS AND SYSTEMS FOR ADJUSTING 7,143,300 B2 11/2006 Potter et al.
1/2010 Ranganathan et al.
POWER CONSUMPTION BASED ON A 7,647,516 B2
FIXED - DURATION POWER OPTION (Continued )
AGREEMENT
FOREIGN PATENT DOCUMENTS
(71 ) Applicant: Lancium LLC , Houston , TX (US) CN 103163904 A 6/2013
KR 20090012523 A 2/2009
(72) Inventors: Michael T. McNamara , Newport WO 2015199629 A1 12/2015
Beach , CA (US ) ; Raymond E. Cline,
Jr. , Houston , TX (US ) OTHER PUBLICATIONS
(73 ) Assignee: Lancium LLC , Houston , TX (US ) Bird et al., “Wind and Solar Energy Curtailment: Experience and
( * ) Notice: Subject to any disclaimer , the term of this Practices in the United States,” National Renewable Energy Lab
patent is extended or adjusted under 35 (NREL ), Technical Report NREL / TP -6A20-60983, Mar. 2014 , 58
U.S.C. 154(b ) by 0 days . pages .
(Continued )
(21) Appl. No.: 16 /702,931
Primary Examiner Christopher E. Everett
( 22 ) Filed : Dec. 4 , 2019 (74 ) Attorney, Agent, or Firm - McDonnell Boehnen
Related U.S. Application Data Hulbert & Berghoff LLP
(60 ) Provisional application No. 62 /927,119 , filed on Oct. (57 ) ABSTRACT
28 , 2019 . Examples relate to adjusting load power consumption based
(51 ) Int. Cl. on a power option agreement. A computing system may
HO2J 3/14 ( 2006.01 ) receive power option data that is based on a power option
HO2J 3/00 ( 2006.01) agreement and specify minimum power thresholds associ
GO6F 1/3203 (2019.01) ated with time intervals. The computing system may deter
(52) U.S. Cl. mine a performance strategy for a load (e.g., set of comput
CPC HO2J 3/14 (2013.01 ); G06F 1/3203 ing systems) based on a combination of the power option
( 2013.01); H02J 3/008 ( 2013.01) data and one or more monitored conditions. The perfor
( 58 ) Field of Classification Search mance strategy may specify a power consumption target for
CPC HO2J 3/14 ; HO2J 3/008; G06F 1/3203 the load for each time interval such that each power con
See application file for complete search history . sumption target is equal to or greater than the minimum
power threshold associated with each time interval. The
( 56 ) References Cited computing system may provide instructions the set of com
U.S. PATENT DOCUMENTS puting systems to perform one or more computational opera
tions based on the performance strategy .
6,288,456 B1 9/2001 Cratty
6,633,823 B2 10/2003 Bartone et al. 20 Claims, 16 Drawing Sheets
POWER 1204
1200
15 he
10
206B
1206
5
1206A 12060
-1202
0 4 8 12 16 20 24 TIME (H )
US 10,608,433 B1
Page 2
( 56 ) References Cited 2016/0198656 A1 7/2016 McNamara et al.
2016/0212954 A1 7/2016 Argento
U.S. PATENT DOCUMENTS 2016/0324077 Al 11/2016 Frantzen et al.
2017/0023969 A1 1/2017 Shows et al.
7,702,931 B2 4/2010 Goodrum et al. 2017/0104336 Al 4/2017 Elbsat et al.
7,779,276 B2 8/2010 Bolan et al. 2017/0261949 A1 9/2017 Hoffmann et al.
7,861,102 B1 12/2010 Ranganathan et al. 2018/0144414 Al 5/2018 Lee et al.
7,921,315 B2 4/2011 Langgood et al. 2018/0202825 A1 7/2018 You et al.
7,970,561 B2 6/2011 Pfeiffer 2018/0240112 Al 8/2018 Castinado et al.
8,001,403 B2 8/2011 Hamilton et al. 2018/0366978 A1 12/2018 Matan et al.
8,006,108 B2 8/2011 Brey et al. 2018/0367320 A1 12/2018 Montalvo
8,214,843 B2 7/2012 Boss et al. 2019/0052094 A1 2/2019 Pmsvvsv et al .
8,374,928 B2 2/2013 Gopisetty et al. 2019/0168630 A1 6/2019 Mrlik et al.
8,447,993 B2 5/2013 Greene et al. 2019/0258307 Al 8/2019 Shaikh et al.
8,571,820 B2 10/2013 Pfeiffer 2019/0280521 Al 9/2019 Lundstrom et al.
8,627,123 B2 1/2014 Jain et al. 2019/0318327 Al 10/2019 Sowell et al .
8,700,929 B1 * 4/2014 Weber G06F 30/13 2019/0324820 Al 10/2019 Krishnan et al.
713/310
8,789,061 B2 7/2014 Pavel et al. OTHER PUBLICATIONS
8,799,690 B2 8/2014 Dawson et al.
9,003,211 B2 4/2015 Pfeiffer EPEX Spot, " How They Occur, What They Mean," https://www .
9,003,216 B2 4/2015 Sankar et al .
9,026,814 B2 5/2015 Aasheim et al. epexspot.com/en/company-info/basics_of_the_power_market/negative_
9,207,993 B2 12/2015 Jain prices, 2018 , 2 pages .
9,218,035 B2 12/2015 Li et al. FinalOffice Action dated Oct. 1, 2019 for U.S. Appl.No. 16 /175,246 ,
9,552,234 B2 1/2017 Boldyrev et al. filed Oct. 30 , 2018, 18 pages .
10,367,353 B1 7/2019 McNamara et al. Ghamkhari et al.,“ Optimal Integration of Renewable Energy Resources
10,367,535 B2 7/2019 Corse et al.
10,444,818 B1 10/2019 McNamara et al. in Data Centers with Behind-the-Meter Renewable Generator,"
10,452,127 B1 10/2019 McNamara et al. Department of Electrical and Computer Engineering Texas Tech
10,497,072 B2 12/2019 Hooshmand et al. University, 2012 , pp . 3340-3444 .
2002/0072868 A1
2003/0074464 A1
6/2002 Bartone et al. International Search Report and Written Opinion of PCT Applica
4/2003 Bohrer et al . tion No. PCT/US2018 /017955, dated Apr. 30, 2018 , 22 pages .
2005/0203761 A1 * 9/2005 Barr G06F 1/26 International Search Report and Written Opinion of PCT Applica
713/320 tion No. PCT/US2018 /017950 , dated May 31 , 2018 , 15 pages .
2006/0161765 Al 7/2006 Cromer et al.
2008/0030078 A1 2/2008 Whitted et al. Non -Final Office Action dated Dec. 5 , 2019 for U.S. Appl. No.
2008/0094797 A1 4/2008 Coglitore et al. 16 /529,360 , filed Aug. 1, 2019 , 72 pages .
2009/0055665 Al 2/2009 Maglione et al. Non- Final Office Action dated Dec. 10 , 2019 for U.S. Appl. No.
2009/0070611 A1 * 3/2009 Bower, III G06F 1/3203 16 / 596,190 , filed Oct. 8 , 2019 , 72 pages.
713/322 Non- Final Office Action dated Nov. 14 , 2019 for U.S. Appl. No.
2009/0089595 A1 * 4/2009 Brey G06F 1/3203 16 / 132,098 , filed Sep. 14 , 2018 , 25 pages.
713/300 Non- Final Office Action dated Nov. 21 , 2019 for U.S. Appl. No.
2010/0211810 A1 8/2010 Zacho
2010/0328849 Al 12/2010 Ewing et al. 16 / 529,402 , filed Aug. 1 , 2019 , 57 pages.
2011/0072289 A1 3/2011 Kato Non -Final Office Action dated Dec. 11 , 2019 on for U.S. Appl. No.
2012/0000121 A1 1/2012 Swann 16 /132,062, filed Sep. 14 , 2018 , 17 pages .
2012/0072745 A1 3/2012 Ahluwalia et al. Non - Final Office Action dated Dec. 10 , 2019 for U.S. Appl. No.
2012/0300524 A1 11/2012 Fornage et al. 16 /528,348 , filed Oct. 8, 2019 , 33 pages .
2012/0324259 Al 12/2012 Aasheim et al . Notice of Allowance dated Apr. 2, 2019, for U.S. Appl. No.
2013/0006401 A1 1/2013 Shan 16 / 175,335, filed Oct. 30 , 2018 , 12 pages .
2013/0063991 Al 3/2013 Xiao et al.
2013/0086404 A1 * 4/2013 Sankar G06F 1/305 Notice of Allowance dated Aug. 15, 2019 , for U.S. Appl. No.
713/324 16/175, 146 , filed Oct. 30 , 2018 , 17 pages .
2013/0187464 A1 7/2013 Smith et al. Notice of Allowance dated Jul. 29 , 2019, for U.S. Appl. No.
2013/0227139 A1 8/2013 Suffling 16 /245,532 , filed Jan. 11, 2019 , 13 pages.
2013/0306276 A1 11/2013 Duchesneau Rahimi, Farrokh, “ Using a Transactive Energy Framework ,” IEEE
2014/0137468 A1 5/2014 Ching Electrification Magazine , Dec. 2016 , pp . 23-29 .
2014/0379156 A1 12/2014 Kamel et al. Soluna ., " Powering the Block Chain ," Aug. 2018 , version 1.1, 29
2015/0121113 Al 4/2015 Ramamurthy et al. pages .
2015/0155712 A1 6/2015 Mondal
2015/0229227 Al 8/2015 Aeloiza et al . Wilson , Joseph Nathanael, “ A Utility- Scale Deployment Project of
2015/0277410 A1 10/2015 Gupta et al. Behind -the -Meter Energy Storage for Use in Ancillary Services,
2015/0278968 A1 10/2015 Steven et al. Energy Resiliency, Grid Infrastructure Investment Deferment, and
2016/0170469 A1 6/2016 Sehgal et al. Demand -Response Integration ," Portland State University , 2016 ,
2016/0172900 A1 6/2016 Welch , Jr. 154 pages.
2016/0187906 A1 * 6/2016 Bodas G05B 15/02
700/287 * cited by examiner
U.S. Patent Mar 31 , 2020 Sheet 1 of 16 US 10.608,433 Bl
GENERATION TRANSMISSION DISTRIBUTION
12 { {3 { }4 15
+ POI STEP DOWN }
++
-
02a 14a ~~
~
-
03a
~
-
06a ~~~
++ ~
1 ~~
--06d
32e 12 { 03e 106e
?
+
2b
+
{ 35a 106f
++
-103b ~
~~
~~
~~
{ 16 ~~
{
103f
+
{ { 2f 332
- 106d
~~
{ 36e
?
++
?
++
~
-
??
*
{ 02c
+
--103c
{ 06f
*
15b { 08e-
+
106f
103g }
102g 302k
~
~
-
~
*
106c
*
**
~
~
-
-
-Q6d
?? ~~
-
~
{ 2d ---{ 33d
336e
+
**
+
106f
??
~
{ 5c 336e
102h {{3 } 106f
~
..
PRIOR ART
FIGURE 1
U.S. Patent Mar. 31, 2020 Sheet 2 of 16 US 10,608,433 B1
}
} }
DISTRIBUTION 206
256
206A
M
R
262
E
AS
M O
T ER C
S O
Y S
N T REOML 260
206A
TRADITIONAL DATACENTER TRADITIONAL DATACENTER TRADITIONAL DATACENTER
260
206A
260
206A
106
?? }
bi $
?? $ { ! & } $ $ }
??? ????? }
*
1 1
1 1
105
254
205 DSOTWENP
TRANSMISSION
}
104a
104 }
FIG2URE
253252253A POI
103 203
1
206A 06A
]
236B 236A 216A +
218 256A 256B 206A
250 213 STA.GENTIO CSOYNSTREOLM ENERGY STORAGE
216C
242 206A
2 3 4 -
212
2 3 4 B 216B 216
---
GENERATION E
P
G Q
E N
O
U E
I
102
R
W
P A
M T
E
210 INO
RTN -232B 214
230B
240C
232A 240A 240B 202
242A 2423-218A2 0A
220
2 0B
2 0A
FLEXIBLE DATACENTER FLEXIBLE DATACENTER FLEXIBLE DATACENTER
220
2 0B
220A
220B
230A 1 1 1
U.S. Patent Mar. 31, 2020 Sheet 3 of 16 US 10,608,433 B1
300
304 308 312
SDUTOANRTIAGAE IUNTSERFEACRE SQYUSETUEM
CRMSOEAYNMSTORTEORML
314
FIG3URE
302 306 310
PROCESOR COMUNICATION INTERFACE ENVIRONMENT
OPAERANTIODNS AMNOADLYUSLIES
U.S. Patent Mar. 31, 2020 Sheet 4 of 16 US 10,608,433 B1
400
408 410 414
COMUNICATION INTERFACE IUNTSERFEACRE SCOYNSTREOML GSETNEARTAITON
412 FIG4URE
401 402 404 406
GPENOERWATEIORN EQUIPMENT POWER ETRANSFORMATION QUIPMENT IGNTERFIACDE INTERFACE BMEH-THETINRD
U.S. Patent Mar. 31, 2020 Sheet 5 of 16 US 10,608,433 B1
510 512 516 518 520 514
500
SQYUSETUEM PCREOCNETSRIANLG UNITS
CPUS
()
PGROACPESHICNSG UNITS
(
GPUS
) ASPLEIC-AITFIONC ICNITRECGRUAITESD SCOYMSPTUEIMNSG SQYUSETUEM
ASICS
)
(
FDALTEAXCIEBNTLER
528
FIG5URE
BMEHTHE-TINRD GRID
SETNOERAGYE SPIYONSWPTEURTM CIONMTUENIRCFATICOEN DATACENTER SCOYNSTREOML DPISOTRIWBUETIRON SYSTEM CSLOYINSMTARETOMEL
502
522 524 526 503 504 506 508
U.S. Patent Mar. 31, 2020 Sheet 6 of 16 US 10,608,433 B1
502 602
504 606
506 608
604
604 508
500
604
604
604
* * * * * * * * * * * * ** * * * * * * *
604
604
604 FIG6AURE
512
508
U.S. Patent Mar. 31, 2020 Sheet 7 of 16 US 10,608,433 B1
604
610 512
608
604
512 FIGOBURE
909
L
604
512
U.S. Patent Mar. 31, 2020 Sheet 8 of 16 US 10,608,433 B1
many
Ww.
PIONPWUETR SYSTEM 502
700 600
WW
714
w
DATACENTER FIG7URE
ww
504
708
www
712
CSONY.TRSOL 710
?
414 SGTA.ETION
702 706
CSOYNSTREOML 300
U.S. Patent Mar. 31, 2020 Sheet 9 of 16 US 10,608,433 B1
8908
Hologie dolore category otolar
Y
804
m w w w mo m ?
300 FIG8URE
www www
ma
008
}
Www an ww wa www wa where www www Wate
Y. 802
806A
U.S. Patent Mar. 31, 2020 Sheet 10 of 16 US 10,608,433 B1
930 932 934 936
902
PSIYONSWPTEURTM DIPSTORIWBUTEIORN SYSTEM CSOYMSPUTEIMNGS DATACENTER CSOYNSTREOML DTARATDAICTEINOTNEARL
924 916
300 312 920B
918
MRAESMTOETRE CSOYNSTREOML SQYUSETUEM FIG9URE
922 920A
500
10 0 SIPYNOSPWTUEMTR DPISTORIWBUETIORN SYSTEM CSOYMSPTUEIMNSG SYSTEM DCAOTANCTERNTOELR DFALTEAXCIENBTLER
502 506 512 504
U.S. Patent Mar. 31, 2020 Sheet 11 of 16 US 10,608,433 B1
1000
MONITORING B - T - M POWER
AVAILABILITY 1010
DETERMINING WHEN A
DATACENTER RAMP - UP 1020
CONDITION IS MET
ENABLING B - T - M POWER
DELIVERY TO ONE OR MORE 1030
COMPUTING SYSTEMS
DIRECTING ONE OR MORE
COMPUTING SYSTEMS TO
PERFORM COMPUTATIONAL 1040
OPERATIONS
FIGURE 10A
U.S. Patent Mar. 31, 2020 Sheet 12 of 16 US 10,608,433 B1
1050
MONITORING B - T - M POWER
AVAILABILITY 1060
DETERMINING WHEN A
DATACENTER RAMP -DOWN 1070
CONDITION IS MET
DISABLING B - T - M POWER
DELIVERY TO ONE OR MORE 1080
COMPUTING SYSTEMS
REMAINING POWERED AND
IN COMMUNICATION WITH
REMOTE MASTER CONTROL 1090
SYSTEM
FIGURE 10B
U.S. Patent Mar. 31, 2020 Sheet 13 of 16 US 10,608,433 B1
1 44 1 46
1142
1 18A 1 18B 1 18C 1 12 1 40 DFUIRA-XTEIODN 1 407 DYNAMIC OPOTWIEONR MODULE EPNOTWIETRY FIG11URE
1106
F
S IER S
T SECEOT
ND THISETRD
COMPUTING SYSTEMS TRADITIONAL DATACENTER 2
1 26 1 28 1 29
-1 04
FIRSETS SECEONTD TSHIERTD
COMPUTING SYSTEMS FLEXIBLE DATACENTER CPRYPTIOCURENSCY COMPUTATIONAL OPAERAMTEIOENRS CWOENDAITHIOENRS CRMSOEAYNMSTORTEORML
1 16A 1 16B 1 16C 1 10 EOAMNPVEOAIRDLANOTYUIMSOLEIDNEST
APVAIOLAWBIELITRY PROIWCERS CSOYMSPUTEIMNSG PARAMETERS
1130
1 02 F
S IER S
T S E C OTN D COMPUTING DATACENTER
THISETRD SYSTEMS FLEXIBLE
1 14A 1 14B 1 14C 1 08 10 1 20 1 2 1 24 310 262
U.S. Patent Mar. 31, 2020 Sheet 14 of 16 US 10,608,433 B1
-1202 )
TIME
H
(
24
120
20
1206
16
1206B 12 FIG12URE
8
4
1206 1206A
1204 0
POWER MW
(
)
15 10 5
U.S. Patent Mar. 31, 2020 Sheet 15 of 16 US 10,608,433 B1
1300
MONITOR A SET OF CONDITIONS
1302
RECEIVE POWER OPTION DATA BASED ,
AT LEAST IN PART, ON A POWER OPTION
AGREEMENT 1304
DETERMINE A PERFORMANCE
STRATEGY FOR A SET OF COMPUTING
SYSTEMS BASED ON A COMBINATION OF
AT LEAST A PORTION OF THE POWER 1306
OPTION DATA AND AT LEAST ONE
CONDITION IN THE SET OF CONDITIONS
PROVIDE INSTRUCTIONS TO THE SET OF
COMPUTING SYSTEMS TO PERFORM
ONE OR MORE COMPUTATIONAL
OPERATIONS BASED ON THE 1308
PERFORMANCE STRATEGY
FIGURE 13
U.S. Patent Mar. 31, 2020 Sheet 16 of 16 US 10,608,433 B1
1400 MONITOR A SET OF CONDITIONS
1402
WHILE MONITORING THE SET OF
CONDITIONS, RECEIVE FIRST POWER
OPTION DATA BASED , AT LEAST IN PART,
ON A POWER OPTION AGREEMENT 1404
RESPONSIVE TO RECEIVING THE FIRST
POWER OPTION DATA , PROVIDE FIRST
CONTROL INSTRUCTIONS FOR A SET OF
COMPUTING SYSTEMS BASED ON A
COMBINATION OF AT LEAST A PORTION ( 1406
OF THE FIRST POWER OPTION DATA
AND AT LEAST ONE CONDITION
WHILE MONITORING THE SET OF
CONDITIONS , RECEIVE SECOND POWER
OPTION DATA BASED , AT LEAST IN PART, 1408
ON THE POWER OPTION AGREEMENT
RESPONSIVE TO RECEIVING THE
SECOND POWER OPTION DATA ,
PROVIDE SECOND CONTROL
INSTRUCTIONS FOR THE SET OF
COMPUTING SYSTEMS BASED ON A 1410
COMBINATION OF AT LEAST A PORTION
OF THE SECOND POWER OPTION DATA
AND AT LEAST ONE CONDITION
FIGURE 14
US 10,608,433 B1
2
METHODS AND SYSTEMS FOR ADJUSTING collectors, switches, transformers , frequency converters,
POWER CONSUMPTION BASED ON A power converters, electrical filters, and / or other station elec
FIXED - DURATION POWER OPTION trical equipmentbefore leaving the wind power station 102c .
AGREEMENT Similarly , at photovoltaic power station 102d , individual
5 photovoltaic panels and /or arrays of photovoltaic panels
CROSS -REFERENCE TO RELATED may include inverters , transformers, frequency converters ,
APPLICATIONS power converters, and /or electrical filters. Energy generated
at each photovoltaic panel and /or array may be collected by
The presentapplication claimspriority to U.S. Provisional distribution lines along the photovoltaic panels and move
Patent Application No. 62/927,119, filed Oct. 28 , 2019, the 10 through collectors, switches, transformers , frequency con
entire contents of which are herein incorporated by refer verters, power converters, electrical filters, and /or other
ence .
station electrical equipment before leaving the photovoltaic
FIELD power station 102d.
15 Each generation station 102a -d may produce AC or DC
This specification relates to power consumption adjust higher electrical current which is then typically stepped up to a
ments when using grid power and/or intermittent behind station .ACForvoltage before leaving the respective generation
example, wind turbines may typically produce
the -meter power.
AC electrical energy at 600V to 700V, which may then be
BACKGROUND 20 stepped up to 34.5 kV before leaving the generation station
102d . In some cases, the voltage may be stepped up multiple
“ Electrical grid ” or “ grid ,” as used herein , refers to a Wide times and to a different voltage before exiting the generation
Area SynchronousGrid (also known as an Interconnection ), station 102c . As another example, photovoltaic arrays may
and is a regional scale or greater electric power grid that that produce DC voltage at 600V to 900V, which is then inverted
operates at a synchronized frequency and is electrically tied 25 to AC voltage and may be stepped up to 34.5 kV before
together during normal system conditions. An electrical grid leaving the generation station 102d . In some cases, the
delivers electricity from generation stations to consumers . voltage may be stepped up multiple times and to a different
An electrical grid includes : (i) generation stations that voltage before exiting the generation station 102d .
produce electrical power at large scales for delivery through Upon exiting the generation segment 102, electrical
the grid , ( ii ) high voltage transmission lines that carry that 30 power generated at generation stations 102a -d passes
power from the generation stations to demand centers, and through a respective Point of Interconnection (“ POI” ) 103
( iii) distribution networks carry that power to individual between a generation station (e.g., 102a -d ) and the rest of
customers . the grid . A respective POI 103 represents the point of
FIG . 1 illustrates a typical electrical grid , such as a North connection between a generation station's (e.g. 102a -d )
American Interconnection or the synchronous grid of Con- 35 equipment and a transmission system (e.g. , transmission
tinental Europe ( formerly known as the UCTE grid ). The segment 104 ) associated with electrical grid . In some cases,
electrical grid of FIG . 1 can be described with respect to the at the POI 103, generated power from generation stations
various segments that make up the grid . 102a -d may be stepped up at transformer systems 103e -h to
A generation segment 102 includes one or more genera high voltage scales suitable for long -distance transmission
tion stations that produce utility -scale electricity (typically 40 along transmission lines 104a . Typically, the generated
> 50 MW ), such as a nuclear plant 102a , a coal plant 102b , electrical energy leaving the POI 103 will be at 115 kV AC
a wind power station ( i.e., wind farm ) 102c, and /or a or above ,but in some cases it may be as low as , for example,
photovoltaic power station ( i.e., a solar farm ) 102d . Gen 69kV for shorter distance transmissions along transmission
eration stations are differentiated from building -mounted lines 104a . Each of transformer systems 103e - h may be a
and other decentralized or local wind or solar power appli- 45 single transformer or may be multiple transformers operat
cations because they supply power at the utility level and ing in parallel or series and may be co -located or located in
scale (> 50 MW ), rather than to a local user or users. The geographically distinct locations. Each of the transformer
primary purpose of generation stations is to produce power systems 103e -h may include substations and other links
for distribution through the grid , and in exchange for pay between the generation stations 102a - d and the transmission
ment for the supplied electricity . Each of the generation 50 lines 104a .
stations 102a -d includes power generation equipment A key aspect of the POI 103 is that this is where
102e -h , respectively, typically capable of supply utility generation -side metering occurs . One or more utility -scale
scale power (> 50 MW ) . For example , the power generation generation -side meters 103a -d (e.g. , settlement meters ) are
equipment 102g at wind power station 102c includes wind located at settlement metering points at the respective POI
turbines, and the power generation equipment 102h at pho- 55 103 for each generation station 102a -d . The utility-scale
tovoltaic power station 102d includes photovoltaic panels. generation -side meters 103a -d measure power supplied
Each of the generation stations 102a -d may further from generation stations 102a -d into the transmission seg
include station electrical equipment 102i-1 respectively . ment 104 for eventual distribution throughout the grid .
Station electrical equipment 102i- 1 are each illustrated in For electricity consumption , the price consumers pay for
FIG . 1 as distinct elements for simplified illustrative pur- 60 power distributed through electric power grids is typically
poses only and may, alternatively or additionally, be distrib composed of, among other costs , Generation, Administra
uted throughout the power generation equipment, 102e -h , tion , and Transmission & Distribution (“ T & D ” ) costs . T & D
respectively . For example , at wind power station 102c, each costs represent a significant portion of the overall price paid
wind turbine may include transformers, frequency convert by consumers for electricity . These costs include capital
ers , power converters , and /or electrical filters. Energy gen- 65 costs (land , equipment, substations, wire , etc. ), costs asso
erated at each wind turbine may be collected by distribution ciated with electrical transmission losses, and operation and
lines along strings of wind turbines and move through maintenance costs .
US 10,608,433 B1
3 4
For utility -scale electricity supply, operators of generation current transformer), scale (typically less than 1600 amps vs.
stations ( e.g., 102a -d ) are paid a variable market price for typically greater than 50 MW ; typically less than 600V vs.
the amount of power the operator generates and provides to typically greater than 14 kV ), primary function (use vs.
the grid , which is typically determined via a power purchase supply metering ), economic purpose (credit against use vs
agreement (PPA ) between the generation station operator 5 payment for power), and location ( in a distribution network
and a grid operator. The amount of power the generation at point of use vs. at a settlement metering point at a Point
station operator generates and provides to the grid is mea of Interconnection between a generation station and a trans
sured by utility -scale generation -side meters (e.g., 103a -d ) mission line ).
at settlement metering points. As illustrated in FIG . 1, the To maintain stability of the grid , the grid operator strives
utility -scale generation -side meters 103a -d are shown on a 10 to maintain a balance between the amount of power entering
low side of the transformer systems 103e -h ), but they may the grid from generation stations (e.g., 102a -d ) and the
alternatively be located within the transformer systems amount of grid power used by loads (e.g. , customers in the
103e -h or on the high side of the transformer systems distribution segment 106 ). In order to maintain grid stability
103e -h . A key aspect of a utility -scale generation - side meter and manage congestion , grid operators may take steps to
is that it is able to meter the power supplied from a specific 15 reduce the supply of power arriving from generation stations
generation station into the grid . As a result, the grid operator (e.g., 102a -d ) when necessary ( e.g., curtailment). Particu
can use that information to calculate and process payments larly , grid operators may decrease the market price paid for
for power supplied from the generation station to the grid . generated power to dis- incentivize generation stations ( e.g.,
That price paid for the power supplied from the generation 102a -d ) from generating and supplying power to the grid . In
station is then subject to T & D costs , as well as other costs, 20 some cases, the market price may even go negative such that
in order to determine the price paid by consumers . generation station operators must pay for power they allow
After passing through the utility -scale generation -side into the grid . In addition, some situations may arise where
meters in the POI 103, the power originally generated at the grid operators explicitly direct a generation station ( e.g.,
generation stations 102a -d is transmitted onto and along the 102a -d ) to reduce or stop the amount of power the station is
transmission lines 104a in the transmission segment 104. 25 supplying to the grid .
Typically, the electrical energy is transmitted as AC at 115 Power market fluctuations, power system conditions (e.g.,
KV + or above, though it may be as low as 69 kV for short power factor fluctuation or generation station startup and
transmission distances . In some cases , the transmission testing), and operational directives resulting in reduced or
segment 104 may include further power conversions to aid discontinued generation all can have disparate effects on
in efficiency or stability . For example , transmission segment 30 renewal energy generators and can occur multiple times in
104 may include high - voltage DC ("HVDC " ) portions a day and last for indeterminate periods of time. Curtail
(along with conversion equipment) to aid in frequency ment, in particular, is particularly problematic.
synchronization across portions of the transmission segment According to the National Renewable Energy Laborato
104. As another example, transmission segment 104 may ry's Technical Report TP -6A20-60983 (March 2014 ):
include transformers to step AC voltage up and then back 35 [C ]urtailment [ is ] a reduction in the output of a generator
down to aid in long distance transmission (e.g., 230 kV, 500 from what it could otherwise produce given available
KV , 765 kV, etc.). resources (e.g., wind or sunlight), typically on an
Power generated at the generation stations 104a -d is involuntary basis . Curtailments can result when opera
ultimately destined for use by consumers connected to the tors or utilities command wind and solar generators to
grid. Once the energy has been transmitted along the trans- 40 reduce output to minimize transmission congestion or
mission segment 104 , the voltage will be stepped down by otherwise manage the system or achieve the optimal
transformer systems 105a -c in the step down segment 105 so mix of resources . Curtailment of wind and solar
that it can move into the distribution segment 106 . resources typically occurs because of transmission con
In the distribution segment 106 , distribution networks gestion or lack of transmission access , but it can also
106a -c take power that has been stepped down from the 45 occur for reasons such as excess generation during low
transmission lines 104a and distribute it to local customers , load periods that could cause baseload generators to
such as local sub - grids (illustrated at 106a), industrial cus reach minimum generation thresholds, because of volt
tomers, including large EV charging networks (illustrated at age or interconnection issues, or to maintain frequency
106b ), and/or residential and retail customers, including requirements, particularly for small, isolated grids .
individual EV charging stations (illustrated at 106c ). Cus- 50 Curtailment is one among many tools to maintain
tomermeters 106 , 106fmeasure the power used by each of system energy balance , which can also include grid
the grid -connected customers in distribution networks 106a capacity, hydropower and thermal generation , demand
c . Customer meters 106d are typically load meters that are response , storage, and institutional changes. Deciding
unidirectional and measure power use. Some of the local which method to use is primarily a matter ofeconomics
customers in the distribution networks 106a -d may have 55 and operational practice.
local wind or solar power systems 106e owned by the “ Curtailment” today does not necessarily mean what it did
customer . As discussed above, these local customer power in the early 2000s. Two separate changes in the electric
systems 106e are decentralized and supply power directly to sector have shaped curtailment practices since that
the customer (s ). Customers with decentralized wind or solar time: the utility -scale deployment of wind power,
power systems 106e may have customer meters 106fthat are 60 which has no fuel cost, and the evolution of wholesale
bidirectional or net-metering meters that can track when the power markets. These simultaneous changes have led
local customer power systems 106e produce power in excess to new operational challenges but have also expanded
of the customer's use, thereby allowing the utility to provide the array of market-based tools for addressing them .
a credit to the customer's monthly electricity bill . Customer Practices vary significantly by region and market design .
meters 106 , 106f differ from utility -scale generation -side 65 In places with centrally -organized wholesale power
meters ( e.g., settlement meters) in at least the following markets and experience with wind power,manual wind
characteristics: design (electro -mechanical or electronic vs energy curtailment processes are increasingly being
US 10,608,433 B1
5 6
replaced by transparent offer-based market mecha agreement. The second power option data specify a second
nisms that base dispatch on economics. Market proto minimum power threshold associated with a second time
cols that dispatch generation based on economics can interval. Responsive to receiving the second power option
also result in renewable energy plants generating less data , the control system is configured to provide second
than what they could potentially produce with available 5 control instructions for the set of computing systems based
wind or sunlight. This is often referred to by grid on a combination of at least a portion of the second power
operators by other terms, such as “ downward dispatch .” data and at least one condition of the set of conditions. The
In places served primarily by vertically integrated second control instructions comprises a second power con
utilities, power purchase agreements (PPAs) between sumption target for the set of computing systems for the
the utility and the wind developer increasingly contain 10 sumption
financial provisions for curtailment contingencies .
second timetarget
interval, and wherein the second power con
is equal to or greater than the second
Some reductions in output are determined by how a wind minimum power threshold associated with the second time
operator values dispatch versus non -dispatch . Other interval.
curtailments of wind are determined by the grid opera In another example, a method involves monitoring, at a
tor in response to potential reliability events. Still other 15 computing system , a set of conditions, and while monitoring
curtailments result from overdevelopment of wind the setof conditions, receiving first power option data based ,
power in transmission -constrained areas. at least in part, on a power option agreement. The first power
Dispatch below maximum output ( curtailment) can be option data specify a first minimum power threshold asso
more of an issue for wind and solar generators than it ciated with a first time interval. The method further involves ,
is for fossil generation units because of differences in 20 responsive to receiving the first power option data , provid
their cost structures . The economics of wind and solar ing first control instructions for a set of computing systems
generation depend on the ability to generate electricity based on a combination ofat least a portion of the first power
whenever there is sufficient sunlight or wind to power option data and at least one condition of the set of condi
their facilities. tions. The first control instructions comprises a first power
Because wind and solar generators have substantial capi- 25 consumption target for the set of computing systems for the
tal costs but no fuel costs ( i.e., minimal variable costs ), first time interval, and the first power consumption target is
maximizing output improves their ability to recover equal to or greater than the first minimum power threshold
capital costs . In contrast, fossil generators have higher associated with the first time interval. The method further
variable costs , such as fuel costs . Avoiding these costs involves , while monitoring the set of conditions, receiving
can , depending on the economics of a specific genera- 30 second power option data based , at least in part, on the
tor, to some degree reduce the financial impact of power option agreement. The second power option data
curtailment, especially if the generator's capital costs specify a second minimum power threshold associated with
are included in a utility's rate base . a second time interval. Themethod also involves, responsive
Curtailment may result in available energy being wasted to receiving the second power option data , providing second
because solar and wind operators have zero variable cost 35 control instructions for the set of computing systems based
(which may not be true to the same extent for fossil on a combination of at least a portion of the second power
generation units which can simply reduce the amount of fuel data and at least one condition of the set of conditions. The
that is being used ). With wind generation , in particular, it second control instructions comprises a second power con
may also take some time for a wind farm to become fully sumption target for the set of computing systems for the
operational following curtailment. As such, until the time 40 second time interval, and the second power consumption
that the wind farm is fully operational, the wind farm may target is equal to or greater than the second minimum power
not be operating with optimum efficiency and / or may not be threshold associated with the second time interval.
able to provide power to the grid . In yet another example , a system is provided . The system
includes a set of computing systems, where the set of
SUMMARY 45 computing systems is configured to perform computational
operations using power from a power grid . The system also
In an example , a system includes a set of computing includes a control system configured to monitor a set of
systems. The set of computing systems is configured to conditions and receive power option data based , at least in
perform computational operations using power from a part, on a power option agreement. The power option data
power grid . The system also includes a control system 50 specify: (i) a set of minimum power thresholds, and ( ii ) a set
configured to monitor a set of conditions and, while moni of time intervals , where each minimum power threshold in
toring the set of conditions, receive first power option data the set of minimum power thresholds is associated with a
based , at least in part, on a power option agreement. The first time interval in the set of time intervals . The control system
power option data specify a first minimum power threshold is further configured to , responsive to receiving the power
associated with a first time interval. The control system is 55 option data , determine a performance strategy for the set of
further configured to provide first control instructions for the computing systems based on a combination of at least a
set of computing systemsbased on a combination of at least portion of the power option data and at least one condition
a portion of the first power option data and at least one in the set of conditions. The performance strategy comprises
condition of the set of conditions responsive to receiving the a power consumption target for the set of computing systems
first power option data . The first control instructions com- 60 for each time interval in the set of time intervals, where each
prises a first power consumption target for the set of com power consumption target is equal to or greater than the
puting systems for the first time interval, and the first power minimum power threshold associated with each time inter
consumption target is equal to or greater than the first val. The control system is also configured to provide instruc
minimum power threshold associated with the first time tions to the set of computing systems to perform one or more
interval. The control system is also configured to , while 65 computational operations based on the performance strategy.
monitoring the set of conditions, receive second power In a further example , non-transitory computer-readable
option data based , at least in part, on the power option medium is described that is configured to store instructions,
US 10,608,433 B1
7 8
that when executed by a computing system , causes the operations to decrease the amount of power generated .
computing system to perform operations consistent with the Furthermore , some situations may even require generation
method steps described above . stations to incur costs in order to offload power to the grid
Other aspects of the present invention will be apparent 5 or to shut down generation temporarily .
from the following description and claims. The volatility in the market price offered for power
supplied to the grid can be especially problematic for some
BRIEF DESCRIPTION OF THE FIGURES types of generation stations. In particular, wind farms and
some other types of renewable resource power producers
FIG . 1 shows a typical electrical grid . may lack the ability to quickly adjust operations in response
FIG . 2 shows a behind -the-meter arrangement with 10 to changes in the market price offered for supplying power
optional grid power, including one or more flexible data to the grid . As a result, power generation and management
centers, according to one or more example embodiments . at some generation stations can be inefficient, which can
FIG . 3 shows a block diagram of a remote master control frequently result in power being sold to the grid at low or
system , according to one or more example embodiments. negative prices. In some situations, a generation station may
FIG . 4 a block diagram of a generation station , according 15 even opt to halt power generation temporarily to avoid such
to one or more example embodiments. unfavorable pricing . As such , the time required to halt and
FIG . 5 shows a block diagram of a flexible datacenter, to restart the power generation at a generation station can
according to one or more example embodiments . reduce the generation station's ability to take advantage of
FIG . 6A shows a structural arrangement of a flexible rising market prices for power supplied to the grid .
datacenter , according to one or more example embodiments . 20 Example embodiments provided herein aim to assist
FIG . 6B shows a set of computing systems arranged in a generation stations in managing power generation opera
straight configuration, according to one or more example tions and avoid unfavorable power pricing situations like
embodiments . those described above . In particular , example embodiments
FIG . 7 shows a control distribution system for a flexible may involve providing a load that is positioned behind -the
datacenter, according to one or more example embodiments . 25 meter (“ BTM ” ) and enabling the load to utilize power
FIG . 8 shows a control distribution system for a fleet of received behind - the -meter at a generation station in a timely
flexible datacenters, according to one or more example manner. As a general rule of thumb , BTM power is not
embodiments . subject to traditional T & D costs .
FIG . 9 shows a queue distribution system for a traditional For purposes herein , a generation station is considered to
datacenter and a flexible datacenter, according to one or 30 be configured for the primary purpose of generating utility
more example embodiments. scale power for supply to the electrical grid (e.g., a Wide
FIG . 10A shows a method of dynamic power consump Area Synchronous Grid or a North American Interconnect).
tion at a flexible datacenter using behind -the-meter power, In one embodi equipment located behind -the -meter
according to one or more example embodiments . (“ BTM equipment” ) is equipment that is electrically con
FIG . 10B shows a method of dynamic power delivery at 35 nected to a generation station's power generation equipment
a flexible datacenter using behind -the-meter power, accord behind ( i.e., prior to ) the generation station's POI with an
ing to one or more example embodiments. electrical grid .
FIG . 11 shows a block diagram of a system for imple In one embodiment, behind- the-meter power (“ BTM
menting power consumption adjustments based on a power power” ) is electrical power produced by a generation sta
option agreement, according to one or more embodiments. 40 tion's power generation equipment and utilized behind ( i.e.,
FIG . 12 shows a graph representing power option data prior to ) the generation station’s POI with an electrical grid .
based on a power option agreement, according to one or In another embodiment, equipment may be considered
more embodiments . behind -the -meter if it is electrically connected to a genera
FIG . 13 shows a method for implementing power con tion station that is subject to metering by a utility -scale
sumption adjustments based on a fixed -duration power 45 generation -side meter ( e.g., settlement meter), and the BTM
option agreement, according to one or more embodiments. equipment receives power from the generation station , but
FIG . 14 shows a method for implementing power con the power received by the BTM equipment from the gen
sumption adjustments based on a dynamic power option eration station has not passed through the utility -scale gen
agreement, according to one or more embodiments . eration -side meter. In one embodiment, the utility -scale
50 generation -side meter for the generation station is located at
DETAILED DESCRIPTION the generation station's POI. In another embodiment, the
utility -scale generation -side meter for the generation station
Disclosed examples will now be described more fully is at a location other than the POI for the generation
hereinafter with reference to the accompanying drawings, in station — for example , a substation between the generation
which some, but not all of the disclosed examples are shown. 55 station and the generation station's POI.
Different examples may be described and should not be In another embodiment, power may be considered
construed as limited to the examples set forth herein . behind -the-meter if it is electrical power produced at a
As discussed above, the market price paid to generation generation station that is subject to metering by a utility
stations for supplying power to the grid often fluctuates due scale generation -side meter ( e.g., settlementmeter), and the
to various factors , including the need to maintain grid 60 BTM power is utilized before being metered at the utility
stability and based on current demand and usage by con scale generation -side meter. In one embodiment, the utility
nected loads in distribution networks. Due to these factors , scale generation -side meter for the generation station is
situations can arise where generation stations are offered located at the generation station's POI. In another embodi
substantially lower prices to deter an over-supply of power ment, the utility -scale generation -side meter for the genera
to the grid . Although these situations typically exist tempo- 65 tion station is at a location other than the POI for the
rarily, generation stations are sometimes forced to either sell generation station - for example , a substation between the
power to the grid at the much lower prices or adjust generation station and the generation station's POI.
US 10,608,433 B1
9 10
In another embodiment, equipment may be considered the economic impact in situations when supplying power to
behind - the-meter if it is electrically connected to a genera the grid would result in the generation station incurring a net
tion station that supplies power to a grid , and the BTM cost.
equipment receives power from the generation station that is Providing BTM power to a load can also benefit the BTM
not subject to T & D charges, but power received from the 5 load operator. A BTM load may be able to receive and utilize
grid that is supplied by the generation station is subject to BTM power received from the generation station at a cost
T & D charges. that is lower than the cost for power from the grid (e.g., at
In another embodiment, power may be considered a customer meter 106 , 1060. This is primarily due to the
behind -the-meter if it is electrical power produced at a avoidance (or significant reduction ) in T & D costs and the
generation station that supplies power to a grid , and the 10 market effects of curtailment. As indicated above , the gen
BTM power is not subject to T & D charges before being used eration station may be willing to divert generated power to
by electrical equipment, but power received from the grid the BTM load rather than supplying the grid due to changing
that is supplied by the generation station is subject to T & D market conditions, or during maintenance periods, or for
charges. other non -market conditions . Thus , some situations may
In another embodiment, equipment may be considered 15 arise where the generation station offers power to the BTM
behind -the-meter if the BTM equipment receives power load at a price that is substantially lower than the price
generated from the generation station and that received available on the grid . Furthermore, in some situations , the
power is not routed through the electrical grid before being BTM load may even be able to obtain and utilize BTM
delivered to the BTM equipment. power from a generation station at no cost or even at
In another embodiment, power may be considered 20 negative pricing since the generation station may rather
behind -the-meter if it is electrical power produced at a supply the BTM load with generated power during a given
generation station , and BTM equipment receives that gen time range instead of paying a higher price for the grid to
erated power, and that generated power received by the take the power or modifying operations to decrease power
BTM equipment is not routed through the electrical grid output.
before being delivered to the BTM equipment. 25 Another example of cost- effective use of BTM power is
For purposes herein , BTM equipment may also be when the generation station 202 is selling power to the grid
referred to as a behind - the -meter load (“ BTM load ” ) when at a negative price that is offset by a production tax credit .
the BTM equipment is actively consuming BTM power. In certain circumstances, the value of the production tax
Beneficially , where BTM power is not subject to tradi credit may exceed the price the generation station 202 would
tional T & D costs, a wind farm or other type of generation 30 have to pay to the grid power to offload generation's station
station can be connected to BTM loads which can allow the 202 generated power . Advantageously, one or more flexible
generation station to selectively avoid the adverse or less datacenters 220 may take the generated power behind - the
than optimal cost str ture occasionally associated with meter, thereby allowing the generation station 202 to pro
supplying power to the grid by shunting generated power to duce and obtain the production tax credit , while selling less
the BTM load . 35 power to the grid at the negative price .
An arrangement that positions and connects a BTM load Another example of cost -effective behind- the -meter
to a generation station can offer several advantages. In such power is when the generation station 202 is selling power to
arrangements, the generation station may selectively choose the grid at a negative price because the grid is oversupplied
whether to supply power to the grid or to the BTM load , or and /or the generation station 202 is instructed to stand down
both . The operator of a BTM load may pay to utilize BTM 40 and stop producing altogether. A grid operator may select
power at a cost less than that charged through a consumer and direct certain generation stations to go offline and stop
meter ( e.g., 106 , 1060 located at a distribution network supplying power to the grid . Advantageously , one or more
(e.g., 106a -c) receiving power from the grid . The operator of flexible datacenters may be used to take power behind -the
a BTM load may additionally or alternatively charge less meter , thereby allowing the generation station 202 to stop
than the market rate to consume excess power generated at 45 supplying power to the grid , but still stay online and make
the generation station during curtailment. As a result , the productive use of the power generated .
generation station may direct generated power based on the Another example of beneficial behind - the -meter power
“ best” price that the generation station can receive during a use is when the generation station 202 is producing power
given time frame, and/or the lowest cost the generation that is , with reference to the grid , unstable, out of phase, or
station may incur from negative market pricing during 50 at the wrong frequency, or the grid is already unstable , out
curtailment. The "best" price may be the highest price that of phase , or at the wrong frequency. A grid operator may
the generation station may receive for its generated power select certain generation stations to go either offline and stop
during a given duration, but can also differ within embodi producing power, or to take corrective action with respect to
ments and may depend on various factors , such as a prior the grid power stability , phase, or frequency. Advanta
PPA . 55 geously, one or more flexible datacenters 220 may be used
In one example , by having a behind -the -meter option to selectively consume power behind -the-meter, thereby
available, a generation station may transition from supplying allowing the generation station 202 to stop providing power
all generated power to the grid to supplying some or all to the grid and / or provide corrective feedback to the grid .
generated power to one ormore BTM loads when the market Another example of beneficial behind - the -meter power
price paid for power by grid operators drops below a 60 use is that cost-effectivebehind -the-meter power availability
predefined threshold ( e.g., the price that the operator of the may occur when the generation station 202 is starting up or
BTM load is willing to pay the generation station for power). testing . Individual equipment in the power generation equip
Thus, by having an alternative option for power consump ment 210 may be routinely offline for installation , mainte
tion ( i.e., one or more BTM loads), the generation station nance , and /or service and the individual units must be tested
can selectively utilize the different options to maximize the 65 prior to coming online as part of overall power generation
price received for generated power. In addition , the genera equipment 210. During such testing or maintenance time,
tion station may also utilize a BTM load to avoid or reduce one or more flexible datacenters may be intermittently
US 10,608,433 B1
11 12
powered by the one or more units of the power generation tions , contractual obligations, etc.) may be used by the
equipment 210 that are offline from the overall power generation station , a BTM load control system , a remote
generation equipment 210 . master control system , or some other system or enterprise , to
Another example of beneficial behind - the-meter power selectively output power to either the grid or to one or more
use is that datacenter control systems at the flexible data- 5 BTM loads in a manner that maximizes revenue to the
centers 220 may quickly ramp up and ramp down power generation station . In such an arrangement, the generation
consumption by computing systems in the flexible datacen station may also be able to supply both the grid and one or
ters 220 based on power availability from the generation more BTM loads simultaneously. In some instances , the
station 202. For instance , if the grid requires additional arrangement may be configured to allow dynamic manipu
power, the generation station 202 can supply the grid with is supplied topercentage
power and signals the demand via a higher local price for 10 lation of the
each
of the overall generated power that
option at a given time. For example , in
power nearly instantly by having active flexible datacenters some time periods, the generation station may supply no
220 quickly ramp down and turn off computing systems (or power to the BTM load.
switch to a stored energy source ), thereby reducing an active
BTM load . 15 In addition , the type of loads that are positioned behind
Another example of beneficial behind-the -meter power the a
-meter can vary within example embodiments . In general,
load that is behind-the-meter may correspond to any type
use is in new photovoltaic generation stations 202. For
example , it is common to design and build new photovoltaic of load capable of receiving and utilizing power behind -the
generation stations with a surplus of power capacity to meter from a generation station . Some examples of loads
account for degradation in efficiency of the photovoltaic 20 include, but are not limited to, datacenters and electric
panels over the life of the generation stations. Excess power vehicle ( EV ) charging stations.
availability at the generation station can occur when there is Preferred BTM loads are loads that can be subject to
excess local power generation and /or low grid demand. In intermittent power supply because BTM power may be
high incident sunlight situations, a photovoltaic generation available intermittently . In some instances, the generation
station 202 may generate more power than the intended 25 station may generate power intermittently. For example,
capacity of generation station 202. In such situations, a wind power station 102c and /or photovoltaic power station
photovoltaic generation station 202 may have to take steps 102d may only generate power when resource are available
to protect its equipment from damage, which may include or favorable . Additionally or alternatively, BTM power
taking one or more photovoltaic panels offline or shunting availability at a generation station may only be available
their voltage to dummy loads or the ground. Advanta- 30 intermittently due to power market fluctuations, power sys
geously, one or more flexible datacenters (e.g., the flexible tem conditions ( e.g., power factor fluctuation or generation
datacenters 220 ) may take power behind - the -meter at the station startup and testing), and /or operational directives
Generations Station 02, thereby allowing the generation from grid operators or generation station operators .
station 202 to operate the power generation equipment 210 Some example embodiments of BTM loads described
within operating ranges while the flexible datacenters 220 35 herein
receive BTM power without transmission or distribution serve asinvolve using one or more computing systems to
a BTM load at a generation station . In particular, the
costs .
Thus, for at least the reasons described herein , arrange computing system or computing systemsmay receive power
ments that involves providing a BTM load as an alternative behind-the -meter from the generation station to perform
option for a generation station to direct its generated power 40 various computational operations , such as processing or
to can serve as a mutually beneficial relationship in which storing information , performing calculations, mining for
both the generation station and the BTM load can economi cryptocurrencies , supporting blockchain ledgers, and /or
cally benefit . The above -noted examples of beneficial use of executing applications, etc.
BTM power are merely exemplary and are not intended to Multiple computing systems positioned behind - the-meter
limit the scope of what one of ordinary skill in the art would 45 may operate as part of a “ flexible” datacenter that is con
recognize as benefits to unutilized BTM power capacity, figured to operate only intermittently and to receive and
BTM power pricing , or BTM power consumption . utilize BTM power to carry out various computational
Within example embodiments described herein , various operations similar to a traditional datacenter. In particular,
types of utility -scale power producers may operate as gen the flexible datacenter may include computing systems and
eration stations 202 that are capable of supplying power to 50 other components (e.g. , support infrastructure, a control
one ormore loads behind -the -meter. For instance, renewable system ) configured to utilize BTM power from one ormore
energy sources (e.g., wind , solar, hydroelectric , wave , water generation stations . The flexible datacenter may be config
current, tidal), fossil fuel power generation sources (coal, ured to use particular load ramping abilities ( e.g., quickly
natural gas ), and other types of power producers ( e.g., increase or decrease power usage ) to effectively operate
nuclear power ) may be positioned in an arrangement that 55 during intermittent periods of time when power is available
enables the intermittent supply of generated power behind from a generation station and supplied to the flexible data
the -meter to one or more BTM loads. One of ordinary skill center behind -the-meter, such as during situations when
in the art will recognize that the generation station 202 may supplying generated power to the grid is not favorable for
vary based on an application or design in accordance with the generation station.
one or more example embodiments. 60 In some instances, the amount of power consumed by the
In addition , the particular arrangement (e.g. , connections) computing systems at a flexible datacenter can be ramped up
between the generation station and one ormore BTM loads and down quickly , and potentially with high granularity (i.e.,
can vary within examples . In one embodiment, a generation the load can be changed in small increments if desired ). This
station may be positioned in an arrangement wherein the may be done based on monitored power system conditions
generation station selectively supplies power to the grid 65 or other information analyses as discussed herein . As recited
and/ or to one or more BTM loads. As such , power cost above , this can enable a generation station to avoid negative
analysis and other factors (e.g., predicted weather condi power market pricing and to respond quickly to grid direc
US 10,608,433 B1
13 14
tives . And by extension , the flexible datacenter may obtain voltages (e.g., typically 4 kV to 26 kV AC ) and sent into the
BTM power at a price lower than the cost for power from the distribution networks, such as distribution network 206 via
grid . distribution line 256. The power on distribution line 256
Various types ofcomputing systems can provide granular may be further stepped down (not shown) before entering
power ramping . Preferably , the computing systems can 5 individual consumer facilities such as a remote master
perform computational tasks that are immune to , or not control system 262 and /or traditional datacenters 260 via
substantially hindered by , frequent interruptions or slow customer meters 206A , which may correspond to customer
downs in processing as the computing systems ramp down meters 106d in FIG . 1, or customer meters 106f in FIG . 1 if
or up . In some embodiments, a control system may be used the respective consumer facility includes a local customer
to activate or de- activate one or more computing systems in 10 power system , such as 106e (not shown in FIG . 2 ).
an array of computing systems. For example , the control Consistent with FIG . 1, power entering the grid from
system may provide control instructions to one or more generation station 202 is metered by a utility -scale genera
blockchain miners (e.g., a group of blockchain miners ), tion - side meter. A utility - scale generation -side meter 253 is
including instructions for powering on or off, adjusting shown on the low side of transformer system 203 and an
frequency of computing systems performing operations 15 alternative location is shown as 253A on the high side of
(e.g., adjusting the processing frequency ), adjusting the transformer system 203. Both locations may be considered
quantity of operations being performed , and when to operatesettlement metering points for the generation station 202 at
within a low power mode ( if available ). the POI 103. Alternatively , a utility -scale generation - side
Within examples , a control system may correspond to a meter for the generation station 202 may be located at
specialized computing system or may be a computing sys- 20 another location consistent with the descriptions of such
tem within a datacenter serving in the role of the control meters provided herein .
system . The location of the control system can vary within Generation station 202 includes power generation equip
examples as well. For instance , the control system may be ment 210 , which may include , as examples, wind turbines
located at a datacenter or physically separate from the and/or photovoltaic panels . Power generation equipment
datacenter. In some examples , the control system may be 25 210 may further include other electrical equipment, includ
part of a network of control systems that manage computa ing but not limited to switches,busses , collectors, inverters ,
tional operations, power consumption , and other aspects of and power unit transformers (e.g., transformers in wind
a fleet of datacenters. The fleet of datacenters may include turbines ).
one or more traditional datacenters and /or flexible datacen As illustrated in FIG . 2 , generation station 202 is config
ters . 30 ured to connect with BTM equipment which may function as
Some embodiments may involve using one or more BTM loads. In the illustrated embodiment of FIG . 2 , the
control systems to direct time- insensitive (e.g., interruptible ) BTM equipment includes flexible datacenters 220. Various
computational tasks to computational har are , such as configurations to supply BTM power flexible datacenters
central processing units (CPUs) and graphics processing 220 within the arrangement of FIG . 2 are described herein .
units (GPUs), sited behind the meter, while other hardware 35 In one configuration , generated power may travel from
is sited in front of the meter (i.e., consuming metered grid the power generation equipment 210 over one or more
power via a customer meter (e.g., 106 , 1060) and possibly connectors 230A , 230B to one or more electrical busses
remote from the behind - the -meter hardware. As such , par 240A , 240B , respectively . Each of the connectors 230A ,
allel computing processes , such as Monte Carlo simulations , 230B may be a switched connector such that power may be
batch processing of financial transactions, graphics render- 40 routed independently to 240A and /or 240B . For illustrative
ing , machine learning, neural network processing , queued purposes only, connector 230B is shown with an open
operations , and oil and gas field simulation models , are good switch , and connector 230A is shown with a closed switch ,
candidates for such interruptible computational operations. but either or both may be reversed in some embodiments.
FIG . 2 shows a behind-the-meter arrangement with Aspects of this configuration can be used in various embodi
optional grid -power, including one or more flexible data- 45 ments when BTM power is supplied without significant
centers, according to one or more example embodiments . power conversion to BTM loads.
Dark arrows illustrate a typical power delivery direction . In various configurations, the busses 240A and 240B may
Consistent with FIG . 1 , the arrangement illustrates a gen be separated by an open switch 240C or combined into a
eration station 202 in the generation segment 102 of a common bus by a closed switch 240C .
Wide -Area Synchronous Grid . The generation station 202 50 In another configuration, generated power may travel
supplies utility -scale power (typically > 50 MW ) via a gen from the power generation equipment 210 to the high side of
eration power connection 250 to the Point of Interconnection a local step - down transformer 214. The generated power
103 between the generation station 202 and the rest of the may then travel from the low side of the local step -down
grid. Typically, the power supplied on connection 250 may transformer 214 over one ormore connectors 232A , 232B to
be at 34.5 kV AC , but it may be higher or lower. Depending 55 the one or more electrical busses 240A , 240B , respectively .
on the voltage at connection 250 and the voltage at trans Each of the connectors 232A , 232B may be a switched
mission lines 104a , a transformer system 203 may step up connector such that power may be routed independently to
the power supplied from the generation station 202 to high 240A and/or 240B . For illustrative purposes only , connector
voltage (e.g., 115 kV + AC ) for transmission over connection 232A is shown with an open switch , and connector 232B is
252 and onto transmission lines 104a of transmission seg- 60 shown with a closed switch , but either or both may be
ment 104. Grid power carried on the transmission segment reversed in some embodiments . Aspects of this configura
104 may be from generation station 202 as well as other tion can be used when it is preferable to connect BTM power
generation stations (not shown ). Also consistentwith FIG . 1 , to the power generation equipment 210 , but the generated
grid power is consumed at one or more distribution net powermust be stepped down prior to use at the BTM loads .
works , including example distribution network 206. Grid 65 In another configuration, generated power may travel
power may be taken from the transmission lines 104a via from the power generation equipment 210 to the low side of
connector 254 and stepped down to distribution network a local step -up transformer 212. The generated power may
US 10,608,433 B1
15 16
then travel from the high side of the local step -up trans through POI 103 and such grid power may power the
former 212 over one or more connectors 234A , 234B to the generation station control system 216 .
one or more electrical busses 240A , 240B , respectively . In some configurations, an energy storage system 218
Each of the connectors 234A , 234B may be a switched ui may be connected to the generation station 202 via connec
connector such that power may be routed independently to tor 218A , which may be a switched connector. For illustra
240A and /or 240B . For illustrative purposes only, both tive purposes only , connector 218A is shown with an open
connectors 234A , 234B are shown with open switches, but switch but in some embodiments it may be closed . The
either or both may be closed in some embodiments. Aspects energy storage system 218 may be connected to bus 240A
of this configuration can be used when it is preferable to and /or bus 240B and store energy produced by the power
connect BTM power to the outbound connector 250 or the 10 generation equipment 210. The energy storage system may
high side of the local step - up transformer 212 . also be isolated from generation station 202 by switch 242A .
In another configuration , generated power may travel In times of need , such as when the power generation
from the power generation equipment 210 to the low side of equipment in an idle or off state and not generating power,
the local step -up transformer 212. The generated power may the energy storage system may feed power to , for example ,
then travel from the high side of the local step -up trans- 15 the flexible datacenters 220. The energy storage system may
former 212 to the high side of local step -down transformer also be isolated from the flexible datacenters 220 by switch
213. The generated power may then travel from the low side 242B .
of the local step -down transformer 213 over one or more In a preferred embodiment, as illustrated, power genera
connectors 236A , 236B to the one or more electrical busses tion equipment 210 supplies BTM power via connector 242
240A , 240B , respectively. Each of the connectors 236A , 20 to flexible datacenters 220. The BTM power used by the
2346 may be a switched connector such that power may be flexible datacenters 220 was generated by the generation
routed independently to 240A and /or 240B . For illustrative station 202 and did not pass through the POI 103 or
purposes only, both connectors 236A , 236B are shown with utility -scale generation -side meter 253, and is not subject to
open switches, but either or both may be closed in some T & D charges . Power received at the flexible datacenters 220
embodiments. Aspects of this configuration can be used 25 may be received through respective power input connectors
when it is preferable to connect BTM power to the outbound 220A . Each of the respective connectors 220A may be
connector 250 or the high side of the local step -up trans switched connector that can electrically isolate the respec
former 212, but the power must be stepped down prior to use tive flexible datacenter 220 from the connector 242. Power
at the BTM loads. equipment 220B may be arranged between the flexible
In one embodiment, power generated at the generation 30 datacenters 220 and the connector 242. The power equip
station 202 may be used to power a generation station ment 220B may include , but is not limited to , power
control system 216 located at the generation station 202 , conditioners, unit transformers, inverters, and isolation
when power is available. The generation station control equipment. As illustrated , each flexible datacenter 220 may
system 216 may typically control the operation of the be served by a respective power equipment 220B . However,
generation station 202. Generated power used at the gen- 35 in another embodiment, one power equipment 220B may
eration station control system 216 may be supplied from bus serve multiple flexible datacenter 220 .
240A via connector 216A and/ or from bus 240B via con In one embodiment, flexible datacenters 220 may be
nector 216B . Each of the connectors 216A , 216B may be a considered BTM equipment located behind -the -meter and
switched connector such that power may be routed indepen electrically connected to the power generation equipment
dently to 240A and /or 240B . While the generation station 40 210 behind ( i.e., prior to ) the generation station's POI 103
control system 216 can consumeBTM power when powered with the rest of the electrical grid .
via bus 240A or bus 240B , the BTM power taken by In one embodiment, BTM power produced by the power
generation station control system 216 is insignificant in generation equipment 210 is utilized by the flexible data
terms of rendering an economic benefit. Further, the gen centers 220 behind ( i.e., prior to ) the generation station's
eration station control system 216 is not configured to 45 POI with an electrical grid .
operate intermittently , as it generally must remain always In another embodiment, flexible datacenters 220 may be
on . Further still, the generation station control system 216 considered BTM equipment located behind -the-meter as the
does not have the ability to quickly ramp a BTM load up or flexible datacenters 220 are electrically connected to the
down. generation station 202, and generation station 202 is subject
In another embodiment, grid power may alternatively or 50 to metering by utility -scale generation -side meter 253 (or
additionally be used to power the generation station control 253A, or another utility -scale generation -side meter ), and
system 216. As illustrated here, metered grid power from a the flexible datacenters 220 receive power from the genera
distribution network , such as distribution network 206 for tion station 202 , but the power received by the flexible
simplicity of illustration purposes only, may be used to datacenters 220 from the generation station 202 has not
power generation station control system 216 over connector 55 passed through a utility - scale generation -side meter. In this
216C . Connector 216C may be a switched connector so that embodiment, the utility -scale generation -sidemeter 253 (or
metered grid power to the generation station control system 253A ) for the generation station 202 is located at the
216 can be switched on or off as needed .More commonly, generation station's 202 POI 103. In another embodiment,
metered grid power would be delivered to the generation the utility -scale generation -side meter for the generation
station control system 216 via a separate distribution net- 60 station 202 is at a location other than the POI for the
work (not shown ), and also over a switched connector. Any generation station 202 — for example, a substation (not
such grid power delivered to the generation station control shown ) between the generation station 202 and the genera
system 216 is metered by a customer meter 206A and subject tion station's POI 103 .
to T & D costs . In another embodiment, power from the generation sta
In another embodiment, when power generation equip- 65 tion 202 is supplied to the flexible datacenters 220 as BTM
ment 210 is in an idle or off state and not generating power, power, where power produced at the generation station 202
grid power may backfeed into generation station 202 is subject to metering by utility -scale generation -side meter
US 10,608,433 B1
17 18
253 (or 253A , or another utility -scale generation -side datacenters 220 , in which case each of the flexible datacen
meter), but the BTM power supplied to the flexible data ters 220 so connected would not be operating as a BTM load .
centers 220 is utilized before being metered at the utility In another embodiment, when power generation equip
scale generation -side meter 253 (or 253A , or another utility ment 210 is in an idle or off state and not generating power,
scale generation -sidemeter ). In this embodiment, the utility- 5 grid power may backfeed into generation station 202
scale generation - side meter 253 (or 253A ) for the generation through POI 103 and such grid power may power the
station 202 is located at the generation station's 202 POI flexible datacenters 220 .
103. In another embodiment, the utility -scale generation The flexible datacenters 220 are shown in an example
side meter for the generation station 202 is at a location other arrangement relative to the generation station 202. Particu
than the POI for the generation station 202 — for example , a 10 larly, generated power from the generation station 202 may
substation (not shown ) between the generation station 202 be supplied to the flexible datacenters 220 through a series
and the generation station's POI 103. of connectors and / or busses (e.g., 232B , 240B , 242 , 220A ).
In another embodiment, flexible datacenters 220 may be As illustrated , in other embodiments , connectors between
considered BTM equipment located behind -the-meter as the power generation equipment 210 and other components
they are electrically connected to the generation station 202 15 may be switched open or closed , allowing other pathways
that supplies power to the grid , and the flexible datacenters for power transfer between the power generation equipment
220 receive power from the generation station 202 that is not 210 and components , including the flexible datacenters 220 .
subject to T & D charges , but power otherwise received from Additionally, the connector arrangement shown is illustra
the grid that is supplied by the generation station 202 is tive only and other circuit arrangements are contemplated
subject to T & D charges. 20 within the scope of supplying BTM power to a BTM load at
In another embodiment, power from the generation sta generation station 202. For example , there may be more or
tion 202 is supplied to the flexible datacenters 220 as BTM fewer transformers, or one ormore of transformers 212, 213,
power, where electrical power is generated at the generation 214 ay be transformer systems with multiple steppings
station 202 that supplies power to a grid , and the generated and /or may include additional power equipment including
power is not subject to T & D charges before being used by 25 but not limited to power conditioners , filters , switches,
flexible datacenters 220,but power otherwise received from inverters , and / or AC /DC -DC /AC isolators . As another
the connected grid is subject to T & D charges . example, metered grid power connections to flexible data
In another embodiment, flexible datacenters 220 may be centers 220 are shown via both 256A and 256B ; however, a
considered BTM equipment located behind -the -meter single connection may connect one or more flexible data
because they receive power generated from the generation 30 centers 220 (or power equipment 220B ) to metered grid
station 202 intended for the grid , and that received power is power and the one or more flexible datacenters 220 (or
not routed through the electrical grid before being delivered power equipment 220B )may include switching apparatus to
to the flexible datacenters 220 . direct BTM power and /or metered grid power to control
In another embodiment, power from the generation sta systems, communication systems, and /or computing sys
tion 202 is supplied to the flexible datacenters 220 as BTM 35 tems as desired .
power , where electrical power is generated at the generation In some examples, BTM powermay arrive at the flexible
station 202 for distribution to the grid , and the flexible datacenters 220 in a three -phase AC format. As such , power
datacenters 220 receive that power, and that received power equipment (e.g. , power equipment 220B ) at one or more of
is not routed through the electrical grid before being deliv the flexible datacenters 220 may enable each flexible data
ered to the flexible datacenters 220 . 40 center 220 to use one or more phases of the power. For
In another embodiment, metered grid power may alter instance , the flexible datacenters 220 may utilize power
natively or additionally be used to power one or more of the equipment (e.g., power equipment 220B , or alternatively or
flexible datacenters 220 , or a portion within one or more of additionally power equipment that is part of the flexible
the flexible datacenters 220. As illustrated here for simplic datacenter 220 ) to convert BTM power received from the
ity ,metered grid power from a distribution network , such as 45 generation station 202 for use at computing systems at each
distribution network 206 ,may be used to power one ormore flexible datacenter 220. In other examples, the BTM power
flexible datacenters 220 over connector 256A and /or 256B . may arrive at one or more of the flexible datacenters 220 as
Each of connector 256A and /or 256B may be a switched DC power. As such , the flexible datacenters 220 may use the
connector so that metered grid power to the flexible data DC power to power computing systems. In some such
centers 220 can be switched on or off as needed . More 50 examples, the DC power may be routed through a DC -to -DC
commonly, metered grid power would be delivered to the converter that is part of power equipment 220B and /or
flexible datacenters 220 via a separate distribution network flexibles datacenter 220 .
(not shown ), and also over switched connectors. Any such In some configurations, a flexible datacenter 220 may be
grid power delivered to the flexible datacenters 220 is arranged to only have access to power received behind -the
metered by customer meters 206A and subject to T & D costs. 55 meter from a generation station 202. In the arrangement of
In one embodiment, connector 256B may supply metered FIG . 2 , the flexible datacenters 220 may be arranged only
grid power to a portion of one or more flexible datacenters with a connection to the generation station 202 and depend
220. For example, connector 256B may supply metered grid solely on power received behind -the -meter from the gen
power to control and /or communication systems for the eration station 202. Alternatively or additionally , the flexible
flexible datacenters 220 that need constant power and cannot 60 datacenters 220 may receive power from energy storage
be subject to intermittent BTM power. Connector 242 may system 218.
supply solely BTM power from the generation station 202 to In some configurations, one or more of the flexible
high power demand computing systems within the flexible datacenters 220 can be arranged to have connections to
datacenters 220 , in which case at least a portion of each multiple sources that are capable of supplying power to a
flexible datacenters 220 so connected is operating as a BTM 65 flexible datacenter 220. To illustrate a first example , the
load . In another embodiment, connector 256A and/or 256B flexible datacenters 220 are shown connected to connector
may supply all power used at one or more of the flexible 242, which can be connected or disconnected via switches to
US 10,608,433 B1
19 20
the energy storage system 218 via connector 218A , the examples, flexible datacenters 220 may be deployed at a
generation station 202 via bus 240B , and grid power via location geographically remote from the generation station
metered connector 256A . In one embodiment, the flexible 202 , while still maintaining a BTM power connection to the
datacenters 220 may selectively use power received behind generation station 202.
the -meter from the generation station 202, stored power 5 In another example arrangement, the generation station
supplied by the energy storage system 218 , and/or grid 202 may be connected to a first BTM load (e.g., a flexible
power. For instance, flexible datacenters 220 may use power datacenter 220) and may supply power to additional BTM
stored in the energy storage system 218 when costs for using loads via connections between the first BTM load and the
power supplied behind - the-meter from the generation sta additional BTM loads (e.g., a connection between a flexible
tion 202 are disadvantageous. By having access to the 10 datacenter 220 and another flexible datacenter 220).
energy storage system 218 available , the flexible datacenters
220 may use the stored power and allow the generation therein The arrangement in FIG . 2 , and components included
station 202 to subsequently refill the energy storage system , are for non - limiting illustration purposes and other
arrangements are contemplated in examples. For instance , in
218 when cost for power behind- the -meter is low . Alterna another
tively , the flexible datacenters 220 may use power from 15 example embodiment, the arrangement of FIG . 2
multiple sources simultaneously to power different compo may include more or fewer components , such as more BTM
nents (e.g. , a first set and a second set of computing loads, different connections between power sources and
systems). Thus, the flexible datacenters 220 may leverage loads, and / or a different number of datacenters. In addition ,
the multiple connections in a manner that can reduce the cost some examplesmay involve one ormore components within
for power used by the computing systems at the flexible 20 the arrangement of FIG . 2 being combined or further
datacenters 220. The flexible datacenters 220 control system divided .
or the remote master control system 262 may monitor power Within the arrangement of FIG . 2 , a control system , such
conditions and other factors to determine whether the flex as the remote master control system 262 or another compo
ible datacenters 220 should use power from either the nent ( e.g ., a control system associated with the grid operator,
generation station 202, grid power , the energy storage sys- 25 the generation station control system 216 , or a datacenter
tem 218 , none of the sources, or a subset of sources during control system associated with a traditional datacenter or
a given time range. Other arrangements are possible as well . one or more flexible datacenters ) may use information to
For example , the arrangement of FIG . 2 illustrates each efficiently manage various operations of some of the com
flexible datacenter 220 as connected via a single connector ponents within the arrangement of FIG . 2. For example , the
242 to energy storage system 218 , generation station 202 , 30 remote master control system 262 or another component
and metered grid power via 256A . However, one or more may manage distribution and execution of computational
flexible datacenters 220 may have independent switched operations at one or more traditional datacenters 260 and/or
connections to each energy source , allowing the one ormore flexible datacenters 220 via one or more information -pro
flexible datacenters 220 to operate from different energy cessing algorithms. These algorithms may utilize past and
sources than other flexible datacenters 220 at the same time. 35 current information in real- time to manage operations of the
The selection of which power source to use at a flexible different components . These algorithms may also make
datacenter ( e.g., the flexible datacenters 220 ) or another type some predictions based on past trends and information
of BTM load can change based on various factors, such as analysis . In some examples, multiple computing systems
the cost and availability of power from both sources, the may operate as a network to process information .
type of computing systems using the power at the flexible 40 Information used to make decisions may include eco
datacenters 220 (e.g., some systems may require a reliable nomic and /or power-related information , such as monitored
source of power for a long period), the nature of the power system conditions. Monitored power system condi
computational operations being performed at the flexible tions may include one or more of excess power generation
datacenters 220 (e.g., a high priority task may require at a generation station 202 , excess power at a generation
immediate completion regardless of cost), and temperature 45 station 202 that a connected grid cannot receive , power
and weather conditions, among other possible factors. As generation at a generation station 202 subject to economic
such , a datacenter control system at the flexible datacenters curtailment, power generation at a generation station 202
220 , the remote master control system 262, or another entity subject to reliability curtailment, power generation at a
(e.g., an operator at the generation station 202 ) may also generation station 202 subject to power factor correction ,
influence and/or determine the source of power that the 50 low power generation at a generation station 202 , start up
flexible datacenters 220 use at a given time to complete conditions at a generation station 202 , transient power
computational operations . generation conditions at a generation station 202, or testing
In some example embodiments, the flexible datacenters conditions where there is an economic advantage to using
220 may use power from the different sources to serve behind- the-meter power generation at a generation station
different purposes. For example, the flexible datacenters 220 55 202. These different monitored power system conditions can
may use metered power from grid power to power one or be weighted differently during processing and analysis.
more systems at the flexible datacenters 220 that are con In some examples, the information can include the cost
figured to be always-on (or almost always on ), such as a for power from available sources (e.g., BTM power at the
control and /or communication system and /or one or more generation station 202 versus metered grid power ) to enable
computing systems (e.g., a set of computing systems per- 60 comparisons to be made which power source costs less. In
forming highly important computational operations). The some instances, the information may include historic prices
flexible datacenters 220 may use BTM power to power other for power to enable the remote master control system 262 or
components within the flexible datacenters 220 , such as one another system to predict potential future prices in similar
or more computing systems that perform less critical com situations ( e.g., the cost of power tends to trend upwards for
putational operations. 65 grid power during warmer weather and peak -use hours ). The
In some examples , one or more flexible datacenters 220 information may also indicate the availability of power from
may be deployed at the generation station 202. In other the various sources (e.g., BTM power at the generation
US 10,608,433 B1
21 22
station 262 , the energy storage system 218 at the generation see one or more operations within the arrangement of FIG .
station 262, and/or metered grid power ). 2. As such , the remote master control system 262may be one
In addition , the information may also include other data , or more computing systems configured to process all, or a
including information associated with operations at compo subset of, the information described above , such as power,
nents within the arrangement. For instance , the information 5 environment, computational characterization , and economic
may include data associated with performance of operations factors to assist with the distribution and execution of
at the flexible datacenters 220 and the traditional datacenters
260 , such as the number of computational tasks currently computinginstance ,
operations among one or more datacenters. For
the remote master control system 262 may be
being performed , the types of tasks being performed (e.g., configured to obtain and delegate computational operations
type of computational operation , time- sensitivity, etc.), the 10 among one or more datacenters
number , types, and capabilities of available computing sys of a variety of factors , includingbased on a weighted analysis
tems, the amount of computational tasks awaiting perfor availability of power, the types andmore one or of the cost and
availability of the
mance, and the types of computing systems at one or more computing systems at each datacenter, current and predicted
datacenters, among others. The information may also
include data specifying the conditions at one or more 15 weather conditions at the different locations of flexible
datacenters (e.g., flexible datacenters 220 ) and generation
datacenters ( e.g. , whether or not the temperatures are in a
desired range , the amount of power available within an stations ( e.g., generation stations 202 ), levels of power
energy storage system such as 218 ), the amount of compu storage available at one or more energy storage systems
tational tasks awaiting performance in the queue of one or ( e.g., energy storage system 218 ), and deadlines and other
more of the datacenters, and the identities of the entities 20 attributes associated with particular computational opera
associated with the computational operations at one ormore tions, among other possible factors. As such , the analysis of
of the datacenters . Entities associated with computational information performed by the remote master control system
operations may be, for example , owners of the datacenters, 262 may vary within examples. For instance , the remote
customers who purchase computational time at the datacen master control system 262 may use real-time information to
ters , or other entities. 25 determine whether or not to route a computational operation
The information used by the remote master control system to a particular flexible datacenter ( e.g., a flexible datacenter
262 or another componentmay include data associated with 220 ) or to transition a computational operation between
the computational operations to be performed , such as datacenters ( e.g., from traditional datacenter 260 to a flexible
deadlines, priorities ( e.g. , high vs. low priority tasks ), cost to datacenter
perform based on required computing systems, the optimal 30 As shown220in). FIG . 2, the generation station 202 may be
computing systems (e.g., CPU vs GPU vs ASIC ; processing able to supply power to the grid and /or BTM loads such as
unit capabilities , speeds, or frequencies, or instructional sets
executable by the processing units ) for performing each flexible datacenters 220. With such a configuration , the
requested computational task , and prices each entity (e.g., BTM loadsstation
generation 202 may selectively provide power to the
and/or the grid based on economic and power
company ) is willing to pay for computational operations to 35 availability
be performed or otherwise supported via computing systems tion 202 mayconsiderations supply power
. For example, the generation sta
to the grid when the price paid
at a traditional datacenter 260 or a flexible datacenter 220 ,
among others . In addition , the information may also include for the power exceeds a particular threshold (e.g., the power
other data ( e.g., weather conditions at locations of datacen price offered by operators of the flexible datacenters 220 ). In
ters or power sources, any emergencies associated with a 40 some instances , the operator of a flexible datacenter and the
datacenter or power source, or the current value of bids operator of a generation station capable of supplying BTM
associated with an auction for computational tasks). power to the flexible datacenter may utilize a predefined
The information may be updated in - real time and used to arrangement (e.g., a contract) that specifies a duration and /or
make the different operational decisions within the arrange price range when the generation station may supply power
ment of FIG . 2. For instance , the information may help a 45 to the flexible datacenter.
component (e.g., the remote master control system 262 or a The remote master control system 262 may be capable of
control system at a flexible datacenter 220 ) determine when directing one or more flexible datacenters 220 to ramp-up or
to ramp up or ramp down power use at a flexible datacenter ramp-down to desired power consumption levels, and /or to
220 or when to switch one ormore computing systems at a control cooperative action ofmultiple flexible datacenters by
flexible datacenter 220 into a low powermode or to operate 50 determining how to power each individual flexible datacen
at a different frequency, among other operational adjust ter 220 in accordance with operational directives.
ments. The information can additionally or alternatively The configuration of the remote master control system
help a component within the arrangement of FIG . 2 to 262 can vary within examples as further discussed with
determine when to transfer computational operations respect to FIGS . 2 , 3 , and 7-9 . The remote master control
between computing systems or between datacenters based 55 system 262 may operate as a single computing system or
on various factors. In some instances, the information may may involve a network of computing systems. Preferably,
also be used to determine when to temporarily stop per the remote master control system 262 is implemented across
forming a computational operation or when to perform a one or more servers in a fault -tolerant operating environ
computational operation atmultiple sites for redundancy or ment that ensures continuous uptime and connectivity by
other reasons. The information may further be used to 60 virtue of its distributed nature. Alternatively, although the
determine when to accept new computational operations remote master control system 262 is shown as a physically
from entities or when to temporarily suspend accepting new separate component arrangement for FIG . 2 , the remote
tasks to be performed due to lack of computing system master control system 262 may be combined with another
availability. component in other embodiments . To illustrate an example,
The remote master control system 262 represents a com- 65 the remotemaster control system 262 may operate as part of
puting system that is capable of obtaining , managing, and a flexible datacenter (e.g., a computing system or a data
using the information described above to manage and over center control system of the flexible datacenter 220 ), includ
US 10,608,433 B1
23 24
ing sharing components with a flexible datacenter , sharing ofways ( e.g., straight configuration , zig -zag configuration )
power with a flexible datacenter, and /or being co - located as further discussed with respect to FIGS. 6A , 6B . Further
with a flexible datacenter. more, although the example arrangement illustrated in FIG .
In addition , the remote master control system 262 may 2 shows configurations where flexible datacenters 220 serve
communicate with components within the arrangement of 5 as BTM loads, other types of loads can be used as BTM
FIG . 2 using various communication technologies, including loads within examples.
wired and wireless communication technologies. For The arrangement of FIG . 2 includes the traditional data
instance , the remote master control system 262 may use centers 260 coupled to metered grid power . The traditional
wired (not illustrated ) or wireless communication to com datacenters 260 using metered grid power to provide com
municate with datacenter control systems or other comput- 10 putational resources to support computational operations.
ing systems at the flexible datacenters 220 and the traditional One or more enterprises may assign computational opera
datacenters 260. The remote master control system 262 may tions to the traditional datacenters 260 with expectations that
also communicate with entities inside or outside the arrange the datacenters reliably provide resources without interrup
ment of FIG . 2 and other components within the arrange tion (i.e., non - intermittently ) to support the computational
ment of FIG . 2 via wired or wireless communication . For 15 operations, such as processing abilities, networking , and /or
instance , the remote master control system 262 may use volatile storage. Similarly , one or more enterprises may also
wireless communication to obtain computational operations request computational operations to be performed by the
from entities seeking support for the computational opera flexible datacenters 220. The flexible datacenters 220 differ
tions at one or more datacenters in exchange for payment. from the traditional datacenters 260 in that the flexible
The remote master control system 262 may communicate 20 datacenters 220 are arranged and /or configured to be con
directly with the entities or may obtain the computational nected to BTM power , are expected to operate intermittently ,
operations from the traditional datacenters 260. For instance , and are expected to ramp load (and thus computational
an entity may submit jobs (e.g., computational operations) to capability ) up or down regularly in response to control
one or more traditional datacenters 260. The remote master directives. In some examples, the flexible datacenters 220
control system 262 may determine that transferring one or 25 and the traditional datacenters 260 may have similar con
more of the computational operations to a flexible datacenter figurations and may only differ based on the source( s) of
220 may better support the transferred computational opera power relied upon to power internal computing systems.
tions. For example , the remote master control system 262 Preferably, however , the flexible datacenters 220 include
may determine that the transfer may enable the computa particular fast load ramping abilities (e.g., quickly increase
tional operations to be completed quicker and /or at a lower 30 or decrease power usage ) and are intended and designed to
cost. In some examples , the remote master control system effectively operate during intermittent periods of time.
262 may communicate with the entity to obtain approval FIG . 3 shows a block diagram of the remote master
prior to transferring the one or more computational opera control system 300 according to one or more example
tions . embodiments. Remote master control system 262 may take
The remote master control system 262 may also commu- 35 the form of remote master control system 300 , or may
nicate with grid operators and /or an operator of generation include less than all components in remote master control
station 202 to help determine power management strategies system 300 , different components than in remote master
when distributing computational operations across the vari control system 300, and /ormore components than in remote
ous datacenters. In addition , the remote master control master control system 300.
system 262 may communicate with other sources, such as 40 The remote master control system 300 may perform one
weather prediction systems, historical and current power or more operations described herein and may include a
price databases , and auction systems, etc. processor 302 , a data storage unit 304 , a communication
In further examples, the remote master control system 262 interface 306 , a user interface 308 , an operations and envi
or another computing system within the arrangement of FIG . ronment analysis module 310 , and a queue system 312. In
2 may use wired or wireless communication to submit bids 45 other examples, the remote master control system 300 may
within an auction that involves a bidder (e.g., the highest include more or fewer components in other possible arrange
bid) obtaining computational operations or other tasks to be ments .
performed . Particularly , the remote master control system As shown in FIG . 3 , the various components ofthe remote
262 may use the information discussed above to develop master control system 300 can be connected via one or more
bids to obtain computing operations for performance at 50 connection mechanisms ( e.g., a connection mechanism
available computing systems at flexible datacenters ( e.g., 314 ). In this disclosure , the term “ connection mechanism ”
flexible datacenters 220 ). means a mechanism that facilitates communication between
In the example arrangement shown in FIG . 2 , the flexible two or more devices, systems, components, or other entities.
datacenters 220 represent example loads that can receive For instance, a connection mechanism can be a simple
power behind - the-meter from the generation station 202. In 55 mechanism , such as a cable, PCB trace , or system bus, or a
such a configuration , the flexible datacenters 220 may obtain relatively complex mechanism , such as a packet-based com
and utilize power behind -the -meter from the generation munication network ( e.g., LAN ,WAN , and /or the Internet).
station 202 to perform various computational operations . In some instances , a connection mechanism can include a
Performance of a computational operation may involve one non-tangible medium (e.g., where the connection is wire
or more computing systems providing resources useful in 60 less ).
the computational operation . For instance, the flexible data As part of the arrangement of FIG . 2, the remote master
centers 220 may include one or more computing systems control system 300 (corresponding to remote master control
configured to store information , perform calculations and /or system 262) may perform a variety of operations, such as
parallel processes, perform simulations, mine cryptocurren management and distribution of computational operations
cies, and execute applications, among other potential tasks . 65 among datacenters, monitoring operational, economic , and
The computing systems can be specialized or generic and environment conditions, and power management. For
can be arranged at each flexible datacenter 220 in a variety instance, the remote master control system 300 may obtain
US 10,608,433 B1
25 26
computational operations from one or more enterprises for weather conditions, and information regarding the different
performance at one or more datacenters. The remote master datacenters (e.g., current workloads at datacenters , types of
control system 300 may subsequently use information to computing systems available within datacenters, price to
distribute and assign the computational operations to one or obtain power at each datacenter , levels of power storage
more datacenters ( e.g., the flexible datacenters 220) that 5 available and accessible at each datacenter, etc.). In an
have the resources ( e.g., particular types of computing example , the communication interface 306 can include a
systems and available power ) available to complete the wired interface , such as an Ethernet interface or a high
computational operations. In some examples, the remote definition serial-digital-interface (HD -SDI). In another
master control system 300 may assign all incoming com example , the communication interface 406 can include a
putational operation requests to the queue system 312 and 10 wireless interface, such as a cellular, satellite , WiMAX , or
subsequently assign the queued requests to computing sys WI-FI interface . A connection can be a direct connection or
tems based on an analysis of current market and power an indirect connection , the latter being a connection that
conditions. passes through and /or traverses one or more components ,
Although the remote master control system 300 is shown such as such as a router, switcher, or other network device .
as a single entity , a network of computing systems may 15 Likewise , a wireless transmission can be a direct transmis
perform the operations of the remote master control system sion or an indirect transmission . The communication inter
300 in some examples. For example , the remote master face 306 may also utilize other types of wireless communi
control system 300 may exist in the form of computing cation to enable communication with datacenters positioned
systems (e.g., datacenter control systems) distributed across at various locations.
multiple datacenters . 20 The communication interface 306 may enable the remote
The remote master control system 300 may include one or master control system 300 to communicate with the com
more processors 302. As such , the processor 302 may ponents of the arrangement of FIG . 2. In addition , the
represent one or more general -purpose processors (e.g., a communication interface 306 may also be used to commu
microprocessor) and /or one or more special-purpose proces nicate with the various datacenters, power sources, and
sors (e.g., a digital signal processor (DSP )). In some 25 different enterprises submitting computational operations
examples, the processor 302 may include a combination of for the datacenters to support.
processors within examples. The processor 302 may per The user interface 308 can facilitate interaction between
form operations , including processing data received from the remote master control system 300 and an administrator
the other components within the arrangement of FIG . 2 and or user, if applicable. As such , the user interface 308 can
data obtained from external sources, including information 30 include input components such as a keyboard, a keypad , a
such as weather forecasting systems, power market price mouse, a touch -sensitive panel, a microphone , and /or a
systems, and other types of sources or databases . camera, and / or output components such as a display device
The data storage unit 304 may include one or more (which , for example, can be combined with a touch -sensitive
volatile , non - volatile, removable, and /or non -removable panel ), a sound speaker, and /or a haptic feedback system .
storage components, such as magnetic , optical, or flash 35 More generally, the user interface 308 can include hardware
storage , and/ or can be integrated in whole or in part with the and /or software components that facilitate interaction
processor 302. As such , the data storage unit 304 may take between remote master control system 300 and the user of
the form of a non -transitory computer-readable storage the system .
medium , having stored thereon program instructions ( e.g., In some examples, the user interface 308 may enable the
compiled or non -compiled program logic and /or machine 40 manual examination and / or manipulation of components
code ) that, when executed by the processor 302 , cause the within the arrangement of FIG . 2. For instance , an admin
remote master control system 300 to perform one or more istrator or user may use the user interface 308 to check the
acts and / or functions , such as those described in this dis status of, or change , one or more computational operations ,
closure. Such program instructions can define and / or be part the performance or power consumption at one or more
of a discrete software application . In some instances , the 45 datacenters, the number of tasks remaining within the queue
remote master control system 300 can execute program system 312 , and other operations. As such , the user interface
instructions in response to receiving an input, such as from 308 may provide remote connectivity to one or more sys
the communication interface 306 , the user interface 308 , or tems within the arrangement of FIG . 2 .
the operations and environment analysis module 310. The The operations and environment analysis module 310
data storage unit 304 may also store other information , such 50 represents a component of the remote master control system
as those types described in this disclosure. 300 associated with obtaining and analyzing information to
In some examples , the data storage unit 304 may serve as develop instructions/directives for components within the
storage for information obtained from one ormore external arrangement of FIG . 2. The information analyzed by the
sources. For example , data storage unit 304 may store operations and environment analysis module 310 can vary
information obtained from one or more of the traditional 55 within examples and may include the information described
datacenters 260, a generation station 202 , a system associ above with respect predicting and / or directing the use of
ated with the grid , and flexible datacenters 220.As examples BTM power. For instance, the operations and environment
only , data storage 304 may include , in whole or in part, local analysis module 310 may obtain and access information
storage, dedicated server -managed storage , network related to the current power state of computing systems
attached storage , and /or cloud -based storage , and /or combi- 60 operating as part of the flexible datacenters 220 and other
nations thereof. datacenters that the remote master control system 300 has
The communication interface 306 can allow the remote access to . This information may be used to determine when
master control system 300 to connect to and /or communicate to adjust power usage or mode of one or more computing
with another component according to one or more protocols. systems. In addition , the remote master control system 300
For instance , the communication interface 306 may be used 65 may provide instructions a flexible datacenter 220 to cause
to obtain information related to current, future , and past a subset of the computing systems to transition into a low
prices for power , power availability , current and predicted power mode to consume less power while still performing
US 10,608,433 B1
27 28
operations at a slower rate . The remote master control 401 , a communication interface 408 , a behind -the-meter
system 300 may also use power state information to cause a interface 406 , a grid interface 404 , a user interface 410 , a
set of computing systems at a flexible datacenter 220 to generation station control system 414 , and power transfor
operate at a higher power consumption mode . In addition , mation equipment 402. The power generation equipment
the remote master control system 300 may transition com 5 210 may take the form of power generation equipment 401 ,
puting systems into sleep states or power on /off based on ormay include less than all components in power generation
information analyzed by the operations and environment equipment 401, different components than in power genera
analysis module 310 . tion equipment 401, and / or more components than in power
In some examples, the operations and environment analy generation equipment 401. Generation station control sys
sis module 310 may use location , weather, activity levels at 10 tem 216 may take the form of generation station control
the flexible datacenters or the generation station , and power system 414 , or may include less than all components in
cost information to determine control strategies for one or generation station control system 414, different components
more components in the arrangement of FIG . 2. For than in generation station control system 414 , and/or more
instance, the remote master control system 300 may use components than in generation station control system 414 .
location information for one or more datacenters to antici- 15 Some or all of the components generation station 400 may
pate potential weather conditions that could impact access to be connected via a communication interface 516. These
power. In addition , the operations and environment analysis components are illustrated in FIG . 4 to convey an example
module 310 may assist the remote master control system 300 configuration for the generation station 400 ( corresponding
determine whether to transfer computational operations to generation station 202 shown in FIG . 2 ). In other
between datacenters based on various economic and power 20 examples, the generation station 400 may include more or
factors . fewer components in other arrangements.
The queue system 312 represents a queue capable of The generation station 400 can correspond to any type of
organizing computational operations to be performed by one grid -connected utility -scale power producer capable of sup
or more datacenters . Upon receiving a request to perform a plying power to one or more loads. The size , amount of
computational operation , the remote master control system 25 power generated , and other characteristics of the generation
300 may assign the computational operation to the queue station 400 may differ within examples. For instance , the
until one or more computing systems are available to generation station 400 may be a power producer that pro
support the computational operation . The queue system 312 vides power intermittently . The power generation may
may be used for organizing and transferring computational depend on monitored power conditions, such as weather at
tasks in real time. 30 the location of the generation station 400 and other possible
The organizational design of the queue system 312 may conditions. As such , the generation station 400 may be a
vary within examples. In some examples, the queue system temporary arrangement, or a permanent facility , configured
312 may organize indications (e.g., tags, pointers) to sets of to supply power. The generation station 400 may supply
computational operations requested by various enterprises . BTM power to one ormore loads and supply metered power
The queue system 312 may operate as a First-In -First-Out 35 to the electrical grid. Particularly , the generation station 400
( FIFO ) data structure . In a FIFO data structure, the first may supply power to the grid as shown in the arrangement
element added to the queue will be the first one to be of FIG . 2 .
removed . As such , the queue system 312 may include one or The power generation equipment 401 represents the com
more queues that operate using the FIFO data structure . ponent or components configured to generate utility -scale
In some examples, one or more queues within the queue 40 power . As such , the power generation equipment 401 may
system 312 may use other designs of queues , including rules depend on the type of facility that the generation station 400
to rank or organize queues in a particular manner that can corresponds to . For instance , the power generation equip
prioritize some sets of computational operations over others . ment 401 may correspond to electric generators that trans
The rules may include one or more of an estimated cost form kinetic energy into electricity . The power generation
and /or revenue to perform each set of computational opera- 45 equipment 401 may use electromagnetic induction to gen
tions, an importance assigned to each set of computational erate power. In other examples, the power generation equip
operations, and deadlines for initiating or completing each ment 401 may utilize electrochemistry to transform chemi
set of computational operations, among others. Examples cal energy into power. The power generation equipment 401
using a queue system are further described below with may use the photovoltaic effect to transform light into
respect to FIG . 9 . 50 electrical energy. In some examples, the power generation
In some examples, the remote master control system 300 equipment 401 may use turbines to generate power . The
may be configured to monitor one ormore auctions to obtain turbines may be driven by, for example, wind, water, steam
computational operations for datacenters to support. Particu or burning gas. Other examples of power production are
larly , the remote master control system 300 may use possible .
resource availability and power prices to develop and submit 55 The communication interface 408 can enable the genera
bids to an external or internal auction system for the right to tion station 400 to communicate with other components
support particular computational operations. As a result , the within the arrangement of FIG . 2. As such , the communi
remote master control system 300 may identify computa cation interface 408 may operate similarly to the commu
tional operations that could be supported at one or more nication interface 306 of the remote master control system
flexible datacenters 220 at low costs . 60 300 and the communication interface 503 of the flexible
FIG . 4 is a block diagram of a generation station 400 , datacenter 500 .
according to one or more example embodiments .Generation The generation station control system 414 may be one or
station 202 may take the form of generation station 400 , or more computing systems configured to control various
may include less than all components in generation station aspects of the generation station 400 .
400, different components than in generation station 400 , 65 The BTM interface 406 is a module configured to enable
and /or more components than in generation station 400. The the power generation equipment 401 to supply BTM power
generation station 400 includes power generation equipment to one ormore loads and may include multiple components .
US 10,608,433 B1
29 30
The arrangement of the BTM interface 406 may differ within deployed at a location near a source of generation station
examples based on various factors, such as the number of power (e.g., near a wind farm or solar farm ). Rapid deploy
flexible datacenters 220 (or 500) coupled to the generation mentmay involve positioning the flexible datacenter 500 at
station 400 , the proximity of the flexible datacenters 220 (or a target location and installing and / or configuring one or
500 ), and the type of generation station 400 , among others. 5 more racks of computing systems within . The racks may
In some examples, the BTM interface 406 may be config include wheels to enable swift movement of the computing
ured to enable power delivery to one or more flexible systems. Although the flexible datacenter 500 could theo
datacenters positioned near the generation station 400. Alter retically be placed anywhere, transmission losses may be
natively , the BTM interface 406 may also be configured to minimized by locating it proximate to BTM power genera
enable power delivery to one or more flexible datacenters 10 tion .
220 (or 500 ) positioned remotely from the generation station The physical construction and layout of the flexible
400 . datacenter 500 can vary . In some instances, the flexible
The grid interface 404 is a module configured to enable datacenter 500 may utilize a metal container (e.g., a metal
the power generation equipment 401 to supply power to the container 602 shown in FIG . 6A ). In general, the flexible
grid andmay include multiple components.As such ,the grid 15 datacenter 500 may utilize some form of secure weather
interface 404 may couple to one ormore transmission lines proof housing designed to protect interior components from
(e.g. , transmission lines 404a shown in FIG . 2) to enable wind , weather , and intrusion . The physical construction and
delivery of power to the grid . layout of example flexible datacenters are further described
The user interface 410 represents an interface that enables with respect to FIGS. 6A -6B .
administrators and /or other entities to communicate with the 20 Within the flexible datacenter 500, various internal com
generation station 400. As such , the user interface 410 may ponents enable the flexible datacenter 500 to utilize power to
have a configuration that resembles the configuration of the perform some form of operations. The power input system
user interface 308 shown in FIG . 3. An operator may utilize 502 is a module of the flexible datacenter 500 configured to
the user interface 410 to control or monitor operations at the receive external power and input the power to the different
generation station 400 . 25 components via assistance from the power distribution sys
The power transformation equipment 402 represents tem 506. As discussed with respect to FIG . 2 , the sources of
equipment that can be utilized to enable power delivery from external power feeding a flexible datacenter can vary in both
the power generation equipment 401 to the loads and to quantity and type (e.g., the generation stations 202 , 400 ,
transmission lines linked to the grid . Example power trans grid -power, energy storage systems). Power input system
formation equipment 402 includes , but is not limited to , 30 502 includes a BTM power input sub -system 522 , and may
transformers, inverters , phase converters , and power condi additionally include other power input sub - systems (e.g., a
tioners . grid -power input sub-system 524 and /or an energy storage
FIG . 5 shows a block diagram of a flexible datacenter 500 , input sub - system 526 ). In some instances, the quantity of
according to one or more example embodiments. Flexible power input sub -systems may depend on the size of the
datacenters 220 may take the form of flexible datacenter 35 flexible datacenter and the number and /or type of computing
500 , or may include less than all components in flexible systems being powered . In an example embodiment, the
datacenter 500 , different components than in flexible data flexible datacenter may use grid power as the primary power
center 500, and/or more components than in flexible data supply.
center 500. In the example embodiment shown in FIG . 5 , the In some embodiments , the power input system 502 may
flexible datacenter 500 includes a power input system 502 , 40 include some or all of flexible datacenter Power Equipment
a communication interface 503 , a datacenter control system 220B . The power input system 502 may be designed to
504, a power distribution system 506 , a climate control obtain power in different forms ( e.g., single phase or three
system 508 , one ormore sets of computing systems512 , and phase behind-the -meter alternating current (“ AC ” ) voltage,
a queue system 514. These components are shown con and /or direct current (“ DC ” ) voltage ). As shown , the power
nected by a communication bus 528. In other embodiments , 45 input system 502 includes a BTM power input sub -system
the configuration of flexible datacenter 500 can differ, 522 , a grid power input sub -system 524 , and an energy input
including more or fewer components. In addition , the com sub -system 526. These sub -systems are included to illustrate
ponents within flexible datacenter 500 may be combined or example power input sub -systems that the flexible datacen
further divided into additional components within other ter 500 may utilize, but other examples are possible . In
embodiments . 50 addition , in some instances, these sub -systemsmay be used
The example configuration shown in FIG . 5 represents simultaneously to supply power to components of the flex
one possible configuration for a flexible datacenter. As such , ible datacenter 500. The sub - systemsmay also be used based
each flexible datacenter may have a different configuration on available power sources.
when implemented based on a variety of factors that may In some implementations, the BTM power input sub
influence its design , such as location and temperature that 55 system 522 may include one or more AC -to -AC step -down
the location , particular uses for the flexible datacenter, transformers used to step down supplied medium - voltage
source of power supplying computing systems within the AC to low voltage AC ( e.g., 120V to 600V nominal) used to
flexible datacenter, design influence from an entity (or power computing systems 512 and/or other components of
entities) that implements the flexible datacenter , and space flexible datacenter 500. The power input system 502 may
available for the flexible datacenter . Thus, the embodiment 60 also directly receive single -phase low voltage AC from a
of flexible datacenter 220 shown in FIG . 2 represents one generation station as BTM power, from grid power , or from
possible configuration for a flexible datacenter out ofmany a stored energy system such as energy storage system 218 .
other possible configurations . In some implementations , the power input system 502 may
The flexible datacenter 500 may include a design that provide single -phase AC voltage to the datacenter control
allows for temporary and /or rapid deployment, setup , and 65 system 504 (and/or other components of flexible datacenter
start time for supporting computational operations . For 500 ) independent of power supplied to computing systems
instance , the flexible datacenter 500 may be rapidly 512 to enable the datacenter control system 504 to perform
US 10,608,433 B1
31 32
management operations for the flexible datacenter 500. For systems 512 based on various factors, such as BTM power
instance, the grid power input sub -system 524 may use grid availability or an operational directive from a generation
power to supply power to the datacenter control system 504 station 262 or 300 control system , a remote master control
to ensure that the datacenter control system 504 can perform system 262 or 300 , or a grid operator. In some examples, the
control operations and communicate with the remote master 5 datacenter control system 504 may provide computational
control system 300 (or 262) during situations when BTM operations to sets of computing systems 512 and modulate
power is not available. As such , the datacenter control power delivery based on priorities assigned to the compu
system 504 may utilize power received from the power input tational operations . For instance , an important computa
system 502 to remain powered to control the operation of tional operation (e.g., based on a deadline for execution
flexible datacenter
performed 500,even ifsystem
by the computing the computational operations
512 are powered inter 10 particular
and/or price paid by an entity) may be assigned to a
mittently. In some instances, the datacenter control system computing system or set of computing systems
512 that
504 may switch into a lower power mode to utilize less the computational has the capacity, computational abilities to support
power while still maintaining the ability to perform some control system 504may operation . In addition , the datacenter
functions. 15 also prioritize power delivery to the
The power distribution system 506 may distribute incom computing system or set of computing systems 512 .
ing power to the various components of the flexible data In some example , the datacenter control system 504 may
center 500. For instance , the power distribution system 506 further provide directives to one ormore computing systems
may direct power (e.g., single-phase or three-phase AC ) to to change operations in some manner. For instance, the
one or more components within flexible datacenter 500. In 20 datacenter control system 504 may cause one or more
some embodiments , the power distribution system 506 may computing systems 512 to operate at a lower or higher
include some or all of flexible datacenter Power Equipment frequency, change clock cycles , or operate in a different
220B power con otion mode (e.g., a low power mode ). These
In some examples , the power input system 502 may abilities may vary depending on types of computing systems
provide three phases of three -phase AC voltage to the power 25 512 available at the flexible datacenter 500. As a result, the
distribution system 506. The power distribution system 506 datacenter control system 504 may be configured to analyze
may controllably provide a single phase of AC voltage to the computing systems 512 available either on a periodic
each computing system or groups of computing systems512 basis ( e.g., during initial set up of the flexible datacenter
disposed within the flexible datacenter 500. The datacenter 500 ) or in another manner (e.g., when a new computational
control system 504 may controllably select which phase of 30
three -phase nominal AC voltage that power distribution operation is assigned to the flexible datacenter 500).
system 506 provides to each computing system 512 or directives receivedcontrol
The datacenter system 504 may also implement
from the remote master control system
groups of computing systems 512. This is one example 262 or 300. For instance , the remote master control system
manner in which the datacenter control system 504 may
modulate power delivery (and load at the flexible datacenter 35 262 or 300 may direct the flexible datacenter 500 to switch
500 ) by ramping -up flexible datacenter 500 to fully opera into a low power mode . As a result, one or more of the
computing systems512 and other componentsmay switch to
tional status, ramping -down flexible datacenter 500 to
offline status (where only datacenter control system 504 the low power mode in response .
remains powered ), reducing load by withdrawing power The datacenter control system 504 may utilize the com
delivery from , or reducing power to , one or more of the 40 munication interface 503 to communicate with the remote
computing systems 512 or groups of the computing systems master control system 262 or 300 , other datacenter control
512 , or modulating power factor correction for the genera systems of other datacenters, and other entities. As such , the
tion station 300 (or 202) by controllably adjusting which communication interface 503 may include components and
phases of three -phase nominal AC voltage are used by one operate similar to the communication interface 306 of the
or more of the computing systems 512 or groups of the 45 remote master control system 300 described with respect to
computing systems 512. The datacenter control system 504 FIG . 4 .
may direct power to certain sets of computing systemsbased The flexible datacenter 500 may also include a climate
on computational operations waiting for computational control system 508 to maintain computing systems 512
resources within the queue system 514. In some embodi within a desired operational temperature range. The climate
ments, the flexible datacenter 500 may receive BTM DC 50 control system 508 may include various components , such
power to power the computing systems 512 . as one or more air intake components, an evaporative
One of ordinary skill in the art will recognize that a cooling system , one or more fans, an immersive cooling
voltage level of three -phase AC voltage may vary based on system , an air conditioning or refrigerant cooling system ,
an application or design and the type or kind of local power and one or more air outtake components. One of ordinary
generation. As such, a type , kind , or configuration of the 55 skill in the art will recognize that any suitable heat extraction
operational AC - to -AC step down transformer (not shown) system configured to maintain the operation of computing
may vary based on the application or design . In addition , the
systems 512 within the desired operational temperature
frequency and voltage level of three -phase AC voltage , range may be used .
single -phase AC voltage, and DC voltage may vary based on The flexible datacenter 500 may further include an energy
the application or design in accordance with one or more 60 storage system 510. The energy storage system 510 may
embodiments . store energy for subsequent use by computing systems 512
As discussed above , the datacenter control system 504 and other components of flexible datacenter 500. For
may perform operations described herein , such as dynami instance , the energy storage system 510 may include a
cally modulating power delivery to one or more of the battery system . The battery system may be configured to
computing systems 512 disposed within flexible datacenter 65 convert AC voltage to DC voltage and store power in one or
500. For instance , the datacenter control system 504 may more storage cells. In some instances, the battery system
modulate power delivery to one or more of the computing may include a DC -to -AC inverter configured to convert DC
US 10,608,433 B1
33 34
voltage to AC voltage, and may further include an AC configuration . This redundant configuration creates a back
phase - converter, to provide AC voltage for use by flexible up that prevents losing progress on the computational opera
datacenter 500 . tion in situations of a computing failure or intermittent
The energy storage system 510 may be configured to operation of one or more computing systems. In addition ,
serve as a backup source of power for the flexible datacenter 5 the computing systems 512 may also perform computational
500. For instance , the energy storage system 510 may operations using a check point system . The check point
receive and retain power from a BTM power source at a low system may enable a first computing system to perform
cost (or no cost at all). This low -cost power can then be used operations up to a certain point (e.g., a checkpoint) and
by the flexible datacenter 500 at a subsequent point, such as switch to a second computing system to continue performing
when BTM power costs more. Similarly , the energy storage 10 the operations from that certain point. The check point
system 510 may also store energy from other sources (e.g., system may also enable the datacenter control system 504 to
grid power ). As such , the energy storage system 510 may be communicate statuses of computational operations to the
configured to use one or more of the sub -systems of the remote master control system 262 or 300. This can further
power input system 502. enable the remote master control system 262 300 to transfer
In some examples , the energy storage system 510 may be 15 ters
external to the flexible datacenter 500. For instance , the
computational operationsbetween different flexible datacen
allowing computing systems at the different flexible
energy storage system 510 may be an external source that datacenters to resume support of computational operations
multiple flexible datacenters utilize for back -up power. based on the check points .
The computing systems 512 represent various types of The queue system 514 may operate similar to the queue
computing systems configured to perform computational 20 system 312 of the remote master control system 300 shown
operations . Performance of computational operations in FIG . 3. Particularly , the queue system 514 may help store
include a variety of tasks that one or more computing and organize computational tasks assigned for performance
systems may perform , such as data sto ge , calculations , at the flexible datacenter 500. In some examples, the queue
application processing , parallel processing , data manipula system 514 may be part of a distributed queue system such
tion , cryptocurrency mining , and maintenance of a distrib- 25 that each flexible datacenter in a fleet of flexible datacenter
uted ledger, among others. As shown in FIG . 5 , the com includes a queue , and each queue system 514 may be able
puting systems 512 may include one or more CPUs 516 , one to communicate with other queue systems. In addition , the
or more GPUs 518, and /or one or more Application -Specific remote master control system 262 or 300 may be configured
Integrated Circuits (ASIC's ) 520. Each type of computing to assign computational tasks to the queues located at each
system 512 may be configured to perform particular opera- 30 flexible datacenter (e.g., the queue system 514 of the flexible
tions or types of operations. datacenter 500 ). As such , communication between the
Due to different performance features and abilities asso remote master control system 262 or 300 and the datacenter
ciated with the different types of computing systems, the control system 504 and/or the queue system 514 may allow
datacenter control system 504 may determine , maintain , organization of computational operations for the flexible
and /or relay this information about the types and/or abilities 35 datacenter 500 to support.
of the computing systems, quantity of each type, and avail FIG . 6A shows another structural arrangement for a
ability to the remote master control system 262 or 300 on a flexible datacenter, according to one or more example
routine basis ( e.g.,periodically or on -demand ). This way, the embodiments . The particular structural arrangement shown
remote master control system 262 or 300 may have current in FIG . 6A may be implemented at flexible datacenter 500 .
information about the abilities of the computing systems 512 40 The illustration depicts the flexible datacenter 500 as a
when distributing computational operations for performance mobile container 702 equipped with the power input system
at one or more flexible datacenters . Particularly , the remote 502 , the power distribution system 506 , the climate control
master control system 262 or 300 may assign computational system 508, the datacenter control system 504 , and the
operations based on various factors, such as the types of computing systems512 arranged on one or more racks 604 .
computing systems available and the type of computing 45 These components of flexible datacenter 500 may be
systems required by each computing operation , the avail arranged and organized according to an example structural
ability of the computing systems, whether computing sys region arrangement. As such , the example illustration rep
tems can operate in a low power mode, and / or power resents one possible configuration for the flexible datacenter
consumption and/ or costs associated with operating the 500 , but others are possible within examples .
computing systems, among others. 50 As discussed above, the structural arrangement of the
The quantity and arrangement of these computing systems flexible datacenter 500 may depend on various factors, such
512 may vary within examples . In some examples, the as the ability to maintain temperature within the mobile
configuration and quantity of computing systems 512 may container 602 within a desired temperature range. The
depend on various factors, such as the computational tasks desired temperature range may depend on the geographical
that are performed by the flexible datacenter 500. In other 55 location of the mobile container 602 and the type and
examples, the computing systems 512 may include other quantity of the computing systems 512 operating within the
types of computing systems as well, such as DSPs, SIMDs, flexible datacenter 500 as well as other possible factors. As
neural processors , and /or quantum processors . such , the different design elements of the mobile container
As indicated above, the computing systems 512 can 602 including the inner contents and positioning of compo
perform various computational operations, including in dif- 60 nents may depend on factors that aim to maximize the use
ferent configurations. For instance, each computing system of space within mobile container 602 , lower the amount of
may perform a particular computational operation unrelated power required to cool the computing systems 512 , and
to the operations performed at other computing systems. make setup of the flexible datacenter 500 efficient. For
Groups of the computing systems 512 may also be used to instance, a first flexible datacenter positioned in a cooler
work together to perform computational operations . 65 geographic region may include less cooling equipment than
In some examples , multiple computing systems may a second flexible datacenter positioned in a warmer geo
perform the same computational operation in a redundant graphic region .
US 10,608,433 B1
35 36
As shown in FIG . 6A , the mobile container 602 may be control system 300 over a networked connection 706 and
a storage trailer disposed on permanent or removable wheels with the datacenter control system 704 over a networked or
and configured for rapid deployment. In other embodiments , other data connection 708 .
the mobile container 602 may be a storage container (not As discussed with respect to FIGS . 2 and 3 , the remote
shown ) configured for placement on the ground and poten 5 master control system 300 can be one or more computing
tially stacked in a vertical or horizontal manner (not shown). systems 710located offsite, but connected via a network con
In still other embodiments , the mobile container 602 may be nection to the datacenter control system 504. The
remote master control system 300 may provide supervisory
an inflatable container, a floating container, or any other type controls or override control of the flexible datacenter 500 or
or kind of container suitable for housing a mobile flexible
datacenter. As such, the flexible datacenter 500 may be 10 a fleet
The of
gridflexible datacenters
operator 702 may(not shownor).more computing
be one
rapidly deployed on site near a source of unutilized behind systems that are configured to control various aspects of the
the-meter power generation . And in still other embodiments , power grid (not independently illustrated ) that receives
the flexible datacenter 500 might not include a mobile power from the generation station . The grid operator 702
container. For example, the flexible datacenter 500 may be 15 may communicate with the generation station control system
situated within a building or another type of stationary 300 over a networked or other data connection 712 .
environment. The datacenter control system 504 may monitor BTM
FIG . 6B shows the computing systems 512 in a straight power conditions at the generation station and determine
line configuration for installation within the flexible data when a datacenter ramp- up condition is met. The BTM
center 500, according to one or more example embodiments . 20 power availability may include one or more of excess local
As indicated above, the flexible datacenter 500 may include power generation , excess local power generation that the
a plurality of racks 604, each of which may include one or grid cannot accept, local power generation that is subject to
more computing systems 512 disposed therein . As discussed economic curtailment, local power generation that is subject
above, the power input system 502 may provide three phases to reliability curtailment, local power generation that is
of AC voltage to the power distribution system 506. In some 25 subject to power factor correction , conditions where the cost
examples, the power distribution system 506 may control for power is economically viable ( e.g., low cost to obtain
lably provide a single phase of AC voltage to each comput power), low priced power, situations where local power
ing system 512 or group of computing systems 512 disposed generation is prohibitively low , start up situations, transient
situations, or testing situations where there is an economic
within the flexible datacenter 500. As shown in FIG . 6B , for 30 advantage
purposes of illustration only, eighteen total racks 604 are power generation to using locally generated behind -the -meter
divided into a first group of six racks 606 , a second group of , specifically power available at little to no
cost and with no associated transmission or distribution
six racks 608 , and a third group of six racks 610 , where each
rack contains eighteen computing systems 512. The power losses or costs . For example , a datacenter control system
may analyze future workload and near term weather condi
distribution system (506 of FIG . 5 ) may, for example , 35 tions at the flexible datacenter.
provide a first phase of three -phase AC voltage to the first
group of six racks 606 , a second phase of three -phase AC be Inmetsome instances, the datacenter ramp -up condition may
if there is sufficient behind -the -meter power avail
voltage to the second group of six racks 608 , and a third ability and there is no operational directive from the